?
Solved

cat command read one file at a time

Posted on 2011-04-28
9
Medium Priority
?
291 Views
Last Modified: 2012-05-11
Currently I have a cat command that reads all the files in a directory, grep the data I want, then output it to a file.

Is there a way to have each file processed 1 at a time?

Below is my code and example output.

I would like the output to be
 
042811104531$  <--this is BEGIN:VCAL output
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN$  <---this is PROID: output
042811104531$ <---this is VERSION:2.0 output
042811104531$   <--this is BEGIN:VCAL output
PRODID:-//Microsoft Corporation//Outlook 11.0 MIMEDIR//EN$    <---this is PROID: output
042811104531$   <---this is VERSION:2.0 output

#process each file
for f in $todaydir
do
	echo "Processing file(s)..."
	#cat -E dfp* | egrep -i -e "VCAL|BEGIN:VEVENT|VERSION|SUMMARY|DESCRIPTION|DTSTART;TZID|DTEND;TZID|LOCATION|VEVENT|TZNAME|TRANSP:OPAQUE" > /root/Desktop/test.txt
	cat -E dfp* | egrep -i -e "BEGIN:VCAL" > /root/Desktop/test.txt
	cat -E dfp* | egrep -i -e "PRODID:" >> /root/Desktop/test.txt
	cat -E dfp* | egrep -i -e "VERSION:2.0" >> /root/Desktop/test.txt
done

Open in new window

#current output below
042811104531$
042811104531$
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN$
PRODID:-//Microsoft Corporation//Outlook 11.0 MIMEDIR//EN$
042811104531$
042811104531$

Open in new window

0
Comment
Question by:SRG041808
  • 5
  • 4
9 Comments
 
LVL 31

Expert Comment

by:farzanj
ID: 35484425
Try something like this:


#!/bin/bash

FOLDER=/path/folder
FILE=/root/Desktop/test.txt
cd $FOLDER

for f in $(ls $FOLDER)
do
   echo "Now processing $f"
   egrep -i -e "VCAL|BEGIN:VEVENT|VERSION|SUMMARY|DESCRIPTION|DTSTART;TZID|DTEND;TZID|LOCATION|VEVENT|TZNAME|TRANSP:OPAQUE" $f >> $FILE
egrep -i -e "BEGIN:VCAL" > /root/Desktop/test.txt
   egrep -i -e "PRODID:" $f >> $FILE
   egrep -i -e "VERSION:2.0" $f >> $FILE
done

Open in new window

0
 

Author Comment

by:SRG041808
ID: 35484554
I think I see what we are trying to do but when I run it the script finds the correct directory and starts "processing files filenamehere" and appears to hang.
0
 

Author Comment

by:SRG041808
ID: 35484566
when I comment out the egrep commands it completes
0
NFR key for Veeam Agent for Linux

Veeam is happy to provide a free NFR license for one year.  It allows for the non‑production use and valid for five workstations and two servers. Veeam Agent for Linux is a simple backup tool for your Linux installations, both on‑premises and in the public cloud.

 
LVL 31

Expert Comment

by:farzanj
ID: 35484612
Which egrep? All of them?

I have provided $f filenames in egrep.  This would hang if the filename is missing OR is an empty string.  Since you see the filename, I assume you are missing the filenames.

Let me know.
0
 

Author Comment

by:SRG041808
ID: 35484684
Yes, I commented out all of them and re-add them as I figure out who is the troubled line...  Attached current script running output and modifications i have made to code.  in the FOR statement using ls or dir does not appear to have an effect...   the files that start with qfp do not need to be read.  
[root@myserver Desktop]# ./reademail.sh 
/data/MailArchive/archivecal/20110428
Now processing dfp3SDu5Fd009202
Now processing dfp3SE1aZ3012340
Now processing /data/MailArchive/archivecal/20110428:
egrep: /data/MailArchive/archivecal/20110428:: No such file or directory
Now processing dfp3SDu5Fd009202
Now processing dfp3SE1aZ3012340
Now processing qfp3SDu5Fd009202
Now processing qfp3SE1aZ3012340
script complete
[root@myserver Desktop]#

Open in new window

todaydir="/data/MailArchive/archivecal/"`eval date +%Y%m%d`
#make sure we found correct directory
echo $todaydir
#go into the directory
cd $todaydir
#process each file
FILE=/root/Desktop/test.txt

for f in $(dir dfp* $todaydir)
do
   echo "Now processing "$f
   cat -E dfp* |egrep -i -e "VCAL|BEGIN:VEVENT|VERSION|SUMMARY|DESCRIPTION|DTSTART;TZID|DTEND;TZID|LOCATION|VEVENT|TZNAME|TRANSP:OPAQUE" $f >> $FILE
	#egrep -i -e "BEGIN:VCAL" > $f
  	#egrep -i -e "PRODID:" $f >> $FILE
  	#egrep -i -e "VERSION:2.0" $f >> $FILE
done

Open in new window

0
 
LVL 31

Expert Comment

by:farzanj
ID: 35484762
There is a problem in the first commented egrep

It should be
egrep -i -e "BEGIN:VCAL" $f >>$FILE
0
 

Author Comment

by:SRG041808
ID: 35485415
Good call.  I fixed that line...  It appears somewhere that is it trying to process the folder name instead of the file name.  

It is also still trying to read the qfp* files instead of the dfp* files
0
 
LVL 31

Accepted Solution

by:
farzanj earned 1000 total points
ID: 35485724
Use this structure

cd $todaydir
for f in $(ls dfp*)
do
   if [[ -f $f ]]
   then
      echo "Now processing "$f
      #EGREP HERE
   fi
done

Open in new window

0
 

Author Closing Comment

by:SRG041808
ID: 35486547
That did the trick thanks!
0

Featured Post

Fill in the form and get your FREE NFR key NOW!

Veeam is happy to provide a FREE NFR server license to certified engineers, trainers, and bloggers.  It allows for the non‑production use of Veeam Agent for Microsoft Windows. This license is valid for five workstations and two servers.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

If you use Debian 6 Squeeze and you are tired of looking at the childish graphical GDM login screen that is used by default, here's an easy way to change it. If you've already tried to change it you've probably discovered that none of the old met…
The purpose of this article is to show how we can create Linux Mint virtual machine using Oracle Virtual Box. To install Linux Mint we have to download the ISO file from its website i.e. http://www.linuxmint.com. Once you open the link you will see …
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial
With just a little bit of  SQL and VBA, many doors open to cool things like synchronize a list box to display data relevant to other information on a form.  If you have never written code or looked at an SQL statement before, no problem! ...  give i…

862 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question