Solved

Applying Logic To A Subset of File

Posted on 2008-10-01
3
302 Views
Last Modified: 2013-12-26
I have a script that searches a large file, and does arithmetic depending on what it finds.

Its a bit script but the bit of interest is:

for FILE in `ls $FILEMASK`
do
  while read line
  do
    if [ $1 = "NUMBER" ]
    then
      TOTAL=`expr $TOTAL + $2`
    fi
  done < $FILE
echo "Total is $TOTAL"
done

This works on all but the very largest of files.

To make it work for even the largest files, I want to egrep the file, and apply the locgic to the result of the egrep.

I guess I could egrep to a temporary file and change the filemask to process just that file, but is there a way to do it on the fly?
0
Comment
Question by:stummj
3 Comments
 
LVL 14

Accepted Solution

by:
sjm_ee earned 500 total points
ID: 22616275
for FILE in `ls $FILEMASK`
do
  egrep PATTERN $FILE | while read line
  do
    if [ $1 = "NUMBER" ]
    then
      TOTAL=`expr $TOTAL + $2`
    fi
  done
echo "Total is $TOTAL"
done
0
 

Author Closing Comment

by:stummj
ID: 31502033
Great. Thanks very much.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 22619259
You'll find that a while loop and expr will be very slow on large files.  *Much*  quicker to do
for file in $FILEMASK

do

  awk '/PATTERN/ && $1=="NUMBER" {t+=$2} END {print "Total is", t}' $file

done

Open in new window

0

Featured Post

Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

This Windows batch file is useful for organizing image files from a digital camera or other source, but can have many other uses.  It simply renames the file(s) to match their create date.  For example, if you took a picture today at 1:40pm and the …
How to remove superseded packages in windows w60 or w61 installation media (.wim) or online system to prevent unnecessary space. w60 means Windows Vista or Windows Server 2008. w61 means Windows 7 or Windows Server 2008 R2. There are various …
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're looking for how to monitor bandwidth using netflow or packet s…

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now