Solved

Applying Logic To A Subset of File

Posted on 2008-10-01
3
303 Views
Last Modified: 2013-12-26
I have a script that searches a large file, and does arithmetic depending on what it finds.

Its a bit script but the bit of interest is:

for FILE in `ls $FILEMASK`
do
  while read line
  do
    if [ $1 = "NUMBER" ]
    then
      TOTAL=`expr $TOTAL + $2`
    fi
  done < $FILE
echo "Total is $TOTAL"
done

This works on all but the very largest of files.

To make it work for even the largest files, I want to egrep the file, and apply the locgic to the result of the egrep.

I guess I could egrep to a temporary file and change the filemask to process just that file, but is there a way to do it on the fly?
0
Comment
Question by:stummj
3 Comments
 
LVL 14

Accepted Solution

by:
sjm_ee earned 500 total points
ID: 22616275
for FILE in `ls $FILEMASK`
do
  egrep PATTERN $FILE | while read line
  do
    if [ $1 = "NUMBER" ]
    then
      TOTAL=`expr $TOTAL + $2`
    fi
  done
echo "Total is $TOTAL"
done
0
 

Author Closing Comment

by:stummj
ID: 31502033
Great. Thanks very much.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 22619259
You'll find that a while loop and expr will be very slow on large files.  *Much*  quicker to do
for file in $FILEMASK

do

  awk '/PATTERN/ && $1=="NUMBER" {t+=$2} END {print "Total is", t}' $file

done

Open in new window

0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Utilizing an array to gracefully append to a list of EmailAddresses
Active Directory replication delay is the cause to many problems.  Here is a super easy script to force Active Directory replication to all sites with by using an elevated PowerShell command prompt, and a tool to verify your changes.
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
With Secure Portal Encryption, the recipient is sent a link to their email address directing them to the email laundry delivery page. From there, the recipient will be required to enter a user name and password to enter the page. Once the recipient …

932 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now