[2 days left] What’s wrong with your cloud strategy? Learn why multicloud solutions matter with Nimble Storage.Register Now

x
?
Solved

Applying Logic To A Subset of File

Posted on 2008-10-01
3
Medium Priority
?
312 Views
Last Modified: 2013-12-26
I have a script that searches a large file, and does arithmetic depending on what it finds.

Its a bit script but the bit of interest is:

for FILE in `ls $FILEMASK`
do
  while read line
  do
    if [ $1 = "NUMBER" ]
    then
      TOTAL=`expr $TOTAL + $2`
    fi
  done < $FILE
echo "Total is $TOTAL"
done

This works on all but the very largest of files.

To make it work for even the largest files, I want to egrep the file, and apply the locgic to the result of the egrep.

I guess I could egrep to a temporary file and change the filemask to process just that file, but is there a way to do it on the fly?
0
Comment
Question by:stummj
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
3 Comments
 
LVL 14

Accepted Solution

by:
sjm_ee earned 2000 total points
ID: 22616275
for FILE in `ls $FILEMASK`
do
  egrep PATTERN $FILE | while read line
  do
    if [ $1 = "NUMBER" ]
    then
      TOTAL=`expr $TOTAL + $2`
    fi
  done
echo "Total is $TOTAL"
done
0
 

Author Closing Comment

by:stummj
ID: 31502033
Great. Thanks very much.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 22619259
You'll find that a while loop and expr will be very slow on large files.  *Much*  quicker to do
for file in $FILEMASK
do
  awk '/PATTERN/ && $1=="NUMBER" {t+=$2} END {print "Total is", t}' $file
done

Open in new window

0

Featured Post

On Demand Webinar: Networking for the Cloud Era

Did you know SD-WANs can improve network connectivity? Check out this webinar to learn how an SD-WAN simplified, one-click tool can help you migrate and manage data in the cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The following is a collection of cases for strange behaviour when using advanced techniques in DOS batch files. You should have some basic experience in batch "programming", as I'm assuming some knowledge and not further explain the basics. For some…
Active Directory replication delay is the cause to many problems.  Here is a super easy script to force Active Directory replication to all sites with by using an elevated PowerShell command prompt, and a tool to verify your changes.
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…
Suggested Courses

649 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question