linux bash script problem

Hi all.

I know I should learn more bash programming and I want to and I will but right now, I'm somewhat ashamed to admit that I need help and
haven't got the time to learn what I need to know.

so, my problem:

I have an application that outputs a log file (rotated daily) of the format:

17-Apr-2006 20:16:28 [proc1] [INFO ] [mainapp] 15845 Request: xxx.xxx.xxx.xxx
17-Apr-2006 20:16:28 [proc1] [INFO ] [mainapp] 15845 Info: blah/blah1/blah2/blah3
17-Apr-2006 20:16:29 [proc1] [INFO ] [mainapp] 15845 Response: 00000000000000000000000000005Ev2 - OK blah/blah1/blah2/blah3

Each successful process on my application results in a set of three lines of code that go into /logs/blah.log, each set of three log entries is linked by the id number, in this case 15845 (names, processes and etc changed to protect the innocent and copyrighted..). However, sometimes my application crashes and in that circumstance, it only writes lines 1 and 2.  

I need a script that will tail, say, the last 50 lines of the file, check that

how the script does this, I dont know.

maybe it will get the order number from the file through grep/awk then grep the log file for that order number (surrounded by spaces) and pipe it through "wc -l" (but this would have a problem if the system was in the middle of processing something and was half way through a set of "three log entries").

I dunno.

any ideas please?

thanks v much for any help offered.

Daryn
darynAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

MikeOM_DBACommented:
This could be done with a simple awk program, but your problem actually resides here:

>>this would have a problem if the system was in the middle of processing something and was half way through a set of "three log entries".

Unless you ignore the "first" group after the "tail" command and the last group at the end of the file.

0
ozoCommented:
perl -ane '$id{$F[5]}++;END{for(keys %id){ print "$_ only $id{$_} lines\n" if $id{$_}<3 } }' /logs/blah.log
0
MikeOM_DBACommented:

-- OR --
tail -50 /logs/blah.log|\
awk 'BEGIN {n=0;f=0}
{ if (n == 0) k=$7;
  print k " " NR " - " $7 ":" $8 " f" f " n" n;
  if ( k == $7 ) n=n+1
  else {
    if ( f == 1 && n < 3 ) {print k " Missing line. " f;}
    n=1;f=1;}
  k = $7;
}'
0
Cloud Class® Course: Ruby Fundamentals

This course will introduce you to Ruby, as well as teach you about classes, methods, variables, data structures, loops, enumerable methods, and finishing touches.

ozoCommented:
Are all ids with the same number guaranteed to be together in the log file, and not mixed with log entries from other ids?
0
darynAuthor Commented:
thats the thing, (due to my not mentioning this somewhat crucial fact) neither your perl script not the bash script from MikeOM_DBA works too well since the id numbers are not sequential, they can be mixed up, ie 2 lines re: 15845, then a line for 15846, then the final line for 15845, then two lines of 15847, then the final two lines of 15846 etc...

apologies for missing out that fact, I've got a stinking cold and no sleep.. :)

0
ahoffmannCommented:
awk '(NR>6){a[$6]++}END{for(n in a){if(a[n]<3){print "bad "n}}}' /logs/blah.log
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux OS Dev

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.