Link to home
Start Free TrialLog in
Avatar of daryn
daryn

asked on

linux bash script problem

Hi all.

I know I should learn more bash programming and I want to and I will but right now, I'm somewhat ashamed to admit that I need help and
haven't got the time to learn what I need to know.

so, my problem:

I have an application that outputs a log file (rotated daily) of the format:

17-Apr-2006 20:16:28 [proc1] [INFO ] [mainapp] 15845 Request: xxx.xxx.xxx.xxx
17-Apr-2006 20:16:28 [proc1] [INFO ] [mainapp] 15845 Info: blah/blah1/blah2/blah3
17-Apr-2006 20:16:29 [proc1] [INFO ] [mainapp] 15845 Response: 00000000000000000000000000005Ev2 - OK blah/blah1/blah2/blah3

Each successful process on my application results in a set of three lines of code that go into /logs/blah.log, each set of three log entries is linked by the id number, in this case 15845 (names, processes and etc changed to protect the innocent and copyrighted..). However, sometimes my application crashes and in that circumstance, it only writes lines 1 and 2.  

I need a script that will tail, say, the last 50 lines of the file, check that

how the script does this, I dont know.

maybe it will get the order number from the file through grep/awk then grep the log file for that order number (surrounded by spaces) and pipe it through "wc -l" (but this would have a problem if the system was in the middle of processing something and was half way through a set of "three log entries").

I dunno.

any ideas please?

thanks v much for any help offered.

Daryn
Avatar of MikeOM_DBA
MikeOM_DBA
Flag of United States of America image

This could be done with a simple awk program, but your problem actually resides here:

>>this would have a problem if the system was in the middle of processing something and was half way through a set of "three log entries".

Unless you ignore the "first" group after the "tail" command and the last group at the end of the file.

Avatar of ozo
perl -ane '$id{$F[5]}++;END{for(keys %id){ print "$_ only $id{$_} lines\n" if $id{$_}<3 } }' /logs/blah.log

-- OR --
tail -50 /logs/blah.log|\
awk 'BEGIN {n=0;f=0}
{ if (n == 0) k=$7;
  print k " " NR " - " $7 ":" $8 " f" f " n" n;
  if ( k == $7 ) n=n+1
  else {
    if ( f == 1 && n < 3 ) {print k " Missing line. " f;}
    n=1;f=1;}
  k = $7;
}'
Are all ids with the same number guaranteed to be together in the log file, and not mixed with log entries from other ids?
Avatar of daryn
daryn

ASKER

thats the thing, (due to my not mentioning this somewhat crucial fact) neither your perl script not the bash script from MikeOM_DBA works too well since the id numbers are not sequential, they can be mixed up, ie 2 lines re: 15845, then a line for 15846, then the final line for 15845, then two lines of 15847, then the final two lines of 15846 etc...

apologies for missing out that fact, I've got a stinking cold and no sleep.. :)

ASKER CERTIFIED SOLUTION
Avatar of ahoffmann
ahoffmann
Flag of Germany image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial