daryn
asked on
linux bash script problem
Hi all.
I know I should learn more bash programming and I want to and I will but right now, I'm somewhat ashamed to admit that I need help and
haven't got the time to learn what I need to know.
so, my problem:
I have an application that outputs a log file (rotated daily) of the format:
17-Apr-2006 20:16:28 [proc1] [INFO ] [mainapp] 15845 Request: xxx.xxx.xxx.xxx
17-Apr-2006 20:16:28 [proc1] [INFO ] [mainapp] 15845 Info: blah/blah1/blah2/blah3
17-Apr-2006 20:16:29 [proc1] [INFO ] [mainapp] 15845 Response: 00000000000000000000000000 005Ev2 - OK blah/blah1/blah2/blah3
Each successful process on my application results in a set of three lines of code that go into /logs/blah.log, each set of three log entries is linked by the id number, in this case 15845 (names, processes and etc changed to protect the innocent and copyrighted..). However, sometimes my application crashes and in that circumstance, it only writes lines 1 and 2.
I need a script that will tail, say, the last 50 lines of the file, check that
how the script does this, I dont know.
maybe it will get the order number from the file through grep/awk then grep the log file for that order number (surrounded by spaces) and pipe it through "wc -l" (but this would have a problem if the system was in the middle of processing something and was half way through a set of "three log entries").
I dunno.
any ideas please?
thanks v much for any help offered.
Daryn
I know I should learn more bash programming and I want to and I will but right now, I'm somewhat ashamed to admit that I need help and
haven't got the time to learn what I need to know.
so, my problem:
I have an application that outputs a log file (rotated daily) of the format:
17-Apr-2006 20:16:28 [proc1] [INFO ] [mainapp] 15845 Request: xxx.xxx.xxx.xxx
17-Apr-2006 20:16:28 [proc1] [INFO ] [mainapp] 15845 Info: blah/blah1/blah2/blah3
17-Apr-2006 20:16:29 [proc1] [INFO ] [mainapp] 15845 Response: 00000000000000000000000000
Each successful process on my application results in a set of three lines of code that go into /logs/blah.log, each set of three log entries is linked by the id number, in this case 15845 (names, processes and etc changed to protect the innocent and copyrighted..). However, sometimes my application crashes and in that circumstance, it only writes lines 1 and 2.
I need a script that will tail, say, the last 50 lines of the file, check that
how the script does this, I dont know.
maybe it will get the order number from the file through grep/awk then grep the log file for that order number (surrounded by spaces) and pipe it through "wc -l" (but this would have a problem if the system was in the middle of processing something and was half way through a set of "three log entries").
I dunno.
any ideas please?
thanks v much for any help offered.
Daryn
perl -ane '$id{$F[5]}++;END{for(keys %id){ print "$_ only $id{$_} lines\n" if $id{$_}<3 } }' /logs/blah.log
-- OR --
tail -50 /logs/blah.log|\
awk 'BEGIN {n=0;f=0}
{ if (n == 0) k=$7;
print k " " NR " - " $7 ":" $8 " f" f " n" n;
if ( k == $7 ) n=n+1
else {
if ( f == 1 && n < 3 ) {print k " Missing line. " f;}
n=1;f=1;}
k = $7;
}'
Are all ids with the same number guaranteed to be together in the log file, and not mixed with log entries from other ids?
ASKER
thats the thing, (due to my not mentioning this somewhat crucial fact) neither your perl script not the bash script from MikeOM_DBA works too well since the id numbers are not sequential, they can be mixed up, ie 2 lines re: 15845, then a line for 15846, then the final line for 15845, then two lines of 15847, then the final two lines of 15846 etc...
apologies for missing out that fact, I've got a stinking cold and no sleep.. :)
apologies for missing out that fact, I've got a stinking cold and no sleep.. :)
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
>>this would have a problem if the system was in the middle of processing something and was half way through a set of "three log entries".
Unless you ignore the "first" group after the "tail" command and the last group at the end of the file.