• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 555
  • Last Modified:

Unix log monitoring

hi,

Is there any simple shell script/ awk script which can monitor a log file (maybe read new lines as they are logged) for a set of predefined errors  and lance an alert ?

I found a few tools but they appeared to require quite a bit of configuration.. I dont want something too sofisticated but something more efficient than a :

grep "memory exception"  server.log

thnx,

sg

0
sgaucho
Asked:
sgaucho
  • 6
  • 5
  • 5
  • +2
2 Solutions
 
adacunhaCommented:
something like following should help you (ksh is only used):

#!/bin/ksh

LogFile=<log file>
ErrorPattern=<error pattern, ex: @(string1|string2|foo*bar)
JobCmd=<cmd to run, ex: print |Mail -s alert root>


tail -f <log file> | while read;do
  eval [[ $REPLY = *$ErrorPattern* ]] && eval $JobCmd  #2nd 'eval' allows use of '|' and more into $JobCmd...
done

Dean.
0
 
ahoffmannCommented:
how about using perl with File::Tail
0
 
ppfoongCommented:

Well, if you don't find SWatch as complicated, you might want to take a look into it.

http://swatch.sourceforge.net/

0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
sgauchoAuthor Commented:
Hi,

Swatch requires perl and when executing it gave me some errors like "Warning: prerequisite Date::Calc failed to load:" which requires additional packages to be installed..

I tried using a counterpart of Swatch which is LogSurfer which also has other complications..  I have a team thats developing a customizes log analyzer but I wanted up and running a simple system instead for waiting another few weeks..
0
 
ppfoongCommented:

You need the following 5 perl modules for Swatch to run:

Time::HiRes
Date::Calc
Date::Format
File::Tail
Date::Parse


And you just need to run the following command to get them installed:

perl -MCPAN -e "shell"
install Time::HiRes
install Date::Calc
install Date::Format
install File::Tail
install Date::Parse
quit

0
 
sgauchoAuthor Commented:
Hi,

I had already tried that. The system is behind a firewall and cannot access the net to download the packages. Further, am not sure whether I need ROOT access (which I dont have) to the machine to install these packages..

thats the reason, I am out looking for something simple and efficient..

tnx
0
 
ppfoongCommented:

Then perhaps:

tail -f server.log | grep "memory exception"  > output.log &


0
 
tel2Commented:
adacunha's solution looks intriguing, but I can't get it to work.  Can anyone explain:
1. ErrorPattern=@(string1|string2|foo*bar)
Some kind of array?
2. *$ErrorPattern*
What's with the "*"s?
0
 
ahoffmannCommented:
> ErrorPattern=<error pattern, ex: @(string1|string2|foo*bar)
# use something like:

ErrorPattern='(string1|string2|foo*bar)'
tail -f <log file> | while read line;do
   egrep "$ErrorPattern" $line && mail -s"pattern match" you@some.where
done
0
 
sgauchoAuthor Commented:
hi,

I tried both adacunha's and ahoffmann┬┤s solutions but still getting nowwhere.. I have the script montoring the logs but its not generating any kind of usefull info...

Basically, when I execute the script, I see a lot of output such as:

egrep: can't open at
egrep: can't open weblogic.servlet.internal.ServletStubImpl$ServletInvocationAction.run(ServletStubImpl.java:971)
egrep: can't open at
egrep: can't open weblogic.servlet.internal.ServletStubImpl.invokeServlet(ServletStubImpl.java:402)
egrep: can't open at

and so on...

any suggestions ? thnx
0
 
adacunhaCommented:
Hi,

I may miss some "
Try this, it works:

#!/bin/ksh

LogFile=/tmp/foolog
ErrorPattern=@(failed|error)
JobCmd='print "$REPLY"|Mail -s alert root'


tail -f $LogFile | while read;do
  eval [[ "$REPLY" = *$ErrorPattern* ]] && eval "$JobCmd"
done
0
 
ahoffmannCommented:
> egrep: can't open at

opps, typed suggestion to fast, try:

   echo "$line" | egrep "$ErrorPattern" && mail -s"pattern match" you@some.where

BTW, for some egrep you probably want to add -q or -s option for this kind of test
0
 
adacunhaCommented:
one correction more:

#!/bin/ksh

LogFile=/tmp/foolog
ErrorPattern=@(failed|error)
JobCmd='print "$REPLY"|Mail -s alert root'


tail -f $LogFile | while read;do
  eval [[ "\$REPLY" = *$ErrorPattern* ]] && eval "$JobCmd"
done
0
 
sgauchoAuthor Commented:
hi,

adacunha┬┤s soln seems to be working but  I am not receiving the alerts with the Subject.. why is that ?

My $JobCmd='print "$REPLY" | /bin/mail -s "PRD: SERVER LOG ALERT"  abc@test.com'

thnx,
0
 
adacunhaCommented:
I prefer use:
Mail -s "my subject" recipient@mydomain
0
 
ahoffmannCommented:
does your /bin/mail work at all to send you mails?
0
 
sgauchoAuthor Commented:
I  have to user /bin/mail and yes it works fine.. I am able to receive mails sent from the script except without the subject.

thnx
0
 
ahoffmannCommented:
depends on your version of /bin/mail, some won't allow a space betwee -s and the text for the subject, use:
  mail -s"your subject" ....
0
 
sgauchoAuthor Commented:
I got it.. Gotta use MAILX and it works..

Just one last question before I close this topic..

Can my ErrorPattern read like this:

ErrorPattern='(Out of memory Exception|LDAP threads|locked|Warning)'  ??

tnx
0
 
adacunhaCommented:
The korn shell pattern is:
ErrorPattern='@(Out of memory Exception|LDAP threads|locked|Warning)'

The equivalent extented regular expression is:
ErrorPattern='(Out of memory Exception|LDAP threads|locked|Warning)'

0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

  • 6
  • 5
  • 5
  • +2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now