Solved

Shell / Perl script to check for a value in a dynamic logfile & send out email once value hit threshold

Posted on 2011-03-03
9
1,337 Views
Last Modified: 2012-05-11
I need to constantly monitor for a value in a dynamically growing log file
& this log file will be rotated as well.

Say logfile name is xx.log & the most recent information is found at the
bottom of the logfile which has entries :

less relevant lines in logfile ...
dd-mmm-yyyy:hh:mm the current number of users :  255
....
dd-mmmyyyy:hh:mm the current number of users : 321
...


So I thought of a Shell ( Perl script welcomed too) script :
usercount=`tail -1f xx.log | grep "number of users" | awk '{print ($9)}' `
if [ $usercount .ge. 300 ]
then
 mailx -s "alert users now at $usercount" myemail@xx.com < /dev/null
fi

Problem is the logfile is dynamically moving & I'm not sure
if "tail -1f ..." to grab the last line of the logfile would work.
Also, not every new line written to the logfile contains
user count, thus I inserted
 ...  | grep "number of user" | ...

the fact the logfile gets rotated is probably not much of an
issue as we'll still be monitoring the same filename as
xx.log is the current logfile that's being written to


I'm not sure if the syntax "tail -1f ..." is supported on my
old HP-UX but I'm certain RHES Linux 4.x supports it.
So ideally the script provided don't make use of "tail -1f ..."


Needed this script urgently
0
Comment
Question by:sunhux
  • 4
  • 3
  • 2
9 Comments
 

Author Comment

by:sunhux
ID: 35028163

One more thing:

instead of using   " awk '{print ($9)}' ", best that the script
search based on the value that appears after "number of users : " 
in case there's variable number of text on that line prior to
"number of users : "
0
 
LVL 26

Accepted Solution

by:
wilcoxon earned 400 total points
ID: 35028642
I'm pretty sure tail -f will fail when the log is rotated (still pointing at the file that was originally opened).

This perl script automatically handles log rotation/truncation.
#!/usr/local/bin/perl

use strict;
use warnings;
use File::Tail;

my $file = File::Tail->new(name => 'xx.log');
my $line;
while (defined($line = $file->read)) {
    chomp $line;
    next unless ($line =~ m{current\s+number\s+of\s+users\s*:\s+(\d+)});
    my $num = $1;
    if ($num >= 300) {
        system("mailx -s 'alert users now at $num' myemail@xx.com < /dev/null");
    }
}

Open in new window

0
 
LVL 77

Assisted Solution

by:arnold
arnold earned 100 total points
ID: 35029223
Tail will not fail as long as the file to which it is attached is not deleted, it can be renamed.
Does whatever logs this have an option to generate an SNMPTRAP?
Does the process that adds these entries presumably syslog event, have an option to generate the email?
If you are using rsyslog instead of syslog, you could use rsyslog options to detect the event and perform/call the action you want.
0
Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
LVL 26

Expert Comment

by:wilcoxon
ID: 35029654
Tail will not "fail" per se but it will also not start reading the newly created file of the same name after the old log is renamed - it will still point to the old log.  That is what I meant by fail (should have been more explicit).
0
 

Author Comment

by:sunhux
ID: 35033666


There's some issue when I ran the script :



./count.pl
Can't locate File/Tail.pm in @INC (@INC contains: /opt/perl/lib/5.8.2/PA-RISC1.1-thread-multi /opt/perl/lib/5.8.2 /opt/perl/lib/site_perl/5.8.2/PA-RISC1.1-thread-multi /opt/perl/lib/site_perl/5.8.2 /opt/perl/lib/site_perl .) at ./orgpl.pl line 5.
BEGIN failed--compilation aborted at ./orgpl.pl line 5.



Think the problem lies with the code:
  use File::Tail;
0
 
LVL 77

Assisted Solution

by:arnold
arnold earned 100 total points
ID: 35033921
This means you do not have the FILE::TAil installed.
perl -MCPAN -e 'install File::Tail;'
This will use CPAN to locate the File::Tail with any/dependency and will compile and install it for you.  you must run it as root.

If you prefer to install the module your self, the module can be downloaded from http://search.cpan.org/~mgrabnar/File-Tail-0.99.3/Tail.pm

Depending on how familiar with perl coding, I would suggest you look at using open, seek, tell while.

Do you need this notification in Real-time or as close to real time i.e. within a minute of the event occurring?
0
 

Author Comment

by:sunhux
ID: 35034109

This HP-UX box is not connected to Internet so looks like compilation failed :
0
 
LVL 77

Expert Comment

by:arnold
ID: 35034352
If you do not have a way to download the module, you should consider using
open, seek, tell, while loop with a check to see whether a new file was created (rotation occured)
What about whether an option exists to generate an event within the system that generates this log entry or whether you have SNMP enabled and it can be polled with that information being one of the responses?
i.e. snmpget <OID for number of logged in users> if this value is 300 or greater, trigger an email.
0
 

Author Closing Comment

by:sunhux
ID: 35035676
excellent
0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Attention: This article will no longer be maintained. If you have any questions, please feel free to mail me. jgh@FreeBSD.org Please see http://www.freebsd.org/doc/en_US.ISO8859-1/articles/freebsd-update-server/ for the updated article. It is avail…
Checking the Alert Log in AWS RDS Oracle can be a pain through their user interface.  I made a script to download the Alert Log, look for errors, and email me the trace files.  In this article I'll describe what I did and share my script.
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

828 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question