Solved

Shell / Perl script to check for a value in a dynamic logfile & send out email once value hit threshold

Posted on 2011-03-03
9
1,325 Views
Last Modified: 2012-05-11
I need to constantly monitor for a value in a dynamically growing log file
& this log file will be rotated as well.

Say logfile name is xx.log & the most recent information is found at the
bottom of the logfile which has entries :

less relevant lines in logfile ...
dd-mmm-yyyy:hh:mm the current number of users :  255
....
dd-mmmyyyy:hh:mm the current number of users : 321
...


So I thought of a Shell ( Perl script welcomed too) script :
usercount=`tail -1f xx.log | grep "number of users" | awk '{print ($9)}' `
if [ $usercount .ge. 300 ]
then
 mailx -s "alert users now at $usercount" myemail@xx.com < /dev/null
fi

Problem is the logfile is dynamically moving & I'm not sure
if "tail -1f ..." to grab the last line of the logfile would work.
Also, not every new line written to the logfile contains
user count, thus I inserted
 ...  | grep "number of user" | ...

the fact the logfile gets rotated is probably not much of an
issue as we'll still be monitoring the same filename as
xx.log is the current logfile that's being written to


I'm not sure if the syntax "tail -1f ..." is supported on my
old HP-UX but I'm certain RHES Linux 4.x supports it.
So ideally the script provided don't make use of "tail -1f ..."


Needed this script urgently
0
Comment
Question by:sunhux
  • 4
  • 3
  • 2
9 Comments
 

Author Comment

by:sunhux
Comment Utility

One more thing:

instead of using   " awk '{print ($9)}' ", best that the script
search based on the value that appears after "number of users : "
in case there's variable number of text on that line prior to
"number of users : "
0
 
LVL 26

Accepted Solution

by:
wilcoxon earned 400 total points
Comment Utility
I'm pretty sure tail -f will fail when the log is rotated (still pointing at the file that was originally opened).

This perl script automatically handles log rotation/truncation.
#!/usr/local/bin/perl

use strict;
use warnings;
use File::Tail;

my $file = File::Tail->new(name => 'xx.log');
my $line;
while (defined($line = $file->read)) {
    chomp $line;
    next unless ($line =~ m{current\s+number\s+of\s+users\s*:\s+(\d+)});
    my $num = $1;
    if ($num >= 300) {
        system("mailx -s 'alert users now at $num' myemail@xx.com < /dev/null");
    }
}

Open in new window

0
 
LVL 76

Assisted Solution

by:arnold
arnold earned 100 total points
Comment Utility
Tail will not fail as long as the file to which it is attached is not deleted, it can be renamed.
Does whatever logs this have an option to generate an SNMPTRAP?
Does the process that adds these entries presumably syslog event, have an option to generate the email?
If you are using rsyslog instead of syslog, you could use rsyslog options to detect the event and perform/call the action you want.
0
 
LVL 26

Expert Comment

by:wilcoxon
Comment Utility
Tail will not "fail" per se but it will also not start reading the newly created file of the same name after the old log is renamed - it will still point to the old log.  That is what I meant by fail (should have been more explicit).
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 

Author Comment

by:sunhux
Comment Utility


There's some issue when I ran the script :



./count.pl
Can't locate File/Tail.pm in @INC (@INC contains: /opt/perl/lib/5.8.2/PA-RISC1.1-thread-multi /opt/perl/lib/5.8.2 /opt/perl/lib/site_perl/5.8.2/PA-RISC1.1-thread-multi /opt/perl/lib/site_perl/5.8.2 /opt/perl/lib/site_perl .) at ./orgpl.pl line 5.
BEGIN failed--compilation aborted at ./orgpl.pl line 5.



Think the problem lies with the code:
  use File::Tail;
0
 
LVL 76

Assisted Solution

by:arnold
arnold earned 100 total points
Comment Utility
This means you do not have the FILE::TAil installed.
perl -MCPAN -e 'install File::Tail;'
This will use CPAN to locate the File::Tail with any/dependency and will compile and install it for you.  you must run it as root.

If you prefer to install the module your self, the module can be downloaded from http://search.cpan.org/~mgrabnar/File-Tail-0.99.3/Tail.pm

Depending on how familiar with perl coding, I would suggest you look at using open, seek, tell while.

Do you need this notification in Real-time or as close to real time i.e. within a minute of the event occurring?
0
 

Author Comment

by:sunhux
Comment Utility

This HP-UX box is not connected to Internet so looks like compilation failed :
0
 
LVL 76

Expert Comment

by:arnold
Comment Utility
If you do not have a way to download the module, you should consider using
open, seek, tell, while loop with a check to see whether a new file was created (rotation occured)
What about whether an option exists to generate an event within the system that generates this log entry or whether you have SNMP enabled and it can be polled with that information being one of the responses?
i.e. snmpget <OID for number of logged in users> if this value is 300 or greater, trigger an email.
0
 

Author Closing Comment

by:sunhux
Comment Utility
excellent
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Using libpcap/Jpcap to capture and send packets on Solaris version (10/11) Library used: 1.      Libpcap (http://www.tcpdump.org) Version 1.2 2.      Jpcap(http://netresearch.ics.uci.edu/kfujii/Jpcap/doc/index.html) Version 0.6 Prerequisite: 1.      GCC …
Over the years I've spent many an hour playing on hardened, DMZ'd servers, with only a sub-set of the usual GNU toy's to keep me company; frequently I've needed to save and send log or data extracts from these server back to my PC, or to others, and…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now