Solved

Shell / Perl script to check for a value in a dynamic logfile & send out email once value hit threshold

Posted on 2011-03-03
9
1,340 Views
Last Modified: 2012-05-11
I need to constantly monitor for a value in a dynamically growing log file
& this log file will be rotated as well.

Say logfile name is xx.log & the most recent information is found at the
bottom of the logfile which has entries :

less relevant lines in logfile ...
dd-mmm-yyyy:hh:mm the current number of users :  255
....
dd-mmmyyyy:hh:mm the current number of users : 321
...


So I thought of a Shell ( Perl script welcomed too) script :
usercount=`tail -1f xx.log | grep "number of users" | awk '{print ($9)}' `
if [ $usercount .ge. 300 ]
then
 mailx -s "alert users now at $usercount" myemail@xx.com < /dev/null
fi

Problem is the logfile is dynamically moving & I'm not sure
if "tail -1f ..." to grab the last line of the logfile would work.
Also, not every new line written to the logfile contains
user count, thus I inserted
 ...  | grep "number of user" | ...

the fact the logfile gets rotated is probably not much of an
issue as we'll still be monitoring the same filename as
xx.log is the current logfile that's being written to


I'm not sure if the syntax "tail -1f ..." is supported on my
old HP-UX but I'm certain RHES Linux 4.x supports it.
So ideally the script provided don't make use of "tail -1f ..."


Needed this script urgently
0
Comment
Question by:sunhux
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
9 Comments
 

Author Comment

by:sunhux
ID: 35028163

One more thing:

instead of using   " awk '{print ($9)}' ", best that the script
search based on the value that appears after "number of users : " 
in case there's variable number of text on that line prior to
"number of users : "
0
 
LVL 26

Accepted Solution

by:
wilcoxon earned 400 total points
ID: 35028642
I'm pretty sure tail -f will fail when the log is rotated (still pointing at the file that was originally opened).

This perl script automatically handles log rotation/truncation.
#!/usr/local/bin/perl

use strict;
use warnings;
use File::Tail;

my $file = File::Tail->new(name => 'xx.log');
my $line;
while (defined($line = $file->read)) {
    chomp $line;
    next unless ($line =~ m{current\s+number\s+of\s+users\s*:\s+(\d+)});
    my $num = $1;
    if ($num >= 300) {
        system("mailx -s 'alert users now at $num' myemail@xx.com < /dev/null");
    }
}

Open in new window

0
 
LVL 78

Assisted Solution

by:arnold
arnold earned 100 total points
ID: 35029223
Tail will not fail as long as the file to which it is attached is not deleted, it can be renamed.
Does whatever logs this have an option to generate an SNMPTRAP?
Does the process that adds these entries presumably syslog event, have an option to generate the email?
If you are using rsyslog instead of syslog, you could use rsyslog options to detect the event and perform/call the action you want.
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 26

Expert Comment

by:wilcoxon
ID: 35029654
Tail will not "fail" per se but it will also not start reading the newly created file of the same name after the old log is renamed - it will still point to the old log.  That is what I meant by fail (should have been more explicit).
0
 

Author Comment

by:sunhux
ID: 35033666


There's some issue when I ran the script :



./count.pl
Can't locate File/Tail.pm in @INC (@INC contains: /opt/perl/lib/5.8.2/PA-RISC1.1-thread-multi /opt/perl/lib/5.8.2 /opt/perl/lib/site_perl/5.8.2/PA-RISC1.1-thread-multi /opt/perl/lib/site_perl/5.8.2 /opt/perl/lib/site_perl .) at ./orgpl.pl line 5.
BEGIN failed--compilation aborted at ./orgpl.pl line 5.



Think the problem lies with the code:
  use File::Tail;
0
 
LVL 78

Assisted Solution

by:arnold
arnold earned 100 total points
ID: 35033921
This means you do not have the FILE::TAil installed.
perl -MCPAN -e 'install File::Tail;'
This will use CPAN to locate the File::Tail with any/dependency and will compile and install it for you.  you must run it as root.

If you prefer to install the module your self, the module can be downloaded from http://search.cpan.org/~mgrabnar/File-Tail-0.99.3/Tail.pm

Depending on how familiar with perl coding, I would suggest you look at using open, seek, tell while.

Do you need this notification in Real-time or as close to real time i.e. within a minute of the event occurring?
0
 

Author Comment

by:sunhux
ID: 35034109

This HP-UX box is not connected to Internet so looks like compilation failed :
0
 
LVL 78

Expert Comment

by:arnold
ID: 35034352
If you do not have a way to download the module, you should consider using
open, seek, tell, while loop with a check to see whether a new file was created (rotation occured)
What about whether an option exists to generate an event within the system that generates this log entry or whether you have SNMP enabled and it can be polled with that information being one of the responses?
i.e. snmpget <OID for number of logged in users> if this value is 300 or greater, trigger an email.
0
 

Author Closing Comment

by:sunhux
ID: 35035676
excellent
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Bash Script to Analyze Oracle Schemas 11 128
Authenticate using sesu from script 7 159
parse a file and get data out 11 138
cscript to activate Windows and Office? 2 155
Using libpcap/Jpcap to capture and send packets on Solaris version (10/11) Library used: 1.      Libpcap (http://www.tcpdump.org) Version 1.2 2.      Jpcap(http://netresearch.ics.uci.edu/kfujii/Jpcap/doc/index.html) Version 0.6 Prerequisite: 1.      GCC …
Active Directory replication delay is the cause to many problems.  Here is a super easy script to force Active Directory replication to all sites with by using an elevated PowerShell command prompt, and a tool to verify your changes.
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

734 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question