Solved

Shell / Perl script to check for a value in a dynamic logfile & send out email once value hit threshold

Posted on 2011-03-03
9
1,345 Views
Last Modified: 2012-05-11
I need to constantly monitor for a value in a dynamically growing log file
& this log file will be rotated as well.

Say logfile name is xx.log & the most recent information is found at the
bottom of the logfile which has entries :

less relevant lines in logfile ...
dd-mmm-yyyy:hh:mm the current number of users :  255
....
dd-mmmyyyy:hh:mm the current number of users : 321
...


So I thought of a Shell ( Perl script welcomed too) script :
usercount=`tail -1f xx.log | grep "number of users" | awk '{print ($9)}' `
if [ $usercount .ge. 300 ]
then
 mailx -s "alert users now at $usercount" myemail@xx.com < /dev/null
fi

Problem is the logfile is dynamically moving & I'm not sure
if "tail -1f ..." to grab the last line of the logfile would work.
Also, not every new line written to the logfile contains
user count, thus I inserted
 ...  | grep "number of user" | ...

the fact the logfile gets rotated is probably not much of an
issue as we'll still be monitoring the same filename as
xx.log is the current logfile that's being written to


I'm not sure if the syntax "tail -1f ..." is supported on my
old HP-UX but I'm certain RHES Linux 4.x supports it.
So ideally the script provided don't make use of "tail -1f ..."


Needed this script urgently
0
Comment
Question by:sunhux
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
9 Comments
 

Author Comment

by:sunhux
ID: 35028163

One more thing:

instead of using   " awk '{print ($9)}' ", best that the script
search based on the value that appears after "number of users : " 
in case there's variable number of text on that line prior to
"number of users : "
0
 
LVL 26

Accepted Solution

by:
wilcoxon earned 400 total points
ID: 35028642
I'm pretty sure tail -f will fail when the log is rotated (still pointing at the file that was originally opened).

This perl script automatically handles log rotation/truncation.
#!/usr/local/bin/perl

use strict;
use warnings;
use File::Tail;

my $file = File::Tail->new(name => 'xx.log');
my $line;
while (defined($line = $file->read)) {
    chomp $line;
    next unless ($line =~ m{current\s+number\s+of\s+users\s*:\s+(\d+)});
    my $num = $1;
    if ($num >= 300) {
        system("mailx -s 'alert users now at $num' myemail@xx.com < /dev/null");
    }
}

Open in new window

0
 
LVL 79

Assisted Solution

by:arnold
arnold earned 100 total points
ID: 35029223
Tail will not fail as long as the file to which it is attached is not deleted, it can be renamed.
Does whatever logs this have an option to generate an SNMPTRAP?
Does the process that adds these entries presumably syslog event, have an option to generate the email?
If you are using rsyslog instead of syslog, you could use rsyslog options to detect the event and perform/call the action you want.
0
[Live Webinar] The Cloud Skills Gap

As Cloud technologies come of age, business leaders grapple with the impact it has on their team's skills and the gap associated with the use of a cloud platform.

Join experts from 451 Research and Concerto Cloud Services on July 27th where we will examine fact and fiction.

 
LVL 26

Expert Comment

by:wilcoxon
ID: 35029654
Tail will not "fail" per se but it will also not start reading the newly created file of the same name after the old log is renamed - it will still point to the old log.  That is what I meant by fail (should have been more explicit).
0
 

Author Comment

by:sunhux
ID: 35033666


There's some issue when I ran the script :



./count.pl
Can't locate File/Tail.pm in @INC (@INC contains: /opt/perl/lib/5.8.2/PA-RISC1.1-thread-multi /opt/perl/lib/5.8.2 /opt/perl/lib/site_perl/5.8.2/PA-RISC1.1-thread-multi /opt/perl/lib/site_perl/5.8.2 /opt/perl/lib/site_perl .) at ./orgpl.pl line 5.
BEGIN failed--compilation aborted at ./orgpl.pl line 5.



Think the problem lies with the code:
  use File::Tail;
0
 
LVL 79

Assisted Solution

by:arnold
arnold earned 100 total points
ID: 35033921
This means you do not have the FILE::TAil installed.
perl -MCPAN -e 'install File::Tail;'
This will use CPAN to locate the File::Tail with any/dependency and will compile and install it for you.  you must run it as root.

If you prefer to install the module your self, the module can be downloaded from http://search.cpan.org/~mgrabnar/File-Tail-0.99.3/Tail.pm

Depending on how familiar with perl coding, I would suggest you look at using open, seek, tell while.

Do you need this notification in Real-time or as close to real time i.e. within a minute of the event occurring?
0
 

Author Comment

by:sunhux
ID: 35034109

This HP-UX box is not connected to Internet so looks like compilation failed :
0
 
LVL 79

Expert Comment

by:arnold
ID: 35034352
If you do not have a way to download the module, you should consider using
open, seek, tell, while loop with a check to see whether a new file was created (rotation occured)
What about whether an option exists to generate an event within the system that generates this log entry or whether you have SNMP enabled and it can be polled with that information being one of the responses?
i.e. snmpget <OID for number of logged in users> if this value is 300 or greater, trigger an email.
0
 

Author Closing Comment

by:sunhux
ID: 35035676
excellent
0

Featured Post

Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Background Still having to process all these year-end "csv" files received from all these sources (including Government entities), sometimes we have the need to examine the contents due to data error, etc... As a "Unix" shop, our only readily …
How to remove superseded packages in windows w60 or w61 installation media (.wim) or online system to prevent unnecessary space. w60 means Windows Vista or Windows Server 2008. w61 means Windows 7 or Windows Server 2008 R2. There are various …
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

615 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question