Link to home
Start Free TrialLog in
Avatar of grazal
grazal

asked on

How to calculate between two given datetime in Perl

Let me explain further:

I have this file as a result of a query about more than a thousand lines.  It has a date and time on each line entry and stored in a file called maillog that looks like the following:

Sep  5 00:00:30ns2 pop3-login: Login: jflanagan [192.168.30.20]....
Sep  5 01:00:02 ns2 pop3-login: Login: vismail [192.168.70.7]...
Sep  5 00:20:28 ns2 sendmail[16116]: m8570ItF016116: Milter add: header: ...
Sep  5 00:00:30 ns2 pop3-login: Login: atinbound [192.168.50.37]...
Sep  5 00:01:35 ns2 pop3-login: Login: pyen [192.168.10.87]....
Sep  5 00:00:36 ns2 sendmail[16143]: m8570Zu2016143: ......

My task is to capture all the data against the current local datetime.  That is, write a Perl script to capture only lines entries  that are less than 30 minutes old compared to  the current local time.   Let say, if I ran a Perl script at local time (e.g. "Sep 5 00:01:00), only get the entries within 30 minutes of the local time.  
How can I represent the date in the maillog file and compare it with the current date.  Now, you have to keep in mind that the datetime is a flat string format, a flat file does not have the same format as the current datetime.  That is, you can manipulate the localtime() into different formats, but how can you manipulate that date in the maillog so that I can compare the two dates (the localtime and the date in the maillog file)?   Can you give me some answers and detailed example script please!!???

Avatar of Adam314
Adam314

You can use the Time::Local module to convert the file time into a unix time (epoch seconds).  Then compare to 30*60 (for 30 minutes).

#!/usr/bin/perl
use strict;
use warnings;
use Time::Local;
my %Months = (Jan => 0, Feb=>1, Mar=>2, Apr=>3, May=>4, Jun=>5, Jul=>6, Aug=>7, Sep=>8, Oct=>9, Nov=>11, Dec=>12);
 
my $year = (localtime())[7];
while(<>) {
	next unless /^(\w+)\s+(\d+)\s+(\d+):(\d+):(\d+)/;
	my $linetime = timelocal($5, $4, $3, $2, $Months{$1}, $year);
	next if time()-$linetime>30*60;
	print $_;
}

Open in new window

Avatar of grazal

ASKER

I copied the codes you gave me, but it gave me no output.  It should have had given me hundreds of lines of output, but in this case nothing came up!  Any suggestions please!

Here's a few excerpt from my input file "maillog":

Oct 15 04:02:56 ns2 pop3-login: Login: mlavarias [192.168.70.53]..
Oct 15 04:02:58 ns2 sendmail[9396]: m95B2lPd009396: from=<dwvelvinm@velvin.com>, size=764, ..
Oct 15 04:02:58 ns2 sendmail[9396]: m95B2lPd009396: Milter: data, discard...
Oct 15 04:02:58 ns2 sendmail[9396]: m95B2lPd009396: discarded...
Oct 15 04:02:58 ns2 pop3-login: Login: pyen [192.168.10.87]...
Oct 15 04:03:01 ns2 sendmail[9686]: m95B31l9009686: mta-pva.cluster2.convio.net [66.45.103.48] did ...
Oct 15 04:03:02 ns2 sendmail[9635]: m95B2oof009635: ruleset=check_mail, .....

My perl script file contains:

#!/usr/bin/perl
use strict;
use warnings;
use Time::Local;
opendir(DIR, "/var/log");
open(READFILE,"/var/log/maillog");
my %Months = (Jan => 0, Feb => 1, Mar => 2, Apr =>3, May => 4, Jun => 5, Jul => 6, Aug => 7, Sep => 8, Oct => 9, Nov => 11, Dec => 12);
my $year = (localtime()) [7];
while (<READFILE>)
{
   next unless /^(\W+)\s+(\d+)\s+(\d+):(\d+)/;
   my $linetime = timelocal($5, $4, $3, $2, $Months{$1}, $year);
   next if time()-$linetime>30*60;
  print $_;
}
There were a few bugs in my previous post... try this:
#!/usr/bin/perl
use strict;
use warnings;
use Time::Local;
open(READFILE,"/var/log/maillog");
my %Months = (Jan => 0, Feb => 1, Mar => 2, Apr =>3, May => 4, Jun => 5, Jul => 6, Aug => 7, Sep => 8, Oct => 9, Nov => 10, Dec => 11);
my $year = (localtime())[5];
while (<READFILE>)
{
    next unless /^(\W+)\s+(\d+)\s+(\d+):(\d+)/;
    my $linetime = timelocal($5, $4, $3, $2, $Months{$1}, $year);
    next if time()-$linetime>30*60;
    print $_;
}
close(READFILE);

Open in new window

Avatar of grazal

ASKER

Here's what happened:  I revised my script per your instruction.  The script ran, but no output.  So, what I did was I commented lines 11 and 13 just to troubleshoot.  
As a result I got the following Error:

Day '' out of range 1..31 at test2.pl line 12

Here's the revised script:
#!usr/bin/perl
use strict;
use warnings;
use Time::Local;
opendir(DIR, "/var/log");
open(READFILE,"/var/log/maillog");
my %Months = (Jan => 0, Feb => 1, Mar => 2, Apr =>3, May => 4, Jun => 5, Jul => 6,
                         Aug => 7, Sep => 8, Oct => 9, Nov => 10, Dec => 11);
my $year = (localtime()) [5];
while (<READFILE>)
{
        # next unless /^(\W+)\s+(\d+)\s+(\d+):(\d+)/;
   my $linetime = timelocal($5, $4, $3, $2, $Months{$1}, $year);
       # next if time()-$linetime>30*60;
  print $_;
}
close(READFILE);

I'm clueless!  
It looks like you commented lines 10 and 12 in the version I posted... line 10 is what parses the line from your file, getting the date from it.  Without that line, it won't work at all.  Line 12 is what skips the lines older than 30 minutes.  Without that, you will get all lines printed.

When I run it on your sample data, I get the output as expected.

If you get no output, it could be your file doesn't have any entries within the last 30 minutes.  If you post your actual file that you are using, I'll take a look.
Avatar of grazal

ASKER

To Adam314:

Please explain to me lines 11 thru 13 of the While loop above?  what does those numbers (e.g. $5, $4,..\W, \d+, etc. supposed to mean?  Sorry for my ignorance...but I'm so new to this!

Thanks
Avatar of grazal

ASKER

Here's some actual contents of the file "maillog".  I use the grep command to capture them, because the actual file contains hundreds of line input.  In fact, lines of input are added every second to this file.

 Oct 10 15:19:33 ns2 pop3-login: Login: maryp [192.168.80.31]
 Oct 10 15:19:33 ns2 pop3-login: Login: dku [192.168.193.124]
 Oct 10 15:19:34 ns2 pop3-login: Login: kmaclean [192.168.5.208]
 Oct 10 15:19:35 ns2 sendmail[19380]: m9AMJW7G019380: from=<>, size=2420, class=0, nrcpts=1, msgid=<01ce01c92b26$45cb1ae0$e905a8c0@LA46>, proto=ESMTP, daemon=MTA, relay=[192.168.5.233]
 Oct 10 15:19:35 ns2 pop3-login: Login: pyen [192.168.10.87]
 Oct 10 15:19:35 ns2 sendmail[19392]: m9AMJW7G019380: to=<maryp@coasteramer.com>, delay=00:00:00, xdelay=00:00:00, mailer=local, pri=32594, dsn=2.0.0, stat=Sent
 Oct 10 15:19:36 ns2 pop3-login: Login: rbhatti [192.168.70.25]
 Oct 10 15:19:37 ns2 last message repeated 2 times
 Oct 10 15:19:37 ns2 pop3-login: Login: pmohabbat [192.168.70.28]
 Oct 10 15:19:37 ns2 pop3-login: Login: rbhatti [192.168.70.25]
 Oct 10 15:19:39 ns2 last message repeated 3 times
 Oct 10 15:19:39 ns2 pop3-login: Login: vismail [192.168.70.7]
 Oct 10 15:19:39 ns2 pop3-login: Login: rbhatti [192.168.70.25]
 Oct 10 15:19:40 ns2 pop3-login: Login: rbhatti [192.168.70.25]
 Oct 10 15:19:40 ns2 pop3-login: Login: srosenthal [192.168.30.54]
 Oct 10 15:19:40 ns2 sendmail[19287]: m9AMJex9019287: [87.126.124.96] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA
 Oct 10 15:19:40 ns2 pop3-login: Login: rbhatti [192.168.70.25]
 Oct 10 15:19:41 ns2 sendmail[19436]: m9AMJevw019436:
 Oct 10 15:20:10 ns2 pop3-login: Login: kmaclean [192.168.5.208]
 Oct 10 15:20:10 ns2 pop3-login: Login: lhernandez [192.168.15.208]
 Oct 10 15:20:11 ns2 pop3-login: Login: maybelle [192.168.70.35]
 Oct 10 15:20:12 ns2 pop3-login: Login: maybelle [192.168.70.35]
 Oct 10 15:20:12 ns2 pop3-login: Login: lcreceiving [192.168.101.155]
 Oct 10 15:20:12 ns2 pop3-login: Login: acaluag [192.168.10.222]
 Oct 10 15:20:12 ns2 sendmail[19589]: m9AMK41G019589: from=<kmaclean@coasteramer.com>, size=24137, class=0, nrcpts=1, msgid=<!~!UENERkVCMDkAAQACAAAAAAAAAAAAAAAAABgAAAAAAAAAk6DvyRr5zEK67tyYpWXWL8KAAAAQAAAAqWVaNEnJ7k2m1Ba4K5TB, proto=ESMTP, daemon=MTA, relay=[192.168.5.208]
 Oct 10 15:20:12 ns2 pop3-login: Login: maybelle [192.168.70.35]
 Oct 10 15:20:12 ns2 sendmail[19607]: m9AMK8AO019607: from=<BuyCostumes.com@nintrilab.com>, size=1378, class=0, nrcpts=1, msgid=<127533888.4755685@datinggathering.com>, bodytype=8BITMIME, proto=SMTP, daemon=MTA, relay=72.37.221.139.rdns.ubiquityservers.com [72.37.221.139] (may be forged)
 Oct 10 15:20:13 ns2 sendmail[19607]: m9AMK8AO019607: Milter: data, discard
 Oct 10 15:20:13 ns2 sendmail[19607]: m9AMK8AO019607: discarded
 Oct 10 15:20:13 ns2 pop3-login: Login: maybelle [192.168.70.35]
 Oct 10 15:20:13 ns2 pop3-login: Login: vismail [192.168.60.7]
 Oct 10 15:20:14 ns2 pop3-login: Login: agutierrez [192.168.70.47]
 Oct 10 15:20:14 ns2 pop3-login: Login: tkonetzny [70.0.132.37]
 Oct 10 15:20:14 ns2 pop3-login: Login: maybelle [192.168.70.35]
 Oct 10 15:20:14 ns2 pop3-login: Login: nweltz [192.168.15.143]
 Oct 10 15:20:15 ns2 pop3-login: Login: maybelle [192.168.70.35]
 Oct 10 15:20:15 ns2 sendmail[19635]: m9AMKFE8019635: <dortega@coasteramer.com>... User unknown

ASKER CERTIFIED SOLUTION
Avatar of Adam314
Adam314

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of grazal

ASKER

This time, it worked!  Thank you so much for your patience and understanding.  Also, thank you for pointing me to the perldoc.perl.org site.  I will use this to learn more on my own.  .....till next time.
Avatar of grazal

ASKER

Hi, this solution failed to work at the turn of the new year.   The $linetime value is blank.   This may have something to do with the year (it's now 2009).  Please how do I fix this bug?????
Has the format of your log changed?  The script gets the time from the system, so as long as your system time is correct, the year being 2009 should not cause any problems.
Avatar of grazal

ASKER

You know what?  I just realized that there's no problem with my code.  The problem was the data file.  The data was mixed in with the 2008 and 2009 data.  So, when my code/program encountered the 2008 data, the results became srewy.  Now that all the data are in the same year (2009), the results are now back to normal.  

Thank you very much for your time.  Sincerely...