grazal
asked on
How to calculate between two given datetime in Perl
Let me explain further:
I have this file as a result of a query about more than a thousand lines. It has a date and time on each line entry and stored in a file called maillog that looks like the following:
Sep 5 00:00:30ns2 pop3-login: Login: jflanagan [192.168.30.20]....
Sep 5 01:00:02 ns2 pop3-login: Login: vismail [192.168.70.7]...
Sep 5 00:20:28 ns2 sendmail[16116]: m8570ItF016116: Milter add: header: ...
Sep 5 00:00:30 ns2 pop3-login: Login: atinbound [192.168.50.37]...
Sep 5 00:01:35 ns2 pop3-login: Login: pyen [192.168.10.87]....
Sep 5 00:00:36 ns2 sendmail[16143]: m8570Zu2016143: ......
My task is to capture all the data against the current local datetime. That is, write a Perl script to capture only lines entries that are less than 30 minutes old compared to the current local time. Let say, if I ran a Perl script at local time (e.g. "Sep 5 00:01:00), only get the entries within 30 minutes of the local time.
How can I represent the date in the maillog file and compare it with the current date. Now, you have to keep in mind that the datetime is a flat string format, a flat file does not have the same format as the current datetime. That is, you can manipulate the localtime() into different formats, but how can you manipulate that date in the maillog so that I can compare the two dates (the localtime and the date in the maillog file)? Can you give me some answers and detailed example script please!!???
I have this file as a result of a query about more than a thousand lines. It has a date and time on each line entry and stored in a file called maillog that looks like the following:
Sep 5 00:00:30ns2 pop3-login: Login: jflanagan [192.168.30.20]....
Sep 5 01:00:02 ns2 pop3-login: Login: vismail [192.168.70.7]...
Sep 5 00:20:28 ns2 sendmail[16116]: m8570ItF016116: Milter add: header: ...
Sep 5 00:00:30 ns2 pop3-login: Login: atinbound [192.168.50.37]...
Sep 5 00:01:35 ns2 pop3-login: Login: pyen [192.168.10.87]....
Sep 5 00:00:36 ns2 sendmail[16143]: m8570Zu2016143: ......
My task is to capture all the data against the current local datetime. That is, write a Perl script to capture only lines entries that are less than 30 minutes old compared to the current local time. Let say, if I ran a Perl script at local time (e.g. "Sep 5 00:01:00), only get the entries within 30 minutes of the local time.
How can I represent the date in the maillog file and compare it with the current date. Now, you have to keep in mind that the datetime is a flat string format, a flat file does not have the same format as the current datetime. That is, you can manipulate the localtime() into different formats, but how can you manipulate that date in the maillog so that I can compare the two dates (the localtime and the date in the maillog file)? Can you give me some answers and detailed example script please!!???
ASKER
I copied the codes you gave me, but it gave me no output. It should have had given me hundreds of lines of output, but in this case nothing came up! Any suggestions please!
Here's a few excerpt from my input file "maillog":
Oct 15 04:02:56 ns2 pop3-login: Login: mlavarias [192.168.70.53]..
Oct 15 04:02:58 ns2 sendmail[9396]: m95B2lPd009396: from=<dwvelvinm@velvin.com >, size=764, ..
Oct 15 04:02:58 ns2 sendmail[9396]: m95B2lPd009396: Milter: data, discard...
Oct 15 04:02:58 ns2 sendmail[9396]: m95B2lPd009396: discarded...
Oct 15 04:02:58 ns2 pop3-login: Login: pyen [192.168.10.87]...
Oct 15 04:03:01 ns2 sendmail[9686]: m95B31l9009686: mta-pva.cluster2.convio.ne t [66.45.103.48] did ...
Oct 15 04:03:02 ns2 sendmail[9635]: m95B2oof009635: ruleset=check_mail, .....
My perl script file contains:
#!/usr/bin/perl
use strict;
use warnings;
use Time::Local;
opendir(DIR, "/var/log");
open(READFILE,"/var/log/ma illog");
my %Months = (Jan => 0, Feb => 1, Mar => 2, Apr =>3, May => 4, Jun => 5, Jul => 6, Aug => 7, Sep => 8, Oct => 9, Nov => 11, Dec => 12);
my $year = (localtime()) [7];
while (<READFILE>)
{
next unless /^(\W+)\s+(\d+)\s+(\d+):(\ d+)/;
my $linetime = timelocal($5, $4, $3, $2, $Months{$1}, $year);
next if time()-$linetime>30*60;
print $_;
}
Here's a few excerpt from my input file "maillog":
Oct 15 04:02:56 ns2 pop3-login: Login: mlavarias [192.168.70.53]..
Oct 15 04:02:58 ns2 sendmail[9396]: m95B2lPd009396: from=<dwvelvinm@velvin.com
Oct 15 04:02:58 ns2 sendmail[9396]: m95B2lPd009396: Milter: data, discard...
Oct 15 04:02:58 ns2 sendmail[9396]: m95B2lPd009396: discarded...
Oct 15 04:02:58 ns2 pop3-login: Login: pyen [192.168.10.87]...
Oct 15 04:03:01 ns2 sendmail[9686]: m95B31l9009686: mta-pva.cluster2.convio.ne
Oct 15 04:03:02 ns2 sendmail[9635]: m95B2oof009635: ruleset=check_mail, .....
My perl script file contains:
#!/usr/bin/perl
use strict;
use warnings;
use Time::Local;
opendir(DIR, "/var/log");
open(READFILE,"/var/log/ma
my %Months = (Jan => 0, Feb => 1, Mar => 2, Apr =>3, May => 4, Jun => 5, Jul => 6, Aug => 7, Sep => 8, Oct => 9, Nov => 11, Dec => 12);
my $year = (localtime()) [7];
while (<READFILE>)
{
next unless /^(\W+)\s+(\d+)\s+(\d+):(\
my $linetime = timelocal($5, $4, $3, $2, $Months{$1}, $year);
next if time()-$linetime>30*60;
print $_;
}
There were a few bugs in my previous post... try this:
#!/usr/bin/perl
use strict;
use warnings;
use Time::Local;
open(READFILE,"/var/log/maillog");
my %Months = (Jan => 0, Feb => 1, Mar => 2, Apr =>3, May => 4, Jun => 5, Jul => 6, Aug => 7, Sep => 8, Oct => 9, Nov => 10, Dec => 11);
my $year = (localtime())[5];
while (<READFILE>)
{
next unless /^(\W+)\s+(\d+)\s+(\d+):(\d+)/;
my $linetime = timelocal($5, $4, $3, $2, $Months{$1}, $year);
next if time()-$linetime>30*60;
print $_;
}
close(READFILE);
ASKER
Here's what happened: I revised my script per your instruction. The script ran, but no output. So, what I did was I commented lines 11 and 13 just to troubleshoot.
As a result I got the following Error:
Day '' out of range 1..31 at test2.pl line 12
Here's the revised script:
#!usr/bin/perl
use strict;
use warnings;
use Time::Local;
opendir(DIR, "/var/log");
open(READFILE,"/var/log/ma illog");
my %Months = (Jan => 0, Feb => 1, Mar => 2, Apr =>3, May => 4, Jun => 5, Jul => 6,
Aug => 7, Sep => 8, Oct => 9, Nov => 10, Dec => 11);
my $year = (localtime()) [5];
while (<READFILE>)
{
# next unless /^(\W+)\s+(\d+)\s+(\d+):(\ d+)/;
my $linetime = timelocal($5, $4, $3, $2, $Months{$1}, $year);
# next if time()-$linetime>30*60;
print $_;
}
close(READFILE);
I'm clueless!
As a result I got the following Error:
Day '' out of range 1..31 at test2.pl line 12
Here's the revised script:
#!usr/bin/perl
use strict;
use warnings;
use Time::Local;
opendir(DIR, "/var/log");
open(READFILE,"/var/log/ma
my %Months = (Jan => 0, Feb => 1, Mar => 2, Apr =>3, May => 4, Jun => 5, Jul => 6,
Aug => 7, Sep => 8, Oct => 9, Nov => 10, Dec => 11);
my $year = (localtime()) [5];
while (<READFILE>)
{
# next unless /^(\W+)\s+(\d+)\s+(\d+):(\
my $linetime = timelocal($5, $4, $3, $2, $Months{$1}, $year);
# next if time()-$linetime>30*60;
print $_;
}
close(READFILE);
I'm clueless!
It looks like you commented lines 10 and 12 in the version I posted... line 10 is what parses the line from your file, getting the date from it. Without that line, it won't work at all. Line 12 is what skips the lines older than 30 minutes. Without that, you will get all lines printed.
When I run it on your sample data, I get the output as expected.
If you get no output, it could be your file doesn't have any entries within the last 30 minutes. If you post your actual file that you are using, I'll take a look.
When I run it on your sample data, I get the output as expected.
If you get no output, it could be your file doesn't have any entries within the last 30 minutes. If you post your actual file that you are using, I'll take a look.
ASKER
To Adam314:
Please explain to me lines 11 thru 13 of the While loop above? what does those numbers (e.g. $5, $4,..\W, \d+, etc. supposed to mean? Sorry for my ignorance...but I'm so new to this!
Thanks
Please explain to me lines 11 thru 13 of the While loop above? what does those numbers (e.g. $5, $4,..\W, \d+, etc. supposed to mean? Sorry for my ignorance...but I'm so new to this!
Thanks
ASKER
Here's some actual contents of the file "maillog". I use the grep command to capture them, because the actual file contains hundreds of line input. In fact, lines of input are added every second to this file.
Oct 10 15:19:33 ns2 pop3-login: Login: maryp [192.168.80.31]
Oct 10 15:19:33 ns2 pop3-login: Login: dku [192.168.193.124]
Oct 10 15:19:34 ns2 pop3-login: Login: kmaclean [192.168.5.208]
Oct 10 15:19:35 ns2 sendmail[19380]: m9AMJW7G019380: from=<>, size=2420, class=0, nrcpts=1, msgid=<01ce01c92b26$45cb1a e0$e905a8c 0@LA46>, proto=ESMTP, daemon=MTA, relay=[192.168.5.233]
Oct 10 15:19:35 ns2 pop3-login: Login: pyen [192.168.10.87]
Oct 10 15:19:35 ns2 sendmail[19392]: m9AMJW7G019380: to=<maryp@coasteramer.com> , delay=00:00:00, xdelay=00:00:00, mailer=local, pri=32594, dsn=2.0.0, stat=Sent
Oct 10 15:19:36 ns2 pop3-login: Login: rbhatti [192.168.70.25]
Oct 10 15:19:37 ns2 last message repeated 2 times
Oct 10 15:19:37 ns2 pop3-login: Login: pmohabbat [192.168.70.28]
Oct 10 15:19:37 ns2 pop3-login: Login: rbhatti [192.168.70.25]
Oct 10 15:19:39 ns2 last message repeated 3 times
Oct 10 15:19:39 ns2 pop3-login: Login: vismail [192.168.70.7]
Oct 10 15:19:39 ns2 pop3-login: Login: rbhatti [192.168.70.25]
Oct 10 15:19:40 ns2 pop3-login: Login: rbhatti [192.168.70.25]
Oct 10 15:19:40 ns2 pop3-login: Login: srosenthal [192.168.30.54]
Oct 10 15:19:40 ns2 sendmail[19287]: m9AMJex9019287: [87.126.124.96] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA
Oct 10 15:19:40 ns2 pop3-login: Login: rbhatti [192.168.70.25]
Oct 10 15:19:41 ns2 sendmail[19436]: m9AMJevw019436:
Oct 10 15:20:10 ns2 pop3-login: Login: kmaclean [192.168.5.208]
Oct 10 15:20:10 ns2 pop3-login: Login: lhernandez [192.168.15.208]
Oct 10 15:20:11 ns2 pop3-login: Login: maybelle [192.168.70.35]
Oct 10 15:20:12 ns2 pop3-login: Login: maybelle [192.168.70.35]
Oct 10 15:20:12 ns2 pop3-login: Login: lcreceiving [192.168.101.155]
Oct 10 15:20:12 ns2 pop3-login: Login: acaluag [192.168.10.222]
Oct 10 15:20:12 ns2 sendmail[19589]: m9AMK41G019589: from=<kmaclean@coasteramer .com>, size=24137, class=0, nrcpts=1, msgid=<!~!UENERkVCMDkAAQAC AAAAAAAAAA AAAAAAABgA AAAAAAAAk6 DvyRr5zEK6 7tyYpWXWL8 KAAAAQAAAA qWVaNEnJ7k 2m1Ba4K5TB , proto=ESMTP, daemon=MTA, relay=[192.168.5.208]
Oct 10 15:20:12 ns2 pop3-login: Login: maybelle [192.168.70.35]
Oct 10 15:20:12 ns2 sendmail[19607]: m9AMK8AO019607: from=<BuyCostumes.com@nint rilab.com> , size=1378, class=0, nrcpts=1, msgid=<127533888.4755685@d atinggathe ring.com>, bodytype=8BITMIME, proto=SMTP, daemon=MTA, relay=72.37.221.139.rdns.u biquityser vers.com [72.37.221.139] (may be forged)
Oct 10 15:20:13 ns2 sendmail[19607]: m9AMK8AO019607: Milter: data, discard
Oct 10 15:20:13 ns2 sendmail[19607]: m9AMK8AO019607: discarded
Oct 10 15:20:13 ns2 pop3-login: Login: maybelle [192.168.70.35]
Oct 10 15:20:13 ns2 pop3-login: Login: vismail [192.168.60.7]
Oct 10 15:20:14 ns2 pop3-login: Login: agutierrez [192.168.70.47]
Oct 10 15:20:14 ns2 pop3-login: Login: tkonetzny [70.0.132.37]
Oct 10 15:20:14 ns2 pop3-login: Login: maybelle [192.168.70.35]
Oct 10 15:20:14 ns2 pop3-login: Login: nweltz [192.168.15.143]
Oct 10 15:20:15 ns2 pop3-login: Login: maybelle [192.168.70.35]
Oct 10 15:20:15 ns2 sendmail[19635]: m9AMKFE8019635: <dortega@coasteramer.com>. .. User unknown
Oct 10 15:19:33 ns2 pop3-login: Login: maryp [192.168.80.31]
Oct 10 15:19:33 ns2 pop3-login: Login: dku [192.168.193.124]
Oct 10 15:19:34 ns2 pop3-login: Login: kmaclean [192.168.5.208]
Oct 10 15:19:35 ns2 sendmail[19380]: m9AMJW7G019380: from=<>, size=2420, class=0, nrcpts=1, msgid=<01ce01c92b26$45cb1a
Oct 10 15:19:35 ns2 pop3-login: Login: pyen [192.168.10.87]
Oct 10 15:19:35 ns2 sendmail[19392]: m9AMJW7G019380: to=<maryp@coasteramer.com>
Oct 10 15:19:36 ns2 pop3-login: Login: rbhatti [192.168.70.25]
Oct 10 15:19:37 ns2 last message repeated 2 times
Oct 10 15:19:37 ns2 pop3-login: Login: pmohabbat [192.168.70.28]
Oct 10 15:19:37 ns2 pop3-login: Login: rbhatti [192.168.70.25]
Oct 10 15:19:39 ns2 last message repeated 3 times
Oct 10 15:19:39 ns2 pop3-login: Login: vismail [192.168.70.7]
Oct 10 15:19:39 ns2 pop3-login: Login: rbhatti [192.168.70.25]
Oct 10 15:19:40 ns2 pop3-login: Login: rbhatti [192.168.70.25]
Oct 10 15:19:40 ns2 pop3-login: Login: srosenthal [192.168.30.54]
Oct 10 15:19:40 ns2 sendmail[19287]: m9AMJex9019287: [87.126.124.96] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA
Oct 10 15:19:40 ns2 pop3-login: Login: rbhatti [192.168.70.25]
Oct 10 15:19:41 ns2 sendmail[19436]: m9AMJevw019436:
Oct 10 15:20:10 ns2 pop3-login: Login: kmaclean [192.168.5.208]
Oct 10 15:20:10 ns2 pop3-login: Login: lhernandez [192.168.15.208]
Oct 10 15:20:11 ns2 pop3-login: Login: maybelle [192.168.70.35]
Oct 10 15:20:12 ns2 pop3-login: Login: maybelle [192.168.70.35]
Oct 10 15:20:12 ns2 pop3-login: Login: lcreceiving [192.168.101.155]
Oct 10 15:20:12 ns2 pop3-login: Login: acaluag [192.168.10.222]
Oct 10 15:20:12 ns2 sendmail[19589]: m9AMK41G019589: from=<kmaclean@coasteramer
Oct 10 15:20:12 ns2 pop3-login: Login: maybelle [192.168.70.35]
Oct 10 15:20:12 ns2 sendmail[19607]: m9AMK8AO019607: from=<BuyCostumes.com@nint
Oct 10 15:20:13 ns2 sendmail[19607]: m9AMK8AO019607: Milter: data, discard
Oct 10 15:20:13 ns2 sendmail[19607]: m9AMK8AO019607: discarded
Oct 10 15:20:13 ns2 pop3-login: Login: maybelle [192.168.70.35]
Oct 10 15:20:13 ns2 pop3-login: Login: vismail [192.168.60.7]
Oct 10 15:20:14 ns2 pop3-login: Login: agutierrez [192.168.70.47]
Oct 10 15:20:14 ns2 pop3-login: Login: tkonetzny [70.0.132.37]
Oct 10 15:20:14 ns2 pop3-login: Login: maybelle [192.168.70.35]
Oct 10 15:20:14 ns2 pop3-login: Login: nweltz [192.168.15.143]
Oct 10 15:20:15 ns2 pop3-login: Login: maybelle [192.168.70.35]
Oct 10 15:20:15 ns2 sendmail[19635]: m9AMKFE8019635: <dortega@coasteramer.com>.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
This time, it worked! Thank you so much for your patience and understanding. Also, thank you for pointing me to the perldoc.perl.org site. I will use this to learn more on my own. .....till next time.
ASKER
Hi, this solution failed to work at the turn of the new year. The $linetime value is blank. This may have something to do with the year (it's now 2009). Please how do I fix this bug?????
Has the format of your log changed? The script gets the time from the system, so as long as your system time is correct, the year being 2009 should not cause any problems.
ASKER
You know what? I just realized that there's no problem with my code. The problem was the data file. The data was mixed in with the 2008 and 2009 data. So, when my code/program encountered the 2008 data, the results became srewy. Now that all the data are in the same year (2009), the results are now back to normal.
Thank you very much for your time. Sincerely...
Thank you very much for your time. Sincerely...
Open in new window