Extract lines based on date from log file.

I have a logfile (log.txt) that is stored in this format:

OfficeAAA.txt
ON 02/16/06 12:17:03    OFF 02/16/06 12:18:03    RUNTIME 00:01:00
ON 02/16/06 12:23:37    OFF 02/16/06 12:23:50    RUNTIME 00:00:13
ON 02/20/06 08:53:36    OFF 02/20/06 09:35:25    RUNTIME 00:41:49
ON 02/27/06 08:53:15    OFF 02/27/06 09:26:05    RUNTIME 00:32:50
ON 03/06/06 08:52:56    OFF 03/06/06 09:34:42    RUNTIME 00:41:46
OfficeBBB.txt
ON 02/02/06 12:04:59    OFF 02/02/06 12:05:06    RUNTIME 00:00:07
ON 02/02/06 12:07:05    OFF 02/02/06 12:07:43    RUNTIME 00:00:38
ON 02/13/06 10:30:17    OFF 02/13/06 11:10:02    RUNTIME 00:39:45
ON 02/20/06 10:30:20    OFF 02/20/06 11:10:06    RUNTIME 00:39:46
OfficeCCC.txt
ON 02/16/06 12:19:41    OFF 02/16/06 12:19:47    RUNTIME 00:00:06
ON 02/16/06 12:23:37    OFF 02/16/06 12:23:50    RUNTIME 00:00:13
ON 02/20/06 08:53:36    OFF 02/20/06 09:35:25    RUNTIME 00:41:49
ON 03/06/06 10:32:29    OFF 03/06/06 11:12:13    RUNTIME 00:39:44

This log file is appended daily (if there was any activity) and I plan to run a script each night after midnight and create a report showing just the previous days activity. So if we say today is 3/7/06, my report should look like this:

OfficeAAA.txt
ON 03/06/06 08:52:56    OFF 03/06/06 09:34:42    RUNTIME 00:41:46
OfficeCCC.txt
ON 03/06/06 10:32:29    OFF 03/06/06 11:12:13    RUNTIME 00:39:44

I'm starting my efforts on it now and expect to have a problem with having the script search for the correct date (it will always be today - 1 day) and looping through the file and keeping the proper office name associated with the lines I extract. I think I have an old question I'll lookup that may help me keep the names and data extract correct. But I need this one working quickly and I'm open to any ideas.
Also just for clarity, I want to extract the data based on the date associated with the ON date, and I do not care about the ON time or OFF date and time.

omcrAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

omcrAuthor Commented:
I have go it working for the most part, here it is:

#!/usr/bin/perl -w
open FINAL,  "log.txt" or die "Cannot open file $!";
open FINISH, ">daily.txt"   or die "Cannot open file $!";

my($dd,$mm,$yy,$day) = (localtime)[3,4,5,6];
my $yesterday = join '/', map sprintf("%02d", $_), ($mm+1,$dd-1,$yy%100);
print "$yesterday\n";

while (<FINAL>){
  print if /\.gen|$today/;
  print FINISH if /\.gen|$today/;
}

The output looks like this:

OfficeAAA.txt
ON 03/06/06 08:52:56    OFF 03/06/06 09:34:42    RUNTIME 00:41:46
OfficeBBB.txt
OfficeCCC.txt
ON 03/06/06 10:32:29    OFF 03/06/06 11:12:13    RUNTIME 00:39:44

So it does what I needed but I don't want OfficeBBB.txt to show up since there is no recent data associated with it (I will be checking 70 offices).




0
omcrAuthor Commented:
Sorry, the code below-

while (<FINAL>){
  print if /\.gen|$today/;
  print FINISH if /\.gen|$today/;
}

Should be-

while (<FINAL>){
  print if /\.gen|$yesterday/;
  print FINISH if /\.gen|$yesterday/;
}
0
omcrAuthor Commented:
Yet another mistake in my post- .gen should be .txt
Here is the whole thing one more time CORRECTED:

#!/usr/bin/perl -w
open FINAL,  "log.txt" or die "Cannot open file $!";
open FINISH, ">daily.txt"   or die "Cannot open file $!";

my($dd,$mm,$yy,$day) = (localtime)[3,4,5,6];
my $yesterday = join '/', map sprintf("%02d", $_), ($mm+1,$dd-1,$yy%100);
print "$yesterday\n";

while (<FINAL>){
  print if /\.txt|$yesterday/;
  print FINISH if /\.txt|$yesterday/;
}
0
Cloud Class® Course: C++ 11 Fundamentals

This course will introduce you to C++ 11 and teach you about syntax fundamentals.

FishMongerCommented:
If you use the strftime function from the POSIX module, you can simplify the date calculation.  Also, you only need to apply the regex once and it would be best to use \Q to quote the slashes in the date.

#!/usr/bin/perl -w

use POSIX 'strftime';
use strict;

open FINAL,  "log.txt" or die "Cannot open file $!";
open FINISH, ">daily.txt"   or die "Cannot open file $!";

$yesterday = strftime("%m/%d/%y", localtime(time - 86400));
print "$yesterday\n";

while (<FINAL>){
  if (/\.txt|\Q$yesterday\E/) {
     print;
     print FINISH if /\.txt|$yesterday/;
  }
}
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
FishMongerCommented:
Oops, cut/paste goof

   print FINISH if /\.txt|$yesterday/;

should have been

   print FINISH;
0
omcrAuthor Commented:
FishMonger,
 Any ideas on how to keep the Offices which had no activity from showing up in my final report,
or a way to print the office name and activity information on the same line in the report, which would cause my current data to print out like this:


OfficeAAA.txt  ON 03/06/06 08:52:56    OFF 03/06/06 09:34:42    RUNTIME 00:41:46
OfficeBBB.txt
OfficeCCC.txt  ON 03/06/06 10:32:29    OFF 03/06/06 11:12:13    RUNTIME 00:39:44

And then I could loop through it and extract the offices with data into another file.
0
Adam314Commented:
Here is my version, based off of FishMonger.  A few changes:
  - Only prints office if there is data for that office
  - Instead of matching /$yesterday/, matches /^ON $yesterday/...
    Without this, if yesterday appeared in the OFF time, but not ON time, it would have been displayed
  - Office could be printed on a seperate line... there are a few lines commented out, uncomment them if you like that better
    And comment the line to print office on the same line

#!/usr/bin/perl -w

use POSIX 'strftime';
use strict;

open FINAL,  "<log.txt" or die "Cannot open file $!";
open FINISH, ">daily.txt"   or die "Cannot open file $!";

my $yesterday = strftime("%m/%d/%y", localtime(time - 86400));
print "Yesterday=$yesterday\n";

my $Office="";
while (<FINAL>){
      if(/^ON/){
            #This is on/off/runtime data
            print "\$_=$_   ON $yesterday\n";
            if($_ =~ /^ON $yesterday/){
                  #Office and Log data on seperate lines
                  #if($Office ne ""){
                  #      print FINISH "$Office\n";
                  #      $Office="";
                  #}
                  #Office and Log data on same line
                  print FINISH "$Office ";
                  print FINISH $_;
            }
      }
      else {
            #This is office data
            $Office=$_;
            chomp($Office);
      }
}
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Perl

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.