Solved

Split file into two files using Perl

Posted on 2006-11-27
14
220 Views
Last Modified: 2010-03-05
I am a Perl neophyte.  I would like to take logs and split them into two files based upon a timestamp in the logs.  The timestamp is in the following format;

11/27/06 3:45:24 PM.

   Basically I want a business hours and a non-business hours file for each day.  I want everything from say 7 AM to 9 PM to be business houts and everything outside of this to be non business hours.  How do I accomplish this?

Perl neophyte
0
Comment
Question by:awakenings
  • 7
  • 3
  • 2
  • +1
14 Comments
 

Author Comment

by:awakenings
ID: 18022491
I will be working in SUSE.
0
 
LVL 84

Assisted Solution

by:ozo
ozo earned 175 total points
ID: 18022614
open B,">business" or die $!;
open N,">non-business" or die $!;
while( <> ){
    if( /\b([7-9]|1[01]):\d\d:\d\d\s*AM\b/i || /\b(12|[1-9]):\d\d:\d\d\s*PM\b/i ){
       print B;
   }else{
      print N;
   }
}
0
 
LVL 17

Assisted Solution

by:mjcoyne
mjcoyne earned 75 total points
ID: 18026807
If you need to further define business hours from non-business hours (for example, all hours on a weekend or a holiday might be considered non-business hours, regardless of the time of day), see Date::Manip (http://www.cise.ufl.edu/~sbeck/DateManip.html).
0
 

Author Comment

by:awakenings
ID: 18027824
Thanks folks!  Things became very busy yesterday and I didn't have time to get around to this.  I'll be testing this morning to make sure I have everything.
0
 

Author Comment

by:awakenings
ID: 18028145
Ozo,

    Please forgive my ignorence on this.  I am no programmer of any sort and I am trying to understand basics.  One of the things I forgot to mention was that I am taking logs from actual files.  I am not sure I see how to do that.  The files have dates so I am looking at the following functions added on to what you suggested.

#!/usr/bin/perl

my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);

$year += 1900;
$mon += 1;
$mday = sprintf("%0.2d", $mday);
$mon = sprintf("%0.2d", $mon);

$date_string = "$mon-$mday-$year";

print "$date_string\n";

open LogFile$date_string\n.txt  (<---  Key part I am not sure of)

open B,">business" or die $!;
open N,">non-business" or die $!;
while( <> ){
    if( /\b([7-9]|1[01]):\d\d:\d\d\s*AM\b/i || /\b(12|[1-9]):\d\d:\d\d\s*PM\b/i ){
       print B;
   }else{
      print N;
   }
}

   Now how do I output this information to another txt file?  I want to run this in cron daily so I don't have to do anything with it.
0
 

Author Comment

by:awakenings
ID: 18028159
Oh... As for  the comment about vacations, Saturdays and Sundays are considered non-business hours as well.

Thanks,

Perl Neophyte
0
 
LVL 84

Assisted Solution

by:ozo
ozo earned 175 total points
ID: 18028396
open LogFile, "<$date_string.txt" or die "can't open $date_string.txt $!"; #  (<---  Key part I am not sure of)

open B,">business" or die $!;
open N,">non-business" or die $!;
while( <LogFile> ){
    if( /\b([7-9]|1[01]):\d\d:\d\d\s*AM\b/i || /\b(12|[1-9]):\d\d:\d\d\s*PM\b/i ){
       print B;
   }else{
      print N;
   }
}
#Although this ignores weekends and holidays
0
How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

 
LVL 39

Assisted Solution

by:Adam314
Adam314 earned 250 total points
ID: 18028925
This version looks at day of the week.  Saturday/Sunday go to non-business file...



use Time::Local;

open LogFile, "<$date_string.txt" or die "can't open $date_string.txt $!"; #  (<---  Key part I am not sure of)
open B,">business" or die $!;
open N,">non-business" or die $!;
while(<LogFile>){
      if(/(\d{2})\/(\d{2})\/(\d{2}) (\d{1,2}):(\d{2}):(\d{2}) (A|P)M/){
            my $hr=$4;
            my $hr+=12 if $7 eq 'P';
            my $datetime = timelocal($6,$5,$hr,$2,$1-1,$3+2000);
            my @localdate=localtime($datetime);
            my $secs=$localdate[0]+60*$localdate[1]+60*60*$localdate[2];
            if(   ($localdate[6] == 0)    #sunday
               || ($localdate[6] == 6)    #saturday
               || ($secs < 7*60*60)       #before 7 am
               || ($secs > 21*60*60) ){   #after 9 pm
                  print N;
            }
            else {
                  print B;
            }
      }
}
close N;
close B;
close LogFile;
0
 
LVL 84

Assisted Solution

by:ozo
ozo earned 175 total points
ID: 18029198
And if you want to define holidays like Good Friday or Thanksgiving or Yom Yippur or Qi Xi you can use Date::Manip with a *Holiday section in the config file to define them
0
 
LVL 17

Assisted Solution

by:mjcoyne
mjcoyne earned 75 total points
ID: 18035397
"And if you want to define holidays like Good Friday or Thanksgiving or Yom Yippur or Qi Xi you can use Date::Manip with a *Holiday section in the config file to define them"

Didn't I already say that?  :)
0
 

Author Comment

by:awakenings
ID: 18038059
Thanks everyone. Work has kept me busy so I've been slow.  I'm getting a permission denied error so I am troubleshooting through that.

Perl Neophyte
0
 

Author Comment

by:awakenings
ID: 18039015
Okay... With some fenegalling I managed to get the file to run.  My file appears as follows.

#!/usr/bin/perl
use Time::Local;

my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);

$year += 1900;
$mon += 1;
$mday = sprintf("%0.2d", $mday);
$mon = sprintf("%0.2d", $mon);

$date_string = "$mon-$mday-$year";

open LogFile, "</home/mwebster/domain/fetch/sys$date_string.txt" or die "can't open $date_string.txt $!";
open sysB,">business" or die $!;
open sysN,">non-business" or die $!;
while(<LogFile>){
     if(/(\d{2})\/(\d{2})\/(\d{2}) (\d{1,2}):(\d{2}):(\d{2}) (A|P)M/){
          my $hr=$4;
          my $hr+=12 if $7 eq 'P';
          my $datetime = timelocal($6,$5,$hr,$2,$1-1,$3+2000);
          my @localdate=localtime($datetime);
          my $secs=$localdate[0]+60*$localdate[1]+60*60*$localdate[2];
          if(   ($localdate[6] == 0)    #sunday
             || ($localdate[6] == 6)    #saturday
             || ($secs < 7*60*60)       #before 7 am
             || ($secs > 21*60*60) ){   #after 9 pm
               print sysN;
          }
          else {
               print sysB;
          }
     }
}

     I can't find where sysN or sysB output the files to.  I need this to end in separate files.

Thanks,
0
 
LVL 39

Accepted Solution

by:
Adam314 earned 250 total points
ID: 18039588
These lines open the files sysN and sysB (named 'business' and 'non-business' in the current directory)
    open sysB,">business" or die $!;
    open sysN,">non-business" or die $!;
if either failed, you will see an error message, and the program won't continue.

then, this line gets the date and time from the file
    if(/(\d{2})\/(\d{2})\/(\d{2}) (\d{1,2}):(\d{2}):(\d{2}) (A|P)M/){

then, these lines convert that date/time in the format of the file to time value (seconds since epoch)
    my $hr=$4;
    my $hr+=12 if $7 eq 'P';
    my $datetime = timelocal($6,$5,$hr,$2,$1-1,$3+2000);
    my @localdate=localtime($datetime);

then, these lines check if it is a business or non-business hours
    my $secs=$localdate[0]+60*$localdate[1]+60*60*$localdate[2];
    if(   ($localdate[6] == 0)    #sunday
       || ($localdate[6] == 6)    #saturday
       || ($secs < 7*60*60)       #before 7 am
       || ($secs > 21*60*60) ){   #after 9 pm

This line prints to the sysN file
    print sysN;

This line prints to the sysB file
    print sysB;
0
 

Author Comment

by:awakenings
ID: 18039724
Ah...  My bad...  I didn't realize how perl was spitting them out.  Thanks!  The rest of the menutia, I can handle.  I'll assign points now.

Awakenings
0

Featured Post

Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

Join & Write a Comment

A year or so back I was asked to have a play with MongoDB; within half an hour I had downloaded (http://www.mongodb.org/downloads),  installed and started the daemon, and had a console window open. After an hour or two of playing at the command …
There are many situations when we need to display the data in sorted order. For example: Student details by name or by rank or by total marks etc. If you are working on data driven based projects then you will use sorting techniques very frequently.…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
You have products, that come in variants and want to set different prices for them? Watch this micro tutorial that describes how to configure prices for Magento super attributes. Assigning simple products to configurable: We assigned simple products…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now