Solved

Split file into two files using Perl

Posted on 2006-11-27
14
221 Views
Last Modified: 2010-03-05
I am a Perl neophyte.  I would like to take logs and split them into two files based upon a timestamp in the logs.  The timestamp is in the following format;

11/27/06 3:45:24 PM.

   Basically I want a business hours and a non-business hours file for each day.  I want everything from say 7 AM to 9 PM to be business houts and everything outside of this to be non business hours.  How do I accomplish this?

Perl neophyte
0
Comment
Question by:awakenings
  • 7
  • 3
  • 2
  • +1
14 Comments
 

Author Comment

by:awakenings
ID: 18022491
I will be working in SUSE.
0
 
LVL 84

Assisted Solution

by:ozo
ozo earned 175 total points
ID: 18022614
open B,">business" or die $!;
open N,">non-business" or die $!;
while( <> ){
    if( /\b([7-9]|1[01]):\d\d:\d\d\s*AM\b/i || /\b(12|[1-9]):\d\d:\d\d\s*PM\b/i ){
       print B;
   }else{
      print N;
   }
}
0
 
LVL 17

Assisted Solution

by:mjcoyne
mjcoyne earned 75 total points
ID: 18026807
If you need to further define business hours from non-business hours (for example, all hours on a weekend or a holiday might be considered non-business hours, regardless of the time of day), see Date::Manip (http://www.cise.ufl.edu/~sbeck/DateManip.html).
0
 

Author Comment

by:awakenings
ID: 18027824
Thanks folks!  Things became very busy yesterday and I didn't have time to get around to this.  I'll be testing this morning to make sure I have everything.
0
 

Author Comment

by:awakenings
ID: 18028145
Ozo,

    Please forgive my ignorence on this.  I am no programmer of any sort and I am trying to understand basics.  One of the things I forgot to mention was that I am taking logs from actual files.  I am not sure I see how to do that.  The files have dates so I am looking at the following functions added on to what you suggested.

#!/usr/bin/perl

my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);

$year += 1900;
$mon += 1;
$mday = sprintf("%0.2d", $mday);
$mon = sprintf("%0.2d", $mon);

$date_string = "$mon-$mday-$year";

print "$date_string\n";

open LogFile$date_string\n.txt  (<---  Key part I am not sure of)

open B,">business" or die $!;
open N,">non-business" or die $!;
while( <> ){
    if( /\b([7-9]|1[01]):\d\d:\d\d\s*AM\b/i || /\b(12|[1-9]):\d\d:\d\d\s*PM\b/i ){
       print B;
   }else{
      print N;
   }
}

   Now how do I output this information to another txt file?  I want to run this in cron daily so I don't have to do anything with it.
0
 

Author Comment

by:awakenings
ID: 18028159
Oh... As for  the comment about vacations, Saturdays and Sundays are considered non-business hours as well.

Thanks,

Perl Neophyte
0
 
LVL 84

Assisted Solution

by:ozo
ozo earned 175 total points
ID: 18028396
open LogFile, "<$date_string.txt" or die "can't open $date_string.txt $!"; #  (<---  Key part I am not sure of)

open B,">business" or die $!;
open N,">non-business" or die $!;
while( <LogFile> ){
    if( /\b([7-9]|1[01]):\d\d:\d\d\s*AM\b/i || /\b(12|[1-9]):\d\d:\d\d\s*PM\b/i ){
       print B;
   }else{
      print N;
   }
}
#Although this ignores weekends and holidays
0
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

 
LVL 39

Assisted Solution

by:Adam314
Adam314 earned 250 total points
ID: 18028925
This version looks at day of the week.  Saturday/Sunday go to non-business file...



use Time::Local;

open LogFile, "<$date_string.txt" or die "can't open $date_string.txt $!"; #  (<---  Key part I am not sure of)
open B,">business" or die $!;
open N,">non-business" or die $!;
while(<LogFile>){
      if(/(\d{2})\/(\d{2})\/(\d{2}) (\d{1,2}):(\d{2}):(\d{2}) (A|P)M/){
            my $hr=$4;
            my $hr+=12 if $7 eq 'P';
            my $datetime = timelocal($6,$5,$hr,$2,$1-1,$3+2000);
            my @localdate=localtime($datetime);
            my $secs=$localdate[0]+60*$localdate[1]+60*60*$localdate[2];
            if(   ($localdate[6] == 0)    #sunday
               || ($localdate[6] == 6)    #saturday
               || ($secs < 7*60*60)       #before 7 am
               || ($secs > 21*60*60) ){   #after 9 pm
                  print N;
            }
            else {
                  print B;
            }
      }
}
close N;
close B;
close LogFile;
0
 
LVL 84

Assisted Solution

by:ozo
ozo earned 175 total points
ID: 18029198
And if you want to define holidays like Good Friday or Thanksgiving or Yom Yippur or Qi Xi you can use Date::Manip with a *Holiday section in the config file to define them
0
 
LVL 17

Assisted Solution

by:mjcoyne
mjcoyne earned 75 total points
ID: 18035397
"And if you want to define holidays like Good Friday or Thanksgiving or Yom Yippur or Qi Xi you can use Date::Manip with a *Holiday section in the config file to define them"

Didn't I already say that?  :)
0
 

Author Comment

by:awakenings
ID: 18038059
Thanks everyone. Work has kept me busy so I've been slow.  I'm getting a permission denied error so I am troubleshooting through that.

Perl Neophyte
0
 

Author Comment

by:awakenings
ID: 18039015
Okay... With some fenegalling I managed to get the file to run.  My file appears as follows.

#!/usr/bin/perl
use Time::Local;

my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);

$year += 1900;
$mon += 1;
$mday = sprintf("%0.2d", $mday);
$mon = sprintf("%0.2d", $mon);

$date_string = "$mon-$mday-$year";

open LogFile, "</home/mwebster/domain/fetch/sys$date_string.txt" or die "can't open $date_string.txt $!";
open sysB,">business" or die $!;
open sysN,">non-business" or die $!;
while(<LogFile>){
     if(/(\d{2})\/(\d{2})\/(\d{2}) (\d{1,2}):(\d{2}):(\d{2}) (A|P)M/){
          my $hr=$4;
          my $hr+=12 if $7 eq 'P';
          my $datetime = timelocal($6,$5,$hr,$2,$1-1,$3+2000);
          my @localdate=localtime($datetime);
          my $secs=$localdate[0]+60*$localdate[1]+60*60*$localdate[2];
          if(   ($localdate[6] == 0)    #sunday
             || ($localdate[6] == 6)    #saturday
             || ($secs < 7*60*60)       #before 7 am
             || ($secs > 21*60*60) ){   #after 9 pm
               print sysN;
          }
          else {
               print sysB;
          }
     }
}

     I can't find where sysN or sysB output the files to.  I need this to end in separate files.

Thanks,
0
 
LVL 39

Accepted Solution

by:
Adam314 earned 250 total points
ID: 18039588
These lines open the files sysN and sysB (named 'business' and 'non-business' in the current directory)
    open sysB,">business" or die $!;
    open sysN,">non-business" or die $!;
if either failed, you will see an error message, and the program won't continue.

then, this line gets the date and time from the file
    if(/(\d{2})\/(\d{2})\/(\d{2}) (\d{1,2}):(\d{2}):(\d{2}) (A|P)M/){

then, these lines convert that date/time in the format of the file to time value (seconds since epoch)
    my $hr=$4;
    my $hr+=12 if $7 eq 'P';
    my $datetime = timelocal($6,$5,$hr,$2,$1-1,$3+2000);
    my @localdate=localtime($datetime);

then, these lines check if it is a business or non-business hours
    my $secs=$localdate[0]+60*$localdate[1]+60*60*$localdate[2];
    if(   ($localdate[6] == 0)    #sunday
       || ($localdate[6] == 6)    #saturday
       || ($secs < 7*60*60)       #before 7 am
       || ($secs > 21*60*60) ){   #after 9 pm

This line prints to the sysN file
    print sysN;

This line prints to the sysB file
    print sysB;
0
 

Author Comment

by:awakenings
ID: 18039724
Ah...  My bad...  I didn't realize how perl was spitting them out.  Thanks!  The rest of the menutia, I can handle.  I'll assign points now.

Awakenings
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
sort hash by values desc 2 186
Filtering a file to table 9 99
perl to convert excel to csv 3 182
Perl string filter 5 73
Email validation in proper way is  very important validation required in any web pages. This code is self explainable except that Regular Expression which I used for pattern matching. I originally published as a thread on my website : http://www…
In the distant past (last year) I hacked together a little toy that would allow a couple of Manager types to query, preview, and extract data from a number of MongoDB instances, to their tool of choice: Excel (http://dilbert.com/strips/comic/2007-08…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Learn how to create flexible layouts using relative units in CSS.  New relative units added in CSS3 include vw(viewports width), vh(viewports height), vmin(minimum of viewports height and width), and vmax (maximum of viewports height and width).

910 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

23 Experts available now in Live!

Get 1:1 Help Now