We help IT Professionals succeed at work.

perl file merge automation

Vlearns
Vlearns asked
on
Hi Experts,

i need your help with automating the following sequence on unix

I have bunch of directories like the following


ls100_ac2               ls100_hk2               ls100_ne1               ls101_ac2               ls101_mud               ls102_bf1               ls103_gq1               test.pl
ls100_ac4               ls100_in                ls100_sg1               ls101_aue               ls101_ne1               ls102_gq1               ls103_mud               test1.pl
ls100_bf1               ls100_in2               ls100_sk1               ls101_bf1               ls101_sg1               ls102_ird               ls103_ne1
ls100_cnb               ls100_ird               ls100_sp1               ls101_gq1               ls101_sp1               ls102_mud               ls104_mud

f

or every directory i do the following

1) enter the directory
2) there is file with the a conf extension in that directory with the same name along with a constant prefix. so directory ls100_ac2 has a file called imap_ls100_ac2.conf.  I create a copy of that file so that   cp  imap_ls100_ac2.conf imap_ls100_ac2.conf.bak
3) there is a test.pl in every directory.
4) i concatenate the file with conf extension to the perl file. eg. cat test.pl  imap_ls100_ac2.conf  >test2.pl
5) run test2.pl and overwite the original file. so perl test2.pl >  imap_ls100_ac2.conf
Comment
Watch Question

CERTIFIED EXPERT

Commented:
Hi Vlearns,

I see you have this in the Perl zone only.  Would you accept a shell script (e.g. bash)?

I don't think I'll make time to write it, but it would be good for experts to know.
CERTIFIED EXPERT

Commented:
I'm confused.  Why are you concatenating imap_ls100_ac2.conf onto test.pl to create test2.pl?  Why have test.pl in each directory (presumably they are the same script)?

Why not do something like this?  Put this in the parent directory and run it (from the same directory) - it will create backups and process all files.
#!/usr/local/bin/perl

use strict;
use warnings;
use File::Copy qw(cp);

# change these as needed
my $prefix = 'imap_';
my $ext = 'conf';

opendir DIR, '.' or die "could not open curr dir: $!";
my @dirs = grep { -d $_ } readdir DIR;
closedir DIR;

foreach my $dir (@dirs) {
    if (-f "$dir/$prefix$dir.$ext") {
        my $fil = "$dir/$prefix$dir.$ext";
        cp $fil, "$fil.bak" or die "could not backup $fil: $!";
        open IN, "$fil.bak" or die "could not read $fil.bak: $!";
        open OUT, '>', $fil or die "could not write $fil: $!";
        while (<IN>) {
            XXX - do whatever test.pl does here - XXX
        }
        close OUT;
        close IN;
    } else {
        warn "$dir/$prefix$dir.$ext does not exist - skipping $dir";
    }
}

Open in new window

ozo
CERTIFIED EXPERT
Most Valuable Expert 2014
Top Expert 2015

Commented:
Do the conf files contain __DATA__ for the perl programs?
If they contain executable code meant to generate other executable code to run next time,
I'd be very curious what the test.pl and conf files in each directory contain.

In any case, your request can be satisfied by:

#!/usr/bin/perl
for my $d ( grep -d,<ls*_*> ){
   rename "$d/imap_$d.conf","$d/imap_$d.conf.bak" or die $!;
   open STDOUT,">$d/test2.pl" or die $!;
   local @ARGV=("$d/test.pl","$d/imap_$d.conf.bak");
   print while <>;
   close STDOUT;
   open STDOUT,">$d/imap_$d.conf";
   do "$d/test2.pl";
}

Author

Commented:
Hi experts

you are right, copying test.pl to every directory is not good, since its the same file

this is the test.pl file:

#!/usr/bin/perl
while( <DATA> ){
    print;
    print qq($1adsOn: false,\n$1adExclusionlist: "RIM",\n$1coreTimeout: 400,\n) if /(\s*)lightsaberFetchOn: true,/
}
__DATA__



This is one of ozo's previous solution. I am 'cat'  ing the *.conf file in each directory to the data section of this file to create test2.pl

any suggestions to improve the solution are appreciated.

CERTIFIED EXPERT

Commented:
Modifying my previous solution slightly, we end up with this.  Let me know if you run into any problems.
#!/usr/local/bin/perl

use strict;
use warnings;

# change these as needed
my $prefix = 'imap_';
my $ext = 'conf';

opendir DIR, '.' or die "could not open curr dir: $!";
my @dirs = grep { -d $_ } readdir DIR;
closedir DIR;

foreach my $dir (@dirs) {
    if (-f "$dir/$prefix$dir.$ext") {
        my $fil = "$dir/$prefix$dir.$ext";
        rename $fil, "$fil.bak" or die "could not backup $fil: $!";
        open IN, "$fil.bak" or die "could not read $fil.bak: $!";
        open OUT, '>', $fil or die "could not write $fil: $!";
        while (<IN>) {
            print OUT $_;
            print OUT qq($1adsOn: false,\n$1adExclusionlist: "RIM",\n$1coreTimeout: 400,\n) if /(\s*)lightsaberFetchOn: true,/
        }
        close OUT;
        close IN;
    } else {
        warn "$dir/$prefix$dir.$ext does not exist - skipping $dir";
    }
}

Open in new window

Author

Commented:
Thanks wilcoxon

your code works! thanks!
in each of tose directories

there is a readme file like

iauto_conf_ls100_sk1.README

This package contains auto config files for IMAP.

-------------------------------------------------------------
Version 1.0.1
* [bug 4743348] Enabling INTLFR and Partner farms - Wave 2 of 3
  - 11

Version 1.0.0
* [bug 4738726] Initial version -- all services disabled


I want to add a new version number to this file like



-------------------------------------------------------------


Version 1.5
* [bug 4743348]  My change

Version 1.0.1
* [bug 4743348] Enabling INTLFR and Partner farms - Wave 2 of 3
  - 11

Version 1.0.0
* [bug 4738726] Initial version -- all services disabled



can i modify your code to do that?
CERTIFIED EXPERT
Commented:
Sure...

This should do it - just alter the $read_* vars at the top if needed.
#!/usr/local/bin/perl

use strict;
use warnings;

# change these as needed
my $dat_pre = 'imap_';
my $dat_ext = 'conf';
my $read_pre = 'iauto_conf_';
my $read_ext = 'README';
my $read_str = "Version 1.5\n* [bug 4743348] My change\n\n";

opendir DIR, '.' or die "could not open curr dir: $!";
my @dirs = grep { -d $_ } readdir DIR;
closedir DIR;

foreach my $dir (@dirs) {
    if (-f "$dir/$dat_pre$dir.$dat_ext") {
        my $fil = "$dir/$dat_pre$dir.$dat_ext";
        rename $fil, "$fil.bak" or die "could not backup $fil: $!";
        open IN, "$fil.bak" or die "could not read $fil.bak: $!";
        open OUT, '>', $fil or die "could not write $fil: $!";
        while (<IN>) {
            print OUT $_;
            print OUT qq($1adsOn: false,\n$1adExclusionlist: "RIM",\n$1coreTimeout: 400,\n) if /(\s*)lightsaberFetchOn: true,/
        }
        close OUT;
        close IN;
    } else {
        warn "$dir/$dat_pre$dir.$dat_ext does not exist - skipping $dir";
    }
    if (-f "$dir/$read_pre$dir.$read_ext") {
        my $fil = "$dir/$read_pre$dir.$read_ext";
        rename $fil, "$fil.bak" or die "could not backup $fil: $!";
        open IN, "$fil.bak" or die "could not read $fil.bak: $!";
        open OUT, '>', $fil or die "could not write $fil: $!";
        my $done;
        while (<IN>) {
            if ($done) {
                print OUT $_;
            } else {
                if (m{^\s*Version\b}) {
                    print OUT $read_str, $_;
                    $done++;
                } else {
                    print OUT $_;
                }
            }
        }
        close OUT;
        close IN;
    } else {
        warn "$dir/$read_pre$dir.$read_ext does not exist - skipping $dir";
    }
}

Open in new window

ozo
CERTIFIED EXPERT
Most Valuable Expert 2014
Top Expert 2015
Commented:
If all the test.pl files are the same, and end with __DATA__, it sounds like you just want
perl -i.bak -ne 'print;print qq($1adsOn: false,\n$1adExclusionlist: "RIM",\n$1coreTimeout: 400,\n) if /(\s*)lightsaberFetchOn: true,/' ls*/imap_*.conf

Author

Commented:
Thank you!

Explore More ContentExplore courses, solutions, and other research materials related to this topic.