Link to home
Start Free TrialLog in
Avatar of richsark
richsarkFlag for United States of America

asked on

Have a perl script, but need it to work on a whole directory not just one file name at a time

Hello,  I have a perl script that I need it to look into a directory named mpl.  right now I run it but it does it for one file at a time. Sucks cause There are a few hundred. So If I can seek hellp to make my perl script open a whole directory and do what it does per one file but in a whole directory.

I need my script adjusted  to look into my directory mpl and read all the zonefiles and change the serial number using date with an extra number 2019011300 ; serial for example

I run the script like this on the below:

./zonechanges.pl db.apptoapp.net.new vipchange

Copy of the script:

#!/usr/bin/perl

open(ZONE,  $ARGV[0]) || die "Zone file error $!";
while ( $line = <ZONE> ) {
        chomp $line;
        push @fileLines, $line;
}
close ZONE;



# Assuming simple CSV of old_ip, new_ip
open(IPMAP, $ARGV[1]) || die "IP Map file error: $!";
while ( $line = <IPMAP> ) {
        chomp $line;
        ($old, $new) = split(/,/, $line);
        $old =~ s/\s+//;
        $new =~ s/\s+//;
        $ip{$old} = $new;
}
close IPMAP;


## For every line
for ($count=0; $count < @fileLines; $count++) {
#       print STDERR "OUTER FOR ($count)\n";
        $line = $fileLines[$count];
#       print STDERR "OUTER FOR have line $line\n";
        ## For every old address in the remap
        for $old (keys(%ip)) {
                #print STDERR "INNER FOR ($old)\n";
                ## If the line matches the old address
                if ( $line =~ /$old/ ) {
                        ($dnsName, @crap) = split(/\s+/, $line);
                        ## insert a new line with a new record or the new sites
                        ## if this is a site specific record
                        if ( $dnsName =~ m/\-mpl|\-jax/ ) {
                                $line =~ s/\-mpl/\-shk/;
                                $line =~ s/\-jax/\-str/;
                                $line =~ s/$old/$ip{$old}/;
                                push @fileLines, $line;

                        ## Othefwise, change the address to new
                        } else {
                                print STDERR "INNER FOR ELSE - line did not match $old\n";
                               $fileLines[$count] =~ s/$old/$ip{$old}/;
                        }
                }
        }
}

#i Dump the updates to STDOUT
foreach $line (@fileLines) {
        print $line."\n";
}

Open in new window

Avatar of David Favor
David Favor
Flag of United States of America image

Normally I write PERL scripts like this....

foreach my $path (@ARGV) {
       if (-e $path) { do_dir($path); }
       else          { do_file($path); }
}

Open in new window


Then in the do_dir() function build a file list by running recursive find or glob() if no recursion is required.
Then in do_dir() call do_file() for each file.
Avatar of richsark

ASKER

Hi, Thank you, so What will I need to replace? Sorry if I sound not savvy. could I ask that you show me a complete update of what you mean so I can understand? Then when I run it, an example of the command line
You'll do this...

1) Move all your current code into do_file().

2) Write a do_dir() function, to produce a file list however seems best to your (find or glob).

Simple to do. Just takes a bit of time.
So...

sub do_file ($) {

     my $path = shift;

     # change $ARGV[1] to $path

     # all your code goes here...

}

function do_dir ($) {

     my $dir = shift;

     # expand $dir to a file @list using find or glob

     foreach my $file (@list) {
         do_file($file);

}

foreach my $path (@ARGV) {
       if (-e $path) { do_dir($path); }
       else          { do_file($path); }
}
Hello David, I appericate you feed back. I am a beginner and not fully to the level where I can run with it myself.
Hello, thanks, I am trying to learn, but I am slow at this. I know the current one works, just one filename at a time.
I know your time is valuable, perhaps you or another team member can paste where the new code needs to fit.
I agree. But I need to learn on my own time. Right now i need to get this working please.  Could you please or anyone make the necessary updates for me please.
Hello. Can I get someone to assist me.  I'm no means Perl ready to solve this case.  I'm seeking help for this script to have a small addition to do its processing inside a directory
Do you want the output just sent to STDOUT (as it currently does), the file edited in place, or a backup and new file created?

Are you opposed to using modules that will make it easier?
Hello. I like for it to work as is. However with modules I would try it for simplistic. What ever you think. I really appreciate this
You've been posting perl questions for 10 years, some of which I've helped you with, and as such it's reasonable to assume that your perl knowledge is greater than you claim.

David has given a reasonable approach to a solution, but I recommend you not use prototypes like he suggests.
Far More than Everything You've Ever Wanted to Know about Prototypes in Perl -- by Tom Christiansen

wilcoxon may be willing to give you a fully tested script that does exactly what you want, but I think you should attempt to write the code yourself and if you need additional help, post your code and specific question and tell us how your code is failing to meet your needs.
How many lines are typically in the Zone file specified by $ARGV[0] (that go into @fileLines)?  What is (roughly) the max lines you've seen?
They are Dns zone files. Thousands of lines. But we only want to change serial number.
Hello FishMonger, I really appreciate all your efforts , I really do. I am autistic and have a learning disability.  I don't like to mention these things and I am trying to do my best.  All you guys here are terrific and I do truly appreciate you all helping me.
Unfortunately I've run out of time tonight and am pretty much unavailable for the next 3 days.  If this question hasn't been answered to your satisfaction by then (or if I get a little free time before then), I'll work out a solution that also makes the script more efficient.
I have a working (php) script that I use which sets the serial number using the exact format you want but builds the host entries based on querying a database.  However, I'm on vacation and won't have access to it until next week.  If wilcoxon hasn't given you his solution by then, I'll see what I can do for you.
Thank you fishmonger, this means lots. . I hope wilcoxon can help soon too. But if anyone else would like to help please
hi /wilcoxon, I got your update. The sooner the better if you can please sir. I truly appreciate this from everyone..
Avatar of noci
noci

I will look into this a bit later.. As it needs some work.
This should do it....
It will update SOA in two cases:
     SOA mast admin serial ....
or
     SOA ........ (
            serial ....
      ....)

Also IO has been slightly made faster, (and simpler) although that should not be a problem anyway.

#!/usr/bin/perl

use strict;
use POSIX;

my $debug = 0;

sub handle_zonefile($$$) {
        my ($file, $ipmap, $outfile) = @_;

        print "Handling: $file -> $outfile\n";
        open(ZONE,  $file) || return "Zone file error $!";
        my $content;
        {
                local $/ = undef;
                $content = <ZONE>;
        }
        close ZONE;
        my @fileLines = split(/\n/,$content);


        # Assuming simple CSV of old_ip, new_ip
        my %ip;
        print "Reading ipmap: $ipmap\n" if $debug;
        open(IPMAP, $ipmap) || return "IP Map file error: $!";
        while ( my $line = <IPMAP> ) {
                chomp $line;
                my ($old, $new) = split(/,/, $line);
                $old =~ s/\s+//g;
                $new =~ s/\s+//g;
                $ip{$old} = $new;
        }
        close IPMAP;

        my $soaseen = 0;
        ## For every line  replace address possibly insert new ones
        foreach my $line (@fileLines) {
                ## For every old address in the remap
                foreach my $old (keys(%ip)) {
                        #print STDERR "INNER FOR ($old)\n";
                        ## If the line matches the old address
                        if ( $line =~ /$old/ ) {
                                print "Changing: %old\n" if $debug;
                                my ($dnsName, @crap) = split(/\s+/, $line);
                                ## insert a new line with a new record or the new sites
                                ## if this is a site specific record
                                if ( $dnsName =~ m/\-mpl|\-jax/ ) {
                                        my $newline = $line;
                                        $newline =~ s/\-mpl/\-shk/;
                                        $newline =~ s/\-jax/\-str/;
                                        $newline =~ s/$old/$ip{$old}/;
                                        push @fileLines, $newline;
                                ## Otherwise, change the address to new
                                } else {
                                        print STDERR "line did not match new-names, replace address: $old\n";
                                        $line =~ s/$old/$ip{$old}/;
                                }
                        }
                }
                if ($soaseen) {
                        print "Handle case 1\n" if $debug;
                        my $serialhead=strftime('%4Y%02m%02d', gmtime());
                        if ( $line =~ /\s*($serialhead[0-9][0-9])\s*/ ) {
                                my $t = substr($1,8,2);
                                print "Match: $1 $t\n" if $debug;
                                $serialhead= sprintf( "%s%02d", $serialhead, $t+1);
                        } else {
                                $serialhead .= '00';
                        }
                        print "Serial: $serialhead\n" if $debug;
                        $line =~ s/[0-9]+/$serialhead/;
                        print "Update: $line\n" if $debug;
                        $soaseen = 0;
                }
                if ($line =~ /\ssoa\s/i) {
                        print "SOA detected: " if $debug;
                        # 2 cases one: SOA master admin (    with serial on new line
                        #              SOA master admin serial .....   all on one line
                        if ($line =~ /\s\(/) {
                                print "Case 1\n" if $debug;
                                $soaseen = 1; # we need to update next line
                        } else {
                                print "Case 2\n" if $debug;
                                my $serialhead=strftime('%4Y%02m%02d', gmtime());
                                if ( $line =~ /\s*($serialhead[0-9][0-9])\s*/ ) {
                                        my $t = substr($1,8,2);
                                        $serialhead= sprintf( "%s%02d", $serialhead, $t+1);
                                } else {
                                        $serialhead .= '00';
                                }
                                $line =~ s/(\s)[0-9]+(\s)/$1$serialhead$2/;
                        }
                }
        }

        print "Create output\n" if $debug;
        my $out = join("\n", @fileLines);
        open(OUT, ">$outfile") || return "Unable to create new file: $!";
        print OUT $out."\n";
        close OUT;
        return "";
}

my $ipmap = shift(@ARGV);       # first argument is IP map
foreach my $arg (@ARGV) {
        if ( -d $arg ) {
                print STDERR "$arg is a directory pleas change command line\n";
        }
        my $msg = handle_zonefile($arg, $ipmap, $arg.".new"), ;
        if ( $msg != "" ) {
                print STDERR "Error on $arg: $msg\n";
        }
}
exit(0);

Open in new window

Oh btw... you need to call it with:

./zonechanges.pl vipchange db.apptoapp.net.new db.*.net.new

(the shell will take care of wildcard expansion).
A new file will be created with .new appended to you get: db.apptoapp.net.new.new  etc.
Hello. I will try this. I assume it will change the serial number to a date type format. Also it will read the entire directory.

So an example what would it look like on the command line. Same as before. Or s switch is added to read a whole directory.  Just want to learn it

./zonechanges.pl db.apptoapp.net.new vipchange

And I thank you.
strftime is a time string formatting routine....
so Yes, it uses gmtime() as the clock time though.
In another comment i told how to call it...  (it was changed to accomodate the use of wildcards...)
If you are in a directory with only zone files:

/where/the/zonechanges.pl /where/ever/the/vipchange  *

(you will need to change the /where..../ parts.
Hi, as you can tell I have a call out in the command line in my original cmd line called vipchange which contains IP's that the code looks into


run line as:
./changezone2.pl vipchange db.testzone.com


The vipchange contents look like

10.172.223.11,10.204.210.11
10.172.223.12,10.204.210.12
10.172.223.21,10.204.210.13
10.172.223.24,10.204.210.14
10.172.223.25,10.204.210.15
10.172.223.26,10.204.210.16
10.172.223.27,10.204.210.17
10.172.223.28,10.204.210.18
10.172.223.29,10.204.210.19
10.172.223.30,10.204.210.20

Open in new window


Just want to make sure it been accounting for and works the same just that we added it to read the whole directory
I copied your code.... (just the declaration has been changed to allo use strinct.)
On closer inspection there was a problem in your code:
say you had: 192.168.1.10 -> 192.168.1.30 in your mappen, then it would also change a recoord with:
192.168.1.100 -> 192.168.1.300  --- not a good plan:
This will fix that:
#!/usr/bin/perl

use strict;
use POSIX;

my $debug = 0;

sub handle_zonefile($$$) {
        my ($file, $ipmap, $outfile) = @_;

        print "Handling: $file -> $outfile\n";
        open(ZONE,  $file) || return "Zone file error $!";
        my $content;
        {
                local $/ = undef;
                $content = <ZONE>;
        }
        close ZONE;
        my @fileLines = split(/\n/,$content);


        # Assuming simple CSV of old_ip, new_ip
        my %ip;
        print "Reading ipmap: $ipmap\n" if $debug;
        open(IPMAP, $ipmap) || return "IP Map file error: $!";
        while ( my $line = <IPMAP> ) {
                chomp $line;
                my ($old, $new) = split(/,/, $line);
                $old =~ s/\s+//g;
                $new =~ s/\s+//g;
                $ip{$old} = $new;
        }
        close IPMAP;

        if ($debug) {
                foreach my $xip (keys(%ip)) {
                        print "$xip -> $ip{$xip}\n";
                }
        }

        my $soaseen = 0;
        ## For every line  replace address possibly insert new ones
        foreach my $line (@fileLines) {
                ## For every old address in the remap
                foreach my $old (keys(%ip)) {
                        #print STDERR "INNER FOR ($old)\n";
                        ## If the line matches the old address
                        if ( $line =~ /\s$old(\s|;|$)/ ) {
                                print "Changing: $old\n" if $debug;
                                my ($dnsName, @crap) = split(/\s+/, $line);
                                ## insert a new line with a new record or the new sites
                                ## if this is a site specific record
                                if ( $dnsName =~ m/\-mpl|\-jax/ ) {
                                        my $newline = $line;
                                        $newline =~ s/\-mpl/\-shk/;
                                        $newline =~ s/\-jax/\-str/;
                                        $newline =~ s/$old/$ip{$old}/;
                                        push @fileLines, $newline;
                                ## Otherwise, change the address to new
                                } else {
                                        print STDERR "line did not match new-names, replace address: $old\n";
                                        $line =~ s/(\s)$old(\s|$|;)/$1$ip{$old}$2/;
                                }
                        }
                }
                if ($soaseen) {
                        print "Handle case 1\n" if $debug;
                        my $serialhead=strftime('%4Y%02m%02d', gmtime());
                        if ( $line =~ /\s*($serialhead[0-9][0-9])\s*/ ) {
                                my $t = substr($1,8,2);
                                print "Match: $1 $t\n" if $debug;
                                $serialhead= sprintf( "%s%02d", $serialhead, $t+1);
                        } else {
                                $serialhead .= '00';
                        }
                        print "Serial: $serialhead\n" if $debug;
                        $line =~ s/[0-9]+/$serialhead/;
                        print "Update: $line\n" if $debug;
                        $soaseen = 0;
                }
                if ($line =~ /\ssoa\s/i) {
                        print "SOA detected: " if $debug;
                        # 2 cases one: SOA master admin (    with serial on new line
                        #              SOA master admin serial .....   all on one line
                        if ($line =~ /\s\(/) {
                                print "Case 1\n" if $debug;
                                $soaseen = 1; # we need to update next line
                        } else {
                                print "Case 2\n" if $debug;
                                my $serialhead=strftime('%4Y%02m%02d', gmtime());
                                if ( $line =~ /\s*($serialhead[0-9][0-9])\s*/ ) {
                                        my $t = substr($1,8,2);
                                        $serialhead= sprintf( "%s%02d", $serialhead, $t+1);
                                } else {
                                        $serialhead .= '00';
                                }
                                $line =~ s/(\s)[0-9]+(\s)/$1$serialhead$2/;
                        }
                }
        }

        print "Create output\n" if $debug;
        my $out = join("\n", @fileLines);
        open(OUT, ">$outfile") || return "Unable to create new file: $!";
        print OUT $out."\n";
        close OUT;
        return "";
}

my $ipmap = shift(@ARGV);       # first argument is IP map
foreach my $arg (@ARGV) {
        if ( -d $arg ) {
                print STDERR "$arg is a directory pleas change command line\n";
        }
        my $msg = handle_zonefile($arg, $ipmap, $arg.".new"), ;
        if ( $msg != "" ) {
                print STDERR "Error on $arg: $msg\n";
        }
}
exit(0);

Open in new window


btw. set the $debug = 0 ==> $debug = 1 will show some more info.
Ok. Here's the debug

root@xjaxlaac9181 zones_testing]# ./changezone3.pl test/db.net401k.com vip1
Handling: vip1 -> vip1.new
Reading ipmap: test/db.net401k.com
 ->
600;Retryafter10min ->
;ZoneRecords ->
@INNSns0.voya.net. ->
www60INA10.160.50.34 ->
@INNSns1.voya.net. ->
@5INA10.160.50.19 ->
;NAMETTLCLASSTYPERDATA ->
;ZoneNSRecords ->
; ->
2015083002;SerialYYMMDDhh ->
;(alsosee/etc/named.conf) ->
864000;Expireafter1day ->
;ZoneMXRecords ->
3600;Refreshafter1hour ->
;CNAMERecords ->
900);MinimumTTLof15minutes ->
$TTL900 ->
@INSOAns0.voya.net.hostmaster.voya.com.( ->
;nameserverdatafilefornet401k.com ->
Changing:
line did not match new-names, replace address:
Unmatched ) in regex; marked by <-- HERE in m/\s900) <-- HERE ;MinimumTTLof15minutes(\s|;|$)/ at ./changezone3.pl line 49.



Also. I had to add <ZONE> back in and <IPMAP> to fix execution errors.  

Appears that the code in find replace is looking for  ip and does not know how to process the text from first few lines


Never gets past that point to actually replace the ip’s


You see any thing to make it better

[code]   !/usr/bin/perl

   use strict;
   use POSIX;

   my $debug = 0;

   sub handle_zonefile($$$) {
   my ($file, $ipmap, $outfile) = @_;

   print "Handling: $file -> $outfile\n";
   open(ZONE, $file) || return "Zone file error $!";
   my $content;
   {
   local $/ = undef;
   $content = ;
   }
   close ZONE;
   my @fileLines = split(/\n/,$content);


   # Assuming simple CSV of old_ip, new_ip
   my %ip;
   print "Reading ipmap: $ipmap\n" if $debug;
   open(IPMAP, $ipmap) || return "IP Map file error: $!";
   while ( my $line = ) {
   chomp $line;
   my ($old, $new) = split(/,/, $line);
   $old =~ s/\s+//g;
   $new =~ s/\s+//g;
   $ip{$old} = $new;
   }
   close IPMAP;

   if ($debug) {
   foreach my $xip (keys(%ip)) {
   print "$xip -> $ip{$xip}\n";
   }
   }

   my $soaseen = 0;
   ## For every line replace address possibly insert new ones
   foreach my $line (@fileLines) {
   ## For every old address in the remap
   foreach my $old (keys(%ip)) {
   #print STDERR "INNER FOR ($old)\n";
   ## If the line matches the old address
   if ( $line =~ /\s$old(\s|;|$)/ ) {
   print "Changing: $old\n" if $debug;
   my ($dnsName, @crap) = split(/\s+/, $line);
   ## insert a new line with a new record or the new sites
   ## if this is a site specific record
   if ( $dnsName =~ m/\-mpl|\-jax/ ) {
   my $newline = $line;
   $newline =~ s/\-mpl/\-shk/;
   $newline =~ s/\-jax/\-str/;
   $newline =~ s/$old/$ip{$old}/;
   push @fileLines, $newline;
   ## Otherwise, change the address to new
   } else {
   print STDERR "line did not match new-names, replace address: $old\n";
   $line =~ s/(\s)$old(\s|$|;)/$1$ip{$old}$2/;
   }
   }
   }
   if ($soaseen) {
   print "Handle case 1\n" if $debug;
   my $serialhead=strftime('%4Y%02m%02d', gmtime());
   if ( $line =~ /\s*($serialhead[0-9][0-9])\s*/ ) {
   my $t = substr($1,8,2);
   print "Match: $1 $t\n" if $debug;
   $serialhead= sprintf( "%s%02d", $serialhead, $t+1);
   } else {
   $serialhead .= '00';
   }
   print "Serial: $serialhead\n" if $debug;
   $line =~ s/[0-9]+/$serialhead/;
   print "Update: $line\n" if $debug;
   $soaseen = 0;
   }
   if ($line =~ /\ssoa\s/i) {
   print "SOA detected: " if $debug;
   # 2 cases one: SOA master admin ( with serial on new line
   # SOA master admin serial ..... all on one line
   if ($line =~ /\s\(/) {
   print "Case 1\n" if $debug;
   $soaseen = 1; # we need to update next line
   } else {
   print "Case 2\n" if $debug;
   my $serialhead=strftime('%4Y%02m%02d', gmtime());
   if ( $line =~ /\s*($serialhead[0-9][0-9])\s*/ ) {
   my $t = substr($1,8,2);
   $serialhead= sprintf( "%s%02d", $serialhead, $t+1);
   } else {
   $serialhead .= '00';
   }
   $line =~ s/(\s)[0-9]+(\s)/$1$serialhead$2/;
   }
   }
   }

   print "Create output\n" if $debug;
   my $out = join("\n", @fileLines);
   open(OUT, ">$outfile") || return "Unable to create new file: $!";
   print OUT $out."\n";
   close OUT;
   return "";
   }

   my $ipmap = shift(@ARGV); # first argument is IP map
   foreach my $arg (@ARGV) {
   if ( -d $arg ) {
   print STDERR "$arg is a directory pleas change command line\n";
   }
   my $msg = handle_zonefile($arg, $ipmap, $arg.".new"), ;
   if ( $msg != "" ) {
   print STDERR "Error on $arg: $msg\n";
   }
   }
   exit(0);[/code]
What do you mean "i had to add the <zone> back in...." ...
Not sure what you attempted to prove....

There was no <zone> missing in the script i put on the code box....
Also FIRST argument = ip mapping table
Next arguments (however many you like are the Zone files...).
Using it bacward (Zone file as IP Map & IPmap as zone file....)..

Easies way to copy: use "select all" beneath the code box,
than on you unix/linux  system (paste it in the an editor window in insert mode)
or cat >filename.pl in a command line window and paste it and then type CTRL/D (Control & D together).
I will attach it  here as well:

call:
./zoneedit.pl vip1 test/db*
zoneedit.pl
Here are the errors

Errors trying to execute .
 
[root@xjaxlaac9181 zones_testing]# ./changezone vips data/internal/prod/jax/db.*
syntax error at ./changezone line 16, near "= ;"
syntax error at ./changezone line 24, near "= ) "
Global symbol "$line" requires explicit package name at ./changezone line 25.
Global symbol "$line" requires explicit package name at ./changezone line 26.
Global symbol "%ip" requires explicit package name at ./changezone line 34.
Global symbol "%ip" requires explicit package name at ./changezone line 35.
Global symbol "@fileLines" requires explicit package name at ./changezone line 41.
Global symbol "%ip" requires explicit package name at ./changezone line 43.
Global symbol "%ip" requires explicit package name at ./changezone line 55.
Global symbol "@fileLines" requires explicit package name at ./changezone line 56.
Global symbol "%ip" requires explicit package name at ./changezone line 60.
Global symbol "@fileLines" requires explicit package name at ./changezone line 100.
Global symbol "$outfile" requires explicit package name at ./changezone line 101.

syntax error at ./changezone line 105, near "}"
./changezone has too many errors.


Also. What does lines 16 and 24 do?
Hi team. I think I got it it working.

Have a few questions though.

I ran the script against some test zones and picked a few IPS to check from the vip list.  I have mixed results.  I have duplicates of the –shk and –str records for all new records .
The record that was –jax and or –mpl did not get updated with the new vip ip .

Also. Can we add logic to only modify what it needs to change rather then the whole list.
Code line 12-19:  (from the earlier script).
        open(ZONE,  $file) || return "Zone file error $!";
        my $content;
        {
                local $/ = undef;
                $content = <ZONE>;
        }
        close ZONE;
        my @fileLines = split(/\n/,$content);

Open in new window

line 12 opens the file
line 15 set the line terminator to undef meaning there is no end of line marker (in a local variable, that dissipates on nest block close '}'),
line 16 read the COMPLETE file in "one" I/O more importantly in one string.   (aka Slurp mode reading).
Then handling the string can be one all at once. (in this case just splitting in lines)   using split.
(done in line 19)

(so the { local .... } construct is important, as is declaring the string variable receiving the content BEFORE the block.

Line 24:
print "Reading ipmap: $ipmap\n" if $debug;

Open in new window

this will print a line (containing the ipmap filename)  if the debug variable is "true" or 1.

I have no idea what you liner 105 is,
In my listing it is:    
print OUT $out."\n";

Open in new window

I don see an error there.

When presenting code please use a code block ("code" from the comment toolbar).
Hello. What do you think is the issue with the code and the results? Can you make it better? I'm open to anything as long as it works.

I ran the script against some test zones and picked a few IPS to check from the vip list.  I have mixed results.  I have duplicates of the –shk and –str records for all new records .
The record that was –jax and or –mpl did not get updated with the new vip ip.
Hi,   did you actualy use the code from this block: #a42837427
of download from here: #a42837691  (and read the description?)....

It doesn't look like it... you show missing parts that are there..., use a different name & different set of argument.
So the first step would be to get the RIGHT code on you system.
I tried the script on my system on some zone files and the were modified....

if a line contains -mpl  then the -mpl is changed to -shk and the address is changed.
if a line contains -jax then the -jax is changed to -str and the address is changed.
And the new record is added to the end of the list.
==> then -mpl & -jax will have no address change (taken from the original code), if those addresses also need  changed, please explain all the rules.
for any other record the ip address is mapped... based date (if not from today) with .00 if the date was from today then one is added to the suffix.
TODAY00 -> TODAY01 etc.  (with today obvious being YYYYMMDD with today's date)
The next script will ALSO update the -mpl of jax address:

#!/usr/bin/perl

use strict;
use POSIX;

my $debug = 0;

sub handle_zonefile($$$) {
        my ($file, $ipmap, $outfile) = @_;

        print "Handling: $file -> $outfile\n";
        open(ZONE,  $file) || return "Zone file error $!";
        my $content;
        {
                local $/ = undef;
                $content = <ZONE>;
        }
        close ZONE;
        my @fileLines = split(/\n/,$content);


        # Assuming simple CSV of old_ip, new_ip
        my %ip;
        print "Reading ipmap: $ipmap\n" if $debug;
        open(IPMAP, $ipmap) || return "IP Map file error: $!";
        while ( my $line = <IPMAP> ) {
                chomp $line;
                my ($old, $new) = split(/,/, $line);
                $old =~ s/\s+//g;
                $new =~ s/\s+//g;
                $ip{$old} = $new;
        }
        close IPMAP;

        if ($debug) {
                foreach my $xip (keys(%ip)) {
                        print "$xip -> $ip{$xip}\n";
                }
        }

        my $soaseen = 0;
        ## For every line  replace address possibly insert new ones
        foreach my $line (@fileLines) {
                ## For every old address in the remap
                foreach my $old (keys(%ip)) {
                        #print STDERR "INNER FOR ($old)\n";
                        ## If the line matches the old address
                        if ( $line =~ /\s$old(\s|;|$)/ ) {
                                print "Changing: $old\n" if $debug;
                                my ($dnsName, @crap) = split(/\s+/, $line);
                                ## insert a new line with a new record or the new sites
                                ## if this is a site specific record
                                if ( $dnsName =~ m/\-mpl|\-jax/ ) {
                                        $line =~ s/$old/$ip{$old}/;                  # also change th IP address on -mpl & -jax record.
                                        my $newline = $line;
                                        $newline =~ s/\-mpl/\-shk/;               # add new -shk / -str records, first make a copy of the line, later append it.
                                        $newline =~ s/\-jax/\-str/;
                                        push @fileLines, $newline;
                                ## Otherwise, change the address to new
                                } else {
                                        print STDERR "line did not match new-names, replace address: $old\n";
                                        $line =~ s/(\s)$old(\s|$|;)/$1$ip{$old}$2/;
                                }
                        }
                }
                if ($soaseen) {
                        print "Handle case 1\n" if $debug;
                        my $serialhead=strftime('%4Y%02m%02d', gmtime());
                        if ( $line =~ /\s*($serialhead[0-9][0-9])\s*/ ) {
                                my $t = substr($1,8,2);
                                print "Match: $1 $t\n" if $debug;
                                $serialhead= sprintf( "%s%02d", $serialhead, $t+1);
                        } else {
                                $serialhead .= '00';
                        }
                        print "Serial: $serialhead\n" if $debug;
                        $line =~ s/[0-9]+/$serialhead/;
                        print "Update: $line\n" if $debug;
                        $soaseen = 0;
                }
                if ($line =~ /\ssoa\s/i) {
                        print "SOA detected: " if $debug;
                        # 2 cases one: SOA master admin (    with serial on new line
                        #              SOA master admin serial .....   all on one line
                        if ($line =~ /\s\(/) {
                                print "Case 1\n" if $debug;
                                $soaseen = 1; # we need to update next line
                        } else {
                                print "Case 2\n" if $debug;
                                my $serialhead=strftime('%4Y%02m%02d', gmtime());
                                if ( $line =~ /\s*($serialhead[0-9][0-9])\s*/ ) {
                                        my $t = substr($1,8,2);
                                        $serialhead= sprintf( "%s%02d", $serialhead, $t+1);
                                } else {
                                        $serialhead .= '00';
                                }
                                $line =~ s/(\s)[0-9]+(\s)/$1$serialhead$2/;
                        }
                }
        }

        print "Create output\n" if $debug;
        my $out = join("\n", @fileLines);
        open(OUT, ">$outfile") || return "Unable to create new file: $!";
        print OUT $out."\n";
        close OUT;
        return "";
}

my $ipmap = shift(@ARGV);       # first argument is IP map
foreach my $arg (@ARGV) {
        if ( -d $arg ) {
                print STDERR "$arg is a directory pleas change command line\n";
        }
        my $msg = handle_zonefile($arg, $ipmap, $arg.".new"), ;
        if ( $msg != "" ) {
                print STDERR "Error on $arg: $msg\n";
        }
}
exit(0);

Open in new window

Thank you. I will test and Revert back. I really appreciate it.
Hi Team, I have some issues, please let me explain more.

Issues,

1.        CNAME record replacement issues
a.       The original record name was stripped of –mpl or –jax
b.       The CNAME record name was changed and not duplicated with new name

 

Original zone int/prod/jax/db.apptoapp.net

;idataservice-rs-suitability             60       IN      CNAME       datapower-mpl

;idataservice-rs-suitability-mpl         60       IN      CNAME       datapower-mpl

;idataservice-rs-suitability-jax         60       IN      CNAME       datapower-mpl

idataservice-rs-suitability                     60       IN      CNAME       datapower-mpl

idataservice-rs-suitability-mpl         60       IN      CNAME       datapower-mpl

idataservice-rs-suitability-jax                 60       IN      CNAME       datapower-mpl

 

New Output file

;idataservice-rs-suitability             60       IN      CNAME       datapower-mpl

;idataservice-rs-suitability         60       IN      CNAME       datapower-mpl

;idataservice-rs-suitability         60       IN      CNAME       datapower-mpl

idataservice-rs-suitability                     60       IN      CNAME       datapower-mpl

idataservice-rs-suitability             60       IN      CNAME       datapower-mpl

idataservice-rs-suitability             60       IN      CNAME       datapower-mpl

;idataservice-rs-suitability         60       IN      CNAME       datapower-shk

;idataservice-rs-suitability         60       IN      CNAME       datapower-shk

idataservice-rs-suitability             60       IN      CNAME       datapower-shk

idataservice-rs-suitability             60       IN      CNAME       datapower-shk

 

 

2.       Odd records not replaced correctly , Appears to still duplicating the records and the original record name was changed .
 
a.       Same zone file record example
                                                               i.      gengmshttp-jax              60       IN        A        10.172.171.162
                                                             ii.      gengmshttp-jax-mpl              60       IN        A        10.170.171.162
                                                           iii.      gengmshttp-jax-jax              60       IN        A        10.172.171.162
b.       New file records
                                                               i.      gengmshttp              60       IN        A        10.205.7.21
                                                             ii.      gengmshttp-mpl              60       IN        A        10.203.7.46
                                                           iii.      gengmshttp-jax              60       IN        A        10.205.7.21
                                                           iv.      gengmshttp              60       IN        A        10.205.7.21
                                                             v.      gengmshttp-shk              60       IN        A        10.203.7.46
                                                           vi.      gengmshttp-shk              60       IN        A        10.203.7.46
                                                          vii.      gengmshttp-str              60       IN        A        10.205.7.21
                                                        viii.      gengmshttp-str              60       IN        A        10.205.7.21

 

3.       Records changed not correct Examples below same zone file.  The source record name was sripped of view name and the ip vips are not updated.
a.       Original records
                                                               i.      hwpayroll               60      IN      A       172.18.242.32
                                                             ii.      hwpayroll-mpl           60      IN      A       10.170.136.32
                                                           iii.      hwpayroll-jax           60      IN      A       172.18.242.32
b.       New zone records
                                                               i.      hwpayroll               60      IN      A       10.205.9.148
                                                             ii.      hwpayroll       60      IN      A       10.170.136.32
                                                           iii.      hwpayroll               60      IN      A       10.205.9.148
                                                           iv.      hwpayroll       60      IN      A       10.170.136.32
                                                             v.      hwpayroll               60      IN      A       10.205.9.148


Please advice
Hello All, Do you need more info from my above issues?

Again, I appreciate your help !!
Yes,
1)  you need not only to tell what happens, but also what you expect instead..  appearantly there are problems, which exactly ARE the problem.
2) Do all input records have the TTL,  & IN on each A, CNAME records.
3) what kind of lines to expect what to update & what NOT to update.    (And maybe explain what you want with clear  examples. )
The meaning of what you expected needs to be derived from your original code.
Hello Noci

2) Do all input records have the TTL,  & IN on each A, CNAME records.

YES

3) what kind of lines to expect what to update & what NOT to update.    (And maybe explain what you want with clear  examples. )
The meaning of what you expected needs to be derived from your original code.

A: The source record name was sripped of view name and the ip vips are not being updated.

Original records

hwpayroll               60      IN      172.18.242.32

 hwpayroll-mpl           60      IN      A       10.170.136.32

 hwpayroll-jax           60      IN      A       172.18.242.32

b.       New zone records after code run

hwpayroll               60      IN      10.205.9.148
hwpayroll       60      IN      A       10.170.136.32
                                                         
hwpayroll               60      IN      A       10.205.9.148

hwpayroll       60      IN      A       10.170.136.32
hwpayroll               60      IN      A       10.205.9.148


I hope the yes about helps a bit. Sorry about going back and fourth.
From the Examples above the rules of engagement seem to be:

CNAME records should be excluded from any change... (any other?)
in A records:  the -mpl & -jax Should be removed and not changed into  (-shk/-str like earlier mentioned).....? and the IP address should get updated.
(I have to assume the loss of the A is unintentional).
Hi.

The A records  should not be Lost . Cnames can be updated but  if the name has mpl in both parts of the record then a new record needs to be created with the star of shk

Hope this helps. I’m anxious for your reply :)
Hello. Wanted to see if you need anything else. Again. I am very appreciative with your time on this
Hello Noci.
I wanted to see if you had an updated code I can try this morning.
Hi, I think the issue maybe on this line 56

                                     
  $line =~ s/$old/$ip{$old}/;   

Open in new window


Maybe need to add $newip?
Hi, I also would like to incorporate this to your code  -l filehandle': True if file is a symbolic link

So I want to  add if statement that checks if it is a SYmlink skip else

#!/usr/bin/perl -w

$dir = "c:\\";
opendir(DIR, $dir) or die "Can't open $name due to $!";

@entries = readdir(DIR);

closedir(DIR);

@sorted = sort(@entries);
foreach $entry (@sorted) {
    $name = $dir . '/' . $entry;
    print "$name   ";

    if (-l $name) {
        print "symbolic link";
    } elsif (-d $name) {
        print "directory";
    }  elsif (-p $name) {
        print "FIFO pipe"; 
    }  elsif (-f $name) {
        print "normal file";
    }  else {
        print "unknown file type";
    } 

} 

Open in new window


that way we dont have to make a long list of files to update and skip
Changed: file & directory handling, next is changing the handing of a zone file.

#!/usr/bin/perl

use strict;
use POSIX;

my $debug = 0;

sub handle_zonefile($$$) {
        my ($file, $ipmap, $outfile) = @_;

        print "Handling: $file -> $outfile\n";
        open(ZONE,  $file) || return "Zone file error ($file) $!\n";
        my $content;
        {
                local $/ = undef;
                $content = <ZONE>;
        }
        close ZONE;
        my @fileLines = split(/\n/,$content);


        # Assuming simple CSV of old_ip, new_ip
        my %ip;
        print "Reading ipmap: $ipmap\n" if $debug;
        open(IPMAP, $ipmap) || return "IP Map file error: ($ipmap) $!\n";
        while ( my $line = <IPMAP> ) {
                chomp $line;
                my ($old, $new) = split(/,/, $line);
                $old =~ s/\s+//g;
                $new =~ s/\s+//g;
                $ip{$old} = $new;
        }
        close IPMAP;

        if ($debug) {
                foreach my $xip (keys(%ip)) {
                        print "$xip -> $ip{$xip}\n";
                }
        }

        my $soaseen = 0;
        ## For every line  replace address possibly insert new ones
        foreach my $line (@fileLines) {
                ## For every old address in the remap
                foreach my $old (keys(%ip)) {
                        #print STDERR "INNER FOR ($old)\n";
                        ## If the line matches the old address
                        if ( $line =~ /\s$old(\s|;|$)/ ) {
                                print "Changing: $old\n" if $debug;
                                my ($dnsName, @crap) = split(/\s+/, $line);
                                ## insert a new line with a new record or the new sites
                                ## if this is a site specific record
                                if ( $dnsName =~ m/\-mpl|\-jax/ ) {
                                        $line =~ s/$old/$ip{$old}/;                  # also change th IP address on -mpl & -jax record.
                                        my $newline = $line;
                                        $newline =~ s/\-mpl/\-shk/;               # add new -shk / -str records, first make a copy of the line, later append it.
                                        $newline =~ s/\-jax/\-str/;
                                        push @fileLines, $newline;
                                ## Otherwise, change the address to new
                                } else {
                                        print STDERR "line did not match new-names, replace address: $old\n";
                                        $line =~ s/(\s)$old(\s|$|;)/$1$ip{$old}$2/;
                                }
                        }
                }
                if ($soaseen) {
                        print "Handle case 1\n" if $debug;
                        my $serialhead=strftime('%4Y%02m%02d', gmtime());
                        if ( $line =~ /\s*($serialhead[0-9][0-9])\s*/ ) {
                                my $t = substr($1,8,2);
                                print "Match: $1 $t\n" if $debug;
                                $serialhead= sprintf( "%s%02d", $serialhead, $t+1);
                        } else {
                                $serialhead .= '00';
                        }
                        print "Serial: $serialhead\n" if $debug;
                        $line =~ s/[0-9]+/$serialhead/;
                        print "Update: $line\n" if $debug;
                        $soaseen = 0;
                }
                if ($line =~ /\ssoa\s/i) {
                        print "SOA detected: " if $debug;
                        # 2 cases one: SOA master admin (    with serial on new line
                        #              SOA master admin serial .....   all on one line
                        if ($line =~ /\s\(/) {
                                print "Case 1\n" if $debug;
                                $soaseen = 1; # we need to update next line
                        } else {
                                print "Case 2\n" if $debug;
                                my $serialhead=strftime('%4Y%02m%02d', gmtime());
                                if ( $line =~ /\s*($serialhead[0-9][0-9])\s*/ ) {
                                        my $t = substr($1,8,2);
                                        $serialhead= sprintf( "%s%02d", $serialhead, $t+1);
                                } else {
                                        $serialhead .= '00';
                                }
                                $line =~ s/(\s)[0-9]+(\s)/$1$serialhead$2/;
                        }
                }
        }

        print "Create output\n" if $debug;
        my $out = join("\n", @fileLines);
        open(OUT, ">$outfile") || return "Unable to create new file: ($outfile) $!\n";
        print OUT $out."\n";
        close OUT;
        return "";
}

sub handle_directory($$) {
        my ($dir,$ipmap) = @_;
        opendir(DIR, $dir) or return "Can't open $dir due to $!\n";

        my @entries = readdir(DIR);
        closedir(DIR);

        my $msg = "";

        my @sorted = sort(@entries);
        foreach my $entry (@sorted) {
                if ($entry =~ /^\./) {
                        next;               # ignore hidden files & directories including . & ..
                }
                my $name = $dir . '/' . $entry;
                print "$name   " if $debug;
                
                if (-l $name) {
                    print "symbolic link" if $debug;
                } elsif (-d $name) {
                    print "directory\n";
                    $msg .= handle_directory($name, $ipmap);
                }  elsif (-p $name) {
                    print "FIFO pipe\n" if $debug; 
                }  elsif (-f $name) {
                    $msg .= handle_zonefile($name, $ipmap, $name.'.new');
                }  else {
                    print "unknown file type\n";
                } 
        }
}

my $ipmap = shift(@ARGV);       # first argument is IP map  getit & remove from work todo list
my $msg = "";
foreach my $arg (@ARGV) {
        if ( -d $arg ) {
                $msg .= handle_directory($arg, $ipmap);
        } elsif (-f $arg) {
                $msg .= handle_zonefile($arg, $ipmap, $arg.".new"), ;
        }
        if ( $msg != "" ) {
                print STDERR "Error on $arg: $msg\n";
        }
}
exit(0);

Open in new window

Ok. So the above resolves the symlink question I had. If you could is there a way to add a comment showing that in the code so I can see it?

Thank you again
ASKER CERTIFIED SOLUTION
Avatar of noci
noci

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Awesome.. and the symlink else skips variable?

Sorry. I didn’t see your comment on the code. Looks like you did add it.

Thank you so much and I will test.
Fantastic work. I thank you very much for your help and contribution