Solved

Modify a text file from the output of another command

Posted on 2004-08-17
18
280 Views
Last Modified: 2013-12-27
I have a directory that is full of files which start with the word db (db.x.x.x).  I have been asked to rename the files x.x.x.db, which is fine (I have that covered).

for i in `ls db.*`
do
mv $i `echo $i | awk '{FS="."; print $2"."$3"."$4"."$1}'`
done

However, when I rename these files, it screws up a config file.  The config file has 2 entries for each of the file names (kind of like a call for them of source).  I need to be able to modify this config file to compensate for the name changes.  I thought about using the output of the above for-do loop to get the new file names and somehow use sed to modify the file, but I can’t get the logic to work properly.  Does anyone have any ideas?

Just in case your curious, this is db zone files for DNS and the config file I’m referring to is /etc/named.conf.  The 2 entries lead with the words “zone” and “file” respectfully.  However, none of this really matter much, because the concept behind the question really has nothing to do with DNS.  I’m just looking for a way to automate the modification of a ascii text file from the output of a file list.
0
Comment
Question by:teckwiz01
  • 8
  • 7
18 Comments
 
LVL 20

Expert Comment

by:tfewster
Comment Utility
So what's wrong with:

cp /etc/named.conf /etc/named.bak
for i in `ls db.*`
do
inew=`echo $i | awk '{FS="."; print $2"."$3"."$4"."$1}'`
  mv $i $inew
  # Now modify /etc/named.conf
  sed -e "s/$i/$inew/" /etc/named.conf > /etc/named.tmp && cp /etc/named.tmp /etc/named.conf
done
0
 
LVL 20

Expert Comment

by:tfewster
Comment Utility
Oops, the "cp" after the "&&" should be on the same line - Or put a "\" at the end of the line with the "&&"
0
 

Author Comment

by:teckwiz01
Comment Utility
Nothing, I guess..  I'll try that an let you know how it works out.  

I had something else that I kind of borrowed from another script that I did with a developer, but I keeped screwing up the logic on it and the resulting file was blank.

I posted it and no one responded in 2 days, so I kind of re-wrote the question and deleted the old one.  It's a good thing I did cause I got an answer from you.  

Just in case your curious, this was the other script.  It was orginally created to change the NS (name server) fields (and modified for the SOA fields also.  I commented out the stuff I didn't think I needed and tryed to write it to handle this, but that's when I keeped screwing up the logic.  

Here it is.  If you have time, take a look and tell me where I went wrong.  I'm just curious why I couldn't get it to work.

##from NS script, ignore##   echo "Enter Old Name Server: \c"
##from NS script, ignore##   read old_name
##from NS script, ignore##   echo "Enter New Name Server: \c"
##from NS script, ignore##   read new_name
##from NS script, ignore##   echo "Change Name Server: $old_name to $new_name? \c"
echo "Change Name Server Config? \c"
read answer
case $answer in
        yes|Yes|YES|Y|y)
        today=`date | awk '{print $2 $3 $6 "_" $4}'| sed 's/://g'`
##from NS script, ignore##   cd /etc/named
##from NS script, ignore##   tar cvf /etc/named_backup/named_$today.tar *
        cd /etc
        cp named.conf named.conf_$today
        if [ $# -eq 0 ]
        then
                set -- db.*
        fi
##from NS script, ignore##    tmp_db=${TMPDIR:-/var/tmp}/tmpdb.$$
        tmp_db=/tmp/namestuff.txt
        perm_db=/etc/named.conf
        cd /extra/backup/named
##from NS script, ignore##   for db in `find . -depth -print`
        for i in `ls db.*`
        do
                old_name=`echo $i`
                echo $old_name
                new_name=`echo $i | awk '{FS="."; print $2"."$3"."$4"."$1}'`
                echo $new_name
##from NS script, ignore##    if grep "NS[    ][      ]*$old_name\." $db > /dev/null
                        if grep "zone \"$old_name" $perm_db > /dev/null
                        then
                                sed "s\(zone \"$old_name\(zone \"1$new_name" $perm_db > $tmp_db
##from NS script, ignore##       sed "s\(NS[     ][      ]*\)$old_name\.\1$new_name." $db > $tmp_db
#from NS script, ignore##                                mv $tmp_db $perm_db
                               mv $tmp_db $db
#                        fi
                        done
                ;;
        *)
                echo Verify information and try again...
                ;;
esac
 

Don't say it!!!  I know the comment lines are annoying.  You can simply copy the script and delete everything that leads with "##from NS script, ignore##".  Inversely, if you un-comment out those lines and delete the ones I added which are obvious replacement for them, you have the script that I use to change the name servers (NS entries - also SOA by replacing NS with SOA) in the zone files.  I left them there just in case someone needs to see what the original script did to figure out what I'm doing wrong in modifying it.   Please help!!  Thanks…


0
 

Author Comment

by:teckwiz01
Comment Utility
What do you mean?

Oops, the "cp" after the "&&" should be on the same line - Or put a "\" at the end of the line with the "&&"
0
 

Author Comment

by:teckwiz01
Comment Utility
I ask because in the example, the cp is on the same line...
 sed -e "s/$i/$inew/" /etc/named.conf > /etc/named.tmp && cp /etc/named.tmp /etc/named.conf
done
0
 
LVL 20

Expert Comment

by:tfewster
Comment Utility
Double Oops - Forget that, when I posted it, it wrapped round so the "cp" was on a seperate line - But it looks fine now?!

I looked at your previous question and couldn't figure out what it was supposed to do!  But now you've restated the problem, I think I get it...

I think "mv $tmp_db $db"  should be "mv $tmp_db $perm_db";  $db is undefined, so you're not saving the changes that the sed command made back to /etc/named.conf.

However, I'm still not sure about the format of your sed command - I've never had to look after a DNS server, so I don't know what the format of a zone record should be; If you want to post a line (with sensitive info XX'd out, naturally) I can take another look at that.
0
 
LVL 48

Expert Comment

by:Tintin
Comment Utility
Not a specific answer to your question, but I would have written:

for i in `ls db.*`
do
mv $i `echo $i | awk '{FS="."; print $2"."$3"."$4"."$1}'`
done

as

for i in db.*
do
  mv $i `echo $i | sed 's/db.\(.*\)/\1.db/'`
done
0
 

Author Comment

by:teckwiz01
Comment Utility
Oh, darn...  I just noticed that... I guess that would explain why the mods were showing up in the files..  Dahhh!!   Thanks for noticing that..

The format is largely insignificant for this task.   I'm only trying to automate a manual process.  There are more things required in the project, but I think that as long as I'm only making changes to the appropriate entries/fields and not the structure, everything should work well.

One thing,  I tried the script you wrote and some of the result where a little strange.
The first part of the script (rename) works fine..
db.10.1.1      to       10.1.1.db
db.10.1.2      to       10.1.2.db
db.10.1.3      to       10.1.3.db
db.10.1.4      to       10.1.4.db
db.10.1.5      to       10.1.5.db
db.10.1.7      to       10.1.7.db

But when it edit the named.conf files, something goes wrong..
Take a look at the output..  In some cases, there is a number at the end (x.x.x.db1).
Actually it looks like it has a problem with the last field double digit number

From:
------

zone "db.10.1.11" {
        type master;
        file "/etc/named/db.10.1.11";
};
 
zone "db.10.1.12" {
        type master;
        file "/etc/named/db.10.1.12";

To:
----
zone "10.1.1.db1" {
        type master;
        file "/etc/named/10.1.1.db1";
};
 
zone "10.1.1.db2" {
        type master;
        file "/etc/named/10.1.1.db2";

What went wrong?

0
Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

 

Author Comment

by:teckwiz01
Comment Utility
Actually, I think I figured out what happens.  When the script run into db.10.1.1, for example, and it searches for that in /etc/named.conf, it finds not just that entry, but also db.10.1.12, db.10.1.13, db.10.1.14, etc...  I think it then changes them and ignore the last digit, hence I get 10.1.1.db1, 10.1.1.db2, 10.1.1.db3, etc..  If I'm right about this, does anyone know how to overcome it?

I'm going to try what you recommended Tintin, but for the rename I think it does the same thing.  If I'm not mistaken, you're using the save in - put it buffer structure in sed (ie: the \1.db/ in sed 's/db.\(.*\)/\1.db/').  I kind of was using something similar in the other script but for the output of the modification of the config file (sed "s\(zone \"$old_name\(zone \"1$new_name" $perm_db > $tmp_db)..
0
 
LVL 20

Expert Comment

by:tfewster
Comment Utility
stupid, stupid tfewster...

sed -e "s/$i\"/$inew\"/" /etc/named.conf   # ensures the pattern is complete, i.e. there is a " at the end

Sorry about that
0
 

Author Comment

by:teckwiz01
Comment Utility
Ok.. That all seems to work.  I only have one more thing on this specific process.  

I've been asked to change the structure of the output to go from db.10.1.1 to 1.1.10.db, instead of 10.1.1.db; changing [mv $i `echo $i | awk '{FS="."; print $2"."$3"."$4"."$1}'`] to [mv $i `echo $i | awk '{FS="."; print $4"."$3"."$2"."$1}'`].  That doesn't seem to be a big deal.  Except there is one scenario that I have not mentioned because I had it covered until now.

Lets say one of the files has only 3 fields instead of 4 (db.192.168).  With the original structure of the script you (tfewster) recommended, this files get's converted to 192.168..db and I added the following to the script to make up for that scenario.

for i in `ls *..db`
do
fixdot=`echo $i | sed "s/\.\./\./g"`
mv $i $new
sed -e "s/$i\"/$fixdot\"/" /etc/named.conf > /etc/named.tmp && cp /etc/named.tmp /etc/named.conf
done

That worked fine because it took the extra dot out from both the renamed zone file and from the config file /etc/named.conf as well.  But now since I have to change the structure of the output, I'm not totally sure how to clean that up.  The file name now becomes .192.168.db (making it a hidden file) as well as the entry in the config file.  I could just simply manually change that since it is only one file, but I'm being give 10 more that have the same structure.  Any suggestions as to how I can clean that up?
0
 
LVL 20

Expert Comment

by:tfewster
Comment Utility
inew=`echo $i | awk 'BEGIN {FS="."} {for (c=NF;c>=2;c--) {printf $c "."}} END {printf $1}'`   # Handles any number of fields in the new format

My brain hurts, but I think this means you can use (my) original script without having to do any tidyup at the end:

cp /etc/named.conf /etc/named.bak
for i in `ls db.*`
do
  inew=`echo $i | awk 'BEGIN {FS="."} {for (c=NF;c>=2;c--) {printf $c "."}} END {printf $1}'`
  mv $i $inew
  # Now modify /etc/named.conf
  sed -e "s/$i\"/$inew\"/" /etc/named.conf > /etc/named.tmp && cp /etc/named.tmp /etc/named.conf
done
0
 

Author Comment

by:teckwiz01
Comment Utility
Sorry..  Hope I'm not the reason.  I wrote it over as you have it above got everything except the end .db in the output.  So now all the outputs are comming out like so:

1.1.10.
2.1.10.
3.1.10.
168.192.

Instead of:

1.1.10.db
2.1.10.db
3.1.10.db
168.192.db

etc.., etc.., etc..

At least I'm not getting the hidden files anymore..  
I'll try something, but if you come up with it first, post it for me please.
Thanks again.. Sorry this is getting so complexed...
0
 
LVL 20

Accepted Solution

by:
tfewster earned 100 total points
Comment Utility
It may be different versions of awk - Try:
inew=`echo $i | awk 'BEGIN {FS="."} {for (c=NF;c>=2;c--) {printf $c "."};print $1}'`

[Not sure if it needs to be "print" or "printf" for the last field, ($1)]
Or, closer to what you had before:
inew=`echo $i | awk '{FS="."; for (c=NF;c>=2;c--) {printf $c "."};print $1}'`
0
 

Author Comment

by:teckwiz01
Comment Utility
We've got a winner!!  

inew=`echo $i | awk 'BEGIN {FS="."} {for (c=NF;c>=2;c--) {printf $c "."};print $1}'`

It looks like it's working 100% now.  I can move on the next part and hopefully get it without needing help.  

Again, thank you very much for everything!!!  
0
 
LVL 20

Expert Comment

by:tfewster
Comment Utility
Cool - But now comes the obligatory nagging to close your older questions by accepting answers to them ;-)

Please see http://www.experts-exchange.com/help.jsp#hs5  and  http://www.experts-exchange.com/help.jsp#hi107  for more info.

You can see your open questions at http://www.experts-exchange.com/QH_2316154.html

0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

When you do backups in the Solaris Operating System, the file system must be inactive. Otherwise, the output may be inconsistent. A file system is inactive when it's unmounted or it's write-locked by the operating system. Although the fssnap utility…
I have been running these systems for a few years now and I am just very happy with them.   I just wanted to share the manual that I have created for upgrades and other things.  Oooh yes! FreeBSD makes me happy (as a server), no maintenance and I al…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now