Link to home
Start Free TrialLog in
Avatar of Watnog
WatnogFlag for Belgium

asked on

Unix ksh: mass replace of multiple values

Hi Experts,

A tough one, I'm afraid.
I need files to be mass changed.
The input file consists of 4 space/tabe seperated values.
Looks like this:

"Current Name"      "New Name"      "NEW USER"      "NEW ID"
MMDX-DE-D-C0973-AMAPS      MMMM-DE-D-C0973-AMAPS      VNXT3299      3291600
...

The file 'to be changed' has its entries in this format:

PR2-028#MMDX-DE-D-C0973-AMAPS
 SCRIPTNAME "/  -job MMDX-DE-D-C0973-AMAPS -user BG060733 -i 05340301 -c C"
...

After the change it should look like this:

PR2-028#MMMM-DE-D-C0973-AMAPS
 SCRIPTNAME "/  -job MMDX-DE-D-C0973-AMAPS -user VNXT3299 -i 3291600 -c C"
 
What needs to happen is this:
- all instances of MMDX-DE-D-C0973-AMAPS need replaced  by MMMM-DE-D-C0973-AMAPS
- the "-user" parameter needs changed from BG060733 to VNXT3299
- the "-i" parameter needs changed from 05340301 to 3291600

The input file has about 1500 lines.
Hope you can help.
Many thanks in advance.
Avatar of Frank Contrepois
Frank Contrepois
Flag of United Kingdom of Great Britain and Northern Ireland image

sed -e 's/MMDX-DE-D-C0973-AMAPS/MMMM-DE-D-C0973-AMAPS/g' \
       -e 's/BG060733/VNXT3299/g' \
       -e 's/05340301/3291600/g'

Open in new window



Try it first on a copy of the file so you don't mess with the real file
Avatar of Watnog

ASKER

Thanks.
To have the job name replaced with the sed \g would be ok.
A global change on user and jobnumber won't work as the old values are not known.

So when the old jobname is found:
- replace with second value of input line
In the line below that occurence:
- replace -job value with 2nd value of input line
- replace -user value with the 3rd value of the input line
- replace -i value with 4th value of input line

This needs to be repeated for each input line.
I imagine the script could walk these lines:

while read LINE; do
Current=$(echo $LINE | awk '{ print $1 }' )
NewJob=$(echo $LINE | awk '{ print $2 }' )
NewUSer=$(echo $LINE | awk '{ print $3 }' )
NewI=$(echo $LINE | awk '{ print $4 }'  )
[sed to find occurence of $Current and replace with $NewJob]
[sed to go one line lower and change -job value with $NewJob, change -user value with $NewUser, change -i value with $NewI]
done < inputfile


Avatar of Watnog

ASKER

#!/bin/ksh
set -x
while read LINE; do
Current=$(echo $LINE | awk '{ print $1 }' )
NewJob=$(echo $LINE | awk '{ print $2 }' )
NewUSer=$(echo $LINE | awk '{ print $3 }' )
NewI=$(echo $LINE | awk '{ print $4 }'  )
#
sed 's/$Current/$NewJob/' file_to_change
sed '/$NewJob/{n; s/-job .* -user/-job $NewJob -user/}' file_to_change  
sed '/$NewJob/{n; s/-user .* -i/-user $NewUser -i/}' file_to_change
sed '/$NewJob/{n; s/-i .* -c/-i $NewI -c/}' file_to_change                         
done < input

The{n; ...} is meant to look in the line below found pattern.
Yet it get a cannot parse error.

Avatar of Watnog

ASKER

Response is a bit meager... I take the blame.

If you could help me on this I can struggle on...
I get stuck on this loop:

#!/bin/ksh
while read LINE
do
      Current=$(echo $LINE | awk '{ print $1 }' )
      New=$(echo $LINE | awk '{ print $2 }' )
      NewUSer=$(echo $LINE | awk '{ print $3 }' )
      NewI=$(echo $LINE | awk '{ print $4 }'  )
      echo Current=$Current
      echo New=$New
      sed "s/$Current/$New/" file_to_be_changed > changed_file
done < input

Only the first line of the input file is processed.
Any idea?


Avatar of simon3270
If you are using "> changed_file", then every loop will overwrite the output file with a new version, so only the last replacement woudl have been kept.

I think awk might be a better tool here:

#!/bin/ksh

while read curr newv newuser newi
do
  awk 'BEGIN{st=0}
$0 ~ /^[^#]*#'${curr}'/{split($0,aa,"#");print aa[1] "#" "'${newv}'";st=1;next}
st == 1{print " SCRIPTNAME \"/  -job '${newv}' -user '${newuser}' -i '${newi}' -c C\"";st=0;next}
{print}' kshrep.dat > kshrep.tmp
  mv kshrep.tmp kshrep.dat
done < kshrep.in

Open in new window


The quotes here are a bit tricky - probably best to cut and paste, and just change the name of the input file (the kshrep.in) and the file to be changed (here kshrep.dat).  Also make sure that the temporary file (kshrep.tmp) doesn't exist before you run the script.

Note that I have used multiple variable names on the "while read" line - that automatically splits the input line into multiple words, and avoids the need for your echo and awk statements.
Avatar of Watnog

ASKER

Cheers, many thanks, give me some time to check into this.
P:-]
Avatar of Watnog

ASKER

If I run this, the kshrep.dat is emptied of all values:


WORKSTATION#
 SCRIPTNAME "/  -job  -user  -i  -c C"
 STREAMLOGON twsuser
 DESCRIPTION "just a description"
 TASKTYPE STH
 RECOVERY STOP

In the 'real' kshrep.dat not all entries are candidate for replacment.
Maybe I should select out just those, and put them in same order as kshrep.in?

Thanks again.
Avatar of Watnog

ASKER

I could easily check that out myself in fact, but I'm a bit out of time, so forgive me.
I'll be back Monday.
It runs through the awk process once for each input line (so might be a little slow!).  The orde of entries in the input file does not matter, and each time it only updates the entry matching the first field of the input line - all other simply pass through unchanged.

Are all of the input lines the same format?  If one is empty, or is a "heading" line, that will confse things.

Are all of the patterns in the kshrep.dat file the same format?  My code assumes that the text to be matched is on a line immediately after a single "#" character (like the example in the original question).  It then assumes that the "SCRIPTNAME" line is always exactly the same format - the script has the SCRIPTNAME, -job, -i and -c C hardcoded, and prints out the values from the input file to fill in the variable data.

Have a great weekend, and let's talk on Monday.
Avatar of Watnog

ASKER

Ok, after tweaking the input file (kshrep.dat) it works, and that's great.
There are 2 issues, of which the first is less important.

1. Not all "scriptname" lines are identical
Apart from:
      SCRIPTNAME "/  -job MMDX-DE-D-C0973-AMAPS -user BG060733 -i 05340301 -c C"
Some have:
      SCRIPTNAME "/  -job ALE-AP-O-INBND_IDEIL -user US039332 -i 14100000 -c C -nobdcwait -flag DISABLE_JOBLOG"
Or:
      SCRIPTNAME "/  -job PPPU-BE-D-NEUPL-LTP-2-0538 -user EC004251 -i 05063700 -c C -nobdcwait"

Those extra parameters always come at the end (after the 'C') and don't need to be changed.
If it would be possible to count this possibilty in, that would be great, else I can treat them seperately changing the:
      st == 1{print " SCRIPTNAME \"/  -job '${newv}' -user '${newuser}' -i '${newi}' -c C\"";st=0;next}
line accordingly. So in fact that is no big deal.

2. This one is a show stopper (I think).
The full job definition has other lines uder "scriptname", as below.
I left those out to keep it simple... :-~

      PR2-028#PPPU-BE-D-PROD-ORDER-AVCHK-0811
       SCRIPTNAME "/  -job PPPU-BE-D-PROD-ORDER-AVCHK-0811 -user CAN01351 -i 16433201 -c C"
       STREAMLOGON maestro
       DESCRIPTION "Mass Availlability Check P0811"
       TASKTYPE SAP
       RECOVERY STOP

I cannot stripe those off on the 'real' file, so this should be taken into account.
The format is always the same:
      WORKSTATION#JOBNAME           # line to change
      SCRIPTNAME                    # line to change
      STREAMLOGON                                #  no change      
      DESCRIPTION                                #  no change
      TASKTYPE                                #  no change
      RECOVERY                              #  no change      

Thanks!      
ASKER CERTIFIED SOLUTION
Avatar of simon3270
simon3270
Flag of United Kingdom of Great Britain and Northern Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of Watnog

ASKER

Cheers. Yes that does work.

Can the JOBNAME be made an exact match?
MMMM-JP-D-ADR409-PBOM_EXTRACT
finds/replaces also
MMMM-JP-D-ADR409-PBOM_EXTRACT1
MMMM-JP-D-ADR409-PBOM_EXTRACT2

Thx.
Avatar of Watnog

ASKER

Thank  you simon3270.
I'm very much helped with  your solution.
Gratefully.
Yes, it can be an exact match:

Change

    $0 ~ /^[^#]*#'${curr}'/{split($0,aa,"#")

to

    $0 ~ /^[^#]*#'${curr}'$/{split($0,aa,"#")

(note the extra $ just before the /{)