Watnog
asked on
Unix ksh: mass replace of multiple values
Hi Experts,
A tough one, I'm afraid.
I need files to be mass changed.
The input file consists of 4 space/tabe seperated values.
Looks like this:
"Current Name" "New Name" "NEW USER" "NEW ID"
MMDX-DE-D-C0973-AMAPS MMMM-DE-D-C0973-AMAPS VNXT3299 3291600
...
The file 'to be changed' has its entries in this format:
PR2-028#MMDX-DE-D-C0973-AM APS
SCRIPTNAME "/ -job MMDX-DE-D-C0973-AMAPS -user BG060733 -i 05340301 -c C"
...
After the change it should look like this:
PR2-028#MMMM-DE-D-C0973-AM APS
SCRIPTNAME "/ -job MMDX-DE-D-C0973-AMAPS -user VNXT3299 -i 3291600 -c C"
What needs to happen is this:
- all instances of MMDX-DE-D-C0973-AMAPS need replaced by MMMM-DE-D-C0973-AMAPS
- the "-user" parameter needs changed from BG060733 to VNXT3299
- the "-i" parameter needs changed from 05340301 to 3291600
The input file has about 1500 lines.
Hope you can help.
Many thanks in advance.
A tough one, I'm afraid.
I need files to be mass changed.
The input file consists of 4 space/tabe seperated values.
Looks like this:
"Current Name" "New Name" "NEW USER" "NEW ID"
MMDX-DE-D-C0973-AMAPS MMMM-DE-D-C0973-AMAPS VNXT3299 3291600
...
The file 'to be changed' has its entries in this format:
PR2-028#MMDX-DE-D-C0973-AM
SCRIPTNAME "/ -job MMDX-DE-D-C0973-AMAPS -user BG060733 -i 05340301 -c C"
...
After the change it should look like this:
PR2-028#MMMM-DE-D-C0973-AM
SCRIPTNAME "/ -job MMDX-DE-D-C0973-AMAPS -user VNXT3299 -i 3291600 -c C"
What needs to happen is this:
- all instances of MMDX-DE-D-C0973-AMAPS need replaced by MMMM-DE-D-C0973-AMAPS
- the "-user" parameter needs changed from BG060733 to VNXT3299
- the "-i" parameter needs changed from 05340301 to 3291600
The input file has about 1500 lines.
Hope you can help.
Many thanks in advance.
ASKER
Thanks.
To have the job name replaced with the sed \g would be ok.
A global change on user and jobnumber won't work as the old values are not known.
So when the old jobname is found:
- replace with second value of input line
In the line below that occurence:
- replace -job value with 2nd value of input line
- replace -user value with the 3rd value of the input line
- replace -i value with 4th value of input line
This needs to be repeated for each input line.
I imagine the script could walk these lines:
while read LINE; do
Current=$(echo $LINE | awk '{ print $1 }' )
NewJob=$(echo $LINE | awk '{ print $2 }' )
NewUSer=$(echo $LINE | awk '{ print $3 }' )
NewI=$(echo $LINE | awk '{ print $4 }' )
[sed to find occurence of $Current and replace with $NewJob]
[sed to go one line lower and change -job value with $NewJob, change -user value with $NewUser, change -i value with $NewI]
done < inputfile
To have the job name replaced with the sed \g would be ok.
A global change on user and jobnumber won't work as the old values are not known.
So when the old jobname is found:
- replace with second value of input line
In the line below that occurence:
- replace -job value with 2nd value of input line
- replace -user value with the 3rd value of the input line
- replace -i value with 4th value of input line
This needs to be repeated for each input line.
I imagine the script could walk these lines:
while read LINE; do
Current=$(echo $LINE | awk '{ print $1 }' )
NewJob=$(echo $LINE | awk '{ print $2 }' )
NewUSer=$(echo $LINE | awk '{ print $3 }' )
NewI=$(echo $LINE | awk '{ print $4 }' )
[sed to find occurence of $Current and replace with $NewJob]
[sed to go one line lower and change -job value with $NewJob, change -user value with $NewUser, change -i value with $NewI]
done < inputfile
ASKER
#!/bin/ksh
set -x
while read LINE; do
Current=$(echo $LINE | awk '{ print $1 }' )
NewJob=$(echo $LINE | awk '{ print $2 }' )
NewUSer=$(echo $LINE | awk '{ print $3 }' )
NewI=$(echo $LINE | awk '{ print $4 }' )
#
sed 's/$Current/$NewJob/' file_to_change
sed '/$NewJob/{n; s/-job .* -user/-job $NewJob -user/}' file_to_change
sed '/$NewJob/{n; s/-user .* -i/-user $NewUser -i/}' file_to_change
sed '/$NewJob/{n; s/-i .* -c/-i $NewI -c/}' file_to_change
done < input
The{n; ...} is meant to look in the line below found pattern.
Yet it get a cannot parse error.
set -x
while read LINE; do
Current=$(echo $LINE | awk '{ print $1 }' )
NewJob=$(echo $LINE | awk '{ print $2 }' )
NewUSer=$(echo $LINE | awk '{ print $3 }' )
NewI=$(echo $LINE | awk '{ print $4 }' )
#
sed 's/$Current/$NewJob/' file_to_change
sed '/$NewJob/{n; s/-job .* -user/-job $NewJob -user/}' file_to_change
sed '/$NewJob/{n; s/-user .* -i/-user $NewUser -i/}' file_to_change
sed '/$NewJob/{n; s/-i .* -c/-i $NewI -c/}' file_to_change
done < input
The{n; ...} is meant to look in the line below found pattern.
Yet it get a cannot parse error.
ASKER
Response is a bit meager... I take the blame.
If you could help me on this I can struggle on...
I get stuck on this loop:
#!/bin/ksh
while read LINE
do
Current=$(echo $LINE | awk '{ print $1 }' )
New=$(echo $LINE | awk '{ print $2 }' )
NewUSer=$(echo $LINE | awk '{ print $3 }' )
NewI=$(echo $LINE | awk '{ print $4 }' )
echo Current=$Current
echo New=$New
sed "s/$Current/$New/" file_to_be_changed > changed_file
done < input
Only the first line of the input file is processed.
Any idea?
If you could help me on this I can struggle on...
I get stuck on this loop:
#!/bin/ksh
while read LINE
do
Current=$(echo $LINE | awk '{ print $1 }' )
New=$(echo $LINE | awk '{ print $2 }' )
NewUSer=$(echo $LINE | awk '{ print $3 }' )
NewI=$(echo $LINE | awk '{ print $4 }' )
echo Current=$Current
echo New=$New
sed "s/$Current/$New/" file_to_be_changed > changed_file
done < input
Only the first line of the input file is processed.
Any idea?
If you are using "> changed_file", then every loop will overwrite the output file with a new version, so only the last replacement woudl have been kept.
I think awk might be a better tool here:
The quotes here are a bit tricky - probably best to cut and paste, and just change the name of the input file (the kshrep.in) and the file to be changed (here kshrep.dat). Also make sure that the temporary file (kshrep.tmp) doesn't exist before you run the script.
Note that I have used multiple variable names on the "while read" line - that automatically splits the input line into multiple words, and avoids the need for your echo and awk statements.
I think awk might be a better tool here:
#!/bin/ksh
while read curr newv newuser newi
do
awk 'BEGIN{st=0}
$0 ~ /^[^#]*#'${curr}'/{split($0,aa,"#");print aa[1] "#" "'${newv}'";st=1;next}
st == 1{print " SCRIPTNAME \"/ -job '${newv}' -user '${newuser}' -i '${newi}' -c C\"";st=0;next}
{print}' kshrep.dat > kshrep.tmp
mv kshrep.tmp kshrep.dat
done < kshrep.in
The quotes here are a bit tricky - probably best to cut and paste, and just change the name of the input file (the kshrep.in) and the file to be changed (here kshrep.dat). Also make sure that the temporary file (kshrep.tmp) doesn't exist before you run the script.
Note that I have used multiple variable names on the "while read" line - that automatically splits the input line into multiple words, and avoids the need for your echo and awk statements.
ASKER
Cheers, many thanks, give me some time to check into this.
P:-]
P:-]
ASKER
If I run this, the kshrep.dat is emptied of all values:
WORKSTATION#
SCRIPTNAME "/ -job -user -i -c C"
STREAMLOGON twsuser
DESCRIPTION "just a description"
TASKTYPE STH
RECOVERY STOP
In the 'real' kshrep.dat not all entries are candidate for replacment.
Maybe I should select out just those, and put them in same order as kshrep.in?
Thanks again.
WORKSTATION#
SCRIPTNAME "/ -job -user -i -c C"
STREAMLOGON twsuser
DESCRIPTION "just a description"
TASKTYPE STH
RECOVERY STOP
In the 'real' kshrep.dat not all entries are candidate for replacment.
Maybe I should select out just those, and put them in same order as kshrep.in?
Thanks again.
ASKER
I could easily check that out myself in fact, but I'm a bit out of time, so forgive me.
I'll be back Monday.
I'll be back Monday.
It runs through the awk process once for each input line (so might be a little slow!). The orde of entries in the input file does not matter, and each time it only updates the entry matching the first field of the input line - all other simply pass through unchanged.
Are all of the input lines the same format? If one is empty, or is a "heading" line, that will confse things.
Are all of the patterns in the kshrep.dat file the same format? My code assumes that the text to be matched is on a line immediately after a single "#" character (like the example in the original question). It then assumes that the "SCRIPTNAME" line is always exactly the same format - the script has the SCRIPTNAME, -job, -i and -c C hardcoded, and prints out the values from the input file to fill in the variable data.
Have a great weekend, and let's talk on Monday.
Are all of the input lines the same format? If one is empty, or is a "heading" line, that will confse things.
Are all of the patterns in the kshrep.dat file the same format? My code assumes that the text to be matched is on a line immediately after a single "#" character (like the example in the original question). It then assumes that the "SCRIPTNAME" line is always exactly the same format - the script has the SCRIPTNAME, -job, -i and -c C hardcoded, and prints out the values from the input file to fill in the variable data.
Have a great weekend, and let's talk on Monday.
ASKER
Ok, after tweaking the input file (kshrep.dat) it works, and that's great.
There are 2 issues, of which the first is less important.
1. Not all "scriptname" lines are identical
Apart from:
SCRIPTNAME "/ -job MMDX-DE-D-C0973-AMAPS -user BG060733 -i 05340301 -c C"
Some have:
SCRIPTNAME "/ -job ALE-AP-O-INBND_IDEIL -user US039332 -i 14100000 -c C -nobdcwait -flag DISABLE_JOBLOG"
Or:
SCRIPTNAME "/ -job PPPU-BE-D-NEUPL-LTP-2-0538 -user EC004251 -i 05063700 -c C -nobdcwait"
Those extra parameters always come at the end (after the 'C') and don't need to be changed.
If it would be possible to count this possibilty in, that would be great, else I can treat them seperately changing the:
st == 1{print " SCRIPTNAME \"/ -job '${newv}' -user '${newuser}' -i '${newi}' -c C\"";st=0;next}
line accordingly. So in fact that is no big deal.
2. This one is a show stopper (I think).
The full job definition has other lines uder "scriptname", as below.
I left those out to keep it simple... :-~
PR2-028#PPPU-BE-D-PROD-ORD ER-AVCHK-0 811
SCRIPTNAME "/ -job PPPU-BE-D-PROD-ORDER-AVCHK -0811 -user CAN01351 -i 16433201 -c C"
STREAMLOGON maestro
DESCRIPTION "Mass Availlability Check P0811"
TASKTYPE SAP
RECOVERY STOP
I cannot stripe those off on the 'real' file, so this should be taken into account.
The format is always the same:
WORKSTATION#JOBNAME # line to change
SCRIPTNAME # line to change
STREAMLOGON # no change
DESCRIPTION # no change
TASKTYPE # no change
RECOVERY # no change
Thanks!
There are 2 issues, of which the first is less important.
1. Not all "scriptname" lines are identical
Apart from:
SCRIPTNAME "/ -job MMDX-DE-D-C0973-AMAPS -user BG060733 -i 05340301 -c C"
Some have:
SCRIPTNAME "/ -job ALE-AP-O-INBND_IDEIL -user US039332 -i 14100000 -c C -nobdcwait -flag DISABLE_JOBLOG"
Or:
SCRIPTNAME "/ -job PPPU-BE-D-NEUPL-LTP-2-0538
Those extra parameters always come at the end (after the 'C') and don't need to be changed.
If it would be possible to count this possibilty in, that would be great, else I can treat them seperately changing the:
st == 1{print " SCRIPTNAME \"/ -job '${newv}' -user '${newuser}' -i '${newi}' -c C\"";st=0;next}
line accordingly. So in fact that is no big deal.
2. This one is a show stopper (I think).
The full job definition has other lines uder "scriptname", as below.
I left those out to keep it simple... :-~
PR2-028#PPPU-BE-D-PROD-ORD
SCRIPTNAME "/ -job PPPU-BE-D-PROD-ORDER-AVCHK
STREAMLOGON maestro
DESCRIPTION "Mass Availlability Check P0811"
TASKTYPE SAP
RECOVERY STOP
I cannot stripe those off on the 'real' file, so this should be taken into account.
The format is always the same:
WORKSTATION#JOBNAME # line to change
SCRIPTNAME # line to change
STREAMLOGON # no change
DESCRIPTION # no change
TASKTYPE # no change
RECOVERY # no change
Thanks!
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Cheers. Yes that does work.
Can the JOBNAME be made an exact match?
MMMM-JP-D-ADR409-PBOM_EXTR ACT
finds/replaces also
MMMM-JP-D-ADR409-PBOM_EXTR ACT1
MMMM-JP-D-ADR409-PBOM_EXTR ACT2
Thx.
Can the JOBNAME be made an exact match?
MMMM-JP-D-ADR409-PBOM_EXTR
finds/replaces also
MMMM-JP-D-ADR409-PBOM_EXTR
MMMM-JP-D-ADR409-PBOM_EXTR
Thx.
ASKER
Thank you simon3270.
I'm very much helped with your solution.
Gratefully.
I'm very much helped with your solution.
Gratefully.
Yes, it can be an exact match:
Change
$0 ~ /^[^#]*#'${curr}'/{split($ 0,aa,"#")
to
$0 ~ /^[^#]*#'${curr}'$/{split( $0,aa,"#")
(note the extra $ just before the /{)
Change
$0 ~ /^[^#]*#'${curr}'/{split($
to
$0 ~ /^[^#]*#'${curr}'$/{split(
(note the extra $ just before the /{)
Open in new window
Try it first on a copy of the file so you don't mess with the real file