Bash: Replace strings in one file with strings in another file

Hi everybody,
I have a textfile with the following structure:

//FileName1:123
//FileName2:431
SomeText

//FileName1:124
//FileName2:432
SomeOtherText

The first lines with in slashes is the name of the file to process,
the number after the colon is the number of the line in the file to
process.

All I want to do now is, to read the textfile above and replace the
string in the mentioned Line of the mentioned File with the text
below the lines beginning with the slashes. A script should read
the first FileName:FileLine combination, jump to the line in the
named file and replace this line. After that, it should process all
other FileName:FileLine lines before it jumps to the next block of
LineLocations and text to replace. (I have just tried it out, with two
"cat"s, but I was frustraded when I found out, that I can't pass
two parameters from one cat to another because of the process
barrier. What is the correct solution for this problem ?

Greetz,

Patrick
LVL 5
elticAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

ozoCommented:
while( <> ){
   if( m#//(.*?):(\d+)# ){
       push @f,\$f{$1}{$2};
   }elsif( @f ){
       push @t,$_;
       $$_ = \$t[-1] for splice @f;
   }
}
{local @ARGV=keys %f; local $^I='';
 while( <> ){
     $_ = ${$f{$ARGV}{$.}} if $f{$ARGV}{$.};
     print;
 }continue{ close ARGV if eof }
}
elticAuthor Commented:
Seems to be difficult.
Could you insert some comments, please ?
I'm not as fit as I shloud in bash programing...
ozoCommented:
#!/usr/bin/perl
#invoke the Perl interpreter
while( <> ){                     #read each line of the input file
   if( m#^//(.*):(\d+)# ){   #when the line starts with // and contains : followed by a number
       push @f,\$f{$1}{$2};  #remember the name between the // and the : and the number after the :
   }elsif( @f ){                    #when we see the first line after the // lines
       push @t,$_;                #remember the text
       $$_ = \$t[-1] for splice @f;  #associate the text with each of the saved name:number valies
   }
}
#now we're done reading the input file
{local @ARGV=keys %f;  #set the list of saved names as the new list of files to read
 local $^I='';    #set flag to edit those files in place
 while( <> ){  #read each line of the list of files
     $_ = ${$f{$ARGV}{$.}} #set this line to the saved text
             if $f{$ARGV}{$.}; #if we have saved a text line for this line number of this file
     print;                          #print the line back to the file
 }continue{ close ARGV if eof }  #reset the line number counter when we reach the end of each file
}
OWASP Proactive Controls

Learn the most important control and control categories that every architect and developer should include in their projects.

elticAuthor Commented:
I'm not fit at perl at all, so I'm not sure if this would solve my
problem, so one more thing (something like sed in perl). My input
file has the following structure:

//FileName1:123
//FileName2:431
#define STR_SomeString = "SomeString";

I just don't want to replace the whole line in my target file,
but replace "SomeString" with STR_SomeString. So, every
line without the slashes has the structure

#define ReplacementText = "TextToReplaceWithBrackets";

By the way, I need to replace the quotation marks, also.
How should I modify my script and how to call it ? (How
does the script knows which file should be used as input file ?)

Patrick
ozoCommented:
calling it from the command line from bash

perl -ne 'if( m#^//(.*):(\d+)# ){ push @f,\$f{$1}{$2}
             }elsif( /#define\s+(.*?)\s*=\s*(.*?);?$/ ){  
             push @t,eval"sub{s/\Q$2\E/$1/g}"; $$_ = $t[-1] for splice @f; }
             END{ local @ARGV=keys %f; local $^I="";
             while( <> ){ &{$f{$ARGV}{$.}} if $f{$ARGV}{$.}; print; }continue{ close ARGV if eof }
             }' inputfile

I'm assuming the "" are part of what you want to replace but the ; is not, so that if line 123 of FileName1 was
#define STR_SomeString = "SomeString";
then after the replacement it would be
#define STR_SomeString = STR_SomeString;
elticAuthor Commented:
Tried it out, but it seems, that it wouldn't work. Here ist my
test data:

text.h:

// test.c:6
#define STR_Text = "Text";

// test.c:7
// test.c:8
#define STR_Text2 = "Text2"

test.c:
Line6: string teststring            = "Text";
Line7: string AnotherText        = "Text2";
Line8: string QuiteAnotherText = "Text2";


Result should be:
test.c:
Line6: string teststring            = STR_Text;
Line7: string AnotherText        = STR_Text2;
Line8: string QuiteAnotherText = STR_Text2;

When I call your script like you wrote above, the perl
interperter waits for input on stdin, so I send the input
file with "<<" to your script. What is wrong with that ?
ozoCommented:
not <<
either

perl -ne 'if( m#^//(.*):(\d+)# ){ push @f,\$f{$1}{$2}
             }elsif( /#define\s+(.*?)\s*=\s*(.*?);?$/ ){  
             push @t,eval"sub{s/\Q$2\E/$1/g}"; $$_ = $t[-1] for splice @f; }
             END{ local @ARGV=keys %f; local $^I="";
             while( <> ){ &{$f{$ARGV}{$.}} if $f{$ARGV}{$.}; print; }continue{ close ARGV if eof }
             }' text.h
or
perl -ne 'if( m#^//(.*):(\d+)# ){ push @f,\$f{$1}{$2}
             }elsif( /#define\s+(.*?)\s*=\s*(.*?);?$/ ){  
             push @t,eval"sub{s/\Q$2\E/$1/g}"; $$_ = $t[-1] for splice @f; }
             END{ local @ARGV=keys %f; local $^I="";
             while( <> ){ &{$f{$ARGV}{$.}} if $f{$ARGV}{$.}; print; }continue{ close ARGV if eof }
             }' < text.h
ozoCommented:
If it finds no lines beginning with // in text.h, it will wait for input on stdin in the edit file stage, since there will be no files to edit.  We could make it print a warning in that case and stop instead

Also, I see that text.h has added a space after the //, which is probably not part of the file name, so
m#^//(.*):(\d+)#
might be better written as
m#^//\s*(.*):(\d+)#
ozoCommented:
and if there can be spaces to the left of the //, you can use
m#^\s*//\s*(.+):(\d+)#
elticAuthor Commented:
One last thing, you got errornous data from me...
The define has no = in it. and no semicolon at the end.
The correct define Statement should be:

#define STR_Text2 "Text2"

What changes have to be made to the regex ?
ozoCommented:
Assuming STR_Text2 will never contain a space
   }elsif( /#define\s+(.*?)\s*=\s*(.*?);?$/ ){
can be replaced by
   }elsif( /#define\s+(.*?)\s+(.*?)$/ ){  

If you can guarantee the quotes, it may be better to use
   }elsif( /#define\s+(.*?)\s+(".*")/ ){
which also ignores extra spaces after the final ", in case there are any in the input  

the ;? makes the ; optional so it wouldn't have hurt to leave that in, but it would add confusion, so I took it out.
elticAuthor Commented:
Works great, nearly, but doesn't work, when there is at least
more than one string per line:

// test.c:6
#define STR_Text = "Text";

// test.c:6
// test.c:7
// test.c:8
#define STR_Text2 = "Text2"

// test.c:6
#define STR_SomeString = "SomeString";
ozoCommented:
Sorry, I didn't think of that.  This should handle that case

perl -ne 'if( m#^\s*//\s*(.+):(\d+)# ){ push @f,\($f{$1}{$2}||=[])
             }elsif( my($s,$r)=/#define\s+(\w+)\s*=?\s*(.*?)\s*$/ ){  
             my $t=sub{s/\Q$r\E/$s/g}; push@$$_,$t for splice @f; }
             END{ local @ARGV=keys %f; local $^I="";
             while( <> ){ for$r(@{$f{$ARGV}{$.}||[]}){&$r} print; }continue{ close ARGV if eof }
             }' < text.h

I also made some minor efficiency modifications, and made the '=' optional, assuming that the '=' should not be part of the text to be replaced, and that the ; would be part of the text to be replaced, I also assumed that STR_Text2 would always look like a C identifier
elticAuthor Commented:
Works now with more than one string, but it seems that this script has
problems replacing strings beginning or ending with spaces, so a string
like
#define STR_Test = "   no data    ";

wouldn't be replaced...
ozoCommented:
Is the semicolon a part of the text to be replaced?   I assumed it was in the code above, so if there is no semicolon in test.c it will not be replaced.
To ignore an optional semicolon
perl -ne 'if( m#^\s*//\s*(.+):(\d+)# ){ push @f,\($f{$1}{$2}||=[])
             }elsif( my($s,$r)=/#define\s+(\w+)\s*=?\s*(.*?)\s*;?\s*$/ ){  
             my $t=sub{s/\Q$r\E/$s/g}; push@$$_,$t for splice @f; }
             END{ local @ARGV=keys %f; local $^I="";
             while( <> ){ for$r(@{$f{$ARGV}{$.}||[]}){&$r} print; }continue{ close ARGV if eof }
             }' < text.h
elticAuthor Commented:
That's the thing...one (last) question:
Could you please modify the script so that it will also replace
a string beginning or ending with less spaces (and just them)
than the original one ?
E.g.
#define ___TestString___ "   TestString   "

will even replace

string test = "                TestString                   ";
But will not replace

string test = "TestString"; or string test = "a          TestString            b";
ozoCommented:
How about
#define ___TestString___ "   Test String   "
replacing
string test = "                Test                String                   ";
?

elticAuthor Commented:
The script should just care about more spaces at the beginning and
at the end of the string, not between, so "   Test String   " would
replace "              Test String             " but not "         Test      String          "
ozoCommented:
perl -ne 'if( m#^\s*//\s*(.+):(\d+)# ){ push @f,\($f{$1}{$2}||=[])
          }elsif( my($s,$r)=/#define\s+(\w+)\s*=?\s*(.*?)\s*;?\s*$/ ){  
          $r="\Q$r\E"; $r=~s/"\\ /" +/; $r=~s/\\ \\"/ +"/;
          my $t=sub{s/$r/$s/g}; push@$$_,$t for splice @f; }
          END{ local @ARGV=keys %f; local $^I="";
          while( <> ){ for$r(@{$f{$ARGV}{$.}||[]}){&$r} print; }continue{ close ARGV if eof }
          }' < text.h

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux OS Dev

From novice to tech pro — start learning today.