Link to home
Start Free TrialLog in
Avatar of traveh
traveh

asked on

Search and replace within BINARY files on Solaris 8

Hi Experts,

I need to perform a search and replace of text on Solaris 8 with the following conditions:
1. Performed on BINARY files (NOT based on the file extension) - WITHOUT CORRUPTING ANY OF THE FILE'S CONTENTS.
2. The text files must be automatically identified.
3. To be recursively performed on all files within a directory.
4. Output a report on exactly were the text was replaced.

Please direct me to a freeware application (preferably GUI app), or send me a script to do this.

Many thanks!

Regards,
Tal.
Avatar of manav_mathur
manav_mathur

>search and replace of text

Do you expect to find text ina binary file??

Manav
>1. Performed on *BINARY files* (NOT based on the file extension) - WITHOUT CORRUPTING ANY OF THE FILE'S CONTENTS.
>2. The *text files* must be automatically identified.

Do you want to do a search on binary files or text files.

Manav
Avatar of traveh

ASKER

Sorry, I forgot to erase that line (2).

I am refering to binary files.

What I want is to find a specific characters string in a binary file, and replace it with another.

For example: search for the characters "xxxxx" and replace them with "yyyyy"

Thanks,
Tal.
Doesnt sed work??

Manav
Avatar of traveh

ASKER

From "man sed":

DESCRIPTION
     The sed utility is a stream editor that reads  one  or  more
     text  files,  makes editing changes according to a script of
     editing commands, and writes the results to standard output.
     The  script  is  obtained  from  either  the  script operand
     string, or a combination of the option-arguments from the -e
     script and -f script_file options.

     The sed utility is a text  editor.  It  cannot  edit  binary
     files  or files containing ASCII NUL (\0) characters or very
     long lines.
ASKER CERTIFIED SOLUTION
Avatar of ahoffmann
ahoffmann
Flag of Germany image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Avatar of yuzh
yuzh

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
previous suggestion in UNIX:

 find / -type f -exec perl -i.bak -pe 's/lamb/sheep/g' {} \; -print >>/tmp/file_replaced

KISS - keep it stupid simple
ahoffmann, your script is really simple.
However, you skip one "grep -i executable" for binary files only.

Wesly
all files are "binary", somehow ;-)
but if you trust /etc/magic you may improve the find with -prune ...
Avatar of traveh

ASKER

Wesly,

The script you sent doesn't work.
I think it's because of the grep...:

NOTES
  /usr/bin/grep
     Lines are limited only by the size of the available  virtual
     memory.  If  there  is a line with embedded nulls, grep will
     only match up to the first null;  if  it  matches,  it  will
     print the entire line.

  /usr/xpg4/bin/grep
     The results are unspecified if  input  files  contain  lines
     longer  than LINE_MAX bytes or contain binary data. LINE_MAX
     is defined in /usr/include/limits.h.

> grep -i executable
Have you try "egrep -i executable"?

Wesly
Have you tried http:12769954 ?
perl has no such limits ;-)
Avatar of traveh

ASKER

I'll try the perl version now...
Well, a couple qualfiers come to mind:

- Hopefully the binary file isn't checksumed in some way such that when you change it, it doesn't corrupt the file.

- Without knowing the binary file format, the convervative approach would be if string-to-be-changed is 10 characters long, the string used to change it must also be 10 characters long.
-Binary_S&R.sh ------
#!/bin/sh

script_name=`basename $0`
if [ $# -ne 2 ]
then
    echo "Syntax Error!"
    echo "Usage:    $script_name <search string> <replace string>"
    exit 1
fi

search_string=$1     # read first parameter
replace_string=$2    # read second parameter

rm -f /tmp/file_replaced   # clean the previous report file

for files in `find <path to the directory> -type f -exec file {} \; | egrep -i "LSB executable" | awk -F: '{print $1}'`
do
   grep $search_string $files > /dev/null 2>&1
   if [ "$?" = "0" ]
   then
      echo $files >> /tmp/file_replaced        # report which file been replaced
      perl -i.bak -pe "s/$search_string/$replace_string/g" $files
   else
      continue
   fi
done
----

ahoffmann's script doesn't check the file type which may change the text files as well.

Wesly
> ahoffmann's script doesn't check the file type which may change the text files as well
see http:#12781981 :-))
> all files are "binary", somehow ;-)
That's the part I'm confused. Is it true?
Oooohhhhhhh ..... don't let ahoffman confuse you.  Technically everything in the computer is binary.  The difference is around what we stupid humans can read.  "Text files" are files you can simply cat (or "type" for Windows folks) and read the contents without seeing any non-printable characters.  Basically "A-Za-z0-9<space><tab><lots of normal characters like .,-_!@#$%^&*()_+><a several other characters>".  When folks say "binary" they usually mean stuff like executable files, shared libraries, compressed files, encrypted files, etc.  Basically when you cat them they should up as junk on the screen.

But technically ahoffman is 100% correct.  In the end, all files on a computer are nothing more then a bunch of ones and zeros on the disk.
Thanks Nukfror.
Then I don't think ahoffman's script meet "executable program".

Wesly
Avatar of traveh

ASKER

As a matter of fact, I was not referring to executable files specifically - but to binary files in general (which is basically every file on the computer, as ahoffman stated) - executables, binary data files, text files, etc...

Executables in specific are "binary files that can be executed".

(I mentioned BINARY in the question to emphasize that I'm not referring to text files).

Regards,
Tal.
Ok.
Either change my script:
for files in `find <path to the directory> -type f -exec file {} \; | egrep -i "LSB executable" | awk -F: '{print $1}'`
to
for files in `find <path to the directory> -type f`

OR
Usage:    $script_name <path> <search string> <replace string>
----
#!/bin/sh
find $1 -type f -exec perl -i.bak -pe "s/$2/$3/g" {} \; -print >>/tmp/file_replaced
---

Wesly
K - if you mean EVERY FILE on the computer then you better be careful.  Changing a "text" file can be VERY different then changing a "binary" file.  Not all binary files are created equal.  If you personally don't understand the internal structure of a binary file and how doing string changes within it will impact it, you're taking a risk.

Just because you can change something *and* keep the file size exactly the same, does not mean you have kept the file from being corrupt.

I suggest you don't just start changing libraries, executable, archive file, etc.
Avatar of traveh

ASKER

Apparently, the last perl script wasn't a really good idea...

After a while I'm getting these:
Can't rename ./AppServer/logs/update/20041122_163144_was50_fp2_solaris_app.defaultApplication_install.log.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak to ./AppServer/logs/update/20041122_163144_was50_fp2_solaris_app.defaultApplication_install.log.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak: File name too long, skipping file.
Can't rename ./AppServer/logs/update/20041122_163226_was50_fp2_solaris_prereq.analysis.engine_install.log.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak to ./AppServer/logs/update/20041122_163226_was50_fp2_solaris_prereq.analysis.engine_install.log.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak.bak: File name too long, skipping file.

etc, etc, etc...
Well, those .bak files are created (perl -i.bak ...) for backup purpose.
Please delete them by the following command:
find . -type f -name "*.bak" -exec rm -f {} \ ;

Then use
----
#!/bin/sh
find $1 -type f -exec perl -i -pe "s/$2/$3/g" {} \; -print >>/tmp/file_replaced
                                  ^^^
                             no .bak here
---------

Wesly
By the way, if you only deal with "text file", then use my script in
https://www.experts-exchange.com/questions/21221025/Search-and-replace-within-files-on-Solaris-8.html
will be safer t avoid destroying the binary program files.

Wesly
Avatar of traveh

ASKER

Is there a way to perform the replace command on all files, except .bak files?
Hi,

  Those .bak files are generate by the script here with perl -i.bak.
You just remove them (as my last post) since they are junk.

Wesly
Avatar of traveh

ASKER

Yes, I understand that...

I was wondering if there was a way to save the backup files while doing the replace, and still get the script to work.

If I could perform the search and replace on all files *except* .bak files, it would would be nice...

Tal.
As your situation, everytime you run the script and generate one copy of .bak.
So you have more than 10 copies for each files replaced.
IMHO, just delete them all since they are junk to you now.
Skip .bak won't help.

Wesly
Avatar of traveh

ASKER

I wasn't referring to the old .bak.bak.bak... files.
I am talking about saving a single ".bak" file for each file...
Avatar of traveh

ASKER

P.S.

Now (with the perl script), the file_replaced file contains the names of ALL files - not just the ones in which the string was replaced...

-Binary_S&R.sh ------
#!/bin/sh

script_name=`basename $0`
if [ $# -ne 3 ]
then
    echo "Syntax Error!"
    echo "Usage:    $script_name <path> <search string> <replace string>"
    exit 1
fi

search_path=$1
search_string=$2     # read first parameter
replace_string=$3    # read second parameter

rm -f /tmp/file_replaced   # clean the previous report file

for files in `find $search_path -type f | grep -v ".bak"`
do
   grep $search_string $files > /dev/null 2>&1
   if [ "$?" = "0" ]
   then
      echo $files >> /tmp/file_replaced        # report which file been replaced
      perl -i.bak -pe "s/$search_string/$replace_string/g" $files
   else
      continue
   fi
done
----

Wesly