Solved

Linux Command Line Search & Replace Script

Posted on 2012-03-15
8
474 Views
Last Modified: 2012-03-23
I need some help writing a bash script (or something else, if suggested) that will scour recursively through all the php files in a folder and subfolders, and do a search & replace.

The text to match is quite long, so I'm guessing a script file is more appropriate than just a command line entry.

Any ideas?
0
Comment
Question by:RKFcomputers
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
  • 2
  • +2
8 Comments
 
LVL 8

Expert Comment

by:eager
ID: 37725868
Take a look at find and sed.  You can create a sed script to do the replacement and drive is using find to select the correct files.
0
 

Author Comment

by:RKFcomputers
ID: 37725962
@eager, Thanks - I know enough to know that find & sed are the likely tools to use, but not enough about how to employ them properly in this case.

(Honestly, we're an SEO company trying to help a friend. I know just enough Linux to be dangerous, as they say.)

Hence my request for help here in crafting the right approach.
0
 
LVL 16

Expert Comment

by:medvedd
ID: 37726182
find . -name "*.php" -print | xargs sed -i 's/foo/bar/g'
0
Are You Headed to Black Hat USA 2017?

Getting ready for Black Hat next week? Kick things off with the WatchGuard Badge Challenge and test your puzzle and cipher skills. Do you have what it takes to earn our limited edition Firebox Badge? Get started today - https://crimsonthorn.net

 

Author Comment

by:RKFcomputers
ID: 37726571
@medvedd, Thank you!

That's a great start ... but what's the best approach when "foo" is actually about 2,000+ characters?

I know how I'd handle it in PHP by putting the strings into variables ... but I'm not up enough on bash scripting to make sure I don't bugger it up.
0
 
LVL 8

Accepted Solution

by:
eager earned 500 total points
ID: 37727010
You can use a sed script.  You don't need to use xargs.  

find . -name \*.php -exec sed -f script.sed {} \;

Where script.sed contains
s/whatever/whichever/

You can use whatever delimiter you wish in place of '/' as long as it doesn't appear in your source.  You can also split your substitution into multiple patterns if that makes things easier.  Each command in the sed script is executed in order for each input line.  

Sed can be touchy at times.  Make a copy of a couple of the files and move them to a sandbox directory for testing.
0
 
LVL 79

Expert Comment

by:arnold
ID: 37727024
perl -ibak -p -e 's/patterntomatch/replacementtext/;' list_of_files
Matched files will be backed up with a .bak suffix.

Are there any other delineations I.e. the item being replaced is enclosed in comment markers //
foo nor pattern have to be the entire text being replaced, but has to be unique enough to match I.e.
Startingstring.*continuationrequiredstring.*anotherrequiredparameter
The problem is if the dt to match spans multiple lines rather being a auto wrapped single line.
In the multiline, additional options have to be used with sed or perl to look for foo to match across line.

In foo and pattern can be used as a variable
foo="pattern"
Etc.

Perl does the find and replace in a single line command.  It can be converted into a perl script with the while loop going line by line matching on the first occurrence of a defined pattern, then trying to compare the reminder to match the entire pattern.
Etc.
0
 
LVL 16

Expert Comment

by:medvedd
ID: 37728860
As @eager mentioned, you can make sed script and put long patterns there.
Be aware that usually GNU sed has a limitation of 4000 bytes for pattern length.
0
 
LVL 25

Expert Comment

by:madunix
ID: 37732164
I use the following script to replace string in directories/subdirectories/file-names/content

#!/bin/bash
ROOT_DIR="/target/directory"  # your target dir
FILTER_FILE="/directory/filter.sed"  # the sed script for renaming

# custom rename function that uses $FILTER_FILE (via sed)
function rename_using_filter {
    CURRENT_NAME="$1"
    NEW_NAME="$(echo $1 | sed -f $FILTER_FILE)"  # derive new name
    if [ "$CURRENT_NAME" != "$NEW_NAME" ]; then  # rename if diff
        mv "$CURRENT_NAME" "$NEW_NAME"
    fi
}

# for each directory, starting from deepest first
while IFS= read -r -d $'\0' DIR_NAME; do
    cd "$DIR_NAME"           # go to target dir

    # for each file/dir at this level
    while IFS= read -r -d $'\0' FILE_NAME; do
        if [ -f "$FILE_NAME" ]; then                # if it's a file
            sed -i -f "$FILTER_FILE" "$FILE_NAME"   # replace content
        fi
        rename_using_filter "$FILE_NAME"  # rename it
    done < <(find . -maxdepth 1 -print0)

    cd - > /dev/null         # back to original dir. Suppress stdout
done < <(find $ROOT_DIR -depth -type d -print0) # get only dirs




In the filter.sed put your strings
# cat filter.sed
s/old1/new1/g;
s/old2/new2/g;
s/old3/new3/g;

s/oldn-1/newn-1/g;
s/oldn/newn/g;
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In a recent article here at Experts Exchange (http://www.experts-exchange.com/articles/18880/PaperPort-14-in-Windows-10-A-First-Look.html), I discussed my nine-month sandbox testing of the Windows 10 Technical Preview, specifically with respect to r…
Recently, I read that Microsoft has analysed statistics for their security intelligence report. It revealed: still, the clear majority of windows users do their daily work as administrator. An administrative account is a burden, security-wise. My ar…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial
Suggested Courses

623 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question