Using a loop in bash to run a command on a list of items

Published:
Updated:
How many times have you wanted to quickly do the same thing to a list but found yourself typing it again and again? I first figured out a small time saver with the up arrow to recall the last command but that can only get you so far if you have a big list and only a little time.  What if you wanted to ping 50 IP addresses to see if they were in use or needed to check to see if 159 different DNS entries all looked up OK using a host command? Sure, you could write a quick script and have it read in a file and do the work but for those little jobs on a big list that you just need once, there may be a quick and easy way...

If you are using a standard, recent Linux distribution, there's a pretty good chance that you're using a bash shell when you open up the terminal window.  When you're staring at that terminal and thinking about your list, you'll find you have a few easy ways to get started.  The first thing to do is get your list in the buffer either by highlighting it so you can do a middle click or selecting it all and picking it up with a copy and paste.  I have found that you can highlight a column of data in OpenOffice, for example, and use it to make your list with a paste into the terminal window.  Once you have your list ready, all you have to do is format it for use and run the loop.

Getting the list formatted:
The loop will like to iterate over a space separated list. If you have data in the buffer, you can get started by typing in a list name as a variable at the prompt and pasting in the data.  So, type something like:
namesList="
at the command prompt and then paste the input into the console.  At the end of the list, put in a closing " and press the <enter> key.  Now, at the prompt, type in:
echo $namesList

Open in new window

and you'll see your space separated list.

You could also build your list from file patterns.  This is especially useful if your loop will need to do renames for example.  You could say fileList="*.htm" and when you echo $fileList, any files that end in .htm will be in the list. The loop to, say, rename the .htm to .html would be pretty straightfoward once the list is created.

Running the loop:
After you have the variable setup to hold the list, you just need to define the loop.  The loop will look something like:
for <name of item> in <name of list>; do <command> <name of item>; done
with all of the above on one line.

An example of doing a host lookup for the list above would look something like this:

namesList="www.microsoft.com www.google.com www.experts-exchange.com www.radsquare.com"
                      for thisName in $namesList; do host $thisName; done

Open in new window


Once you have the basics of the loop down, you could use more complicated loops to help quickly format the output as well.  Here's an example that just lists the IP addresses for the data returned above:

for thisName in $namesList; do host $thisName | awk -F" " '{print $NF}' | egrep -v "\.$"; done

Open in new window


Conclusion:
A do/while loop in a terminal window is a very fast way to quickly do the same operation on a longer list of items.  This is especially useful for doing a quick check on a list or reformatting it for a one-off situation where writing a script doesn't seem to make sense. The next time you have a long list of items and need to do the same commands, give the loop a try!

p.s.--don't use a loop to do deletes. One unexpected delete could cause lots of headaches!
0
4,665 Views

Comments (4)

CERTIFIED EXPERT

Commented:
The biggest limitation with this methodology is that the space character is a separator, so no list item can contain internal whitespace.  Is there a way to change the separator?  I'm thinking yes, but I forget exactly what it is (could the author maybe add that to the article?)

Using the author's example, a potentially more reliable method is to simply:

cat | awk '{print "host " $0}' | sh -x 2> /dev/null | awk -F" " '{print $NF}' | egrep -v "\.$"

(then hit paste, ctrl-d)

-Jon
CERTIFIED EXPERT

Commented:
"for" ignores quotes (single or double) in its input, AFAIK.

-Jon

Commented:
The separator is set with the $IFS shell variable.
The default value for IFS is space, tab and newline.

Remind that $IFS affects the whole shell, not just the for command.

Author

Commented:
Right, I would consider actually writing a short shell script if you need to do things like account for spaces, comments, special characters, etc. in the data or need to use a something other than a space to separate a field.  I generallyy use this command line script for quckly checking a list of host names against internal DNS.  If you did want to change the separator with IFS, you just need to save it first and then export it again when you're done.  You could do this with something like oldIFS=$IFS; export IFS=","; <your stuff here>; export IFS=$oldIFS and still change the separator just for the that run.  This example uses a comma as the separator.  The

Have a question about something in this article? You can receive help directly from the article author. Sign up for a free trial to get started.