Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1078
  • Last Modified:

Undesirable early termination - Bash shell scripting while loop

I've created a bash shell script that loops through a command file.  This command file performs a range of tasks across a range of servers.

WORKS: If I place 'echo' in front of the commands within the loop, then it will show a simulated execution of all tasks on all servers in the list.

DOESNT WORK: After I remove the 'echo' then it will successfully perform all the tasks for the first server, then never do anything for the subsequent servers!

A heavily edited version for clarity and security of the code is as follows ... ( these are the major utilities used in actual script )

while read recLabel recSourcePath recDestinationServer recDestinationPath recDestinationFile
do
  ssh ${recDestinationServer} mkdir ${recDestinationPath}
  ssh ${recDestinationServer} rm ${recDestinationPath}/*.tar.gz
  rsync -baz -e ssh ${recSourcePath }/${recDestinationFile} username@${recDestinationServer }:${recDestinationPath }
done < ${parmFileList}

... sample parmFileList ...
app1 /opt/app1/data webserver1 /tmp/update 1.tar.gz
app1 /opt/app1/data webserver2 /tmp/update 1.tar.gz
app1 /opt/app2/data appserver1 /tmp/update 3.tar.gz
app1 /opt/app2/data appserver2 /tmp/update 3.tar.gz

Any thoughts?

I've put in place a temporary solution, which was to cut all the loop code and create another script.  I call that script from the loop context with a split of the pid using '&'
This of course works, as parallel; however, I'm still curious why it won't run through the whole parmFileList as this is posted.




0
lennagy
Asked:
lennagy
  • 4
  • 3
  • 3
  • +1
1 Solution
 
glassdCommented:
Just a thought, have you got another read command inside the loop?
0
 
lennagyAuthor Commented:
No, sorry, no 'read' statements in the whole file.  It's a parameter and configfile-based script.

I am thinking that perhaps the ssh command to 'mkdir' or 'rm' files that can fail if the directory is already created or no files exist to wipe out is somehow jinxing the loop.  Of course, those commands can fail; however, trapping the error from the remote system is more complex than $?, which only tests a successful connection to the remote server.

I should try this again without those two statements.  
0
 
ahoffmannCommented:
replace
     ssh ${recDestinationServer} rm ${recDestinationPath}/*.tar.gz
by
     ssh ${recDestinationServer} rm ${recDestinationPath}/\*.tar.gz
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
lennagyAuthor Commented:
Thanks, ahoffman.  I didn't need to escape that because it actually did perform the 'rm' if files existed. Unless you think that causes the problem of the 'while' loop terminating after the first iteration of the loop?
0
 
NopiusCommented:
You don't need to 'mkdir' then 'rm ' with rsync, just use destination pathnames with trailong slash: /tmp/update/
instead of 'rm' you may use --inplace rsync option. In that case destination dir will be automatically created for you.

while read recLabel recSourcePath recDestinationServer recDestinationPath recDestinationFile
do
  rsync --inplace -baz -e ssh ${recSourcePath}/${recDestinationFile} username@${recDestinationServer}:${recDestinationPath}/
done < ${parmFileList}

also enshure that you have correct user private key files and ssh_config file
0
 
ahoffmannCommented:
> ..  did perform the 'rm' if files existed.  ..
could be the problem, for example is your shell runs with -x
I'd always quote *all* variables.
0
 
NopiusCommented:
ahoffman, any errors inside loop are ignored, even with -x flag.
Such glob expansion will be treated as illegal 'ssh' usage and returns exit code non-zero. Then  shell code continues and  loop goes to the next read line.
So, that script should either work or not work for each loop invocation. Probably something wrong either with ssh identities or with destination paths.
0
 
ahoffmannCommented:
which shell with -x continues (in a loop) if any used command does not return 0?
0
 
lennagyAuthor Commented:

private key and ssh are working fine otherwise. Besides, with ssh improperly configured it wouldn't work at all, and it is now working as I said on different PIDs &.  I've taken the 'exact' same code and put into another script, called it from the loop, and forcing it to 'wait' for the child PIDs. All goes smoothly.  

ahoffman: Tried your suggestion, unfortunately, the same results

Nopius: I will try the rsync approach today...

0
 
NopiusCommented:
ahoffman: standard Bourne Shell

$ uname -a
SunOS sf250 5.10 Generic sun4u sparc SUNW,Sun-Fire-V250

 $ which sh
/usr/bin/sh

$ cat ../bad_loop.sh
for i in *
do
  cp $i
done

$ sh -x ../bad_loop.sh
+ cp a
cp: Insufficient arguments (1)
Usage: cp [-f] [-i] [-p] [-@] f1 f2
       cp [-f] [-i] [-p] [-@] f1 ... fn d1
       cp -r|-R [-H|-L|-P] [-f] [-i] [-p] [-@] d1 ... dn-1 dn
+ cp b
cp: Insufficient arguments (1)
Usage: cp [-f] [-i] [-p] [-@] f1 f2
       cp [-f] [-i] [-p] [-@] f1 ... fn d1
       cp -r|-R [-H|-L|-P] [-f] [-i] [-p] [-@] d1 ... dn-1 dn
+ cp c
cp: Insufficient arguments (1)
Usage: cp [-f] [-i] [-p] [-@] f1 f2
       cp [-f] [-i] [-p] [-@] f1 ... fn d1
       cp -r|-R [-H|-L|-P] [-f] [-i] [-p] [-@] d1 ... dn-1 dn
+ cp d
cp: Insufficient arguments (1)
Usage: cp [-f] [-i] [-p] [-@] f1 f2
       cp [-f] [-i] [-p] [-@] f1 ... fn d1
       cp -r|-R [-H|-L|-P] [-f] [-i] [-p] [-@] d1 ... dn-1 dn
0
 
lennagyAuthor Commented:
Thanks for your help.  Still uncertain as to the direct cause, but using rsync to manage the remote directories works nicely.

Thanks again!
0

Featured Post

Free Backup Tool for VMware and Hyper-V

Restore full virtual machine or individual guest files from 19 common file systems directly from the backup file. Schedule VM backups with PowerShell scripts. Set desired time, lean back and let the script to notify you via email upon completion.  

  • 4
  • 3
  • 3
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now