script to transfer file from one system to multiple system at once

Hi Everyone,

I am trying to transfer some file & could be folder in future..

At the moment i am using scp
Host A to Host1
Host A to Host2
Host A to Host3
Host A to Host4
Host A to Host5
Host A to Host6
Host A to Host7
Host A to Host8
Host A to Host10
Host A to Host11
-->This process is time consuming..

At moment i am doing like this

HostA$scp file1 user@Host1:/userhome/user/
HostA$scp file1 user@Host2:/userhome/user/
HostA$scp file1 user@Host3:/userhome/user/
HostA$scp file1 user@Host4:/userhome/user/
HostA$scp file1 user@Host5:/userhome/user/
HostA$scp file1 user@Host6:/userhome/user/
HostA$scp file1 user@Host7:/userhome/user/
HostA$scp file1 user@Host8:/userhome/user/
HostA$scp file1 user@Host9:/userhome/user/
HostA$scp file1 user@Host10:/userhome/user/
HostA$scp file1 user@Host12:/userhome/user/

I would like to make this automated so in one go I can able to transfer file1 to all  Hosts (1-12)

I have a one useraccount with same username & password on all the Hosts (1-12)

Pls let me know if you need further information

Javeed A

Who is Participating?

[Webinar] Streamline your web hosting managementRegister Today

arnoldConnect With a Mentor Commented:
The simplest of things.

./script filename if you are within the directory or
./script /full/path/to/filename
if [ -z "$1" ]; then
echo "Usage: $0 filename"
exit 1;
if [ ! -e "$1"] ; then 
echo "The filename you provided: $1: does not exist"
exit 2
scp $1 user@Host1:/userhome/user/
scp $1 user@Host2:/userhome/user/
scp $1 user@Host3:/userhome/user/
scp $1 user@Host4:/userhome/user/
scp $1 user@Host5:/userhome/user/
scp $1 user@Host6:/userhome/user/
scp $1 user@Host7:/userhome/user/
scp $1 user@Host8:/userhome/user/
scp $1 user@Host9:/userhome/user/
scp $1 user@Host10:/userhome/user/
scp $1 user@Host12:/userhome/user/

Open in new window

are they same file ??

why are u not using a script from each server host 1 to host 11 and they will put data from host A rather then A is sending data into all those server

you can use RSYNC script with cron tab..

you will need somethign like this


TODAY=`date ‘+%A’`

rsync -e ssh -avz –delete uer@HOSTA:/file/to-copy $DESTROOT/store-fle

if you use key base authentication,then you can do this without even typign username and password

The new generation of project management tools

With’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

javeedabdulAuthor Commented:
The link is not complete.


I need a procedure to happen from Host A to Host (1-2) that's the requirement.
The file is not transfer every day it just once in a time.
so when ever a file need to be transfer i would like to run a script on Host A and that should authenticate and transfer file on all 12 host ( one by one ) or all at the same time

let me know if you need more information

# This is a script to copy files from one host to a group of hosts

# There are three variables accepted via commandline
# $1 = first parameter (/source_path/source_filename)
# $2 = second parameter (/target_directory/)
# $3 = third paramter (file that contains list of hosts)


if [ -f $SOURCEFILE ]
   printf "File found, preparing to transfer\n"
   while read server
      scp -p $SOURCEFILE ${server}:$TARGETDIR
   done < $HOSTFILE
   printf "File \"$SOURCEFILE\" not found\n"
   exit 0
exit 0

ref :
If you want to use scp, you will need to use key based authentication in order not to need to type in the password every time.

then have a script that as a line for each remote host
scp -o batchmode=yes $1 <user>@<host>:/<some>/<path>/

where $1 is the file name you pass in when you run the script
eg. save the script file as
./ CopyThisFile.txt

If all systems are in one location and you have NFS share exported from HOSTA to all others, you could have a script on all others to monitor the NFS share for file changes and after some validation/sanity checks etc. the script will propagate the changes it sees on the NFS share.
using puppet/SVN might also be a way to push the changes to all hosts.
A plain SCP can under certain unique conditions be unexpected depending on the files you are transferring i.e. the partition to which the file is being SCPed runs out of the space and the transferred file is runcated or zero length.
To add to arnold:

If you can nfs, then why not just setup a perment mount point from the other servers back to the main server, this way they all read from the one location over nfs from the master, no copy needed.  
javeedabdulAuthor Commented:
I am not an expert in linux,  plus I don't need a permanent setup.. I need temp setup to my work easier,

Suggest a simple script which does the job for me


javeedabdulAuthor Commented:

I will give a try to the script,
But it kind of doesn't achieve what i had hoped it would. I am looking for a way to do multiple transfers in parallel to save time. That is, from a source to multiple destinations at the same time. Is this possible ?

might also suggest you put each scp command in backround and redirect stderr to an errorfile to see what if  anything fails.

scp $1 user@Host1:/userhome/user/  2>&1 >> /tmp/scp error &
note that with this setup, you do not have any checks for errors or sanity checks nor dealing with issues such connection drop during transfer, or power failure.
IMHO if this is a configuration file, a script on each system should check whether the data is valid i.e. use cksum/md5sum to make sure the data sent matches the data received. Also have a check to make sure important parameters/data within the configuration do not suddenly go absent because of a typo. etc. Once validation is completed, the file can be "deployed" on the individual system.
parallel ssh, perhaps?

On Ubuntu ...

$ sudo aptitude install pssh
$ cat hosts.txt
Host1 user
Host2 user
Host3 user
Host4 user
Host5 user
Host6 user
Host7 user
Host8 user
Host9 user
Host10 user
Host11 user
Host12 user
$ parallel-scp --hosts hosts.txt file1 /userhome/user/

Open in new window

javeedabdulAuthor Commented:
Thanks very much

I really need this procedure implemented.I will get back on this asap..

Have few other critical question which i am working on this weekend and will follow on this one. once i sought those out.

Javeed A
javeedabdulAuthor Commented:
error downlaoding the pakage..

#sudo aptitude install pssh
The following NEW packages will be installed:
The following partially installed packages will be configured:
0 packages upgraded, 1 newly installed, 0 to remove and 8 not upgraded.
Need to get 28.3 kB of archives. After unpacking 201 kB will be used.
Get:1 natty/universe pssh all 2.1.1-1 [28.3 kB]
Fetched 28.3 kB in 0s (77.2 kB/s)
Selecting previously deselected package pssh.
(Reading database ... 236459 files and directories currently installed.)
Unpacking pssh (from .../archives/pssh_2.1.1-1_all.deb) ...
Processing triggers for man-db ...
Setting up gdk-pixbuf-psd (20081217) ...
/var/lib/dpkg/info/gdk-pixbuf-psd.postinst: 33: cannot create /usr/lib/gtk-2.0/2.10.0/loader-files.d/gdk-pixbuf-psd.loaders: Directory nonexistent
/var/lib/dpkg/info/gdk-pixbuf-psd.postinst: 33: gdk-pixbuf-query-loaders: not found
dpkg: error processing gdk-pixbuf-psd (--configure):
 subprocess installed post-installation script returned error exit status 127
Setting up pssh (2.1.1-1) ...
Processing triggers for python-support ...
Errors were encountered while processing:
E: Sub-process /usr/bin/dpkg returned an error code (1)
A package failed to install.  Trying to recover:
Setting up gdk-pixbuf-psd (20081217) ...
/var/lib/dpkg/info/gdk-pixbuf-psd.postinst: 33: cannot create /usr/lib/gtk-2.0/2.10.0/loader-files.d/gdk-pixbuf-psd.loaders: Directory nonexistent
/var/lib/dpkg/info/gdk-pixbuf-psd.postinst: 33: gdk-pixbuf-query-loaders: not found
dpkg: error processing gdk-pixbuf-psd (--configure):
 subprocess installed post-installation script returned error exit status 127
Errors were encountered while processing:
This is unrelated to pssh, you have a broken system. Try running ...
sudo aptitude -f install 

Open in new window

javeedabdulAuthor Commented:
Good script but not what i like to acheive. will ask this question once i go through some scripting tutorials
All Courses

From novice to tech pro — start learning today.