Suggestions regarding copying files in UNIX

Hi,

Right now we are copying files from windows to UNIX using R-sync utility . But  sometimes it is getting hung up due to overload of data.

 We are using R-sync to compare checksum and date of the files to copy over.

Is  there any other best utility to replace R-sync.
new_perl_userAsked:
Who is Participating?

[Webinar] Streamline your web hosting managementRegister Today

x
 
parparovConnect With a Mentor Commented:
from rsync's manual:
            --files-from=FILE       read list of source-file names from FILE

Open in new window

0
 
parparovCommented:
rsync has an option --bwlimit=XXXX to limit its data transfer rate. The limit is given in kilobytes. Use --bwlimit=10000 to limit to 10 (9.7 to be precise) Mbyte/sec.
0
 
gheistCommented:
rsync will only access disk and do very little over network connection if file sets (or their big parts) are same, but keeping reading all data at full disk speed at the same time.
If this influences the rest of the system you can "renice" rsync process on UNIX.

Please tell us version of FreeBSD you are using, version of rsync, and version of SSH.
Also would be nice to know command line used to invoke rsync if it needs some parameter tuning.
0
Learn to develop an Android App

Want to increase your earning potential in 2018? Pad your resume with app building experience. Learn how with this hands-on course.

 
new_perl_userAuthor Commented:
Hi.

Below is the command we are using and invoking it through a script.

$RSYNC ${DRYRUN} ${OPTIONS} --filter="exclude ${TEMPDIR}" -e "ssh -i ${KEY}" ${STRING}:${DIR}/ ${LOCALDIR} |$AEE ${TMPLOG} >> ${LOGFILE} 2>&1

Is there a way ( like kind of alert messages) to know when the rsync breaks /stops copying files.
0
 
parparovCommented:
we need to see what ${OPTIONS} are during the actual run.
To see if rsync exited abnormally, test $? value after rsync's execution. If everything was alright, then it is 0, otherwise it contains the error code of the problem rsync encountered.
0
 
new_perl_userAuthor Commented:
options being used are as follows.

OPTIONS=' -rltzc --chmod=u+rwx,go=rx -u -K -v -i  --exclude=Thumbnail --exclude=Thumbs.db '

And where should we actually use this "test $?".  Will it be able to write to a log file.
0
 
parparovCommented:
you may add the --bwlimit I posted earlier to OPTIONS sequence.
Right after the rsync execution, add something like that:
if [ ! $? ]
  then
	echo "`date` Rsync exited abnormally with code $?" >> /your/log/file/location
fi

Open in new window

0
 
new_perl_userAuthor Commented:
Thank you Will try..
0
 
new_perl_userAuthor Commented:
Hi,
One more quick question. Will the R-sync be able to copy files  by reading a file with all the filenames it should copy.
0
All Courses

From novice to tech pro — start learning today.