Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

Rsync progress question

Posted on 2013-01-28
8
Medium Priority
?
637 Views
Last Modified: 2013-01-31
I’m backing up my VM’s from my storage server to external HDD for offsite backup.  I’m trying to monitor how the progress is going, but despite what appears to be the right parameter, I’m not seeing it.  I’ve created a script with a command for each of the dirs that uses a supplied parameter (the date stamp on the backup dir) for each directory.  At the end I want it to shoot me an email.  My command is called backup.sh.  I’m running it as root, sh backup.sh 2013-01-25_04-00-00

The backup.sh looks like this inside:

rsync -r -v --progress /mnt/storage/nfs/ocl_nfs_1/backups/server1/server1-$1 /media/external/server1 | mail -s "RSync Job Log" iamhere@work.ca

(more server backups follow this one, each using the same command with the names changed.)

I’m sure there’s a better way to do this.  I’m running this on Openfiler 2.99 2.6.32-131.17.1.el6-0.11.smp.gcc4.4.x86_64.  (based on rPath)

also, if there's a faster way to transfer to a USB device other than Rynsc that will still mail me a log when completed I'd appreciate any additional input.
0
Comment
Question by:clicker666
  • 4
  • 3
8 Comments
 
LVL 39

Expert Comment

by:Aaron Tomosky
ID: 38829743
Way outside the scope of this question:
If you moved to a zfs based storage (zfsguru.com is what I recommend but there are others) you can make snapshots and send them to another zfs box. For non zfs stuff you can rsync with snapshots like so:
http://forums.freebsd.org/showthread.php?t=11971

Just something I mention to people running NSF shares on non-zfs storage like openfiler.
0
 
LVL 1

Author Comment

by:clicker666
ID: 38830608
I found my own answer.  This parameter seems to break the onscreen progress indicators.   | mail -s "RSync Job Log" iamhere@work.ca  Since it wasn't actually sending me email I suppose I can live without it for now.
0
 
LVL 1

Author Comment

by:clicker666
ID: 38837246
I've requested that this question be closed as follows:

Accepted answer: 0 points for clicker666's comment #a38830608

for the following reason:

Although I fixed my own problem, it stops me from receiving email logs of the jobs.  I will have to research and revisit that side - but for now I will manually run the jobs and monitor their progress.
0
Get your Conversational Ransomware Defense e‑book

This e-book gives you an insight into the ransomware threat and reviews the fundamentals of top-notch ransomware preparedness and recovery. To help you protect yourself and your organization. The initial infection may be inevitable, so the best protection is to be fully prepared.

 
LVL 21

Expert Comment

by:Daniel McAllister
ID: 38835262
Sorry I came late to this one... hope I'm not too late...

Your issue is that you want the output to go both to the screen and to the mail program.

At first glance, I'd suggest using the 'tee' utility (reads standard in, sends a copy to stdout, and another copy to a specified file).... but I don't think that will work, as the real "break" comes from the fact that your output is being directed to a pipe at all.

Still, without having a system to try it on, I might suggest something like:
rsync -r -v --progress /mnt/storage/nfs/ocl_nfs_1/backups/server1/server1-$1 /media/external/server1 | tee /tmp/mailout.tmp ; mail -s "RSync Job Log" iamhere@work.ca < /tmp/mailout.tmp

This would allow the interactive output to the screen, while still capturing it for output to the mail program.

Just a thought...

Dan
IT4SOHO

PS: I won't object to just closing the question ... but if the solution works, it would be sporting to award points :-)
0
 
LVL 1

Author Comment

by:clicker666
ID: 38836533
IT4SOHO almost has it.  Can we reopen?

That does send me an email when done, the only issue is that it comes in as Subject: RSync Job Log, but the attachment gets renamed as RSync Job Log.dat.  Is there a way to specify the attachment name in the command line?  It's almost perfect.
0
 
LVL 21

Expert Comment

by:Daniel McAllister
ID: 38837247
I can stop the cancellation request....

Dan
IT4SOHO
0
 
LVL 21

Accepted Solution

by:
Daniel McAllister earned 1000 total points
ID: 38837290
Hmmm... I'm actually a little surprised that the e-mail content is showing up as an attachment...

Perhaps if we "seed the file" a little (that is, create some interesting first lines)...

Try something like:
echo "" > /tmp/mailout.tmp
echo "Starting backup with rsync at `date +"%r %Z"` >> /tmp/mailout.tmp
rsync -r -v --progress /mnt/storage/nfs/ocl_nfs_1/backups/server1/server1-$1 /media/external/server1 | tee -a /tmp/mailout.tmp
echo "" >> /tmp/mailout.tmp
mail -s "RSync Job Log" iamhere@work.ca < /tmp/mailout.tmp

Once you get this working, you may want to remove the /tmp file at the end :-)

Dan
IT4SOHO
0
 
LVL 1

Author Comment

by:clicker666
ID: 38839483
That works.
0

Featured Post

What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Google Drive is extremely cheap offsite storage, and it's even possible to get extra storage for free for two years.  You can use the free account 15GB, and if you have an Android device..when you install Google Drive for the first time it will give…
This article shows how to use a free utility called 'Parkdale' to easily test the performance and benchmark any Hard Drive(s) installed in your computer. We also look at RAM Disks and their speed comparisons.
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
This tutorial will walk an individual through locating and launching the BEUtility application to properly change the service account username and\or password in situation where it may be necessary or where the password has been inadvertently change…
Suggested Courses

824 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question