?
Solved

Backup a remote directory to local computer

Posted on 2004-08-04
14
Medium Priority
?
597 Views
Last Modified: 2010-04-21
I'm sure there is an easy or elegant way to do this:

/Remote_NIX_Machine/directory/ [some files and directories]

/Local_unix_Machine/a_directory/

I would like to run a cronjob/script on the local machine which backs up the files and directories from the remote machine. The files will already have been compressed so I just need to grab them. Ideally, I'd like an optional flag to ignore files with the same name/path as files (from paths given) on the local machine. I have root access, but the remote files all belong to me anyway.
0
Comment
Question by:macgruder
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 7
  • 5
  • 2
14 Comments
 
LVL 21

Expert Comment

by:tfewster
ID: 11718606
Which Unix?  This sounds like a job for rsync!
0
 
LVL 4

Author Comment

by:macgruder
ID: 11718712
The remote machine is Linux. The local is OS 10.3 which has rsync.
0
 
LVL 4

Author Comment

by:macgruder
ID: 11718850
What exact rsync command should I give if I can ssh into the remote server?
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 21

Expert Comment

by:tfewster
ID: 11718987
Please see jlevies answer in http://www.experts-exchange.com/Operating_Systems/Linux/Q_20718942.html

Note that this will update any files that have changed, and you may want to leave out the --delete flag!
0
 
LVL 4

Author Comment

by:macgruder
ID: 11719054
$ rsync --rsh=ssh --delete -Cav master-copy/* user-name@remote.dom.tld:/path-to/slave-copy

That is slightly different in that it backs up to the remote copy, rather than backing up FROM the remote copy. I'm guessing I would change the -Cav option. I'm looking at 3:30am, but if you know the exact command it would be helpful.
0
 
LVL 21

Expert Comment

by:tfewster
ID: 11719227
Oops, I misread your previous comment - I assumed you were going to ssh into the remote server to run the command...Just reverse the source & target, e.g. rsync [options]  Server:/path/test_file  /local_dir

From http://rsync.samba.org/ftp/rsync/rsync.html -
CONNECTING TO AN RSYNC SERVER OVER A REMOTE SHELL PROGRAM
rsync -av --rsh="ssh -l ssh-user" rsync-user@host::module[/path] local-path
0
 
LVL 4

Author Comment

by:macgruder
ID: 11719281
Thanks that seems close. I'll look at it tomorrow. Just a couple of questions:

does the rysnc-user have to be set up on the remote server?
will the above synchronize things?
why are there two colons after host?

Thanks,
0
 
LVL 21

Accepted Solution

by:
tfewster earned 400 total points
ID: 11719762
The two colons are used when the remote system is an rsync server (which allows rules, different users etc. to be set up). You probably don't want that level of complexity, so just use the single-colon syntax.

If you have the same login on both systems, you don't need to specify a user
rsync -Cav  Server:/path/test_file  /local_dir

The sync copies new files, updates existing files(disable with --ignore-existing)  and (optionally) deletes files that no longer exist on the source system. I can't test it at the moment, but the above syntax with a test file is safe to play with (or use the -n option, which just simulates what would be done)
0
 
LVL 38

Expert Comment

by:yuzh
ID: 11722602
"backup
Remote_NIX_Machine/directory/ [some files and directories]
/Local_unix_Machine/a_directory/"

You need to use ssh + tar to copy the files accross for the first time
(for a empty target dir, rsyn runs very slow!)

ssh  login@remotebox "cd /directory; tar cf - filenames dirnames" | (cd /a_directory ; tar xvf -)

IF you want to copy everything under directory, use:
ssh  login@remotebox "cd /directory; tar cf - ." | (cd /a_directory ; tar xvf -)

To achieve faster backup and avoid have to setup ssh login without password,
you can use NFS mount to mount the remote file system to the local machine
and then use rsync.

DIRECTORY=/mount-point-remote-dir
BACKDIR=/target-dir
 /usr/local/bin/rsync --delete -az ${DIRECTORY} ${BACKDIR}

You can do a search at EE to find more infor about how to setup ssh without password,
I have answered it a few times.

0
 
LVL 4

Author Comment

by:macgruder
ID: 11727379
tfewster,

Thanks, I'm very nearly there with:
rsync -av --delete --rsh="ssh -l me" host:/some_path/dir/files /a_path/dir

The main problem is that I'm prompted for the ssh password. If I put this into a script is there anyway to have the password automatically sent?
0
 
LVL 4

Author Comment

by:macgruder
ID: 11727442
The answer to that maybe here:
http://linuxproblem.org/art_9.html
0
 
LVL 21

Expert Comment

by:tfewster
ID: 11731753
Once again, I apologise for missing a key point - that you hadn't got an autologin/trust/keypair relationship set up already.  If you've got it working now, great  - If not, we should get this question reopened and I'm sure yuzh will be able to help you out.
0
 
LVL 38

Expert Comment

by:yuzh
ID: 11732498
Tim, thanks for your good words!

macgruder,
    If you still need help, just post a comment (don't need to reopen it). I think I can
help you out.

   apart from setup ssh without password, you can also use expect script to handle
the password.
    see:
    http://www.experts-exchange.com/Networking/Unix_Networking/Q_20795862.html
   
   In case you need help on setup ssh without password, have a look at:
    http://www.experts-exchange.com/Networking/Linux_Networking/Q_20677059.html


0
 
LVL 4

Author Comment

by:macgruder
ID: 11732669
Thanks a lot guys. The link
http://linuxproblem.org/art_9.html
ended up working perfectly. So everything is working perfectly.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A metadevice consists of one or more devices (slices). It can be expanded by adding slices. Then, it can be grown to fill a larger space while the file system is in use. However, not all UNIX file systems (UFS) can be expanded this way. The conca…
My previous tech tip, Installing the Solaris OS From the Flash Archive On a Tape (http://www.experts-exchange.com/articles/OS/Unix/Solaris/Installing-the-Solaris-OS-From-the-Flash-Archive-on-a-Tape.html), discussed installing the Solaris Operating S…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

649 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question