Solved

Backup a remote directory to local computer

Posted on 2004-08-04
14
571 Views
Last Modified: 2010-04-21
I'm sure there is an easy or elegant way to do this:

/Remote_NIX_Machine/directory/ [some files and directories]

/Local_unix_Machine/a_directory/

I would like to run a cronjob/script on the local machine which backs up the files and directories from the remote machine. The files will already have been compressed so I just need to grab them. Ideally, I'd like an optional flag to ignore files with the same name/path as files (from paths given) on the local machine. I have root access, but the remote files all belong to me anyway.
0
Comment
Question by:macgruder
  • 7
  • 5
  • 2
14 Comments
 
LVL 20

Expert Comment

by:tfewster
ID: 11718606
Which Unix?  This sounds like a job for rsync!
0
 
LVL 4

Author Comment

by:macgruder
ID: 11718712
The remote machine is Linux. The local is OS 10.3 which has rsync.
0
 
LVL 4

Author Comment

by:macgruder
ID: 11718850
What exact rsync command should I give if I can ssh into the remote server?
0
 
LVL 20

Expert Comment

by:tfewster
ID: 11718987
Please see jlevies answer in http://www.experts-exchange.com/Operating_Systems/Linux/Q_20718942.html

Note that this will update any files that have changed, and you may want to leave out the --delete flag!
0
 
LVL 4

Author Comment

by:macgruder
ID: 11719054
$ rsync --rsh=ssh --delete -Cav master-copy/* user-name@remote.dom.tld:/path-to/slave-copy

That is slightly different in that it backs up to the remote copy, rather than backing up FROM the remote copy. I'm guessing I would change the -Cav option. I'm looking at 3:30am, but if you know the exact command it would be helpful.
0
 
LVL 20

Expert Comment

by:tfewster
ID: 11719227
Oops, I misread your previous comment - I assumed you were going to ssh into the remote server to run the command...Just reverse the source & target, e.g. rsync [options]  Server:/path/test_file  /local_dir

From http://rsync.samba.org/ftp/rsync/rsync.html -
CONNECTING TO AN RSYNC SERVER OVER A REMOTE SHELL PROGRAM
rsync -av --rsh="ssh -l ssh-user" rsync-user@host::module[/path] local-path
0
 
LVL 4

Author Comment

by:macgruder
ID: 11719281
Thanks that seems close. I'll look at it tomorrow. Just a couple of questions:

does the rysnc-user have to be set up on the remote server?
will the above synchronize things?
why are there two colons after host?

Thanks,
0
Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 20

Accepted Solution

by:
tfewster earned 100 total points
ID: 11719762
The two colons are used when the remote system is an rsync server (which allows rules, different users etc. to be set up). You probably don't want that level of complexity, so just use the single-colon syntax.

If you have the same login on both systems, you don't need to specify a user
rsync -Cav  Server:/path/test_file  /local_dir

The sync copies new files, updates existing files(disable with --ignore-existing)  and (optionally) deletes files that no longer exist on the source system. I can't test it at the moment, but the above syntax with a test file is safe to play with (or use the -n option, which just simulates what would be done)
0
 
LVL 38

Expert Comment

by:yuzh
ID: 11722602
"backup
Remote_NIX_Machine/directory/ [some files and directories]
/Local_unix_Machine/a_directory/"

You need to use ssh + tar to copy the files accross for the first time
(for a empty target dir, rsyn runs very slow!)

ssh  login@remotebox "cd /directory; tar cf - filenames dirnames" | (cd /a_directory ; tar xvf -)

IF you want to copy everything under directory, use:
ssh  login@remotebox "cd /directory; tar cf - ." | (cd /a_directory ; tar xvf -)

To achieve faster backup and avoid have to setup ssh login without password,
you can use NFS mount to mount the remote file system to the local machine
and then use rsync.

DIRECTORY=/mount-point-remote-dir
BACKDIR=/target-dir
 /usr/local/bin/rsync --delete -az ${DIRECTORY} ${BACKDIR}

You can do a search at EE to find more infor about how to setup ssh without password,
I have answered it a few times.

0
 
LVL 4

Author Comment

by:macgruder
ID: 11727379
tfewster,

Thanks, I'm very nearly there with:
rsync -av --delete --rsh="ssh -l me" host:/some_path/dir/files /a_path/dir

The main problem is that I'm prompted for the ssh password. If I put this into a script is there anyway to have the password automatically sent?
0
 
LVL 4

Author Comment

by:macgruder
ID: 11727442
The answer to that maybe here:
http://linuxproblem.org/art_9.html
0
 
LVL 20

Expert Comment

by:tfewster
ID: 11731753
Once again, I apologise for missing a key point - that you hadn't got an autologin/trust/keypair relationship set up already.  If you've got it working now, great  - If not, we should get this question reopened and I'm sure yuzh will be able to help you out.
0
 
LVL 38

Expert Comment

by:yuzh
ID: 11732498
Tim, thanks for your good words!

macgruder,
    If you still need help, just post a comment (don't need to reopen it). I think I can
help you out.

   apart from setup ssh without password, you can also use expect script to handle
the password.
    see:
    http://www.experts-exchange.com/Networking/Unix_Networking/Q_20795862.html
   
   In case you need help on setup ssh without password, have a look at:
    http://www.experts-exchange.com/Networking/Linux_Networking/Q_20677059.html


0
 
LVL 4

Author Comment

by:macgruder
ID: 11732669
Thanks a lot guys. The link
http://linuxproblem.org/art_9.html
ended up working perfectly. So everything is working perfectly.
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

Let's say you need to move the data of a file system from one partition to another. This generally involves dismounting the file system, backing it up to tapes, and restoring it to a new partition. You may also copy the file system from one place to…
Using libpcap/Jpcap to capture and send packets on Solaris version (10/11) Library used: 1.      Libpcap (http://www.tcpdump.org) Version 1.2 2.      Jpcap(http://netresearch.ics.uci.edu/kfujii/Jpcap/doc/index.html) Version 0.6 Prerequisite: 1.      GCC …
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now