macgruder
asked on
Backup a remote directory to local computer
I'm sure there is an easy or elegant way to do this:
/Remote_NIX_Machine/direct
/Local_unix_Machine/a_dire
I would like to run a cronjob/script on the local machine which backs up the files and directories from the remote machine. The files will already have been compressed so I just need to grab them. Ideally, I'd like an optional flag to ignore files with the same name/path as files (from paths given) on the local machine. I have root access, but the remote files all belong to me anyway.
/Remote_NIX_Machine/direct
/Local_unix_Machine/a_dire
I would like to run a cronjob/script on the local machine which backs up the files and directories from the remote machine. The files will already have been compressed so I just need to grab them. Ideally, I'd like an optional flag to ignore files with the same name/path as files (from paths given) on the local machine. I have root access, but the remote files all belong to me anyway.
tfewster
Which Unix? This sounds like a job for rsync!
ASKER
The remote machine is Linux. The local is OS 10.3 which has rsync.
ASKER
What exact rsync command should I give if I can ssh into the remote server?
Please see jlevies answer in https://www.experts-exchange.com/questions/20718942/Backup-over-ssh-only-new-or-edited-files-recursive.html
Note that this will update any files that have changed, and you may want to leave out the --delete flag!
Note that this will update any files that have changed, and you may want to leave out the --delete flag!
ASKER
$ rsync --rsh=ssh --delete -Cav master-copy/* user-name@remote.dom.tld:/
That is slightly different in that it backs up to the remote copy, rather than backing up FROM the remote copy. I'm guessing I would change the -Cav option. I'm looking at 3:30am, but if you know the exact command it would be helpful.
That is slightly different in that it backs up to the remote copy, rather than backing up FROM the remote copy. I'm guessing I would change the -Cav option. I'm looking at 3:30am, but if you know the exact command it would be helpful.
Oops, I misread your previous comment - I assumed you were going to ssh into the remote server to run the command...Just reverse the source & target, e.g. rsync [options] Server:/path/test_file /local_dir
From http://rsync.samba.org/ftp/rsync/rsync.html -
CONNECTING TO AN RSYNC SERVER OVER A REMOTE SHELL PROGRAM
rsync -av --rsh="ssh -l ssh-user" rsync-user@host::module[/p
From http://rsync.samba.org/ftp/rsync/rsync.html -
CONNECTING TO AN RSYNC SERVER OVER A REMOTE SHELL PROGRAM
rsync -av --rsh="ssh -l ssh-user" rsync-user@host::module[/p
ASKER
Thanks that seems close. I'll look at it tomorrow. Just a couple of questions:
does the rysnc-user have to be set up on the remote server?
will the above synchronize things?
why are there two colons after host?
Thanks,
does the rysnc-user have to be set up on the remote server?
will the above synchronize things?
why are there two colons after host?
Thanks,
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
"backup
Remote_NIX_Machine/directo
/Local_unix_Machine/a_dire
You need to use ssh + tar to copy the files accross for the first time
(for a empty target dir, rsyn runs very slow!)
ssh login@remotebox "cd /directory; tar cf - filenames dirnames" | (cd /a_directory ; tar xvf -)
IF you want to copy everything under directory, use:
ssh login@remotebox "cd /directory; tar cf - ." | (cd /a_directory ; tar xvf -)
To achieve faster backup and avoid have to setup ssh login without password,
you can use NFS mount to mount the remote file system to the local machine
and then use rsync.
DIRECTORY=/mount-point-rem
BACKDIR=/target-dir
/usr/local/bin/rsync --delete -az ${DIRECTORY} ${BACKDIR}
You can do a search at EE to find more infor about how to setup ssh without password,
I have answered it a few times.
Remote_NIX_Machine/directo
/Local_unix_Machine/a_dire
You need to use ssh + tar to copy the files accross for the first time
(for a empty target dir, rsyn runs very slow!)
ssh login@remotebox "cd /directory; tar cf - filenames dirnames" | (cd /a_directory ; tar xvf -)
IF you want to copy everything under directory, use:
ssh login@remotebox "cd /directory; tar cf - ." | (cd /a_directory ; tar xvf -)
To achieve faster backup and avoid have to setup ssh login without password,
you can use NFS mount to mount the remote file system to the local machine
and then use rsync.
DIRECTORY=/mount-point-rem
BACKDIR=/target-dir
/usr/local/bin/rsync --delete -az ${DIRECTORY} ${BACKDIR}
You can do a search at EE to find more infor about how to setup ssh without password,
I have answered it a few times.
ASKER
tfewster,
Thanks, I'm very nearly there with:
rsync -av --delete --rsh="ssh -l me" host:/some_path/dir/files /a_path/dir
The main problem is that I'm prompted for the ssh password. If I put this into a script is there anyway to have the password automatically sent?
Thanks, I'm very nearly there with:
rsync -av --delete --rsh="ssh -l me" host:/some_path/dir/files /a_path/dir
The main problem is that I'm prompted for the ssh password. If I put this into a script is there anyway to have the password automatically sent?
ASKER
The answer to that maybe here:
http://linuxproblem.org/art_9.html
http://linuxproblem.org/art_9.html
Once again, I apologise for missing a key point - that you hadn't got an autologin/trust/keypair relationship set up already. If you've got it working now, great - If not, we should get this question reopened and I'm sure yuzh will be able to help you out.
Tim, thanks for your good words!
macgruder,
If you still need help, just post a comment (don't need to reopen it). I think I can
help you out.
apart from setup ssh without password, you can also use expect script to handle
the password.
see:
https://www.experts-exchange.com/questions/20795862/SSH-Login-Script.html
In case you need help on setup ssh without password, have a look at:
https://www.experts-exchange.com/questions/20677059/ssh-not-permitting-login-without-password-despite-key-generation.html
macgruder,
If you still need help, just post a comment (don't need to reopen it). I think I can
help you out.
apart from setup ssh without password, you can also use expect script to handle
the password.
see:
https://www.experts-exchange.com/questions/20795862/SSH-Login-Script.html
In case you need help on setup ssh without password, have a look at:
https://www.experts-exchange.com/questions/20677059/ssh-not-permitting-login-without-password-despite-key-generation.html
ASKER
Thanks a lot guys. The link
http://linuxproblem.org/art_9.html
ended up working perfectly. So everything is working perfectly.
http://linuxproblem.org/art_9.html
ended up working perfectly. So everything is working perfectly.