[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

Using the rsync command to synchronize files

Posted on 2011-05-03
10
Medium Priority
?
796 Views
Last Modified: 2012-05-11
I was wondering whether there's a way to get the rsync command to actually synchronize files rather than just supporting a one-way mirror. For example, if I push files A, B and C on HOST1 to HOST2, then change file B on HOST2 before pushing the files again, the copy of file B on HOST1 will overwrite the changes I made to file B on HOST2. That is not the behavior I'm looking for. I need a method that will merge the changes between the two hosts. I've heard of Unison, but there's no binary for AIX, and I won't be compiling any sources.
 
0
Comment
Question by:babyb00mer
  • 4
  • 2
  • 2
  • +2
10 Comments
 
LVL 30

Expert Comment

by:Kerem ERSOY
ID: 35517656
Hi,

rsync will not do a bi-directional replication. At most you can add -u switch which will prevent the newer files to be overwritten by rsync.

For unison I am afraid currently no binaries available that I am aware of but you might like to compile your own.

Cheers,
K.
0
 
LVL 30

Expert Comment

by:Kerem ERSOY
ID: 35517680
All I know about the binary distribution is a very old AIX 4.3 binary. I am not sure if you can run it over your system though.

Here's the link for the old binary:

http://www.bullfreeware.com/download/aix43/unison-2.9.1.0.exe

Cheers,
K.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 35517829
I suppose NFS is out of the question?
0
NEW Veeam Agent for Microsoft Windows

Backup and recover physical and cloud-based servers and workstations, as well as endpoint devices that belong to remote users. Avoid downtime and data loss quickly and easily for Windows-based physical or public cloud-based workloads!

 
LVL 18

Expert Comment

by:liddler
ID: 35688218
How about something like running a cronjob, say every minute on each server
on each server, detect a file newer than 1 minute old and rsync it to the other machine

find /myDir -mtime -1 -exec rsync -e ssh -aup /myDir/{} \; otherserver:/myDir  

I haven't checked the syntax, so you will need to test it.

Obviously this might have problems if the same file is updated on both servers within the same minute.... You could increase granularity by doing something like
touch /tmp/flag
sleep 20
find /myDir -newer /tmp/flag............
To check the last 20 seconds, you would probably run this not a a cron job, but in an infinite loop, as cron normally is no less that 1 minute

0
 
LVL 6

Accepted Solution

by:
Tomunique earned 1336 total points
ID: 35689053
As KeremE said.  if the -u flag will prevent newer files from being over-written.
Then run rsync on both systems, pointing to each other.

A will copy to B
B will copy to A


whomever is the most recent version, the other side won't overwrite because of the -u, but the local rsync will copy  it back over to the other system.
0
 

Author Comment

by:babyb00mer
ID: 35701765
Hmmm. That looks like a viable solution. Given that scenario, Is it possible to use the --files-from option? I'd like to set up the command so that it's reading the source files/directories from a file on the disk. I thought something like this might work:

rsync -av --files-from=/directory/file --rsync-path=/usr/bin/rsync user@node:

0
 
LVL 30

Assisted Solution

by:Kerem ERSOY
Kerem ERSOY earned 664 total points
ID: 35702233
I was thinking as Tominque suggested but I was afraid that the second computer rsync operation would disrupt all the files in the first computer. But I've tried myself and observed none would be overwritten

So you can put a command like this in your cron in both systems:

Server1:
rsync -avuz  /some/path  user@server1:/some/path

Server2:
rsync -avuz  /some/path  user@server2:/some/path

Cheers,
K.
0
 
LVL 30

Expert Comment

by:Kerem ERSOY
ID: 35702243
Of course it will also require you to have a passswordles ssh accounts set up at each end.
0
 
LVL 6

Assisted Solution

by:Tomunique
Tomunique earned 1336 total points
ID: 35705532
The --files should work.
I suspect that it would just be used instead of an `ls` of the directory.
It should still do timestamp checks to determine differences.  The fact that it got the file from a file  list, vs a directory list shouldn't matter.

That said, I've not done it before.
0
 

Author Closing Comment

by:babyb00mer
ID: 35725403
The comments by the experts were quite helpful. Incidentally, while I could run many of their examples from the command line of my Sun SPARC running an old version of Solaris 10, the rsync command with the -e option and reading the file list from standard input failed when invoked from within a shell script. Regardless, the solution worked on the target system, so I chalked it up to a quirk in the OS or the shell I was using on the Sun system.
0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Compliance and data security require steps be taken to prevent unauthorized users from copying data.  Here's one method to prevent data theft via USB drives (and writable optical media).
Often times it's very very easy to extend a volume on a Linux instance in AWS, but impossible to shrink it. I wanted to contribute to the experts-exchange community a way of providing a procedure that works on an AWS instance. It can also be used on…
The Task Scheduler is a powerful tool that is built into Windows. It allows you to schedule tasks (actions) on a recurring basis, such as hourly, daily, weekly, monthly, at log on, at startup, on idle, etc. This video Micro Tutorial is a brief intro…
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial
Suggested Courses
Course of the Month20 days, 3 hours left to enroll

873 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question