Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Need to sync two remote file systems from a unix box.

Posted on 2003-11-06
15
Medium Priority
?
2,190 Views
Last Modified: 2013-12-06
I have two snap servers, each has about 6 gb of data in a directory that I would like to replicate to the other.  The two snaps and the unix (sco) are on the same wan.  No problem to connect, all three see each other.  I need to use the unix box because I want the replication to take place each night and it is the only operating system that stays running through the night.

Rsync won't work because it requires a client and server. I only have one OS to make all the descisions and move the new/changed files.

Any idea what I can use? I have scoured the net for days and cannot find something so simple....

I'm giving max points because I need a valid working answer, not just a direction to looking.

Thanks.

0
Comment
Question by:jasgot
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 6
  • 3
15 Comments
 
LVL 38

Accepted Solution

by:
yuzh earned 2000 total points
ID: 9699135
If you only want to mirror a data dir, rsync should be able to do the job. and
you don't need to have a client and server setup.

For security reason, you should use ssh + rsync to do the job.

Have a look at the following page, it is talking about Solaris, but you can implement it
in SCO:

http://oldlook.experts-exchange.com/Operating_Systems/Solaris/Q_20349722.html

also:
http://www.mids.org/pay/mn/801/rsync.html
http://www.tek-tips.com/gpviewthread.cfm/qid/693807/pid/54/lev2/3/lev3/20
http://www.aplawrence.com/Bofcusm/2154.html

You can also find a lot of example about how to use "rsync + ssh" at EE.

Just read through the docs, if you want more help, post a comment.
0
 
LVL 62

Expert Comment

by:gheist
ID: 9699449
Do not fear if rsync takes long for first time - if 90% of your data stays over the day, rsync will not transfer that.
You can start rsync server from command line or via sudo without background daemon running.
ssh and rsync are to be found at caldera.com/skunkware
0
 
LVL 38

Expert Comment

by:yuzh
ID: 9699670
For the first time, it is better to use ssh + tar to do the job, and  then start from the 2nd use
ssh + rsync it will run a lot faster !
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 

Author Comment

by:jasgot
ID: 9700508
The data has already been manually replicated, so there should be very few files to handle.

So you are telling me I can run

rsync //snap1/dirx //snap2/dirx
rsync //snap2/diry //snap1/diry

without the rsyncd running anywhere and the source AND destination do not have to be a xNIX box?  This goes against everything I have read for weeks......
0
 
LVL 62

Expert Comment

by:gheist
ID: 9700820
rsync --rsh="/usr/bin/ssh -l luser /usr/bin/sudo /usr/local/bin/rsync --daemon --server ." -vz root@remotehost::module_from_remote_config/relative_path /somewhere/here
0
 
LVL 38

Expert Comment

by:yuzh
ID: 9700972
"The data has already been manually replicated, so there should be very few files to handle."

That's a good news! rsync will take a very short time to mirror the two dirs.
I use rsync to mirror more than 120 GB date every night, and I 'm happy with
the result.

If you two boxes are in the same network to can use NFS mount to mount the
remote box to the one which the backup script will be run and make sure it has
root write permission to the remote filesystem , and than use, the
follow systax (use the real path to rsync instead of /usr/local/bin)

 /usr/local/bin/rsync --delete -az ${DIRECTORY} ${BACKDIR}

If you can not use NFS mount, you need to use ssh + rsync:

/usr/local/bin/rsync -e ssh --delete -az root@remote:/home/yuzh/test1 /home/yuzh/test1
http://oldlook.experts-exchange.com/Operating_Systems/Solaris/Q_20505177.html

man rsync

to learn more
0
 

Author Comment

by:jasgot
ID: 9701611
The two drives that are not running an OS are on different subnets of our network, all within a VPN.

I can get to any device on the other network by IP, I have not tried to do it by \\UNC.  I think I can NFS mount to \\IPADDRESS\share.

Next,

I am in a secure VPN, can I skip the ssh?  I am not even sure if the snap drive allows ssh.



0
 
LVL 38

Expert Comment

by:yuzh
ID: 9706089
IF you can use NFS mount, then you don't need to use ssh, the syntax would looks like: (eg, mirror /export/data):

cd /export
/usr/local/bin/rsync --delete -az data /IP/share

In case you need to use ssh, you need to install ssh in both boxes. please read throught the example we have posted.



0
 

Author Comment

by:jasgot
ID: 9708603
SSH is not an option on a SNAP server. www.snapappliance.com So I hope I can NFS through the gateway on our WAN VPN.

Your example above makes me think you beleive the export data is on the same box as the rsync utility.  Please be mindful of my first post indicating that the syncronization has to be managed from the ONLY xNIX box and the two file systems that need to be replicated are snap servers without any type of OS that I can manage.
0
 

Author Comment

by:jasgot
ID: 9708771
OK. I'm going to test the rsync right now. I'll be right back.
0
 
LVL 38

Expert Comment

by:yuzh
ID: 9712471
"Your example above makes me think you beleive the export data is on the same box as the rsync utility.  Please be mindful of my first post indicating that the syncronization has to be managed from the ONLY xNIX box and the two file systems that need to be replicated are snap servers without any type of OS that I can manage. "

When you setup NFS mount to mount the remote on the machine you run rsync (if you want to write to
the mount filesystem, the NFS file system has to be exported with READ and write permission)

rsync treat the two dirs as local dir.
very easy!
0
 

Author Comment

by:jasgot
ID: 9714607
As it turns out, RSYNC is not installed on this SCO box, and to get it installed, would require installing many service level patches and the GCC.  Since this box is supported by another party, I have to see if they will make sure it is at current service levels and if I can install the GCC.

Unless I can find a compiled version of RSYNC for SCO OS 5.0.5.

0
 
LVL 38

Expert Comment

by:yuzh
ID: 9715072
0
 

Author Comment

by:jasgot
ID: 9732501
Although I am unable to get rsync installed.  This would have indeed work well for me.
0
 
LVL 62

Expert Comment

by:gheist
ID: 9746088
gnu packages need recent libm.so, ie libm, libc and kernel, they may break a lot of apps (but never NFS mounts)
if you have two nfs mounts syncing by any means will transfer all data to both nfs servers, you need rsync right where your data lies, rsync computes checksums of data blocks and transfers them over wires where rsync protocol lies (ssh, rsh etc). so it comes out that you need rsync daemon available where your "master" data lies, if this is not an option, you transfer all.
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When you do backups in the Solaris Operating System, the file system must be inactive. Otherwise, the output may be inconsistent. A file system is inactive when it's unmounted or it's write-locked by the operating system. Although the fssnap utility…
In tuning file systems on the Solaris Operating System, changing some parameters of a file system usually destroys the data on it. For instance, changing the cache segment block size in the volume of a T3 requires that you delete the existing volu…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

715 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question