Windows robocopy / mir  or rsync or ? to sync 'open files' reliably in real-time to remote site with little bandwidth

sunhux
sunhux used Ask the Experts™
on
I have a 180GB share drive in Production site where users publish or
upload .xps & .pdf files to it.  I have a 30Mbps WAN link to a remote
(& supposedly 'hot' ) DR site.

As these files are being created/written, they're probably 'open'
or 'locked' but I would still want them to be 'sync'ed over to remote
end asap.

Should there be a failure in copying, the tool should resume back on
its own.

So, is Robocopy or rsync or ...... (recommend which other Opensource freeware tool)....
the best for this case?  May not necessarily have a permanent VPN ?


Must the remote end share out a file service to the local server to
map to (preferably not), as this NetBIOS file sharing over WAN takes
a lot of resource.  Pls recommend
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Have you considered adding the /R:0 switch to your robocopy command to bypass the files that are open?

Example:

robocopy D:\HR_data Z:\HR_data /R:0 /MIR

Robocopy will provide you a report upon completion. Once the files are closed, you can rerun the command so it can pick up any differentials missed since the initial sync.
Commented:
use /ZB switch (use restartable mode; if access denied use Backup mode)
/r:1 /w:1 (retry once, wait 1 second)

if you launch a robocopy job from the source site, you can consider using /maxage:1 to check only files that are less than 1 day old

/MIR for mirroring

repeat the job every x time (depending on the requirements) and log to a local file

remote end needs a share to copy to

Author

Commented:
Many thanks chaps.

>remote end needs a share to copy to
If the remote end has a storage with NFS, can I map to the
NFS share & robocopy to it?

If there's no share, what other means we can enable writing to the
remote end?  (use 'scp' or 'sftp' or VPN tunnel ...... ?? )

>use /ZB switch (use restartable mode; if access denied use Backup mode)
So if the robocopy sync'ing run into errors, it will restart itself & if 'access
denied' (what's this??), it will retry sync'ing using "backup" mode (what's this?)?

Will 'Backup' mode retains the creation/modification dates+timings &
the ownership (+security settings ) of the files sync'ed across?
Success in ‘20 With a Profitable Pricing Strategy

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

Commented:
ntfs share: yes, if the firewall allows it

if there is no share, you can consider FTP or HTTP upload but then robocopy cannot be used

from http://ss64.com/nt/robocopy.html
/B (backup mode) will allow Robocopy to override file and folder permission settings (ACLs).

if the target uses the same SIDS, then yes (check /COPY:DATS)

Author

Commented:
I have Robocopy version XP010 meant for MS® Windows® Server 2003,
MS® Windows® XP and MS® Windows® 2000.  What's the version for
MS Windows 2008 Server R2 (x64) & what's the url to download it ?

Can the following switches help better, ie /MON:1 (after 1 change) & /MOT:0.01 :
Use the /MON:n switch to specify the minimum number of changes that must occur
  before running again.
Use the /MOT:m switch to specify the minimum time, in minutes, that must elapse
  before running again.
Commented:
it's included in 2008R2

the MON and MOT switches should work but haven't tested them as of yet
but if you implement that, monitor your system resources

Author

Commented:
>ntfs share: yes, if the firewall allows it
I meant NFS (Netwk File Share, not ntfs which is Windows new tech filesystem)

Is there any way I can use ssh+scp+rsync jointly to sync over delta changes
& if so what's the exact syntax ?
I would use DFS replication

if you were on Linux, you could use incrond and rsync/unison

while robocopy is great in a LAN environment, latency will have a significant effect on performance.

you could also look at synctoy from Microsoft.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial