Need to keep files synchronized over vpn tunnel

Posted on 2009-07-01
Last Modified: 2012-05-07
I have a client who wants to syncrhonize files over a permanent branch office ipsec vpn tunnel.
For example, if he makes a change to a file at Site A, the change will be propogated to the same file at Site B.
Also, if he adds a fiile at Site A, the file should be added to Site B.
Of course, if he deletes a file at Site A, the file should be deleted at Site B.
This applies in the reverse direction too, i.e.  changes at Site B are replicated to Site A.

The primary purpose of ths is to back up files, but I do not want to use a backup program because I want the client to be be able to see the state of the files just by looking at the filesyste; not just a big. bkf or .arc or other backup file.

The file sync will  be between a 2003 SBS Std. Server and an XP machine (not sure if Home or Pro).

This program must be user friendlyl, as a non-technical user will be using this.

Software that is free is preferred.

Thank  you!
Question by:cliffordgormley
  • 3
  • 2
LVL 14

Expert Comment

by:Hedley Phillips
ID: 24761349
Hi, We use and recommend RoboCopy.

Let me find the details of another question where I have detailed the application and its use...
LVL 14

Accepted Solution

Hedley Phillips earned 455 total points
ID: 24761361
Here we are, I have copied and pasted my long comment in the other question.

I will also give you the link to that question so that you can see other members recommendations as well as mine, just because I love RoboCopy doesn't mean it will suit everyone.:

RoboCopy is a Windows DOS command and stands for Robust File Copy and is way more robust at remote data backups than XCopy.

have a read of the Wiki:

It used to be available as part of the Windows Resource Kit, and was introduced as a standard feature of Windows Vista and Windows Server 2008.

A quick quote from the Wiki site which also contains a list of all the switches.

"Robocopy is designed for reliable mirroring of directories or directory trees. It has features to ensure all NTFS attributes and properties are copied, and includes additional restart code for network connections subject to disruption."


Small (76Kb exe) and FREE,  that we stuff into the same directory as the batch file and the log files so we don't even need to bother with adding it to the environment variables as I am lazy.

An example of the batch file:

ROBOCOPY \\appserv1\C$\XPP\xz\scripts\ C:\XPP\xz\scripts\ /E /B /PURGE /R:3 /W:3  /NP /TEE /NJH /NJS /NS

Basically it takes the format of

ROBOCOPY <source> <destination> switches

The switches we use here are:

/E which copies subdirectories, even empty ones
/B Opens files in backup mode. Backup mode allows files to be opened without security restrictions, but requires additional rights on both the source and destination systems.
/PURGE  Deletes files no longer in the source location, this makes sure we have a mirror copy of the source
/R:3 Tells Robocopy to retry 'n' times before giving up in the event of error (default is 1 million). Handy otherwise it will sit trying to copy a file a million times. We found that 3 is a decent number as it caters for network blips
/W:3  Wait time between retries (default is 30 seconds). This gives us 3 seconds between retries and in conjunction with the previous switch, 3 retries over a period of 9 seconds.

Now we are on to Logging switches which we use to give us readable and useful log files

/NP No progress  don't display % copied.
/TEE Output to console window, as well as the log file. (so the user knows that something is happening, all about user confidence)
/NJH  No job header. Keeps the logs easy to read

There are plenty of other switches you can use but these do exactly what we want.

I run this batch file from a scheduled task:

"C:\Program Files\RoboCopy\Xpp_Sync\XPP_Sync.bat" > XPP_RoboCopy_%date:/=-%.log 2>&1

and what this gives me is a daily log file dated as such, ie: XPPRoboCopy_day-month-year.log

eg: XPP_RoboCopy_30-06-2009.log

I will paste an extract of the log file so you can see what sort of information you can get out of it in the code window:

As you can see it reports if the file is newer, older or an Extra file.

Ok, how and why did we get to be using Robocopy?

We started off using xcopy but found it totally and utterly lacking in the capabilities we required which were namely to be able to tolerate network outages and pickup where it had left off due to timestamps.

We then moved to RSync, a Windows port of an Unix utility. This started off well but was a right !"!*!£ to get working. It also didn't deal with large volumes of data over a long period of time (eg I also use Robocopy to copy my SAN Snapshot from one LUN to another and this is well over 1TB). I also found it didn't work well over the ADSL line and would find data not being backed up but no obvious reason for it.

This is when we moved over to Robocopy and after extensive testing have found it to be the utility of choice.

As I said in an earlier post, it is very user friendly as it can be tucked away in a batch file and run from either a shortcut or scheduled task.

Hope this helps. If you need any more info just say.
   ROBOCOPY     ::     Robust File Copy for Windows     ::     Version XP010
  Started : Tue Jun 30 17:01:00 2009
   Source : \\appserv1\C$\XPP\xz\procs\
     Dest : C:\XPP\xz\procs\
    Files : *.*
  Options : *.* /NS /NJS /TEE /S /E /COPY:DAT /PURGE /B /NP /R:3 /W:3 
          *EXTRA File                   1bgq_config.sde
            Older                       _bgq_config.sde
            Older                       _im_config.sde
            Older                       _sap_config.sde
            Older                       1im_config.sde
            Older                       1sap_config.sde
                        \\appserv1\C$\XPP\xz\sys\Daily Logs\
            New File                    xppws.2009-Jun-29.xml
            Older                       omnimark.lib
etc etc

Open in new window


Author Comment

ID: 24777182
AWESOME answer!!!  Thank you!

Author Closing Comment

ID: 31598745
Best answer I have ever gotten on EE
LVL 14

Expert Comment

by:Hedley Phillips
ID: 24783573
Glad it was of use. We really like Robocopy here. I hope you find it as useful.

Thanks for the points.

Featured Post

Complete VMware vSphere® ESX(i) & Hyper-V Backup

Capture your entire system, including the host, with patented disk imaging integrated with VMware VADP / Microsoft VSS and RCT. RTOs is as low as 15 seconds with Acronis Active Restore™. You can enjoy unlimited P2V/V2V migrations from any source (even from a different hypervisor)

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

VM backups can be lost due to a number of reasons: accidental backup deletion, backup file corruption, disk failure, lost or stolen hardware, malicious attack, or due to some other undesired and unpredicted event. Thus, having more than one copy of …
In this article we will learn how to backup a VMware farm using Nakivo Backup & Replication. In this tutorial we will install the software on a Windows 2012 R2 Server.
This tutorial will walk an individual through configuring a drive on a Windows Server 2008 to perform shadow copies in order to quickly recover deleted files and folders. Click on Start and then select Computer to view the available drives on the se…
This tutorial will walk an individual through setting the global and backup job media overwrite and protection periods in Backup Exec 2012. Log onto the Backup Exec Central Administration Server. Examine the services. If all or most of them are stop…

856 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question