[Last Call] Learn about multicloud storage options and how to improve your company's cloud strategy. Register Now


Need to keep files synchronized over vpn tunnel

Posted on 2009-07-01
Medium Priority
Last Modified: 2012-05-07
I have a client who wants to syncrhonize files over a permanent branch office ipsec vpn tunnel.
For example, if he makes a change to a file at Site A, the change will be propogated to the same file at Site B.
Also, if he adds a fiile at Site A, the file should be added to Site B.
Of course, if he deletes a file at Site A, the file should be deleted at Site B.
This applies in the reverse direction too, i.e.  changes at Site B are replicated to Site A.

The primary purpose of ths is to back up files, but I do not want to use a backup program because I want the client to be be able to see the state of the files just by looking at the filesyste; not just a big. bkf or .arc or other backup file.

The file sync will  be between a 2003 SBS Std. Server and an XP machine (not sure if Home or Pro).

This program must be user friendlyl, as a non-technical user will be using this.

Software that is free is preferred.

Thank  you!
Question by:cliffordgormley
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
LVL 14

Expert Comment

by:Hedley Phillips
ID: 24761349
Hi, We use and recommend RoboCopy.

Let me find the details of another question where I have detailed the application and its use...
LVL 14

Accepted Solution

Hedley Phillips earned 1820 total points
ID: 24761361
Here we are, I have copied and pasted my long comment in the other question.

I will also give you the link to that question so that you can see other members recommendations as well as mine, just because I love RoboCopy doesn't mean it will suit everyone.:


RoboCopy is a Windows DOS command and stands for Robust File Copy and is way more robust at remote data backups than XCopy.

have a read of the Wiki: http://en.wikipedia.org/wiki/Robocopy

It used to be available as part of the Windows Resource Kit, and was introduced as a standard feature of Windows Vista and Windows Server 2008.

A quick quote from the Wiki site which also contains a list of all the switches.

"Robocopy is designed for reliable mirroring of directories or directory trees. It has features to ensure all NTFS attributes and properties are copied, and includes additional restart code for network connections subject to disruption."


Small (76Kb exe) and FREE,  that we stuff into the same directory as the batch file and the log files so we don't even need to bother with adding it to the environment variables as I am lazy.

An example of the batch file:

ROBOCOPY \\appserv1\C$\XPP\xz\scripts\ C:\XPP\xz\scripts\ /E /B /PURGE /R:3 /W:3  /NP /TEE /NJH /NJS /NS

Basically it takes the format of

ROBOCOPY <source> <destination> switches

The switches we use here are:

/E which copies subdirectories, even empty ones
/B Opens files in backup mode. Backup mode allows files to be opened without security restrictions, but requires additional rights on both the source and destination systems.
/PURGE  Deletes files no longer in the source location, this makes sure we have a mirror copy of the source
/R:3 Tells Robocopy to retry 'n' times before giving up in the event of error (default is 1 million). Handy otherwise it will sit trying to copy a file a million times. We found that 3 is a decent number as it caters for network blips
/W:3  Wait time between retries (default is 30 seconds). This gives us 3 seconds between retries and in conjunction with the previous switch, 3 retries over a period of 9 seconds.

Now we are on to Logging switches which we use to give us readable and useful log files

/NP No progress  don't display % copied.
/TEE Output to console window, as well as the log file. (so the user knows that something is happening, all about user confidence)
/NJH  No job header. Keeps the logs easy to read

There are plenty of other switches you can use but these do exactly what we want.

I run this batch file from a scheduled task:

"C:\Program Files\RoboCopy\Xpp_Sync\XPP_Sync.bat" > XPP_RoboCopy_%date:/=-%.log 2>&1

and what this gives me is a daily log file dated as such, ie: XPPRoboCopy_day-month-year.log

eg: XPP_RoboCopy_30-06-2009.log

I will paste an extract of the log file so you can see what sort of information you can get out of it in the code window:

As you can see it reports if the file is newer, older or an Extra file.

Ok, how and why did we get to be using Robocopy?

We started off using xcopy but found it totally and utterly lacking in the capabilities we required which were namely to be able to tolerate network outages and pickup where it had left off due to timestamps.

We then moved to RSync, a Windows port of an Unix utility. This started off well but was a right !"!*!£ to get working. It also didn't deal with large volumes of data over a long period of time (eg I also use Robocopy to copy my SAN Snapshot from one LUN to another and this is well over 1TB). I also found it didn't work well over the ADSL line and would find data not being backed up but no obvious reason for it.

This is when we moved over to Robocopy and after extensive testing have found it to be the utility of choice.

As I said in an earlier post, it is very user friendly as it can be tucked away in a batch file and run from either a shortcut or scheduled task.

Hope this helps. If you need any more info just say.
   ROBOCOPY     ::     Robust File Copy for Windows     ::     Version XP010
  Started : Tue Jun 30 17:01:00 2009
   Source : \\appserv1\C$\XPP\xz\procs\
     Dest : C:\XPP\xz\procs\
    Files : *.*
  Options : *.* /NS /NJS /TEE /S /E /COPY:DAT /PURGE /B /NP /R:3 /W:3 
          *EXTRA File                   1bgq_config.sde
            Older                       _bgq_config.sde
            Older                       _im_config.sde
            Older                       _sap_config.sde
            Older                       1im_config.sde
            Older                       1sap_config.sde
                        \\appserv1\C$\XPP\xz\sys\Daily Logs\
            New File                    xppws.2009-Jun-29.xml
            Older                       omnimark.lib
etc etc

Open in new window


Author Comment

ID: 24777182
AWESOME answer!!!  Thank you!

Author Closing Comment

ID: 31598745
Best answer I have ever gotten on EE
LVL 14

Expert Comment

by:Hedley Phillips
ID: 24783573
Glad it was of use. We really like Robocopy here. I hope you find it as useful.

Thanks for the points.

Featured Post

Tech or Treat! - Giveaway

Submit an article about your scariest tech experience—and the solution—and you’ll be automatically entered to win one of 4 fantastic tech gadgets.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Storage devices are generally used to save the data or sometime transfer the data from one computer system to another system. However, sometimes user accidentally erased their important data from the Storage devices. Users have to know how data reco…
Backups and Disaster RecoveryIn this post, we’ll look at strategies for backups and disaster recovery.
This tutorial will walk an individual through locating and launching the BEUtility application to properly change the service account username and\or password in situation where it may be necessary or where the password has been inadvertently change…
This tutorial will walk an individual through the process of installing of Data Protection Manager on a server running Windows Server 2012 R2, including the prerequisites. Microsoft .Net 3.5 is required. To install this feature, go to Server Manager…

650 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question