Solved

Need to keep files synchronized over vpn tunnel

Posted on 2009-07-01
5
430 Views
Last Modified: 2012-05-07
Hello,
I have a client who wants to syncrhonize files over a permanent branch office ipsec vpn tunnel.
For example, if he makes a change to a file at Site A, the change will be propogated to the same file at Site B.
Also, if he adds a fiile at Site A, the file should be added to Site B.
Of course, if he deletes a file at Site A, the file should be deleted at Site B.
This applies in the reverse direction too, i.e.  changes at Site B are replicated to Site A.

The primary purpose of ths is to back up files, but I do not want to use a backup program because I want the client to be be able to see the state of the files just by looking at the filesyste; not just a big. bkf or .arc or other backup file.

The file sync will  be between a 2003 SBS Std. Server and an XP machine (not sure if Home or Pro).


This program must be user friendlyl, as a non-technical user will be using this.

Software that is free is preferred.

Thank  you!
0
Comment
Question by:cliffordgormley
  • 3
  • 2
5 Comments
 
LVL 14

Expert Comment

by:Hedley Phillips
ID: 24761349
Hi, We use and recommend RoboCopy.

Let me find the details of another question where I have detailed the application and its use...
0
 
LVL 14

Accepted Solution

by:
Hedley Phillips earned 455 total points
ID: 24761361
Here we are, I have copied and pasted my long comment in the other question.

I will also give you the link to that question so that you can see other members recommendations as well as mine, just because I love RoboCopy doesn't mean it will suit everyone.:

http://www.experts-exchange.com/Storage/Backup_Restore/Q_24533331.html


RoboCopy is a Windows DOS command and stands for Robust File Copy and is way more robust at remote data backups than XCopy.

have a read of the Wiki: http://en.wikipedia.org/wiki/Robocopy

It used to be available as part of the Windows Resource Kit, and was introduced as a standard feature of Windows Vista and Windows Server 2008.

A quick quote from the Wiki site which also contains a list of all the switches.

"Robocopy is designed for reliable mirroring of directories or directory trees. It has features to ensure all NTFS attributes and properties are copied, and includes additional restart code for network connections subject to disruption."

Robocopy:

Small (76Kb exe) and FREE,  that we stuff into the same directory as the batch file and the log files so we don't even need to bother with adding it to the environment variables as I am lazy.

An example of the batch file:

ROBOCOPY \\appserv1\C$\XPP\xz\scripts\ C:\XPP\xz\scripts\ /E /B /PURGE /R:3 /W:3  /NP /TEE /NJH /NJS /NS

Basically it takes the format of

ROBOCOPY <source> <destination> switches

The switches we use here are:

/E which copies subdirectories, even empty ones
/B Opens files in backup mode. Backup mode allows files to be opened without security restrictions, but requires additional rights on both the source and destination systems.
/PURGE  Deletes files no longer in the source location, this makes sure we have a mirror copy of the source
/R:3 Tells Robocopy to retry 'n' times before giving up in the event of error (default is 1 million). Handy otherwise it will sit trying to copy a file a million times. We found that 3 is a decent number as it caters for network blips
/W:3  Wait time between retries (default is 30 seconds). This gives us 3 seconds between retries and in conjunction with the previous switch, 3 retries over a period of 9 seconds.

Now we are on to Logging switches which we use to give us readable and useful log files

/NP No progress  don't display % copied.
/TEE Output to console window, as well as the log file. (so the user knows that something is happening, all about user confidence)
/NJH  No job header. Keeps the logs easy to read

There are plenty of other switches you can use but these do exactly what we want.

I run this batch file from a scheduled task:

"C:\Program Files\RoboCopy\Xpp_Sync\XPP_Sync.bat" > XPP_RoboCopy_%date:/=-%.log 2>&1

and what this gives me is a daily log file dated as such, ie: XPPRoboCopy_day-month-year.log

eg: XPP_RoboCopy_30-06-2009.log

I will paste an extract of the log file so you can see what sort of information you can get out of it in the code window:

As you can see it reports if the file is newer, older or an Extra file.

Ok, how and why did we get to be using Robocopy?

We started off using xcopy but found it totally and utterly lacking in the capabilities we required which were namely to be able to tolerate network outages and pickup where it had left off due to timestamps.

We then moved to RSync, a Windows port of an Unix utility. This started off well but was a right !"!*!£ to get working. It also didn't deal with large volumes of data over a long period of time (eg I also use Robocopy to copy my SAN Snapshot from one LUN to another and this is well over 1TB). I also found it didn't work well over the ADSL line and would find data not being backed up but no obvious reason for it.

This is when we moved over to Robocopy and after extensive testing have found it to be the utility of choice.

As I said in an earlier post, it is very user friendly as it can be tucked away in a batch file and run from either a shortcut or scheduled task.

Hope this helps. If you need any more info just say.
-------------------------------------------------------------------------------

   ROBOCOPY     ::     Robust File Copy for Windows     ::     Version XP010

-------------------------------------------------------------------------------

 

  Started : Tue Jun 30 17:01:00 2009

 

   Source : \\appserv1\C$\XPP\xz\procs\

     Dest : C:\XPP\xz\procs\

 

    Files : *.*

            

  Options : *.* /NS /NJS /TEE /S /E /COPY:DAT /PURGE /B /NP /R:3 /W:3 

 

------------------------------------------------------------------------------

 

\\appserv1\C$\XPP\xz\sys\

                        \\appserv1\C$\XPP\xz\sys\cnv\

                        \\appserv1\C$\XPP\xz\sys\config\

          *EXTRA File                   1bgq_config.sde

            Older                       _bgq_config.sde

            Older                       _im_config.sde

            Older                       _sap_config.sde

            Older                       1im_config.sde

            Older                       1sap_config.sde

                        \\appserv1\C$\XPP\xz\sys\config\keyboards\

                        \\appserv1\C$\XPP\xz\sys\Daily Logs\

            New File                    xppws.2009-Jun-29.xml

                        \\appserv1\C$\XPP\xz\sys\fmts\

                        \\appserv1\C$\XPP\xz\sys\fmts\genlcat\

                        \\appserv1\C$\XPP\xz\sys\fmts\mathcat\

                        \\appserv1\C$\XPP\xz\sys\fmts\sd\

                        \\appserv1\C$\XPP\xz\sys\fmts\tablcat\

                        \\appserv1\C$\XPP\xz\sys\gr\

                        \\appserv1\C$\XPP\xz\sys\gr\batik\

                        \\appserv1\C$\XPP\xz\sys\gr\batik\extensions\

                        \\appserv1\C$\XPP\xz\sys\gr\batik\lib\

                        \\appserv1\C$\XPP\xz\sys\gr\sdi\

                        \\appserv1\C$\XPP\xz\sys\gr\tgc\

                        \\appserv1\C$\XPP\xz\sys\gr\tgc\fonts\

                        \\appserv1\C$\XPP\xz\sys\gr\tgc\fonts\etc\

                        \\appserv1\C$\XPP\xz\sys\html\

                        \\appserv1\C$\XPP\xz\sys\ml\

            Older                       omnimark.lib

                        \\appserv1\C$\XPP\xz\sys\ml\dtd\

                        \\appserv1\C$\XPP\xz\sys\ml\dtd\book\

 

etc etc

Open in new window

0
 

Author Comment

by:cliffordgormley
ID: 24777182
AWESOME answer!!!  Thank you!
0
 

Author Closing Comment

by:cliffordgormley
ID: 31598745
Best answer I have ever gotten on EE
0
 
LVL 14

Expert Comment

by:Hedley Phillips
ID: 24783573
Glad it was of use. We really like Robocopy here. I hope you find it as useful.

Thanks for the points.
0

Featured Post

Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

Join & Write a Comment

Suggested Solutions

VM backups can be lost due to a number of reasons: accidental backup deletion, backup file corruption, disk failure, lost or stolen hardware, malicious attack, or due to some other undesired and unpredicted event. Thus, having more than one copy of …
Create your own, high-performance VM backup appliance by installing NAKIVO Backup & Replication directly onto a Synology NAS!
To efficiently enable the rotation of USB drives for backups, storage pools need to be created. This way no matter which USB drive is installed, the backups will successfully write without any administrative intervention. Multiple USB devices need t…
This tutorial will show how to configure a single USB drive with a separate folder for each day of the week. This will allow each of the backups to be kept separate preventing the previous day’s backup from being overwritten. The USB drive must be s…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now