Backing Up Data Remotely

I've solved this problem a few ways in the past.

1) have the user work via terminal services/rdp to a local server/workstation in the office

2) use xxcopy to do an incremental file copy.  This compares the local files, with the ones on the server, and only copies ones that have been modified.  I normally set to check based on file size and date.  http://www.xxcopy.com

3) for someone like a graphic designer that is working with extremely large files we have the user burn a cd and mail it weekly instead of trying to sync

getdropbox, is it userfirendly and xxcopy, is this more like a command script utility then anything?  I want to be able to get her something that is user friendly to use because she is not that tech savvy.  

with xxcopy I just give the user a shortcut that has the command like a batch file.

By looking at that dropbox, it looks like it syncs data to an online server but not my server here on my network?  Thats what i would like it to do is somehow sync with my network if possible?

xxcopy looks a little off to me.  i went to the site but am having trouble knowing how to use it.

Have you guys ever used Robocopy by any chance and if so, what are your thoughts?

xxcopy is basically just a more powerful version of Robocopy.  

xxcopy "c:\my folder" "\\servera\share1" /clone

That will copy any file from "c:\my folder" that a) does not exist in "\\servera\share1" b) that has a newer version in "c:\my folder" based on date/time or file size

If I remember correctly the reason I stopped using robocopy had something to do with attributes and permissions.

I see, so does xxcopy have the ability to notice anything changed locally like either moved, deleted, renamed, etc and then will reflect it over to the network?

How would i go about entering in that command line you just used into the xxcopy program and then save it.  I would prefer for it to copy everything up at first but anything last modified to only copy stuff up like incrementals, know what i mean?

Do you know if its a faster program or does it take a while everytime she will use it over VPN?

You can share dropbox folders between 2 accounts

so you get a work account and a user account create a folder on user share it with the work account and it gets synced between both and on the server

Or use the same account on 2 computers it also synces accros them, and keeps a copy on the server

the command I would use is basically what I posted before.  I would just give the user a batch file with the command.  

xxcopy isn't  slow.  I've used it over vpn a bunch of times.  Really just depends on the upstream bandwidth your remote user has.

xRMG...So your saying I can use Dropbox to work and sync from her file locally to her laptop and somehow connect it to work with my server here on the network?  Do you know where i can find the documentation to set it up as you are talking about here?

Mik...do I enter in and use that command you had given me within the XXconsole?

Its as simple as making an account for dropbox (getdropbox.com)

Installing the dropbox client on the laptop set it up etc.

Install dropbox on the server use the same account you used on the laptop and it will keep copies on both computers and the server (so if you need it on an other worker install it and it will sync automaticly)

https://www.getdropbox.com/help/4

This is also interesting for you :

Does Dropbox always upload/download the entire file any time a change is made?

Dropbox tries to be as smart as possible about uploading for the best possible performance. Before transferring a file, we compare the new file to the previous version and only send the piece of the file that changed. This is called a "binary diff" and works on any file type. Dropbox compresses files before transferring them as well. You also never have to worry about Dropbox reuploading a file - it's smart about this, too.

you xxcopy should be able to use it inside any dos window

If you want something with lots of options and has a good interface try Microsoft SyncToy version 2. It's free and it works well.
http://www.microsoft.com/DownLoads/details.aspx?familyid=C26EFA36-98E0-4EE9-A7C5-98D0592D8C52&displaylang=en
It will give your end user a good indication of success or failure, allowing you to take further action if necssary.

The alternative that I would recommend for supporting remote end users is a Terminal Server or full Citrix solution.
This allows all your data to be stored and accessed centrally.
It is not a cheap solution however but is  far more secure.

Robocopy is great.
SyncToy is like a cut down verison of Robocopy with a easy to use interface put on top.
Great for normal users.
Robocopy is to powerful to put in the hands of ordinary users.
The wrong switch can do serious damage.

Just to add to all of the previous suggestions, I use and highly recommend RoboCopy. I am in exactly the same situation as you with remote Workers who need access to files on our servers and were finding they took ages to open and save.

RoboCopy is a Windows DOS command and stands for Robust File Copy and is way more robust at remote data backups than XCopy.

have a read of the Wiki: http://en.wikipedia.org/wiki/Robocopy

It used to be available as part of the Windows Resource Kit, and was introduced as a standard feature of Windows Vista and Windows Server 2008.

A quick quote from the Wiki site which also contains a list of all the switches.

"Robocopy is designed for reliable mirroring of directories or directory trees. It has features to ensure all NTFS attributes and properties are copied, and includes additional restart code for network connections subject to disruption."

We have one command here that takes a copy of the users files from his local DMS respository (the software belongs to one of our customers) and backs this up, along with other files to his local external USB drive and then copies everything over his ADSL connection to our servers here via VPN. This runs everyday without fail.

If you would like some example commands that we use here I would be more than happy to paste them up and explain what the switches do.

Oh, forgot to say, Robocopy can be run from a Batch file and we have ours set up as a scheduled task. All the user needs to do is leave his PC on at night, it syncs and then shuts it down for him.

He also has a desktop shortcut if he wants to backup instantly.

The beauty of Robocopy and it being scriptable is that we can also include commands in the batch file to start and stop remote services before and after a sync when backing up system files from our Primary Application server to the others.

Can you tell that I love it!

Yeah, i think it may be obvious that your a fan of Robocopy, lol, but I do appreciate it.  These are all alot of good suggestuions.  right now im trying out DropBox and Robocopy.  xxcopy im not really sure how to use that exactly i mean i know i can create a batch file but I think there are alot of switches and im not sure what she will need specifically for her saving up to my network.  Im also going to see if i can try a couple of others suggested as well.  

The good thing i found out about Dropbox is that when i created, saved, modified etc on the local laptop, i was able to see by RDP of the file server the folder i was needing it to save it mirrored it within seconds and the thing is i didnt have to do anything.  She doesnt even need to be in VPN for that to work just as long as i guess both computers are on and linked to the dropbox account, it works nice.  the only fall whch is not bad is that you only get 2GB of space for free and have to pay monthly for the rest, lol.  nit picky i know but its still a factor because she has between 5 and 7 GB of data.  in either case, ill test these out.

I would really like to know more about xxcopy though if it works as well as some say.  Robocopy, again, same insite.

Question...especially for Mr-Madcowz, lol, for Robocopy, under / COPY Copy File Information section, there are a few attributes listed there and by defauly D - file Data is checked off but I see others which are:

S - file Security (NTFS-ACLs)
O - file Ownership
U - file Auditing information

Which ones do i need to check specifically to have the correct switches in place and what do each mean is you could please explain?

Robocopy:

Small (76Kb exe) that we stuff into the same directory as the batch file and the log files so we don't even need to bother with adding it to the environment variables as I am lazy.

An example of the batch file:

ROBOCOPY \\appserv1\C$\XPP\xz\scripts\ C:\XPP\xz\scripts\ /E /B /PURGE /R:3 /W:3  /NP /TEE /NJH /NJS /NS

Basically it takes the format of

ROBOCOPY <source> <destination> switches

The switches we use here are:

/E which copies subdirectories, even empty ones
/B Opens files in backup mode. Backup mode allows files to be opened without security restrictions, but requires additional rights on both the source and destination systems.
/PURGE  Deletes files no longer in the source location, this makes sure we have a mirror copy of the source
/R:3 Tells Robocopy to retry 'n' times before giving up in the event of error (default is 1 million). Handy otherwise it will sit trying to copy a file a million times. We found that 3 is a decent number as it caters for network blips
/W:3  Wait time between retries (default is 30 seconds). This gives us 3 seconds between retries and in conjunction with the previous switch, 3 retries over a period of 9 seconds.

Now we are on to Logging switches which we use to give us readable and useful log files

/NP No progress  don't display % copied.
/TEE Output to console window, as well as the log file. (so the user knows that something is happening, all about user confidence)
/NJH  No job header. Keeps the logs easy to read

There are plenty of other switches you can use but these do exactly what we want.

I run this batch file from a scheduled task:

"C:\Program Files\RoboCopy\Xpp_Sync\XPP_Sync.bat" > XPP_RoboCopy_%date:/=-%.log 2>&1

and what this gives me is a daily log file dated as such, ie: XPPRoboCopy_day-month-year.log

eg: XPP_RoboCopy_30-06-2009.log

I will paste an extract of the log file so you can see what sort of information you can get out of it in the code window:

As you can see it reports if the file is newer, older or an Extra file.

Ok, how and why did we get to be using Robocopy?

We started off using xcopy but found it totally and utterly lacking in the capabilities we required which were namely to be able to tolerate network outages and pickup where it had left off due to timestamps.

We then moved to RSync, a Windows port of an Unix utility. This started off well but was a right !"!*!£ to get working. It also didn't deal with large volumes of data over a long period of time (eg I also use Robocopy to copy my SAN Snapshot from one LUN to another and this is well over 1TB). I also found it didn't work well over the ADSL line and would find data not being backed up but no obvious reason for it.

This is when we moved over to Robocopy and after extensive testing have found it to be the utility of choice.

As I said in an earlier post, it is very user friendly as it can be tucked away in a batch file and run from either a shortcut or scheduled task.

Hope this helps. If you need any more info just say.

-------------------------------------------------------------------------------
   ROBOCOPY     ::     Robust File Copy for Windows     ::     Version XP010
-------------------------------------------------------------------------------
 
  Started : Tue Jun 30 17:01:00 2009
 
   Source : \\appserv1\C$\XPP\xz\procs\
     Dest : C:\XPP\xz\procs\
 
    Files : *.*
	    
  Options : *.* /NS /NJS /TEE /S /E /COPY:DAT /PURGE /B /NP /R:3 /W:3 
 
------------------------------------------------------------------------------
 
\\appserv1\C$\XPP\xz\sys\
	            	\\appserv1\C$\XPP\xz\sys\cnv\
	            	\\appserv1\C$\XPP\xz\sys\config\
	  *EXTRA File 			1bgq_config.sde
	    Older     			_bgq_config.sde
	    Older     			_im_config.sde
	    Older     			_sap_config.sde
	    Older     			1im_config.sde
	    Older     			1sap_config.sde
	            	\\appserv1\C$\XPP\xz\sys\config\keyboards\
	            	\\appserv1\C$\XPP\xz\sys\Daily Logs\
	    New File  			xppws.2009-Jun-29.xml
	            	\\appserv1\C$\XPP\xz\sys\fmts\
	            	\\appserv1\C$\XPP\xz\sys\fmts\genlcat\
	            	\\appserv1\C$\XPP\xz\sys\fmts\mathcat\
	            	\\appserv1\C$\XPP\xz\sys\fmts\sd\
	            	\\appserv1\C$\XPP\xz\sys\fmts\tablcat\
	            	\\appserv1\C$\XPP\xz\sys\gr\
	            	\\appserv1\C$\XPP\xz\sys\gr\batik\
	            	\\appserv1\C$\XPP\xz\sys\gr\batik\extensions\
	            	\\appserv1\C$\XPP\xz\sys\gr\batik\lib\
	            	\\appserv1\C$\XPP\xz\sys\gr\sdi\
	            	\\appserv1\C$\XPP\xz\sys\gr\tgc\
	            	\\appserv1\C$\XPP\xz\sys\gr\tgc\fonts\
	            	\\appserv1\C$\XPP\xz\sys\gr\tgc\fonts\etc\
	            	\\appserv1\C$\XPP\xz\sys\html\
	            	\\appserv1\C$\XPP\xz\sys\ml\
	    Older     			omnimark.lib
	            	\\appserv1\C$\XPP\xz\sys\ml\dtd\
	            	\\appserv1\C$\XPP\xz\sys\ml\dtd\book\
 
etc etc

                                              

1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:
37:
38:
39:
40:
41:
42:
43:
44:
45:
46:
47:

hey, why would i want to do this option here:

/B Opens files in backup mode. Backup mode allows files to be opened without security restrictions, but requires additional rights on both the source and destination systems.

Where would i go to schedule the task using what you used ("C:\Program Files\RoboCopy\Xpp_Sync\XPP_Sync.bat" > XPP_RoboCopy_%date:/=-%.log 2>&1) after i have created the .BAT file?

Also, this was never answered but what do  these particullar switches mean and do i require them?

sorry, here they are:

S - file Security (NTFS-ACLs)
O - file Ownership
U - file Auditing information

Hi,

Use the /B switch to override ACLs and give the administrator access to all files, even if the ACL does not permit.

To schedule the task go to Start - Settings - Control panel - Scheduled Tasks and "Add Scheduled Task"

And those switches:

S File security. This attribute copies the NTFS Access Control Lists (ACL).
O File ownership. This attribute copies the NTFS file ownership information.
U File auditing. This attribute copies the NTFS file auditing information.

Note: Source and destination volumes must both be NTFS to copy Security, Ownership or Auditing information but that is pretty much a given these days.

the source path, is that the same as my local folder on my laptop and the target/destination would be the folder on my server i would want to sync too?