Windows 2003 file store off site

philipfarnes
philipfarnes used Ask the Experts™
on
Hello,

I have got a Windows 2003 server with about 1Tb worth of files store data.   This includes departmental shares, home directories and roaming profiles.  At the moment i am doing a weekly backup, and trying to do a xcopy batch file that copies files that have changed to another site via a 5mb link.

can you give me any advice of a better method.  I have tried double take and that does not seem to work very well, and other copying software i have tried, but it seems to take so long scanning directories to before copying.

Thanks

Phil
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®

Commented:
Try robocopy from the windows 2003 resource kit, it can do lots of good stuff, much better than xcopy!

From http://en.wikipedia.org/wiki/Robocopy

"Robocopy is notable for capabilities above and beyond the built-in Windows copy and xcopy commands, including the following:
Ability to tolerate network interruptions and resume copying. (incomplete files are marked with a date stamp of 1980-01-01 and contain a recovery record so Robocopy knows where to continue from)
Ability to correctly copy file data, attributes and preserve original timestamps, as well as NTFS ACLs, owner information, and audit information using command line switches. (/COPYALL or /COPY:) Copying folder timestamps is also possible in later versions (/DCOPY:T).
Ability to assert the Windows NT "backup right" (/B) so an administrator may copy an entire directory, including files denied readability to the administrator.
Persistence by default, with a programmable number of automatic retries if a file cannot be opened.
A "mirror" mode, which keeps trees in sync by optionally deleting files out of the destination that are no longer present in the source.
Ability to skip files that already appear in the destination folder with identical size and timestamp.
Ability to copy large numbers of files that would otherwise crash the built-in XCOPY utility.[citation needed]
A continuously updated command-line progress indicator.
Ability to copy file and folder names exceeding 256 characters — up to a theoretical limit of 32,000 characters — without errors.[1]
Supports multithreaded copying. (Windows 7 only) [2]
Supports a return code[3] on program termination for batch file usage. This feature was unfortunately dropped in Version XP026 (RobocopyGUI) - its return code is always 0 (zero).
Notably, Robocopy will silently fail to copy open files. The so-called Backup mode is an administrative privilege that allows Robocopy to override permissions settings (specifically, NTFS ACLs) for the purpose of making backups. The Windows Volume Shadow Copy service is the only Windows subsystem that can copy open files. Robocopy does not implement accessing the Volume Shadow Copy service in any way, inhibiting its usefulness as a backup utility for volumes that may be in use. However, one can use separate utilities such as VSHADOW or DISKSHADOW (included with Windows Server 2008) to create a shadow copy of a given volume with which to backup using Robocopy."

Cheers
Dave

Commented:
A rough calculation estimates that copying 1TB across a 5MB link would take about 2 days. Even doing just those new / modified files is going to be a large amount of data, and that is likely to be growing constantly.
If you want to optimise this, then there are 2 possible good solutions:
1) Use a good backup suite to do backups to disk using a remote agent & maximum compression.
2) Get WAN devices like Packeteer or the Riverbed Steelhead applicances to compress and accelerate the data transfer across the link.

Also consider breaking the data down into smaller chunks & robocopying 20% each day?
Commented:
"A rough calculation estimates that copying 1TB across a 5MB link would take about 2 days"
...and that is without any other traffic on the link - I wondered about this too :)
Cheers

Dave

Commented:
Also consider ADP (Avian Datagram Protocol) for file transfer.
http://www.faqs.org/rfcs/rfc1149.html
http://www.faqs.org/rfcs/rfc2549.html

However, my original suggestions above may be more practical.
SnibborgOwner

Commented:
You might think about completely rethinking your backup methodology and offsite the whole thing.  Have a look at http://www.intechnology.com.

I have used them a number of times and found their service very effective.

Snibborg

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial