I need to copy a large number of files between servers

ross13
ross13 used Ask the Experts™
on
Whats the best tool that I can use to copy about 1TB of data from one nas drive to another?
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Business Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018
Commented:
I think I would try RoboCopy for this.

https://technet.microsoft.com/library/cc733145.aspx
*** Hopeleonie ***IT Manager
Commented:
Robocopy is the best tool for me.

robocopy.exe "Source Path" "Destination Path" /E /ZB /R:5 /W:5 /ETA /LOG:"Path for log \log.txt"
Gerwin Jansen, EE MVETopic Advisor
Most Valuable Expert 2016
Commented:
robocopy is a Windows utility, is this question about Windows servers?

If this were about Unix/Linux I'd suggest rsync instead

And what nas drives is this about? Nas drives themselves can have the rsync/copy capability as well, you could copy directly from one nas to the other.
Exploring ASP.NET Core: Fundamentals

Learn to build web apps and services, IoT apps, and mobile backends by covering the fundamentals of ASP.NET Core and  exploring the core foundations for app libraries.

yo_beeDirector of Information Technology
Commented:
I like robocopy ,but my preferred switches are

/E   -for sub-direcoreies

/XO - exclude files that are older on the source (not changed on the source)

/purge - this removes files from the destination directory that no longer exist in the source. I use this only for the purpose you are looking to do.

/Mir - is a good one for your purpose also. It combines /e /xo and /purge

/w:0 - wait time between. Errors. I set mine to zero because 99.9999% it is an open locked file that will not clear in  30 seconds (default setting). So setting this to 0 it will try again immediately

/r:0 - this is the number of retries when a error happens and as mentioned majority of the time the file will not be accessed in 30 seconds so why retry it.

/tee - this is to see the progress of robocopy. If you do not use this switch you will just see a flashing cursor and not really know where you are at regarding directories.

/np. - I use this switch only if I am logging the job. This will not show percentage of the file that have been copied. Very helpful to keep the log smaller and cleaner

/log:filepath. - self explanatory

With all those switches I then schedule multiple passes on the volume to catch all changes.
Dirk MareSystems Engineer (Acting IT Manager)
Commented:
Can one of the NAS devices connect as a USB device. Then you should be able to copy directly between devices via USB this would be the fastest option. I have never done or attemted this but ex colleague (according to him) has done this once. I would think that this would be device dependent to work. Otherwise use robocopy as suggested with a dedicated workstation or server to do the copy.

Also, some NAS have a built-in FTP server and client. You can start the FTP server on one side and use it on the other side to copy the files between them, without using your pc.

DitkMare

Author

Commented:
Thanks for the help
JohnBusiness Consultant (Owner)
Most Valuable Expert 2012
Expert of the Year 2018

Commented:
You are very welcome and good luck with moving large numbers of files

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial