eemmpph
asked on
What is the best way to use RoboCopy to mirror client files/folders (and their security settings) from Server1 to Server2?
The goal: Copy client files/folders (and their security settings) from Server1 to Server2. Note: Server1 and Server2 are on the same network.
The problem: Due to the collective size (1 TB) of data to be copied, it would take days to complete over a network connection.
Possible steps:
Part A - Goal: Copy Server1 data to external disk drive (Drive Letter H)
Step 1: Attach a USB 3.0 external disk drive to Server1.
Step 2: Robocopy Server1 (shared folder "Clients") to external disk drive.
Step 3: When complete, disconnect external disk drive.
Part B - Goal: Copy External disk drive data (Drive Letter H) to Server2
Step 4: Attach external disk drive to Server2.
Step 5: Robocopy external disk drive data to Server2 (shared folder "Clients")
Step 6: When complete, disconnect external disk drive.
I have reviewed the robocopy command here: https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/robocopy
To ensure I do this correctly, I have two questions for you regarding Robocopy:
Question 1: What would comprise the robocopy command line, to accomplish Step 2 (Server1 to USB drive)?
Question 2: What would comprise the robocopy command line, to accomplish Step 5 (USB drive to Server2)?
Thanks for your help!
The problem: Due to the collective size (1 TB) of data to be copied, it would take days to complete over a network connection.
Possible steps:
Part A - Goal: Copy Server1 data to external disk drive (Drive Letter H)
Step 1: Attach a USB 3.0 external disk drive to Server1.
Step 2: Robocopy Server1 (shared folder "Clients") to external disk drive.
Step 3: When complete, disconnect external disk drive.
Part B - Goal: Copy External disk drive data (Drive Letter H) to Server2
Step 4: Attach external disk drive to Server2.
Step 5: Robocopy external disk drive data to Server2 (shared folder "Clients")
Step 6: When complete, disconnect external disk drive.
I have reviewed the robocopy command here: https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/robocopy
To ensure I do this correctly, I have two questions for you regarding Robocopy:
Question 1: What would comprise the robocopy command line, to accomplish Step 2 (Server1 to USB drive)?
Question 2: What would comprise the robocopy command line, to accomplish Step 5 (USB drive to Server2)?
Thanks for your help!
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
I second the multiple runs... Once you get to a point where you can keep users off for a couple hours (holiday?) you finish the migration and change the pointers. The issue with swapping servers is the myriad of user created shortcuts that break if the new server has a different name.
ASKER
The network is only 1 GB. Yes, I am doing a migration. I have severe time contraints (1.5 days) doing the copy from server1 to server2 live.
The first time for robocopy will take quite a while, but after that, only changes. So I don't mind copying the files twice.
The first time for robocopy will take quite a while, but after that, only changes. So I don't mind copying the files twice.
ASKER
After Step 2 is run several times (Monday thru Friday), Server 1 will be shut down.
Server 2 will be up and running, ready to receive the data on Saturday morning in Step 5.
Hence, the use of fast USB 3.0 external disk drive.
Server 2 will be up and running, ready to receive the data on Saturday morning in Step 5.
Hence, the use of fast USB 3.0 external disk drive.
what happens if you get cyclic redundancy check failed error while data copying on external HDD in between since data size is 1 TB and this is quiet possible due to high I/O usage during copy / paste operations specifically on usb drives
In that case you should use SATA SSD disk which can do your work faster without issues instead of usb disk
I always seen poor performance with external usb disks
In that case you should use SATA SSD disk which can do your work faster without issues instead of usb disk
I always seen poor performance with external usb disks
The network is faster than using USB. ALWAYS. It is more reliable, and can be managed without physically moving any cables. 1 TB isn't that much data for the network. Your gigabit network can move 1 TB of data in less than 4 hours. Bottlenecks will be file systems and storage read and write performance. Note that millions of small files take much longer to copy than a large file, even if the total amount of data is similar.
I personally like the /mir switch, because it does delete files on the target that no longer exist on the source, such as temp files and files deliberately deleted.
I personally like the /mir switch, because it does delete files on the target that no longer exist on the source, such as temp files and files deliberately deleted.
ASKER
According to my calculation for step 5, it will take 5 hours to robocopy 605 GB of data from the USB 3.0 external drive to Server2.
If your servers will support the transfer, the gigabit network will do all the copying in under 4 hours. If it takes 5 hours to copy from external drive to server, it could take another 5 hours to copy from server 1 to the USB drive. Now you are at 10 hours total using USB, in a best case scenario. Any delays due to file system overhead in creating the directory entries, updating metadata, etc. will slow everything down, but you have to do it all twice when using he external drive.
ASKER
I do thank you Kevihhsieh, Mahesh and Scott Silva for your input (and additional observations).
My questions were answered when Mahesh provided the robocopy command lines information.
The users will be off the network entirely for 48 hours.
At the same time the robocopy is occuring (from usb drive to Server 2), mailbox migration (50 GB) is also occuring "over the network".
My questions were answered when Mahesh provided the robocopy command lines information.
The users will be off the network entirely for 48 hours.
At the same time the robocopy is occuring (from usb drive to Server 2), mailbox migration (50 GB) is also occuring "over the network".
ASKER
Thank you!
11 TB over a congested gigabit network only took 4 days. Unless you have other file transfers going on, and you're copying to and from single spinning disks, you'll finish copying 1 TB in under a day.
Make sure you use the robocopy.exe from the Server 2016 system, as that has multi-threading. The robocopy on Server 2008 does not have multi-threading and the smaller files will slow your single threaded copy speeds by a very noticeable amount. Verify that your version of robocopy has the /mt option.
Make sure you use the robocopy.exe from the Server 2016 system, as that has multi-threading. The robocopy on Server 2008 does not have multi-threading and the smaller files will slow your single threaded copy speeds by a very noticeable amount. Verify that your version of robocopy has the /mt option.
What is the ultimate goal? If you're doing a migration, you will probably want to do multiple rounds of robocopy. The first one will be slow copying over all of the files, but all subsequent runs will copy over onky changed files.