Need Robocopy Expert to Evaluate Robocopy Syntax
Hello,
I want to use Robocopy to copy files from a networked drive labeled M to an external USB drive attached to the server labeled Z.
I also want Robocopy to skip certain files, file types, and folders.
The M drive has roughly 73,000 files in 1,400 folders that total 420 GB of data.
I came up with the following code which I saved as a bat file. I added the /L at the end to test the command and saved the output to the file. This drive has all of my digital photos and videos on it and I want to be sure I do not screw this up. I think the code is working, but would feel better to have one or two Robocopy experts look at it first. Here is the code:
"C:\CopyRite XP\robocopy.exe" M:\ V:\M_Copy\ /V /TS /S /E /COPY:DAT /ZB /NP /R:1 /W:5 /DCOPY:T /LOG:"C:\Mdrive Copy Log.txt" /TEE /XF *.swp *.dmp *.tmp pagefile.sys hiberfil.sys Thumbs.db /XD "M:\RECYCLER" "M:\System temp" "M:\Temporary Internet Files" "M:\System Volume Information" "M:\wsus" "M:\ZZZ Delete files locally" "M:\ZZ Weight" /L
Here is the end of the log file:
Total Copied Skipped Mismatch FAILED Extras
Dirs : 1406 1400 6 0 0 0
Files : 73444 72717 727 0 0 0
Bytes : 418.331 g 416.160 g 2.170 g 0 0 0
Times : 0:02:16 0:00:00 0:00:00 0:02:16
From the above it looks like files and folders were skipped. The top of the log file had this in it which was encouraging:
Started : Wed Feb 15 21:12:28 2012
Source : M:\
Dest : V:\M_Copy\
Files : *.*
Exc Files : pagefile.sys
hiberfil.sys
Thumbs.db
*.swp
*.dmp
*.tmp
Exc Dirs : M:\RECYCLER
M:\System temp
M:\Temporary Internet Files
M:\System Volume Information
M:\wsus
M:\ZZZ Delete files locally
M:\ZZ Weight
Options : *.* /V /TS /L /TEE /S /E /COPY:DAT /DCOPY:T /ZB /NP /R:1 /W:5
Finally, here is the detail of one skipped file: I notice that skipped files/folders are labeled "named"
named 1.9 g 2012/02/16 02:23:37 pagefile.sys
Question #1: Do you see any problems in the code or is it good to go? I would cut out the /L at the end before running.
Question #2: Please confirm that there is no way that this code would delete anything off the source drive.
Question #3: I plan to schedule this to run a few times a week. After the first long run, will Robocopy skip all identical files and only copy new ones?
Question #4: Since all of the data on this drive are my pictures and movies, I do not want to have any strange permission or security attributes. If my server completely melted, I want to be able to plug the USB into any PC and copy the files to the PC and have full access. Will the above code do this?
Let me know if you have any questions. Unfortunately, my server only has USB 2.0 ports. Unless my old version of Robocopy can multi-thread, this is going to take a long, long time for the first run.
Thanks!
Chip
I want to use Robocopy to copy files from a networked drive labeled M to an external USB drive attached to the server labeled Z.
I also want Robocopy to skip certain files, file types, and folders.
The M drive has roughly 73,000 files in 1,400 folders that total 420 GB of data.
I came up with the following code which I saved as a bat file. I added the /L at the end to test the command and saved the output to the file. This drive has all of my digital photos and videos on it and I want to be sure I do not screw this up. I think the code is working, but would feel better to have one or two Robocopy experts look at it first. Here is the code:
"C:\CopyRite XP\robocopy.exe" M:\ V:\M_Copy\ /V /TS /S /E /COPY:DAT /ZB /NP /R:1 /W:5 /DCOPY:T /LOG:"C:\Mdrive Copy Log.txt" /TEE /XF *.swp *.dmp *.tmp pagefile.sys hiberfil.sys Thumbs.db /XD "M:\RECYCLER" "M:\System temp" "M:\Temporary Internet Files" "M:\System Volume Information" "M:\wsus" "M:\ZZZ Delete files locally" "M:\ZZ Weight" /L
Here is the end of the log file:
Total Copied Skipped Mismatch FAILED Extras
Dirs : 1406 1400 6 0 0 0
Files : 73444 72717 727 0 0 0
Bytes : 418.331 g 416.160 g 2.170 g 0 0 0
Times : 0:02:16 0:00:00 0:00:00 0:02:16
From the above it looks like files and folders were skipped. The top of the log file had this in it which was encouraging:
Started : Wed Feb 15 21:12:28 2012
Source : M:\
Dest : V:\M_Copy\
Files : *.*
Exc Files : pagefile.sys
hiberfil.sys
Thumbs.db
*.swp
*.dmp
*.tmp
Exc Dirs : M:\RECYCLER
M:\System temp
M:\Temporary Internet Files
M:\System Volume Information
M:\wsus
M:\ZZZ Delete files locally
M:\ZZ Weight
Options : *.* /V /TS /L /TEE /S /E /COPY:DAT /DCOPY:T /ZB /NP /R:1 /W:5
Finally, here is the detail of one skipped file: I notice that skipped files/folders are labeled "named"
named 1.9 g 2012/02/16 02:23:37 pagefile.sys
Question #1: Do you see any problems in the code or is it good to go? I would cut out the /L at the end before running.
Question #2: Please confirm that there is no way that this code would delete anything off the source drive.
Question #3: I plan to schedule this to run a few times a week. After the first long run, will Robocopy skip all identical files and only copy new ones?
Question #4: Since all of the data on this drive are my pictures and movies, I do not want to have any strange permission or security attributes. If my server completely melted, I want to be able to plug the USB into any PC and copy the files to the PC and have full access. Will the above code do this?
Let me know if you have any questions. Unfortunately, my server only has USB 2.0 ports. Unless my old version of Robocopy can multi-thread, this is going to take a long, long time for the first run.
Thanks!
Chip
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Qcubed,
I ran your batch file. First off, the change you made to logging really sped this up. The whole thing ran in 16 seconds as opposed to 2 minutes and 16 seconds!
Second, I notice that my version of Robocopy is XP026. Is this the correct copy to use for multi-threading? Maybe W7 has the correct version installed already?
Attached is the log. I see I goofed somehow. You already had the ZZ folders listed - so they were excluded twice. Not sure how I missed that - must be I need to drink more coffee. I will delete my duplicate entries.
I am also going to delete the L command at the end and run the batch file over the server. I will time this operation then try your idea over the network with one of my other drives.
Mdrive-Copy-Log-2012-16-02-09.32.txt
I ran your batch file. First off, the change you made to logging really sped this up. The whole thing ran in 16 seconds as opposed to 2 minutes and 16 seconds!
Second, I notice that my version of Robocopy is XP026. Is this the correct copy to use for multi-threading? Maybe W7 has the correct version installed already?
Attached is the log. I see I goofed somehow. You already had the ZZ folders listed - so they were excluded twice. Not sure how I missed that - must be I need to drink more coffee. I will delete my duplicate entries.
I am also going to delete the L command at the end and run the batch file over the server. I will time this operation then try your idea over the network with one of my other drives.
Mdrive-Copy-Log-2012-16-02-09.32.txt
ASKER
Update - so far 67 GB after about 1 hour. At this rate, this is going to take another 5-6 hours.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
oBdA
Thank you for your continued support. I will add the new code you suggested to the script. Before starting this copy I went into properties of the USB drive, added everyone under security, and gave it full control.
You also suggested that I:
1) Change /ZB to /Z
2) Add /nfl /ndl
Do you suggest I stop the current copy, modify the script, and re-start? So far I am at 31,500 files and 111 GB. We are almost half way through the files, but 20% of the data. It hasn't reached my video folder yet that has 900 files that are 90 GB.
In terms of Windows 7, I have never used a command prompt on this computer or run a batch file. I see four options under Windows PowerShell:
Windows PowerShell (x86)
Windows PowerShell ISE (x86)
Windows PowerShell ISE
Windows PowerShell
I tried one and it seems that I can get Robocopy through this.
How would I modify the batch file to work on my W7 machine?
Thank you for your continued support. I will add the new code you suggested to the script. Before starting this copy I went into properties of the USB drive, added everyone under security, and gave it full control.
You also suggested that I:
1) Change /ZB to /Z
2) Add /nfl /ndl
Do you suggest I stop the current copy, modify the script, and re-start? So far I am at 31,500 files and 111 GB. We are almost half way through the files, but 20% of the data. It hasn't reached my video folder yet that has 900 files that are 90 GB.
In terms of Windows 7, I have never used a command prompt on this computer or run a batch file. I see four options under Windows PowerShell:
Windows PowerShell (x86)
Windows PowerShell ISE (x86)
Windows PowerShell ISE
Windows PowerShell
I tried one and it seems that I can get Robocopy through this.
How would I modify the batch file to work on my W7 machine?
At this point, I'd keep it running. Even if it doesn't copy the files again, it'd still need to scan 32000 files, which takes a bit of time, too.
Just remove the "/ZB" completely, I don't think either option is necessary in your case (and /Z is the one that creates a bit of overhead).
The command prompt works the same in Windows 7 as in earlier versions. Either enter cmd.exe in the Run menu, or start it from the Accessories folder under All Programs.
Power Shell is not the new command prompt, it's a completely different shell.
Batch scripts are still interpreted by the old command prompt, not by Powershell.
The script should run just fine on Windows 7 as well.
Just remove the "/ZB" completely, I don't think either option is necessary in your case (and /Z is the one that creates a bit of overhead).
The command prompt works the same in Windows 7 as in earlier versions. Either enter cmd.exe in the Run menu, or start it from the Accessories folder under All Programs.
Power Shell is not the new command prompt, it's a completely different shell.
Batch scripts are still interpreted by the old command prompt, not by Powershell.
The script should run just fine on Windows 7 as well.
ASKER
Thank you both for your awesome help. I am going to post another follow-on question about Robocopy that either of you could answer. It should be a lot easier to answer thanks to the work you did here. At some point I am going to run the batch file to backup over the network. We will see if it is any faster. It has been over 6 hours and it is about 85% done.
ASKER
I do love this site. I post a technical question at 1 am, get up in the morning and have two very detailed answers. :)
oBdA: You wrote:
#4: "/copy:dat" won't copy NTFS permissions; just make sure the root of the target drive has Full permissions for Everyone before you start the copy...
I will be rotating several removable drives as part of my backup strategy. Is there a way to add a command to Qcubed's batch file that ensures the USB drive has correct permission?
Qcubed, Thank you so much for your bat file. I am going to try it now and will post the results. I made one minor edit to it. I added the following to the very end before the /L because these folders are not hidden or system folders:
/XD "M:\wsus" "M:\ZZZ Delete files locally" "M:\ZZ Weight"
Also, my PC is running Windows 7 64 bit and my network is in theory gigabit. I said in theory because I have never tested the speed but the Dell PowerConnect switch is gigabit as is my NIC and all the wiring is with Cat6. If I wanted to test your idea of copying the files across the network, I assume I will need to make a few changes
When I map a network drive the path to M is "\\theoracle2\M drive". Is the following edit correct? What about the space between M and drive? does the one set of quotation marks cover this?
Edit #1: Change SET "SrcDir=M:" to SET "SrcDir=\\theoracle2\M drive:"
Edit #2: Make sure the USB drive is assigned the letter V when connected to my PC
Edit #3: Change the path to the LogFile so it stores it in a folder on M called RoboLogs:
SET "LogFile=\\theoracle2\M drive:\RoboLogs\Mdrive_Cop
Edit #4: Add some command to enable multi-threading. I have no idea what this would be.
Thank you both again for your help!