Robocopy copy subfolders in pharses.

Perhaps I could use robocopy to copy all subdirectories and securities to the new shared first and I could use another tool to copy the subfolders in pharses.  

Anyone knows the command parameters to copy the subfolder names and securities only?

I believe robocopy already does this by default. Still, you could force it to copy by folder. Make a .bat file of this code:

for /d %%a in ("\\sourceserver\share1\*") do (
  ECHO robocopy "%%~a" "\\destinationserver\share1\%%~na" /MIR /SEC /LOG+:c:\copy.log 

)

Select allOpen in new window

Note: The code is in test mode for visual validation. Open a Command Window and run it.

When you are satisfied how it shows the command, remove the ECHO prefix to run it for real.

Copy folder security is /dcopy:dat

Unless you have the really old robocopy that doesn't have multithreading, you don't need to copy it in chunks.  You're just wasting time.  I used to do it that way back when it was single threaded, but I opened several windows and ran those "chunks" simultaneously.  When multithreaded robocopy came out, I just copied it to all the other older systems so I could run multithread robocopy on the older platforms to save me from manually chunking the copies.  Ever since it became multithreaded, it doesn't make sense to chunk robocopy.

If you're thinking of speeding it up, change the defaults to /W:0 and /R:0.  There's absolutely no good reason to wait for a locked file for the default 30 seconds and retry it 3 times.  It's unlikely to be freed up in that short a time.  It's better to skip over those files and just run robocopy immediately after it completes.  That introduces a longer wait to do the retry and it actually completes faster and gets more of those previously opened files copied.

ROBOCOPY "\\sourceserver\share1\" "\\destinationserver\share1" /MIR /SEC /dcopy:dat /w:0 /r:0 /LOG:c:\copy.log

User files are more likely to be released several hours later or the next day, than within 90 seconds.  That's a lot of wasted time for each file locked in use and introduces a huge delay for every locked file.  That was one of the 1st things I did on my 3rd try when I first used robocopy from the resource kit and saw all those delays pop up.  My initial tests of 300 GB without the /w:0 and /r:0 took over 2 weeks.  When I saw all those delays in the logs, I started to look for ways to speed it up.  Initially I shortened the wait and reduced the retries, but evenutually settled on my standard default 0 settings.  It only took a few days when I set it to /r:0 /w:0 and copied it to a different full backup. Running it again to an exsiting backup, robocopy would finish in under 1-2 hours.  I'd compare the logs and see that most of the skipped files would eventually get copied on the next pass.  By the end of the next day, all the previously skipped files eventually got copied, while some new files were skipped.

I always ran 2 robocopies, back to back to get most of the skipped files on that same day. They should just change those 2 defaults to 0 for robocopy, or remove it completely.  It's a stupid antiquated and useless default setting.  It might be good for 1 GB of file copies but wastes a lot of time for large file systems.

Also, you must run robocopy back to back to correctly set the time stamps of directories. During the first pass, it copies the directory and time stamp, but each file added to the directory updates that time to the current access time as each new file accesses the folder.  During the 2nd pass, most of those file updates are done, so the folder dates get updated to the correct date.  Unfortunately, you still have a few new files that may have gotten copied to some folders on the 2nd pass.

I don't recall if rsync in Linux/Unix did the same thing, but I'm used to running my robocopy and rsync back to back for all initial copies to quickly catch the changes that happen during the initial copy.

Thanks, everyone!