a better copy routine (better than xcopy or robocopy)

I recently updated a sync computer batch file from xcopy to robocopy.  This resolved some problems but I discovered that robocopy does not copy open files as xcopy did. Does anybody know of a copy routine that can run from a batch file that copies open files. The biggest problem is MS Word keeps files open in the background even after it is closed. It would be great if it has a copy version option (copy newer files to a new version number instead of overwriting the older file) and multi-thread capability. Free or low cost would also be good.

LVL 39
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Adam BrownSr Solutions ArchitectCommented:
You would need file sync software to do this or you would need to implement DFS-R. https://www.tgrmn.com/ sells ViceVersa, which is a fairly simple solution for this, but I don't know if it does file versioning like you specify. That is a fairly complicated feature that would probably cost a lot more.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Dr. KlahnPrincipal Software EngineerCommented:
Roadkil's Unstoppable Copy has an option to continue trying to copy files if a copy fails.  This is a better option than brute-force copying an open file, imo, as there is no way to differentiate between a file that is "open" because a program hasn't released it vs. one that is really open for active access.
Ben Personick (Previously QCubed)Lead Network EngineerCommented:
What Robocop options are you using?

 The  /B Option is the "Backup" mode which is intended to open files using a backup method so that you can copy opened files too.

Try adding that to your Robocopy command and see if it solves your issue.
Become a Microsoft Certified Solutions Expert

This course teaches how to install and configure Windows Server 2012 R2.  It is the first step on your path to becoming a Microsoft Certified Solutions Expert (MCSE).

Ben Personick (Previously QCubed)Lead Network EngineerCommented:
Also, you may hear /B doesn't work with Opened files, that depends on what you mean by opened files;  Word and excel actually just lock the original file for writing (but not reading) and then edit a temporary file which is the "Opened" file.  In my experience /B works for those.

However, there are files that get opened and edited directly, which it may not work for.

Based on this run the Robocopy command with the /B setting and then parse the output for locked files into a separate Log.

If those files are truly needed (not temp ~ files which you should not bother copying over anyway) then you either:

A) Use xCopy by parsing the log of locked files and copying them using xcopy.
B) Use an alternative copy process that is shadow-copy aware (I did a search and there are several but I have no experience with them)
C) Create a shadow Copy and mount it, and then robocopy that.

Code to create a log of just the files which are skipped due to the File being locked:

REM Your existing script Example
SET "_YourRobocopyLogPathAndName=C:\Some\Path\LogFile.log"
SET "_YourRobocopyErrorLogPathAndName=C:\Some\Path\ErrorLogFile.log"

REM Add these Options as part of your existing Robocopy Command
[Your Robocopy Command] /B /NP /V /log+:"%_YourRobocopyLogPathAndName%"

Rem parse the resulting log looking for errors:
FindString " ERROR " "%_YourRobocopyLogPathAndName%" >> "%_YourRobocopyErrorLogPathAndName%"

Open in new window

That should output the failed Entries to a log file which you can parse to determine if you really need to copy the remaining failed items.

If you determine you do need these files on each run you can take one of the three actions I referenced above as A/B/C

If A you could parse the results of the file in-line and call XCopy directly from the existing script, and I or another expert would just fill in that last bit if you can not..

If B) there are plenty of options I found by googling but I am not familiar with them)

If C) There are methods to script Creating and mounting a Volume Shadow copy so it can be the source of your Robocopy, it's a much more lengthy process though so I or another expert could help you with it if you decided to go that route.
thenelsonAuthor Commented:
What Robocop options are you using?
:: Copy options :
      /S            :: copy Subdirectories, but not empty ones.
      /COPY:DAT      :: what to COPY for files (default is /COPY:DAT).
      /FFT            :: assume FAT File Times (2-second granularity).
      /Z            :: copy files in restartable mode.
:: Retry Options :
      /R:100      :: number of Retries on failed copies: default 1 million.
      /W:5            :: Wait time between retries: default is 30 seconds.

I'll try the /B option and look into the other suggestions.
Ben Personick (Previously QCubed)Lead Network EngineerCommented:
So this has all the options of your original script, although /R:10 /W:2 should be plenty!!  (While you can change it back to /R:100 /W:5, I suggest you don't as this will mean spending over 8 minutes on every file that has trouble copying, if that is only 7 files you have delayed your script by an hour.. :( )

The finding of the error file is now done in-line and put in to the second log file, so you will have a clog which has all details, and one with just the error files.

REM Your existing script Example
SET "_YourRobocopyLogPathAndName=C:\Some\Path\LogFile.log"

SET "_YourRobocopyErrorLogPathAndName=C:\Some\Path\ErrorLogFile.log"

SET "_RobocopyOptions= /S /COPY:DAT /FFT /R:10 /W:2 /Z /B /NP /V /TEE /log+:"%_YourRobocopyLogPathAndName%""
REM Now doing the Error Finding to the separate Log in-line:

ROBOCOPY "[YOUR SOURCE]" "[YOUR DESTINATION]" * %_RobocopyOptions% | FIND " ERROR " >> "%_YourRobocopyErrorLogPathAndName%"


REM Typing Error Files to Screen:
TYPE "%_YourRobocopyErrorLogPathAndName%"

Open in new window

thenelsonAuthor Commented:
According to tgrmn.com ViceVersa pro does file versioning, ViceVersa plus does not do file versioning. They both do the other things I want. I also like that they will back up to an ftp site. Unfortunately it's not free.

Dr. Klahn,
At first glance it looks like Roadkil's Unstoppable Copy would do what I want except file versioning and it's free.

I added the /B option and changed to /R:10 /W:2.  Your idea of piping to the find command to display ERROR lines is very clever. I'll give that a try.

I'm going to spend some time researching this this weekend. Thanks for all your input.
thenelsonAuthor Commented:
I have found freefilesync at www.freefilesync.org. It will do mirroring, two way syncs, versioning, ftp transfers, create batch jobs from its GUI and real time synchronization. Very impressive and its freeware/donationware. I'm going to look at it more closely.
robocopy can't copy open files, no matter which options you use.  You just have to keep running it until, eventually, the file is no longer open, then it will be copied.  Unfortunately, it won't work for system files.

If you need a scripted command line, hobocopy will work if you enable Volume Shadow Service, but it hasn't been updated in 6 years.   It should still work with Server 2012 and Windows 10, but may eventually stop working on newer versions.
Sam Simon NasserIT Support ProfessionalCommented:
I use FastCopy for copying files, it;s just GREAT. you can (using GUI) specify swap size, copy, more, diff, and other options
fastcopy.jpgthere is command line option, check them and you'll be satisfied, here https://ipmsg.org/tools/fastcopy.html.en and scroll to section 7

Ex) Copy C:\Test Folder to D:\Backup Folder by "diff(Size/Date)" mode is...
  FastCopy.exe /cmd=diff "C:\Test Folder" /to="D:\Backup Folder\"

Open in new window

thenelsonAuthor Commented:
Adam's suggestion started me searching for sync software and I found FreeFileSync. When I was using xcopy and robocopy syncing the two computers took 5-10 minutes while FreeFileSync took about 8 seconds. I now have FreeFileSync setup to run in the background of one computer and automatically sync the computers 300 seconds (5 minutes) after a change on either one with copying versions of changes and deletions to an FTP site and send a backup of all changes with versioning to another FTP site. I am real pleased with FreeFileSync and made a donation.

Ben's suggestions has given me more understanding of Robocopy even though I am no longer using it.

Fastcopy (Sam;s suggestion) looks very interesting but I am very happy with FreeFileSync.

Thanks to all of you!!
Ben Personick (Previously QCubed)Lead Network EngineerCommented:
Glad to Help :)
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Batch

From novice to tech pro — start learning today.