Improve company productivity with a Business Account.Sign Up

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 353
  • Last Modified:

Robocopy help - copy last changes

Hi Experts,

for my fileserver migration I used robocopy \\source \\destination /MIR /SEC /SECFIX /COPYALL
Some Shares are very big and it takes hours to migrate them.
Can you help me with the command to just copy the delta of a share to the new location ?
0
Eprs_Admin
Asked:
Eprs_Admin
  • 9
  • 8
  • 7
  • +1
4 Solutions
 
oBdACommented:
robocopy will by default skip files that are already in the target (same size, same time stamp), unless you specify /IS (include Same), so usually, you just run the same command again.
As to your command line:
* /sec is redundant, it's already included in /copyall.
* If you didn't change the security on any folders already in the target, you don't need /secfix.
* You should add /r:___ (retry, default 1000000) and /w:___ (wait in seconds, default 30) to sensible values (according to your environment), otherwise a single file that can't be copied will basically stop the whole operation (if the connection is stable, /r:0 should do it).
* if you write to a log file, include /np (no progress) as option, otherwise the log file will contain control characters.
So try something like this:
robocopy.exe \\source \\destination /MIR /COPYALL /r:0

Open in new window

0
 
VB ITSSpecialist ConsultantCommented:
Funny that this question popped up on my feed, I'm using robocopy right now to do some file copies..

Did you use the /R:0 and /W:0 switches? If not the copy job will by default retry a million times (!) with 30 seconds between each retry. Also good practice to use the /LOG switch to output the results to a text file as this can increase performance.
0
 
Eprs_AdminSystem ArchitectAuthor Commented:
when I use /MIR again, then at he whole destination will be deleted -> are you joking ???
0
Simplify Active Directory Administration

Administration of Active Directory does not have to be hard.  Too often what should be a simple task is made more difficult than it needs to be.The solution?  Hyena from SystemTools Software.  With ease-of-use as well as powerful importing and bulk updating capabilities.

 
VB ITSSpecialist ConsultantCommented:
Also, if you run Robocopy from your 2012 server you can look at using the /MT switch (multi-threading) which will significantly improve copying performance. A good base value to start from would be /MT:32 but I wouldn't go any higher than /MT:64 as there's been reports of network connectivity dropouts when copying large files.
0
 
Eprs_AdminSystem ArchitectAuthor Commented:
once again, I need a command to just copy the delta information to my dest share.
0
 
VB ITSSpecialist ConsultantCommented:
Sorry I just realized what you mean by 'delta copy'. In this case I always opt for the /XO switch as this will leave the files in the destination if it already exists and is the same date or newer than the source.
0
 
oBdACommented:
No, I am not joking, because /mir will not "[delete] the whole destination"; where did you get this idea from?
/mir makes sure that the destination is a mirror of the destination; it will (only) delete files/folders in the destination that have been deleted in the target since the initial copy.
0
 
oBdACommented:
VB ITS,
using /xo is not required in most cases. Unlike other copy commands, robocopy will by default skip files that are already in the target.
/xo will only apply if a file in the source has been restored to a former date than the one during the first copy had. With a /mir, it can produce erroneous/unintended results; example:
1. User edits file F.
2. The initial copy is run.
3. User isn't happy with his changes in 1, and restores F with an older version.
4. A delta copy with /xo is run: since F now has an older date than the one in the target, it will be skipped, the new file (with the incorrect changes) remains in the target.
5. The user is unhappy when the switch to the new file server is done.
0
 
VB ITSSpecialist ConsultantCommented:
This is all assuming the user has been working out of the old share and not out of the the new share location. Let's wait for further clarification from Eprs Admin before anything else. You make some valid points though.
0
 
oBdACommented:
if the user is already working from the new share, a /mir from the old share would be destructive with or without /xo, since any new file or folder created in the new share would be deleted ...
0
 
VB ITSSpecialist ConsultantCommented:
I probably should have been clearer earlier - I meant to say I would opt for the /XO switch over the /MIR switch in this scenario, i.e. no /MIR switch whatsoever.
0
 
oBdACommented:
In this case, the result wouldn't be erroneous (since you don't want the folders to be mirrored), but the scenario I described above would still make the user unhappy - his restored file won't be on the new server.
And running a delta copy without the /mir will make even more people unhappy after the switch: all files and folders that they thought they deleted will reappear, and all files and folders they renamed or moved will now be there twice, once with the new name and once with the old name.
0
 
QlemoBatchelor, Developer and EE Topic AdvisorCommented:
Obviously we need more info about what you have done already before being able to provide reasonable advice.  To me it sounds like you started the migration, but were unhappy about the time it took, and stopped it. People are already working with the migrated data, and maybe with the old shares too.
0
 
Eprs_AdminSystem ArchitectAuthor Commented:
Hi obda,
I did /MIR and later on again.
And the the first what robocopy is doing, it deletes the data no the target.
100% sure.
0
 
Eprs_AdminSystem ArchitectAuthor Commented:
so, my first command was :

robocopy \\source \\target /MIR /SEC /SECFIX /COPYALL

Now I need a command to copy the delta of source to target.
Still the share SOURCE is active. Therefore I need to copy all files which have been changed since first command.
0
 
VB ITSSpecialist ConsultantCommented:
Are users still working from the SOURCE data or have they been working from the DESTINATION since the original Robocopy?
0
 
Eprs_AdminSystem ArchitectAuthor Commented:
tonight they do not work anymore on source
0
 
oBdACommented:
I've used "robocopy /mir" in more file server migrations than I can remember, including folders with TBs of data, and it most certainly does not delete data on the target before the delta copy.
It might recopy the data if the target folder doesn't support the NTFS high resolution time stamps (which can happen, for example, with Linux CIFS storages); this can be fixed with /FFT.
If your first copy was done before the DST switch, try /DST.
Note that you can add the /L (List only), which will only "pretend" to copy, but not actually do anything.
Here's a test script that will create a test folder, create a single file in this folder, copy this folder into a target location, copy it a second time, and then delete the test folders again.
@echo off
setlocal
set TestSource=C:\RCTest
set TestTarget=D:\RCTest
set RCOptions=/mir /njh /v /np
if not exist "%TestSource%" md "%TestSource%"
>"%TestSource%\test.txt" echo TestFile
echo ========== Robocopy FIRST run: ==============================
robocopy.exe "%TestSource%" "%TestTarget%" %RCOptions%
echo ========== Robocopy SECOND run: ==============================
robocopy.exe "%TestSource%" "%TestTarget%" %RCOptions%
echo Hit any key to remove the test folders, or ^<Ctrl-C^> to stop.
pause >nul
if exist "%TestSource%" rd /s /q "%TestSource%"
if exist "%TestTarget%" rd /s /q "%TestTarget%"

Open in new window

The output should look something like this:
========== Robocopy FIRST run: ==============================

          New Dir          1    C:\RCTest\
            New File                  10        test.txt

------------------------------------------------------------------------------

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :         1         1         0         0         0         0
   Files :         1         1         0         0         0         0
   Bytes :        10        10         0         0         0         0
   Times :   0:00:00   0:00:00                       0:00:00   0:00:00


   Speed :               10000 Bytes/sec.
   Speed :               0.572 MegaBytes/min.
   Ended : Tue Dec 16 14:13:53 2014
 
========== Robocopy SECOND run: ==============================

                           1    C:\RCTest\
                  same                10        test.txt

------------------------------------------------------------------------------

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :         1         0         0         0         0         0
   Files :         1         0         1         0         0         0
   Bytes :        10         0        10         0         0         0
   Times :   0:00:00   0:00:00                       0:00:00   0:00:00
   Ended : Tue Dec 16 14:13:53 2014

Hit any key to remove the test folders, or <Ctrl-C> to stop.

Open in new window

In the second log, note lines 22, where it classifies the text file found as "same", and line 28, where there is a 1 in the "Skipped" column (which means it was not deleted/recopied), while it was in "Copied" in line 10 during the first run.
0
 
Eprs_AdminSystem ArchitectAuthor Commented:
ok OBDA,
then I try some tests later on with /MIR again.
But in my case I also have to copy all security information /SEC /SECFIX and /COPYALL

When I start this cmd again can I use it like this :
robocopy \\source \\target /MIR /SEC /njh /v /np /COPYALL
0
 
Eprs_AdminSystem ArchitectAuthor Commented:
Please check this ticket, here the guy wrote /MIR deletes all folders on the target dir.
And once I have seen it.
http://www.experts-exchange.com/OS/Microsoft_Operating_Systems/Server/Windows_Server_2008/Q_28509155.html

Or has it to do with the other switches ?
0
 
VB ITSSpecialist ConsultantCommented:
running a delta copy without the /mir will make even more people unhappy after the switch: all files and folders that they thought they deleted will reappear, and all files and folders they renamed or moved will now be there twice, once with the new name and once with the old name.
Good point, I didn't think about this.

@Eprs Admin, oBdA is correct. The /MIR switch doesn't delete the entire contents of the destination. What it does delete though are files/folders in the destination that no longer exist in the source.

Also, no need to run both the /SEC and /COPYALL switches together as the /SEC switch is equivalent to /COPY:DATS
I would also leave out the /SECFIX switch since the /COPYALL switch will also copy over the security permissions, just makes it redundant really.
0
 
Eprs_AdminSystem ArchitectAuthor Commented:
ok now I tested my command and you are right.

With /MIR the files not delted on source, but files which are not on the source but in the target, they will be deleted.
It really creates a mirror of source.

Instead of /SEC /SECFIX /COPYALL I can use just /COPYALL ?
0
 
VB ITSSpecialist ConsultantCommented:
That's correct. The /SEC switch is equivalent to /COPY:DATS whereas /COPYALL is equivalent to /COPY:DATSOU

/SECFIX is usually only used to fix security permissions if they go out of whack. We can always run this later without it copying over any files if you run into issues with the security on the destination files/folders.
0
 
oBdACommented:
What Svet Paperov was referring to is that if you do the (first) /mir, anything already in the target folder will (probably) be deleted, because it's not in the source and considered "extra". This has nothing to do with robocopy deleting the target folder before a delta copy.
Note that you can always add /L to your switches and rerun the command (stop with Ctrl-C, or use /lev:x to only "copy" the first x levels) to see what it would do. If you add /v as well, you should have lots of "same" classifications.

As to the command line, please read my comments about the command line in http:#a40500359 again; I already mentioned that /sec is included in /copyall.
/sec and /secfix are independent; /secfix will make robocopy check the security of all files again; by default, it does not check for changed security (only for changed size or changed time), so once a file or folder is already there in the same version, a changed ACL will not be copied over without /secfix. Unless you did extensive changes to existing security settings, don't use it, or it will slow down the copy.
I'd use something this for starters (and run the command on either source or target, instead of a third machine):
robocopy.exe \\source \\target /MIR /COPYALL /r:0 /mt /np /tee /log:"C:\Temp\robocopy.log"

Open in new window

Note that /mt is only available since Windows 7/Server 2008 R2.
Add /v at your discretion.
If you want to use a log file with "/log:", you can use /tee to see the console output as well.
/njh in my script is No Job Header, that was just there to keep the output short for the test script.
/v (verbose) will list files classified as "same" (which it will skip); depends on you whether you want that or consider it information overflow. I've used it in the test script to make it obvious that the file wasn't recopied.

Here's a test script for extra, same, newer, and older file classes:
@echo off
setlocal
set TestSource=C:\RCTest
set TestTarget=D:\RCTest
set RCOptions=/copyall /mir /njh /v /np
if not exist "%TestSource%" md "%TestSource%"
>"%TestSource%\test-stays-the-same.txt" echo TestFile
>"%TestSource%\test-will-be-modified.txt" echo TestFile
>"%TestSource%\test-will-be-removed-from-source.txt" echo TestFile
>"%Temp%\test-will-be-replaced-with-older.txt" echo TestFile
ping.exe -n 3 localhost >NUL
>"%TestSource%\test-will-be-replaced-with-older.txt" echo TestFile
echo ========== Robocopy FIRST run: ==============================
robocopy.exe "%TestSource%" "%TestTarget%" %RCOptions%
>>"%TestSource%\test-will-be-modified.txt" echo Content added after second run.
copy "%Temp%\test-will-be-replaced-with-older.txt" "%TestSource%"
del "%TestSource%\test-will-be-removed-from-source.txt" echo TestFile
echo ========== Robocopy SECOND run: ==============================
robocopy.exe "%TestSource%" "%TestTarget%" %RCOptions%
echo Hit any key to remove the test folders, or ^<Ctrl-C^> to stop.
pause >nul
if exist "%TestSource%" rd /s /q "%TestSource%"
if exist "%TestTarget%" rd /s /q "%TestTarget%"

Open in new window

0
 
Eprs_AdminSystem ArchitectAuthor Commented:
ok now I tested some settings.

robocopy \\source \\target /mir /copyall /njh /np /v /log:C:\Temp\log.txt
This command works well and all my security settings also copied.

And of course the List command works well -> Thanks for this:
robocopy \\source \\target /mir /copyall /njh /np /v /L /log:C:\Temp\log.txt

I think we can close now this ticket.
Thanks a lot.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Simplify Active Directory Administration

Administration of Active Directory does not have to be hard.  Too often what should be a simple task is made more difficult than it needs to be.The solution?  Hyena from SystemTools Software.  With ease-of-use as well as powerful importing and bulk updating capabilities.

  • 9
  • 8
  • 7
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now