Backup with Robocopy

Hello,

I am wanting to configure a backup job to copy a shared directory from one external HD to another on the same machine.  Initially, I want to create an exact copy of the existing 300 GB shared folder on the backup HD, then run a daily scheduled task with robocopy to only backup files that have changed since the full backup.  I am somewhat familiar with robocopy but have had some issues with it, especially copying 300 GB of data at once.

Thanks,
Mike
cheesebugahAsked:
Who is Participating?

Improve company productivity with a Business Account.Sign Up

x
 
Marc LConnect With a Mentor Senior Infrastructure EngineerCommented:
I would use the following in a scheduled task (task requires elevation):

robocopy <source> <destination> /MIR /COPYALL /ZB /R:5 /V /FP /Log:<path to log file>

/MIR - mirrors the directory tree, deletes destination files and folders that don't exist in the source
/COPYALL - copies all file information (data, attributes, time stamps, security, owner, auditing)
/ZB - uses restart mode, and if files are in use, copies using backup mode
/R:5 - sets retries to 5 (default 1,000,000), so if it runs into any issues you still get as much of the backup as possible

For logging:
/V - verbose
/fp - full path names
/log - sets a path for the log file to be written
0
 
sirbountyConnect With a Mentor Commented:
schtasks /create /ru system /sc daily /st 08:00 /tn Backup /tr "robocopy H:\Folder T:\Folder /e /s /mir"

Should be all you need...(run from an elevated cmd prompt)
0
 
cheesebugahAuthor Commented:
So I am creating a command prompt task to run robocopy, right?
0
Upgrade your Question Security!

Your question, your audience. Choose who sees your identity—and your question—with question security.

 
cheesebugahAuthor Commented:
What would I actually put in the Run: and Start in: textboxes?  This is what is currently in those textboxes:

Run:  C:\WINDOWS\system32\cmd.exe

Start in:  C:\WINDOWS\system32
0
 
Marc LConnect With a Mentor Senior Infrastructure EngineerCommented:
You could run robocopy directly:

Run: c:\windows\system32\robocopy.exe
Start in: c:\windows\system32
Arguments: <source> <destination> /MIR /COPYALL /ZB /R:5 /V /FP /Log:<path to log file>

Or you could save the command in a script
save 'robocopy <source> <destination> /MIR /COPYALL /ZB /R:5 /V /FP /Log:<path to log file>' as the first line in a text file, and rename it to c:\script.bat
Run: c:\script.bat
Start in: c:\
0
 
NVITCommented:
If you want to keep the folder timestamps, also use the /DCOPY:T switch
0
 
cheesebugahAuthor Commented:
I am backing up this location, E:\User Files.  The User Files root directory is not present on the destination.  Do I have to create the F:\User Files root folder, or will robocopy create it if I use:

robocopy "E:\User Files" "F:\User Files" /switches

Thanks,
0
 
NVITCommented:
Add trailing backslash:
robocopy "E:\User Files" "F:\User Files\" /switches
0
 
Marc LSenior Infrastructure EngineerCommented:
It will automatically create the destination directory
0
 
cheesebugahAuthor Commented:
Will the log files be created each time this runs or overwrite each time?
0
 
NVITCommented:
/log: overwrites
/log+: appends
0
 
Marc LSenior Infrastructure EngineerCommented:
It overwrites it each time, if you use the parameter /Log+:<log file> instead it will append it to the existing log file
0
 
cheesebugahAuthor Commented:
Would there be a selection that creates a new log file for each run?
0
 
NVITCommented:
If you want a unique log name each time you run the backup, you could use something like:
/log:Backup-%date:/=_%-%time::=_%.txt

Open in new window

0
 
cheesebugahAuthor Commented:
I'll just append to the existing.  The /MIR switch is not going to delete anything from the source will it?
0
 
NVITCommented:
/MIR deletes, if any, at the target. So that the target is a mirror of the source.

I find that trying it out on a small, test folder helps build confidence that it will work.
0
 
cheesebugahAuthor Commented:
First run on an almost 300 GB folder is scheduled for tonight.  We shall see what happens.  I will update this post when the verdict comes in.  Thank you guys for all of your help.  It is greatly appreciated.  Keep up the great work.

Thanks,
Mike
0
 
sirbountyCommented:
Depending on the system specs, you might also consider multi-threading to speed up the transfer.
Add a /MT:16 (8 is the default if no value supplied).
0
 
cheesebugahAuthor Commented:
For some reason the task did not even run over the weekend?
0
 
sirbountyCommented:
There is a file that should give you details on the reasoning located in
%systemroot%\tasks\SCHEDLGU.TXT

Perform a search of the string
[ ***** Most recent entry is above this line ***** ]

and you should be able to pinpoint the record of the event...
0
 
cheesebugahAuthor Commented:
Sorry, but that file is not present.  I must have screwed up the scheduling somehow?  Although at this point it is not clear to me in looking at it?
0
 
cheesebugahAuthor Commented:
In that location you mentioned, it says the task ran at its scheduled time on Friday, yet there is no log file or the mirroring of the shared folder as expected?
0
 
cheesebugahAuthor Commented:
The only thing I can think of is that this task is being run under a command prompt and when CMD opens, it cannot find the script file to run because it is set to run on the root of C: in the task?
0
 
cheesebugahAuthor Commented:
The script file is located in the root of C:, but when CMD opens, I don't think it opens to that location and would have to be instructed to go to that location?  I don't know, just my thought on it.
0
 
sirbountyCommented:
My preference is to schedule it from a cmd line, which helps (me) avoid some of these hiccups.
Yet, if your script is in the root, and it's simply calling robocopy - that file is in your path, so it shouldn't cause any problems.
Try heading your script off with some logging:

echo.  Beginning backup script >> c:\backups\Logfile.txt

Does that file get created and populated?

Another thing that comes to mind - make sure the task is set to run as highest elevation - there's a checkmark on the settings for that...
0
 
cheesebugahAuthor Commented:
I did a test run with a simple text file located in a folder on the source.  I could see the command prompt briefly run, but no log file and no mirroring of the folder and file?  Here is my simple script:

robocopy "E:\Test 1" "F:\Test 1\" /MIR /COPYALL /ZB /R:5 /V /FP /Log+:C:\Robocopy\Logs\UserFilesBU.log

Am I missing something here?

Thanks,
0
 
NVITCommented:
What do your scheduled task options look like?
0
 
sirbountyCommented:
Is it set up to run with highest privileges?
0
 
NVITCommented:
Program: robocopy
Add arguments: "E:\Test 1" "F:\Test 1\" /MIR /COPYALL /ZB /R:5 /V /FP /Log+:C:\Robocopy\Logs\UserFilesBU.log

As sirbounty mentioned, set "Run with highest privileges"
0
 
cheesebugahAuthor Commented:
Well, as it turns out, you have to put quotes around the path in the Run: textbox.  When creating the task, that is not automatically done.  I guess it is assumed you will not have any spaces in the path?  Anyway, my test run ran perfectly with the quotes around it and we should be good to go.  I don't think this is something that can complete in 12 hours though, so I'll have to schedule it again for this weekend.  I will let you guys know.  Thank you for all of your help.  It is greatly appreciated.

Thanks,
Mike
0
 
NVITCommented:
Just a friendly reminder, Mike.
Do you need to keep the folder timestamps? If so, add the /DCOPY:T switch. Else, the folders will have the current date.
0
 
cheesebugahAuthor Commented:
Does this apply to the files contained within those folders also?
0
 
cheesebugahAuthor Commented:
Any particular sequence to apply that switch in the command?
0
 
NVITCommented:
It applies to the folders only. It shouldn't matter where you put the switch.
0
 
cheesebugahAuthor Commented:
What about the /W:   switch?  Might be advantageous?
0
 
NVITCommented:
I'm pondering the utility of that switch... Has there ever been a time I wanted to keep the file dates but not the folders'? No. Maybe they were rushed to get it out. IDK... Have a nice day, all.
0
 
NVITCommented:
I think /W is used when files might be open, e.g. worked on by users. In your case, the /W: switch has no use.
0
 
cheesebugahAuthor Commented:
I believe the /w switch is the wait time between retries if I am not mistaken?  The default is 30 seconds.
0
 
sirbountyCommented:
Correct, default wait is 30 seconds.
/ZB is backup/restartable mode meaning it should be able to deal with file locks, in most, but possibly not all, cases.
0
 
NVITCommented:
/Wait between /Retries. Yes. Some files, when opened by users and processes may be locked. If robocopy can't copy these, it will try after the /W:timeout

In your case, cheese, I believe you are copying files you know are free.

For cases like this, I set it to /W:2 /R:2. This makes the job finish faster. Also, use the /LOG switch. Then, you can review any /W and /R entries later, if any, and copy those manually.
0
 
cheesebugahAuthor Commented:
Hello,

I am finally getting back to this topic after being sick for several days.  The /DCOPY:T switch was causing the script to hang.  I had to take it out.  Might have something to do with an older version of robocopy?  I don't know.  Anyway, the script is still running as of this morning.  What is weird is in the log file, it shows the /PURGE switch, which is not in the actual command?  Anyone know why this switch would mysteriously show up in the log file?  

Thanks,
0
 
sirbountyCommented:
If you're using /MIR switch, it's basically the same thing as using /E /PURGE
and is translated that way in the log file.
0
 
cheesebugahAuthor Commented:
Okay, cool.  I can't believe the thing is still running and only about a quarter of the way finished after 4 and a half days!  I guess that's what you're going to get on an 11 year old system.

Thanks
0
 
sirbountyCommented:
were you able to use multi-threading (/MT:xx)
0
 
cheesebugahAuthor Commented:
I did not include that in my script.  I bought a USB 3.0 adapter card for the server, but have had all kinds of issues with this server, so I am reluctant to install it, seeing as how things are running okay now.  What kind of performance improvements have you seen with this switch?

Thanks
0
 
sirbountyCommented:
I've not done a side-by-side comparison, but can only say that it has helped in scenarios where the system had enough mojo to enable it....
0
 
cheesebugahAuthor Commented:
Thanks for all of your help.  This seems to have done the trick.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.