Solved

Speeding up Robocopy

Posted on 2010-09-16
12
1,131 Views
Last Modified: 2012-06-21
I am using robocopy to run some backups to a couple of NAS devices.

I switched to that after about 1 million people on here told me to make the change from xcopy.

This process takes FOREVER now. I am running it on Server 2003, so as far as I know it can't multi-thread.

How do you speed up Robocopy?
0
Comment
Question by:Sleestack90
  • 7
  • 3
  • 2
12 Comments
 
LVL 11

Expert Comment

by:willettmeister
ID: 33691530
what switches are you using to make the copy.  My intial thought is to cut down the number of retires and the wait time between retries.
0
 

Author Comment

by:Sleestack90
ID: 33691746
I have attached the script, have a look.
@echo off

REM Get the date for log files

@For /F "tokens=2,3,4 delims=/ " %%A in ('Date /t') do @(

    Set Month=%%A

    Set Day=%%B

    Set Year=%%C

)

set year=%Year:~2%

@echo FILESERVER DATA backup process for %Month%/%Day%/%Year%.



set LogFile="G:\FILESERVER_Backup_Logs\%Month%%Day%%Year%_NAS01_FILESERVER_DATA.log"

@echo Please wait: DATA backup to NAS01 in PROGRESS!

@echo Copying DATA to NAS02 and logging to \\FILESERVER\FILESERVER_Backup_Logs\

robocopy d:\data \\NAS01\DATA /zb /mir /NP /NDL /NFL /r:1 /w:1 /log:%LogFile%





if errorlevel 16 set result="***FATAL ERROR***" & goto nas01

if errorlevel 15 set result="OKCOPY + FAIL + MISMATCHES + XTRA" & goto nas01

if errorlevel 14 set result="FAIL + MISMATCHES + XTRA" & goto nas01

if errorlevel 13 set result="OKCOPY + FAIL + MISMATCHES" & goto nas01

if errorlevel 12 set result="FAIL + MISMATCHES" & goto nas01

if errorlevel 11 set result="OKCOPY + FAIL + XTRA" & goto nas01

if errorlevel 10 set result="FAIL + XTRA" & goto nas01

if errorlevel 9 set result="OKCOPY + FAIL" & goto nas01

if errorlevel 8 set result="FAIL" & goto nas01

if errorlevel 7 set result="OKCOPY + MISMATCHES + XTRA" & goto nas01

if errorlevel 6 set result="MISMATCHES + XTRA" & goto nas01

if errorlevel 5 set result="OKCOPY + MISMATCHES" & goto nas01

if errorlevel 4 set result="MISMATCHES" & goto nas01

if errorlevel 3 set result="OKCOPY + XTRA" & goto nas01

if errorlevel 2 set result="XTRA" & goto nas01

if errorlevel 1 set result="OKCOPY" & goto nas01

if errorlevel 0 set result="--no change--" & goto nas01

:nas01



blat -t DOMAIN-InformationServices@DOMAIN.com -f Backup6@DOMAIN.com  -s "FILESERVER DATA Backup to NAS01 finished with a result of %result%" -server EXCHANGE -body "FILESERVER DATA Backup to NAS01 completed with a result of %result%. Logs are available at \\FILESERVER\FILESERVER_Backup_Logs"



@echo DATA backup to NAS01 complete!



set LogFile="G:\FILESERVER_Backup_Logs\%Month%%Day%%Year%_NAS02_FILESERVER_DATA.log"

@echo Please wait: DATA backup to NAS02 in PROGRESS!

@echo Copying DATA to NAS02 and logging to \\FILESERVER\FILESERVER_Backup_Logs\

robocopy d:\data \\NAS02\DATA /zb /mir /NP /NDL /NFL /r:1 /w:1 /log:%LogFile%



if errorlevel 16 set result="***FATAL ERROR***" & goto nas02

if errorlevel 15 set result="OKCOPY + FAIL + MISMATCHES + XTRA" & goto nas02

if errorlevel 14 set result="FAIL + MISMATCHES + XTRA" & goto nas02

if errorlevel 13 set result="OKCOPY + FAIL + MISMATCHES" & goto nas02

if errorlevel 12 set result="FAIL + MISMATCHES" & goto nas02

if errorlevel 11 set result="OKCOPY + FAIL + XTRA" & goto nas02

if errorlevel 10 set result="FAIL + XTRA" & goto nas02

if errorlevel 9 set result="OKCOPY + FAIL" & goto nas02

if errorlevel 8 set result="FAIL" & goto nas02

if errorlevel 7 set result="OKCOPY + MISMATCHES + XTRA" & goto nas02

if errorlevel 6 set result="MISMATCHES + XTRA" & goto nas02

if errorlevel 5 set result="OKCOPY + MISMATCHES" & goto nas02

if errorlevel 4 set result="MISMATCHES" & goto nas02

if errorlevel 3 set result="OKCOPY + XTRA" & goto nas02

if errorlevel 2 set result="XTRA" & goto nas02

if errorlevel 1 set result="OKCOPY" & goto nas02

if errorlevel 0 set result="--no change--" & goto nas02

:nas02



blat -t DOMAIN-InformationServices@DOMAIN.com -f Backup7@DOMAIN.com  -s "FILESERVER DATA Backup to NAS02 finished with a result of %result%" -server EXCHANGE -body "FILESERVER DATA Backup to NAS02 completed with a result of %result%. Logs are available at \\FILESERVER\FILESERVER_Backup_Logs"





@echo DATA backup to NAS02 complete!

Open in new window

0
 
LVL 11

Expert Comment

by:willettmeister
ID: 33692344
Your script looks pretty good.  The only things that I can say is that my version of robocopy does support mutlithreading using the /mt switch.  
0
 

Author Comment

by:Sleestack90
ID: 33692704
willettmeister: you must be running it on Windows Vista, Windows 7, or Windows Server 2008 then?

If not, I would REALLY like to know what version you are running, cause if there is one that supports multi -threading for Server 2003 .... that would solve me problem for sure.
0
 
LVL 11

Expert Comment

by:willettmeister
ID: 33693633
yep you are right.  It looks like the 2003 is not multi-threaded but the version I have is.  sorry
0
 

Author Comment

by:Sleestack90
ID: 33693812
no worries. I am wondering if it makes sense to break the backup up into multiple parts running at the same time.....

like folder that start with A-G, H-N, O-T, U-Z.....

This should cut the time by 1/4....or at least i'd think it would.
0
U.S. Department of Agriculture and Acronis Access

With the new era of mobile computing, smartphones and tablets, wireless communications and cloud services, the USDA sought to take advantage of a mobilized workforce and the blurring lines between personal and corporate computing resources.

 
LVL 63

Expert Comment

by:SysExpert
ID: 33695193
Any errors in the log file ?
0
 

Author Comment

by:Sleestack90
ID: 33695652
SysExpert:Yes and no....there are some errors, but these are related to files being open or standard stuff like that.

The problem is that there is ALOT of data to transfer.....
0
 
LVL 63

Accepted Solution

by:
SysExpert earned 500 total points
ID: 33697063
the /mir options will take a while to do the first time, especially if retry/ resync options are on.

But once it is done, after that it should run quickly since the data changes should be relatively minor, unless you have huge files that change often.

0
 

Author Comment

by:Sleestack90
ID: 33703506
SysExpert: understood i will give it time
0
 

Author Comment

by:Sleestack90
ID: 33736921
So ok.

New direction. I broke up each copy of each directory into seperate jobs, examples:

NAS01\USER = Job 1
NAS02\USER = Job 2

NAS01\DATA = Job 3
NAS02\DATA = Job 4

Each job looks very similar to the attached code snipet.

The problem now is....i can't tell is Robocopy is hanging or what...cause it gets through alot of the job, but it doesn't send the email like it's susposed to...and it never quite closes out....

Suggestions?
@echo off

REM Get the date for log files

@For /F "tokens=2,3,4 delims=/ " %%A in ('Date /t') do @(

    Set Month=%%A

    Set Day=%%B

    Set Year=%%C

)

set year=%Year:~2%

@echo FILESERVER DATA backup process for %Month%/%Day%/%Year%.



set LogFile="G:\FILESERVER_Backup_Logs\%Month%%Day%%Year%_NAS01_FILESERVER_DATA.log"

@echo Please wait: DATA backup to NAS01 in PROGRESS!

@echo Copying DATA to NAS02 and logging to \\FILESERVER\FILESERVER_Backup_Logs\

robocopy d:\data \\NAS01\DATA /zb /mir /NP /NDL /NFL /r:1 /w:1 /log:%LogFile%





blat -t DOMAIN-InformationServices@DOMAIN.com -f Backup6@DOMAIN.com  -s "FILESERVER DATA Backup to NAS01 finished with a error level of %errorlevel%" -server EXCHANGE -body "FILESERVER DATA Backup to NAS01 completed with a error level of %errorlevel%. Logs are available at \\FILESERVER\FILESERVER_Backup_Logs"



@echo DATA backup to NAS01 complete!

Open in new window

0
 

Author Comment

by:Sleestack90
ID: 33737048
well ok, one of these jobs sent an email...so maybe the jobs are hanging.
0

Featured Post

New My Cloud Pro Series - organize everything!

With space to keep virtually everything, the My Cloud Pro Series offers your team the network storage to edit, save and share production files from anywhere with an internet connection. Compatible with both Mac and PC, you're able to protect your content regardless of OS.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Microservice architecture adoption brings many advantages, but can add intricacy. Selecting the right orchestration tool is most important for business specific needs.
Finding original email is quite difficult due to their duplicates. From this article, you will come to know why multiple duplicates of same emails appear and how to delete duplicate emails from Outlook securely and instantly while vital emails remai…
This tutorial will walk an individual through the steps necessary to enable the VMware\Hyper-V licensed feature of Backup Exec 2012. In addition, how to add a VMware server and configure a backup job. The first step is to acquire the necessary licen…
This tutorial will walk an individual through the process of configuring basic necessities in order to use the 2010 version of Data Protection Manager. These include storage, agents, and protection jobs. Launch Data Protection Manager from the deskt…

911 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

24 Experts available now in Live!

Get 1:1 Help Now