Backup script not overwritting files on WD WorldBook

Posted on 2010-08-13
Last Modified: 2012-05-10
I have the following backup script that is not overwritting files:

@echo off

REM Mapping network drives
net use U: /d /y
net use x: /d /y

REM Assigning the network drives to the NAS devices
net use U: \\NAS2\data
net use x: \\NAS3\data

set LogFile2Data="G:\FILESERVER_Backup_Logs\NAS2Databackuplog.log"

set LogFile3Data="G:\FILESERVER_Backup_Logs\NAS3Databackuplog.log"

REM Copies the DATA folder to Network Attached Storage device NAS2 and writes the log file on the G: drive
xcopy D:\Data U:\ /s/e/c/d/h/y >>%LogFile2Data% 2>&1
blat.exe - -body "FILESERVER DATA Backup to NAS2 completed with %Errorlevel% errors. The log can be viewed at \\FILESERVER\FILESERVER_Backup_Logs" -server EXCHANGESERVER -f -t -s "FILESERVER DATA Backup to NAS2 finished with %Errorlevel% errors"

REM Copies the DATA folder to Network Attached Storage device NAS3 and writes the log file on the G: drive
xcopy D:\Data X:\ /s/e/c/d/h/y >>%LogFile3Data% 2>&1
blat.exe - -body "FILESERVER DATA Backup to NAS3 completed with %Errorlevel% errors. The log can be viewed at \\FILESERVER\FILESERVER_Backup_Logs" -server EXCHANGESERVER -f -t -s "FILESERVER DATA Backup to NAS3 finished with %Errorlevel% errors"

REM Backup Complete!

The log file shows: insufficient disk space.

The backup is something like 385 GBs
The NAS drive is roughly 450 GBs

There should be plenty of room for this backup.
Question by:Sleestack90
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 5
  • 5

Expert Comment

ID: 33429745
It's worth using robocopy instead, just more reliable and many more options

Author Comment

ID: 33430263
I understand that idea, however I have several thing that are working just fine with xcopy.

Are you suggesting that will correct my problem? Will I need to rewrite alot of this in order for it to work? I will have to change over alot of scripts of this is the case, so I'd rather not.

Expert Comment

ID: 33430626
I've just run into problems with xcopy ages age with large folders and deep paths so moved to robocopy and haven't looked back, not sure why this isn't working for you, if anything I would expect xcopy to miss files rather than copy too much.  It may just be that it's creating duplicates or something, not sure.  Assuming your sizes given are correct it must have something to do with the underlying algorithms in xcopy in my opinion as your switches are correct.

Here's the equivalent robocopy syntax for what you've got, /log+ appends to the existing log, /log overwrites it, so up to you.  It would be worth it at least running a once off test with robocopy to see what the resulting disk usage is.  /e is for sub folders, /xo excludes old files so only changed files are copied.  If you have open files in the source you may want to add /r:3 as by default it will retry copying a file a million times before giving up!

robocopy d:\data u: /e /xo /log+:G:\FILESERVER_Backup_Logs\NAS2Databackuplog.log
Forrester Webinar: xMatters Delivers 261% ROI

Guest speaker Dean Davison, Forrester Principal Consultant, explains how a Fortune 500 communication company using xMatters found these results: Achieved a 261% ROI, Experienced $753,280 in net present value benefits over 3 years and Reduced MTTR by 91% for tier 1 incidents.


Author Comment

ID: 33430708
Hmmm....your right, it definately can't hurt it to run it once.

I will work on this asap.


Expert Comment

ID: 33430777
O, another thought is to use /purge - this removes files in the destination folder that no longer exists in the source.  This may actually be the cause or your issue...

Author Comment

ID: 33430864
@ebooyens:: can you provide me with an example of how I would use /purge in my script please?

Expert Comment

ID: 33430876
robocopy d:\data u: /e /xo /purge /log+:G:\FILESERVER_Backup_Logs\NAS2Databackuplog.log
LVL 11

Expert Comment

by:Ben Personick
ID: 33432197
You should use Robocopy to perform reliable backups, please try the attached script

Please note that I am adding the number to the NAS by specifying it as a second input when I call the copy job here:

FOR %%A IN (%SrcDirectories%) DO call :Start-DoCopy-Loop %%A "2"
SET "SkipLoop=NO"
FOR %%A IN (%SrcDirectories%) DO call :Start-DoCopy-Loop %%A "3"

:: Script: RoboCopy_Backup-Any-Drive.bat                                               			  ::
:: Version: 4.3                                                                                   ::
:: Copyright: Ben Personick                                                                       ::
:: Date: 2010-07-12                                                                               ::
::                                                                                                ::
::                                                                                                ::
:: Desc: Copies Local System settings, and documents to a backup location on the HD it's run from ::
::                                                                                                ::
	CALL :Start-Set-Date
	SET "IDL=nul"
	SET "ScriptName=RoboCopy-Backup"
	SET "Preamble=%ScriptName% --"
	CALL :Start-Echo " ---- Begining %ScriptName% ON %TDate% AT %TTime% ---- "

:: This section sets up default variable settings and sets the logging directory and file name.   ::
	:: Sets Relative Paths used to Copy later.  												  ::
	SET "SrcDrv=D:"
	:: Set the following Variable to hold a list of Directories under %SrcDrv%		 			  ::
	:: NOTE: Encapsulate Directories with double Quotes and separate them with Commas
	:: CAREFUL: Ensure not to leave directories below set as NOMATCH, it will terminate the copy job prematurely!
	:: NOTE: To skip SET SrcDirectories=NOMATCH
	:: eg. Usage: SET SrcDirectories="Data", "Data2", "This\That\Another", "This\That\Some Other"
	SET SrcDirectories="Data"
	::The Following Variable Sets the Folder path to store the user files in
	:: If DrvDst is Set to NOMATCH it will find the drive the batch file is double-clicked from.  ::
	SET "DrvDst=\\NAS"
	SET "DstShare=Data"
	:: This section finds the Drive The Batch File is run from for use later in the script        ::
	IF /I "%DrvDst%"=="NOMATCH"	Call :Start-Find-DL-Run-From "%DrvDst%"
	SET "DrvDst=%TDrvFind%"&SET "TDrvFind=NOMATCH"
	CALL :Start-Echo "Drive Run From is %DrvDst%"
	:: This Section Sets up logging variables.													  ::
	SET "IDir=%DrvDst%\Admin\%ScriptName%\Logs"
	SET "ILog=%ScriptName%_%TDate%_%TTime%.Log"
	CALL :Start-Echo "ILOG = %ILOG%"
	SET "IDL=%IDir%\%ILog%"
	IF /I "%IDL%" NEQ "nul" IF /I "%IDL%" NEQ "" CALL :Start-Create-Dir "%IDL%"
	:: This Section Sets Default Values for match variables, don't change except for testing.	  ::
	Set vTemp=NOMATCH
	SET RoBoVista=NO
	SET SkipLoop=NO


	CALL :Start-Set-Date
	CALL :Start-Echo " "
	CALL :Start-Echo "------------------------------------------------------"
	CALL :Start-Echo "BEGIN %ScriptName% Script on %TDate% AT %TTime%"
	CALL :Start-Echo "------------------------------------------------------"
	CALL :Start-Echo " "

:: This section finds the version of the operating system for use later in the script             ::

		FOR /F "Tokens=4 Delims=[] " %%V IN ('Ver ^| find "Version"') DO CALL :Start-Ver-Match "%%V"
		GOTO End-Ver-Match
			SET TVer=%~1
			Call :Start-Echo "Windows Version is: %TVer%"
			IF /I "%TVer%" GTR "3" IF /I "%TVer:~3,1%" NEQ "." SET "OSVer=Win9x"
			IF /I "%TVer%"=="3.10.528" SET "OSVer=NT3.1"
			IF /I "%TVer%"=="3.50.807" SET "OSVer=NT3.5"
			IF /I "%TVer%"=="3.51.1057" SET "OSVer=NT3.51"
			IF /I "%TVer%"=="4.0.1381" SET "OSVer=NT4.0"
			IF /I "%TVer%"=="5.0.2195" SET "OSVer=2K"
			IF /I "%TVer%"=="5.1.2600" SET "OSVer=XP"
			IF /I "%TVer%"=="5.2.3790" SET "OSVer=2K3"
			IF /I "%TVer:~0,7%"=="6.0.600" SET "OSVer=Vista"&SET "RoBoVista=GO"
			IF /I "%TVer%"=="6.0.6001" SET "OSVer=Vista"&SET "RoBoVista=GO"&SET OSSP=SP1
			IF /I "%TVer%"=="6.0.6002" SET "OSVer=Vista"&SET "RoBoVista=GO"&SET OSSP=SP2
			IF /I "%TVer:~0,7%"=="6.1.760" SET "OSVer=7"&SET "RoBoVista=GO"
		CALL :Start-Echo "End-Ver-Match"
		IF "%RoBoVista%" NEQ "GO" GOTO 2K8?-end
			CALL :Start-Echo "2k8?-start"
			FOR /F "Tokens=5" %%R IN ('reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion" /v ProductName ^| FIND /I "2008"') DO SET "OSVer=2K8"
			::START /WAIT REGEDIT.EXE /E "%Temp%.\%%~n0.dat" "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion" 
			::  FOR /F "tokens=2 delims==" %%A IN ('TYPE "%Temp%.\%%~n0.dat" ^| FIND /I "ProductName" ^| Find /I "2008"') DO SET OSVer=2K8

		CALL :Start-Echo "Service pack: %OSSP%"
		IF /I "%OSSP%"=="NOMATCH" IF /I "%TVer%" GEQ "5.1.2600" FOR /F "Tokens=5" %%R IN ('reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion" /v CSDVersion 2^>^&1 ^| FIND /I "CSDVersion"') DO SET "OSSP=SP%%R"
	CALL :Start-Echo "End Find SP"

:: This section finds Whether the Operating system is 32bit or 64bit for use later in the script. ::
:: This section is intended to determin if the OS is Desktop or Server for differing installs.    ::
		CALL :Start-Echo "Checking if Desktop OS"
		CALL :Start-Echo "DTOSRun = %DTOSRun%, OSVer = %OSVer%, TVer = %TVer%"
		CALL :Start-Echo "Set DTOSRun to GO Only if OSVer is a desktop OS"
		IF /I "%OSVer%"=="XP" SET "DTOSRun=GO"
		IF /I "%OSVer%"=="Vista" SET "DTOSRun=GO"
		IF /I "%OSVer%"=="7" SET "DTOSRun=GO"
	CALL :Start-Echo "OS Version = %OSVer%"
	CALL :Start-Echo "OS SP Version = %OSSP%"
	CALL :Start-Echo "OS Architechture = %OSbit% bit"
	CALL :Start-Echo "OS Desktop (NO or GO) = %DTOSRun%"



		CALL :Start-Set-Date
		CALL :Start-Echo "Start-DoCopy ON %TDate% AT %TTime%"
		IF /I "%RoBoVista%" NEQ "GO" SET "RoBoVista=/ZB /E /MIR /LOG+:%IDL% /TEE /NP /R:2 /W:2"
		IF /I "%RoBoVista%"=="GO" SET "RoBoVista=/ZB /E /DCOPY:T /MIR /LOG+:%IDL% /TEE /NP /R:2 /W:2"
		REM Below is the For Loop, %%A is the term pulled from the set to pass to the command loop
		REM  The terms are located within the open and close parenthesis, and quotes allow spaces to be ignored
		REM  If Quotes were not used, the spaces would be seen as a delimiter just as commas are.
		FOR %%A IN (%SrcDirectories%) DO call :Start-DoCopy-Loop %%A "2"
		SET "SkipLoop=NO"
		FOR %%A IN (%SrcDirectories%) DO call :Start-DoCopy-Loop %%A "3"
		GOTO End-DoCopy-Loop
			IF /I "%~1"=="NOMATCH" SET "SkipLoop=YES"
			IF /I "%SkipLoop%"=="YES" GOTO :EOF
			CALL :Start-Set-Date
			REM This is the command loop, everythign inside this loop will be preocessed as a separate batch file
			REM we use %~1 for the variable the Tilde says to strip quotes off the passed text XD
			CALL :Start-Echo "ON %TDate% AT %TTime% Start-%~1"
			SET "TURL=%SrcDrv%\%~1"
			SET "TDst=%DrvDst%%~2\%~1"
			Robocopy "%TURL%" "%TmpDstDrv%" * %RoBoVista%

	CALL :Start-Set-Date
	CALL :Start-Echo "End-DoCopy ON %TDate% AT %TTime%"

GOTO :End-Script

:: This section is intended to determin if the OS is Desktop or Server for differing installs.    ::
GOTO End-Subs
GOTO End-Script
	GOTO :End-Set-Date
		SET TDate=%date:~10,4%-%date:~4,2%-%date:~7,2%
		FOR /F "Tokens=1-3 Delims=:" %%T IN ("%TIME: =0%") DO SET "TTime=%%T.%%U.%%V"
	GOTO End-Echo
		Set TEcho=%~1
		ECHO %Preamble% %TEcho% --
		IF EXIST "%IDir%" ECHO %Preamble% %TEcho% -- >> %IDL%
	GOTO End-Find-DL-Run-From
		SET "TDrvFind=%~1"
		FOR %%A IN (A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z) DO call :Start-DL-Loop %%A
			If "%TDrvFind%" NEQ "NOMATCH" GOTO :EOF
			SET "TDrv=%1:"
			FOR /F %%a IN ('Dir /D "%TDrv%\" 2^>^&1 ^| FIND /I " Directory of " ^| Find /I "%TDrv%\"') DO SET "TDrvFind=%TDrv%"
	GOTO End-Create-Dir
		SET TDir=%~1
		:: CALL :Start-Echo "%TDir%"
		CALL :Start-Echo "Checking For %TDir%"
		IF EXIST "%TDir%" CALL :Start-Echo "%TDir% -- Found!"
		CALL :Start-Echo "Creating %TDir%"		
		FOR /F "Tokens=*" %%D IN ('MD "%TDir%" 2^>^&1') DO SET "MDOutput=%%D"&IF /I "%D" NEQ "A subdirectory or file %TDir% already exists." CALL :Start-Echo "The Command 'MD %TDir%' Completed with Error: %%D"
		IF EXIST "%TDir%" CALL :Start-Echo "Created %TDir%"


	CALL :Start-Set-Date
	CALL :Start-Echo "!!--!! %ScriptName% Exiting ON %TDate% AT %TTime% !!--!!

Open in new window

LVL 11

Expert Comment

by:Ben Personick
ID: 33435524
oh FYI you want to use MIR, it copies AND purges.

Expert Comment

ID: 33435675
Yeah, /MIR is the same as /e /purge
LVL 11

Expert Comment

by:Ben Personick
ID: 33435740
I mention that becausr my script already includes the /MIR option.

Author Comment

ID: 33446105
So after looking at all of this...I am getting ready to test it. It looks like I might just be able to use the following:

robocopy d:\data u: /e /xo /r:3 /purge /log:%LogFile2Data% 2>&1

So this SHOULD copy the data from D: to U: with all the sub folder, while only getting the files that have changed....and if a file is open it will retry copying it 3 times. Then it will write the log file without appending.

****Question: with the purge option...what happens if someone deletes a file or folder in the source that you want to get back....but after you've used the /purge deletes this in the destination right? So a user deletes a folder by mistake last night, your backup runs, and all of a sudden you don't have that information anymore because it deletes the copy of the folder you had in backup until yesterday since it no long exsist in the source destination....?

SO maybe my whole problem isn't xcopy (although I understand everyone thinks I need to move to robocopy) problem SOUNDS like I just don't have enough space on one of my NAS drives because my backup script doesn't delete old files, so it continues to collect files that are probably no longer in use or any good.

I believe I need something that compares the dates on a deleted file and if the source has been gone for 30 will purge the destination file as well. IS this possible with a time stamp or something? If so any code examples please?

LVL 11

Expert Comment

by:Ben Personick
ID: 33447019
Well first off you didn't use /ZB if you don't you aren't copying files through he network in a restartable form (the B will switch to normal 'backup' form if the first copy attempt fails in order to get soft-locked files)

secondly /XO is unnessessary, Robocopy ojnly copies changed files.

Thirdly if you want to copy all files and THEN delete files OLDER than N days

First you run the normal copy job without any /MIR or /Purge options

THEN you run the same job only you specify /MinAge:n and use the /NoCopy /Purge options.

that will tell robocopy only to delete files not in the source and older than X days.

Of course if you want to actually tell when a file was deleted then you can't use this option unles you DON'T Copy file time in step 1.  if you are really trying to run backups you might consider using NTBackup instead.

Onhe Final Note the 2>&1 redirection is pointless here as you are already logging both StdIn and StdOut by virtue of the /Log option..

Here are the commands:

Step1: Robocopy "D:\Data" "U:" * /COPY:DASO /E /ZB /Log+%LogFileData% R:3 W:2

Step2: Robocopy "D:\Data" "U:" * /NoCopy /Purge /MinAge:30 /S /ZB /Log+%LogFileData% /R:3 /W:2

Author Comment

ID: 33448346
@QCubed: I am looking over this and trying take all of that in (being that I am new to scripting).

Will the code you provided still pass error levels back so when I email the results of the log file %errorlevel% to myself I will continue to provide me with that information?

LVL 11

Accepted Solution

Ben Personick earned 500 total points
ID: 33449039
if you are trying to capture error levels in another part than you should be able to do so either the redirection.  Error level codes are separate information left over in the environment after a command executes.

  if you are trying to capture error levels into a log file to then parse the log file to look for them, then yet you will need to leave the 2>&1 in assuming ti worked for you that way before. 2>&1 says to take StdError and pipe the output to StandardOut instead..

However it was my understanding that Errorlevels were transient data set int he environment on completion of a command and not part of the StdError output.

Expert Comment

ID: 33455513
Sleestack, so the solution to your problem is using the 2>&1 to report error level codes or did you assign the solution to that comment by accident?  QCubed's comments are helpful indeed but I think you owe me at least a few points for pointing out that your not deleting files in the destination which seems to be the main issue here.  Thanks.

Featured Post

Space-Age Communications Transitions to DevOps

ViaSat, a global provider of satellite and wireless communications, securely connects businesses, governments, and organizations to the Internet. Learn how ViaSat’s Network Solutions Engineer, drove the transition from a traditional network support to a DevOps-centric model.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The following is a collection of cases for strange behaviour when using advanced techniques in DOS batch files. You should have some basic experience in batch "programming", as I'm assuming some knowledge and not further explain the basics. For some…
When you receive another warning that your shared drive is almost full and you have asked your users to clean out old files again and again, here is a single command that may help. This command will place all the files that have not been used rec…
Attackers love to prey on accounts that have privileges. Reducing privileged accounts and protecting privileged accounts therefore is paramount. Users, groups, and service accounts need to be protected to help protect the entire Active Directory …

756 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question