Backup script not overwritting files on WD WorldBook

Posted on 2010-08-13
Medium Priority
Last Modified: 2012-05-10
I have the following backup script that is not overwritting files:

@echo off

REM Mapping network drives
net use U: /d /y
net use x: /d /y

REM Assigning the network drives to the NAS devices
net use U: \\NAS2\data
net use x: \\NAS3\data

set LogFile2Data="G:\FILESERVER_Backup_Logs\NAS2Databackuplog.log"

set LogFile3Data="G:\FILESERVER_Backup_Logs\NAS3Databackuplog.log"

REM Copies the DATA folder to Network Attached Storage device NAS2 and writes the log file on the G: drive
xcopy D:\Data U:\ /s/e/c/d/h/y >>%LogFile2Data% 2>&1
blat.exe - -body "FILESERVER DATA Backup to NAS2 completed with %Errorlevel% errors. The log can be viewed at \\FILESERVER\FILESERVER_Backup_Logs" -server EXCHANGESERVER -f Backup6@DOMAIN.com -t DOMAIN-InformationServices@DOMAIN.com -s "FILESERVER DATA Backup to NAS2 finished with %Errorlevel% errors"

REM Copies the DATA folder to Network Attached Storage device NAS3 and writes the log file on the G: drive
xcopy D:\Data X:\ /s/e/c/d/h/y >>%LogFile3Data% 2>&1
blat.exe - -body "FILESERVER DATA Backup to NAS3 completed with %Errorlevel% errors. The log can be viewed at \\FILESERVER\FILESERVER_Backup_Logs" -server EXCHANGESERVER -f Backup7@DOMAIN.com -t DOMAIN-InformationServices@DOMAIN.com -s "FILESERVER DATA Backup to NAS3 finished with %Errorlevel% errors"

REM Backup Complete!

The log file shows: insufficient disk space.

The backup is something like 385 GBs
The NAS drive is roughly 450 GBs

There should be plenty of room for this backup.
Question by:Sleestack90
  • 6
  • 5
  • 5

Expert Comment

ID: 33429745
It's worth using robocopy instead, just more reliable and many more options

Author Comment

ID: 33430263
I understand that idea, however I have several thing that are working just fine with xcopy.

Are you suggesting that will correct my problem? Will I need to rewrite alot of this in order for it to work? I will have to change over alot of scripts of this is the case, so I'd rather not.

Expert Comment

ID: 33430626
I've just run into problems with xcopy ages age with large folders and deep paths so moved to robocopy and haven't looked back, not sure why this isn't working for you, if anything I would expect xcopy to miss files rather than copy too much.  It may just be that it's creating duplicates or something, not sure.  Assuming your sizes given are correct it must have something to do with the underlying algorithms in xcopy in my opinion as your switches are correct.

Here's the equivalent robocopy syntax for what you've got, /log+ appends to the existing log, /log overwrites it, so up to you.  It would be worth it at least running a once off test with robocopy to see what the resulting disk usage is.  /e is for sub folders, /xo excludes old files so only changed files are copied.  If you have open files in the source you may want to add /r:3 as by default it will retry copying a file a million times before giving up!

robocopy d:\data u: /e /xo /log+:G:\FILESERVER_Backup_Logs\NAS2Databackuplog.log
Get expert help—faster!

Need expert help—fast? Use the Help Bell for personalized assistance getting answers to your important questions.


Author Comment

ID: 33430708
Hmmm....your right, it definately can't hurt it to run it once.

I will work on this asap.


Expert Comment

ID: 33430777
O, another thought is to use /purge - this removes files in the destination folder that no longer exists in the source.  This may actually be the cause or your issue...

Author Comment

ID: 33430864
@ebooyens:: can you provide me with an example of how I would use /purge in my script please?

Expert Comment

ID: 33430876
robocopy d:\data u: /e /xo /purge /log+:G:\FILESERVER_Backup_Logs\NAS2Databackuplog.log
LVL 15
ID: 33432197
You should use Robocopy to perform reliable backups, please try the attached script

Please note that I am adding the number to the NAS by specifying it as a second input when I call the copy job here:

FOR %%A IN (%SrcDirectories%) DO call :Start-DoCopy-Loop %%A "2"
SET "SkipLoop=NO"
FOR %%A IN (%SrcDirectories%) DO call :Start-DoCopy-Loop %%A "3"

:: Script: RoboCopy_Backup-Any-Drive.bat                                               			  ::
:: Version: 4.3                                                                                   ::
:: Copyright: Ben Personick                                                                       ::
:: Date: 2010-07-12                                                                               ::
::                                                                                                ::
::                                                                                                ::
:: Desc: Copies Local System settings, and documents to a backup location on the HD it's run from ::
::                                                                                                ::
	CALL :Start-Set-Date
	SET "IDL=nul"
	SET "ScriptName=RoboCopy-Backup"
	SET "Preamble=%ScriptName% --"
	CALL :Start-Echo " ---- Begining %ScriptName% ON %TDate% AT %TTime% ---- "

:: This section sets up default variable settings and sets the logging directory and file name.   ::
	:: Sets Relative Paths used to Copy later.  												  ::
	SET "SrcDrv=D:"
	:: Set the following Variable to hold a list of Directories under %SrcDrv%		 			  ::
	:: NOTE: Encapsulate Directories with double Quotes and separate them with Commas
	:: CAREFUL: Ensure not to leave directories below set as NOMATCH, it will terminate the copy job prematurely!
	:: NOTE: To skip SET SrcDirectories=NOMATCH
	:: eg. Usage: SET SrcDirectories="Data", "Data2", "This\That\Another", "This\That\Some Other"
	SET SrcDirectories="Data"
	::The Following Variable Sets the Folder path to store the user files in
	:: If DrvDst is Set to NOMATCH it will find the drive the batch file is double-clicked from.  ::
	SET "DrvDst=\\NAS"
	SET "DstShare=Data"
	:: This section finds the Drive The Batch File is run from for use later in the script        ::
	IF /I "%DrvDst%"=="NOMATCH"	Call :Start-Find-DL-Run-From "%DrvDst%"
	SET "DrvDst=%TDrvFind%"&SET "TDrvFind=NOMATCH"
	CALL :Start-Echo "Drive Run From is %DrvDst%"
	:: This Section Sets up logging variables.													  ::
	SET "IDir=%DrvDst%\Admin\%ScriptName%\Logs"
	SET "ILog=%ScriptName%_%TDate%_%TTime%.Log"
	CALL :Start-Echo "ILOG = %ILOG%"
	SET "IDL=%IDir%\%ILog%"
	IF /I "%IDL%" NEQ "nul" IF /I "%IDL%" NEQ "" CALL :Start-Create-Dir "%IDL%"
	:: This Section Sets Default Values for match variables, don't change except for testing.	  ::
	Set vTemp=NOMATCH
	SET RoBoVista=NO
	SET SkipLoop=NO


	CALL :Start-Set-Date
	CALL :Start-Echo " "
	CALL :Start-Echo "------------------------------------------------------"
	CALL :Start-Echo "BEGIN %ScriptName% Script on %TDate% AT %TTime%"
	CALL :Start-Echo "------------------------------------------------------"
	CALL :Start-Echo " "

:: This section finds the version of the operating system for use later in the script             ::

		FOR /F "Tokens=4 Delims=[] " %%V IN ('Ver ^| find "Version"') DO CALL :Start-Ver-Match "%%V"
		GOTO End-Ver-Match
			SET TVer=%~1
			Call :Start-Echo "Windows Version is: %TVer%"
			IF /I "%TVer%" GTR "3" IF /I "%TVer:~3,1%" NEQ "." SET "OSVer=Win9x"
			IF /I "%TVer%"=="3.10.528" SET "OSVer=NT3.1"
			IF /I "%TVer%"=="3.50.807" SET "OSVer=NT3.5"
			IF /I "%TVer%"=="3.51.1057" SET "OSVer=NT3.51"
			IF /I "%TVer%"=="4.0.1381" SET "OSVer=NT4.0"
			IF /I "%TVer%"=="5.0.2195" SET "OSVer=2K"
			IF /I "%TVer%"=="5.1.2600" SET "OSVer=XP"
			IF /I "%TVer%"=="5.2.3790" SET "OSVer=2K3"
			IF /I "%TVer:~0,7%"=="6.0.600" SET "OSVer=Vista"&SET "RoBoVista=GO"
			IF /I "%TVer%"=="6.0.6001" SET "OSVer=Vista"&SET "RoBoVista=GO"&SET OSSP=SP1
			IF /I "%TVer%"=="6.0.6002" SET "OSVer=Vista"&SET "RoBoVista=GO"&SET OSSP=SP2
			IF /I "%TVer:~0,7%"=="6.1.760" SET "OSVer=7"&SET "RoBoVista=GO"
		CALL :Start-Echo "End-Ver-Match"
		IF "%RoBoVista%" NEQ "GO" GOTO 2K8?-end
			CALL :Start-Echo "2k8?-start"
			FOR /F "Tokens=5" %%R IN ('reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion" /v ProductName ^| FIND /I "2008"') DO SET "OSVer=2K8"
			::START /WAIT REGEDIT.EXE /E "%Temp%.\%%~n0.dat" "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion" 
			::  FOR /F "tokens=2 delims==" %%A IN ('TYPE "%Temp%.\%%~n0.dat" ^| FIND /I "ProductName" ^| Find /I "2008"') DO SET OSVer=2K8

		CALL :Start-Echo "Service pack: %OSSP%"
		IF /I "%OSSP%"=="NOMATCH" IF /I "%TVer%" GEQ "5.1.2600" FOR /F "Tokens=5" %%R IN ('reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion" /v CSDVersion 2^>^&1 ^| FIND /I "CSDVersion"') DO SET "OSSP=SP%%R"
	CALL :Start-Echo "End Find SP"

:: This section finds Whether the Operating system is 32bit or 64bit for use later in the script. ::
:: This section is intended to determin if the OS is Desktop or Server for differing installs.    ::
		CALL :Start-Echo "Checking if Desktop OS"
		CALL :Start-Echo "DTOSRun = %DTOSRun%, OSVer = %OSVer%, TVer = %TVer%"
		CALL :Start-Echo "Set DTOSRun to GO Only if OSVer is a desktop OS"
		IF /I "%OSVer%"=="XP" SET "DTOSRun=GO"
		IF /I "%OSVer%"=="Vista" SET "DTOSRun=GO"
		IF /I "%OSVer%"=="7" SET "DTOSRun=GO"
	CALL :Start-Echo "OS Version = %OSVer%"
	CALL :Start-Echo "OS SP Version = %OSSP%"
	CALL :Start-Echo "OS Architechture = %OSbit% bit"
	CALL :Start-Echo "OS Desktop (NO or GO) = %DTOSRun%"



		CALL :Start-Set-Date
		CALL :Start-Echo "Start-DoCopy ON %TDate% AT %TTime%"
		IF /I "%RoBoVista%" NEQ "GO" SET "RoBoVista=/ZB /E /MIR /LOG+:%IDL% /TEE /NP /R:2 /W:2"
		IF /I "%RoBoVista%"=="GO" SET "RoBoVista=/ZB /E /DCOPY:T /MIR /LOG+:%IDL% /TEE /NP /R:2 /W:2"
		REM Below is the For Loop, %%A is the term pulled from the set to pass to the command loop
		REM  The terms are located within the open and close parenthesis, and quotes allow spaces to be ignored
		REM  If Quotes were not used, the spaces would be seen as a delimiter just as commas are.
		FOR %%A IN (%SrcDirectories%) DO call :Start-DoCopy-Loop %%A "2"
		SET "SkipLoop=NO"
		FOR %%A IN (%SrcDirectories%) DO call :Start-DoCopy-Loop %%A "3"
		GOTO End-DoCopy-Loop
			IF /I "%~1"=="NOMATCH" SET "SkipLoop=YES"
			IF /I "%SkipLoop%"=="YES" GOTO :EOF
			CALL :Start-Set-Date
			REM This is the command loop, everythign inside this loop will be preocessed as a separate batch file
			REM we use %~1 for the variable the Tilde says to strip quotes off the passed text XD
			CALL :Start-Echo "ON %TDate% AT %TTime% Start-%~1"
			SET "TURL=%SrcDrv%\%~1"
			SET "TDst=%DrvDst%%~2\%~1"
			Robocopy "%TURL%" "%TmpDstDrv%" * %RoBoVista%

	CALL :Start-Set-Date
	CALL :Start-Echo "End-DoCopy ON %TDate% AT %TTime%"

GOTO :End-Script

:: This section is intended to determin if the OS is Desktop or Server for differing installs.    ::
GOTO End-Subs
GOTO End-Script
	GOTO :End-Set-Date
		SET TDate=%date:~10,4%-%date:~4,2%-%date:~7,2%
		FOR /F "Tokens=1-3 Delims=:" %%T IN ("%TIME: =0%") DO SET "TTime=%%T.%%U.%%V"
	GOTO End-Echo
		Set TEcho=%~1
		ECHO %Preamble% %TEcho% --
		IF EXIST "%IDir%" ECHO %Preamble% %TEcho% -- >> %IDL%
	GOTO End-Find-DL-Run-From
		SET "TDrvFind=%~1"
		FOR %%A IN (A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z) DO call :Start-DL-Loop %%A
			If "%TDrvFind%" NEQ "NOMATCH" GOTO :EOF
			SET "TDrv=%1:"
			FOR /F %%a IN ('Dir /D "%TDrv%\" 2^>^&1 ^| FIND /I " Directory of " ^| Find /I "%TDrv%\"') DO SET "TDrvFind=%TDrv%"
	GOTO End-Create-Dir
		SET TDir=%~1
		:: CALL :Start-Echo "%TDir%"
		CALL :Start-Echo "Checking For %TDir%"
		IF EXIST "%TDir%" CALL :Start-Echo "%TDir% -- Found!"
		CALL :Start-Echo "Creating %TDir%"		
		FOR /F "Tokens=*" %%D IN ('MD "%TDir%" 2^>^&1') DO SET "MDOutput=%%D"&IF /I "%D" NEQ "A subdirectory or file %TDir% already exists." CALL :Start-Echo "The Command 'MD %TDir%' Completed with Error: %%D"
		IF EXIST "%TDir%" CALL :Start-Echo "Created %TDir%"


	CALL :Start-Set-Date
	CALL :Start-Echo "!!--!! %ScriptName% Exiting ON %TDate% AT %TTime% !!--!!

Open in new window

LVL 15
ID: 33435524
oh FYI you want to use MIR, it copies AND purges.

Expert Comment

ID: 33435675
Yeah, /MIR is the same as /e /purge
LVL 15
ID: 33435740
I mention that becausr my script already includes the /MIR option.

Author Comment

ID: 33446105
So after looking at all of this...I am getting ready to test it. It looks like I might just be able to use the following:

robocopy d:\data u: /e /xo /r:3 /purge /log:%LogFile2Data% 2>&1

So this SHOULD copy the data from D: to U: with all the sub folder, while only getting the files that have changed....and if a file is open it will retry copying it 3 times. Then it will write the log file without appending.

****Question: with the purge option...what happens if someone deletes a file or folder in the source that you want to get back....but after you've used the /purge feature...it deletes this in the destination right? So a user deletes a folder by mistake last night, your backup runs, and all of a sudden you don't have that information anymore because it deletes the copy of the folder you had in backup until yesterday since it no long exsist in the source destination....?

SO maybe my whole problem isn't xcopy (although I understand everyone thinks I need to move to robocopy).......my problem SOUNDS like I just don't have enough space on one of my NAS drives because my backup script doesn't delete old files, so it continues to collect files that are probably no longer in use or any good.

I believe I need something that compares the dates on a deleted file and if the source has been gone for 30 days...it will purge the destination file as well. IS this possible with a time stamp or something? If so any code examples please?

LVL 15
ID: 33447019
Well first off you didn't use /ZB if you don't you aren't copying files through he network in a restartable form (the B will switch to normal 'backup' form if the first copy attempt fails in order to get soft-locked files)

secondly /XO is unnessessary, Robocopy ojnly copies changed files.

Thirdly if you want to copy all files and THEN delete files OLDER than N days

First you run the normal copy job without any /MIR or /Purge options

THEN you run the same job only you specify /MinAge:n and use the /NoCopy /Purge options.

that will tell robocopy only to delete files not in the source and older than X days.

Of course if you want to actually tell when a file was deleted then you can't use this option unles you DON'T Copy file time in step 1.  if you are really trying to run backups you might consider using NTBackup instead.

Onhe Final Note the 2>&1 redirection is pointless here as you are already logging both StdIn and StdOut by virtue of the /Log option..

Here are the commands:

Step1: Robocopy "D:\Data" "U:" * /COPY:DASO /E /ZB /Log+%LogFileData% R:3 W:2

Step2: Robocopy "D:\Data" "U:" * /NoCopy /Purge /MinAge:30 /S /ZB /Log+%LogFileData% /R:3 /W:2

Author Comment

ID: 33448346
@QCubed: I am looking over this and trying take all of that in (being that I am new to scripting).

Will the code you provided still pass error levels back so when I email the results of the log file %errorlevel% to myself I will continue to provide me with that information?

LVL 15

Accepted Solution

Ben Personick (Previously QCubed) earned 2000 total points
ID: 33449039
if you are trying to capture error levels in another part than you should be able to do so either the redirection.  Error level codes are separate information left over in the environment after a command executes.

  if you are trying to capture error levels into a log file to then parse the log file to look for them, then yet you will need to leave the 2>&1 in assuming ti worked for you that way before. 2>&1 says to take StdError and pipe the output to StandardOut instead..

However it was my understanding that Errorlevels were transient data set int he environment on completion of a command and not part of the StdError output.

Expert Comment

ID: 33455513
Sleestack, so the solution to your problem is using the 2>&1 to report error level codes or did you assign the solution to that comment by accident?  QCubed's comments are helpful indeed but I think you owe me at least a few points for pointing out that your not deleting files in the destination which seems to be the main issue here.  Thanks.

Featured Post

Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

AutoHotkey is an excellent, free, open source programming/scripting language for Windows. It started out as a keyboard/mouse macros product, but has expanded into a robust language. This article provides an introduction to it, with links to addition…
This is a fine trick which I've found useful many times, when you just don't want to accidentally run a batch script or the commands needs administrator rights.
This video tutorial shows you the steps to go through to set up what I believe to be the best email app on the android platform to read Exchange mail.  Get the app on your phone: The first step is to make sure you have the Samsung Email app on your …
Is your organization moving toward a cloud and mobile-first environment? In this transition, your IT department will encounter many challenges, such as navigating how to: Deploy new applications and services to a growing team Accommodate employee…
Suggested Courses

621 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question