Solved

Backup script not overwritting files on WD WorldBook

Posted on 2010-08-13
16
353 Views
Last Modified: 2012-05-10
I have the following backup script that is not overwritting files:

@echo off

REM Mapping network drives
net use U: /d /y
net use x: /d /y


REM Assigning the network drives to the NAS devices
net use U: \\NAS2\data
net use x: \\NAS3\data


set LogFile2Data="G:\FILESERVER_Backup_Logs\NAS2Databackuplog.log"

set LogFile3Data="G:\FILESERVER_Backup_Logs\NAS3Databackuplog.log"


REM Copies the DATA folder to Network Attached Storage device NAS2 and writes the log file on the G: drive
xcopy D:\Data U:\ /s/e/c/d/h/y >>%LogFile2Data% 2>&1
blat.exe - -body "FILESERVER DATA Backup to NAS2 completed with %Errorlevel% errors. The log can be viewed at \\FILESERVER\FILESERVER_Backup_Logs" -server EXCHANGESERVER -f Backup6@DOMAIN.com -t DOMAIN-InformationServices@DOMAIN.com -s "FILESERVER DATA Backup to NAS2 finished with %Errorlevel% errors"


REM Copies the DATA folder to Network Attached Storage device NAS3 and writes the log file on the G: drive
xcopy D:\Data X:\ /s/e/c/d/h/y >>%LogFile3Data% 2>&1
blat.exe - -body "FILESERVER DATA Backup to NAS3 completed with %Errorlevel% errors. The log can be viewed at \\FILESERVER\FILESERVER_Backup_Logs" -server EXCHANGESERVER -f Backup7@DOMAIN.com -t DOMAIN-InformationServices@DOMAIN.com -s "FILESERVER DATA Backup to NAS3 finished with %Errorlevel% errors"


REM Backup Complete!


The log file shows: insufficient disk space.

The backup is something like 385 GBs
The NAS drive is roughly 450 GBs

There should be plenty of room for this backup.
0
Comment
Question by:Sleestack90
  • 6
  • 5
  • 5
16 Comments
 
LVL 4

Expert Comment

by:ebooyens
ID: 33429745
It's worth using robocopy instead, just more reliable and many more options
0
 

Author Comment

by:Sleestack90
ID: 33430263
I understand that idea, however I have several thing that are working just fine with xcopy.

Are you suggesting that will correct my problem? Will I need to rewrite alot of this in order for it to work? I will have to change over alot of scripts of this is the case, so I'd rather not.
0
 
LVL 4

Expert Comment

by:ebooyens
ID: 33430626
I've just run into problems with xcopy ages age with large folders and deep paths so moved to robocopy and haven't looked back, not sure why this isn't working for you, if anything I would expect xcopy to miss files rather than copy too much.  It may just be that it's creating duplicates or something, not sure.  Assuming your sizes given are correct it must have something to do with the underlying algorithms in xcopy in my opinion as your switches are correct.

Here's the equivalent robocopy syntax for what you've got, /log+ appends to the existing log, /log overwrites it, so up to you.  It would be worth it at least running a once off test with robocopy to see what the resulting disk usage is.  /e is for sub folders, /xo excludes old files so only changed files are copied.  If you have open files in the source you may want to add /r:3 as by default it will retry copying a file a million times before giving up!

robocopy d:\data u: /e /xo /log+:G:\FILESERVER_Backup_Logs\NAS2Databackuplog.log
0
 

Author Comment

by:Sleestack90
ID: 33430708
Hmmm....your right, it definately can't hurt it to run it once.

I will work on this asap.

0
 
LVL 4

Expert Comment

by:ebooyens
ID: 33430777
O, another thought is to use /purge - this removes files in the destination folder that no longer exists in the source.  This may actually be the cause or your issue...
0
 

Author Comment

by:Sleestack90
ID: 33430864
@ebooyens:: can you provide me with an example of how I would use /purge in my script please?
0
 
LVL 4

Expert Comment

by:ebooyens
ID: 33430876
robocopy d:\data u: /e /xo /purge /log+:G:\FILESERVER_Backup_Logs\NAS2Databackuplog.log
0
 
LVL 11

Expert Comment

by:Ben Personick
ID: 33432197
You should use Robocopy to perform reliable backups, please try the attached script


Please note that I am adding the number to the NAS by specifying it as a second input when I call the copy job here:

FOR %%A IN (%SrcDirectories%) DO call :Start-DoCopy-Loop %%A "2"
SET "SkipLoop=NO"
FOR %%A IN (%SrcDirectories%) DO call :Start-DoCopy-Loop %%A "3"


::------------------------------------------------------------------------------------------------::
:: Script: RoboCopy_Backup-Any-Drive.bat                                               			  ::
:: Version: 4.3                                                                                   ::
:: Copyright: Ben Personick                                                                       ::
:: Date: 2010-07-12                                                                               ::
::                                                                                                ::
::                                                                                                ::
:: Desc: Copies Local System settings, and documents to a backup location on the HD it's run from ::
::                                                                                                ::
::------------------------------------------------------------------------------------------------::
:Begin-Script
	ECHO Off
	CALL :Start-Set-Date
	SET "IDL=nul"
	SET "ScriptName=RoboCopy-Backup"
	SET "Preamble=%ScriptName% --"
	CALL :Start-Echo " ---- Begining %ScriptName% ON %TDate% AT %TTime% ---- "

::------------------------------------------------------------------------------------------------::
:: This section sets up default variable settings and sets the logging directory and file name.   ::
::------------------------------------------------------------------------------------------------::
:Start-Prep
	::--------------------------------------------------------------------------------------------::
	:: Sets Relative Paths used to Copy later.  												  ::
	::--------------------------------------------------------------------------------------------::	
	SET "SrcDrv=D:"
	
	::--------------------------------------------------------------------------------------------::
	:: Set the following Variable to hold a list of Directories under %SrcDrv%		 			  ::
	::--------------------------------------------------------------------------------------------::
	:: NOTE: Encapsulate Directories with double Quotes and separate them with Commas
	:: CAREFUL: Ensure not to leave directories below set as NOMATCH, it will terminate the copy job prematurely!
	:: NOTE: To skip SET SrcDirectories=NOMATCH
	:: eg. Usage: SET SrcDirectories="Data", "Data2", "This\That\Another", "This\That\Some Other"
	SET SrcDirectories="Data"
	
	::The Following Variable Sets the Folder path to store the user files in
	::--------------------------------------------------------------------------------------------::
	:: If DrvDst is Set to NOMATCH it will find the drive the batch file is double-clicked from.  ::
	::--------------------------------------------------------------------------------------------::
	SET "DrvDst=\\NAS"
	SET "DstShare=Data"
	
	::--------------------------------------------------------------------------------------------::
	:: This section finds the Drive The Batch File is run from for use later in the script        ::
	::--------------------------------------------------------------------------------------------::
	IF /I "%DrvDst%"=="NOMATCH"	Call :Start-Find-DL-Run-From "%DrvDst%"
	SET "DrvDst=%TDrvFind%"&SET "TDrvFind=NOMATCH"
	CALL :Start-Echo "Drive Run From is %DrvDst%"
	IF /I "%DstDrv%"=="NOMATCH" ECHO THE BATCH WAS NOT RUN FROM A HARD DRIVE!!&GOTO :End-Script
	
	::--------------------------------------------------------------------------------------------::
	:: This Section Sets up logging variables.													  ::
	::--------------------------------------------------------------------------------------------::
	SET "IDir=%DrvDst%\Admin\%ScriptName%\Logs"
	SET "ILog=%ScriptName%_%TDate%_%TTime%.Log"
	CALL :Start-Echo "ILOG = %ILOG%"
	SET "IDL=%IDir%\%ILog%"
	IF /I "%IDL%" NEQ "nul" IF /I "%IDL%" NEQ "" CALL :Start-Create-Dir "%IDL%"
	
	::--------------------------------------------------------------------------------------------::
	:: This Section Sets Default Values for match variables, don't change except for testing.	  ::
	::--------------------------------------------------------------------------------------------::
	SET OSVer=NOMATCH
	Set vTemp=NOMATCH
	SET OSSP=NOMATCH
	SET OSbit=NOMATCH
	SET OSSPExit=NOEXIT
	SET RoBoVista=NO
	SET SkipLoop=NO

:End-Prep

:Start-Begin-Log
	CALL :Start-Set-Date
	CALL :Start-Echo " "
	CALL :Start-Echo "------------------------------------------------------"
	CALL :Start-Echo "BEGIN %ScriptName% Script on %TDate% AT %TTime%"
	CALL :Start-Echo "------------------------------------------------------"
	CALL :Start-Echo " "
:End-Begin-Log



::------------------------------------------------------------------------------------------------::
:: This section finds the version of the operating system for use later in the script             ::
::------------------------------------------------------------------------------------------------::
:Start-Find-OS-Info

	:Start-FIND-Ver
		FOR /F "Tokens=4 Delims=[] " %%V IN ('Ver ^| find "Version"') DO CALL :Start-Ver-Match "%%V"
		GOTO End-Ver-Match
		:Start-Ver-Match
			SET TVer=%~1
			Call :Start-Echo "Windows Version is: %TVer%"
			IF /I "%TVer%" GTR "3" IF /I "%TVer:~3,1%" NEQ "." SET "OSVer=Win9x"
			IF /I "%TVer%"=="3.10.528" SET "OSVer=NT3.1"
			IF /I "%TVer%"=="3.50.807" SET "OSVer=NT3.5"
			IF /I "%TVer%"=="3.51.1057" SET "OSVer=NT3.51"
			IF /I "%TVer%"=="4.0.1381" SET "OSVer=NT4.0"
			IF /I "%TVer%"=="5.0.2195" SET "OSVer=2K"
			IF /I "%TVer%"=="5.1.2600" SET "OSVer=XP"
			IF /I "%TVer%"=="5.2.3790" SET "OSVer=2K3"
			IF /I "%TVer:~0,7%"=="6.0.600" SET "OSVer=Vista"&SET "RoBoVista=GO"
			IF /I "%TVer%"=="6.0.6001" SET "OSVer=Vista"&SET "RoBoVista=GO"&SET OSSP=SP1
			IF /I "%TVer%"=="6.0.6002" SET "OSVer=Vista"&SET "RoBoVista=GO"&SET OSSP=SP2
			IF /I "%TVer:~0,7%"=="6.1.760" SET "OSVer=7"&SET "RoBoVista=GO"
			
			GOTO :EOF
		:End-Ver-Match
		CALL :Start-Echo "End-Ver-Match"
		
		IF "%RoBoVista%" NEQ "GO" GOTO 2K8?-end
		:2k8?-start
			CALL :Start-Echo "2k8?-start"
			FOR /F "Tokens=5" %%R IN ('reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion" /v ProductName ^| FIND /I "2008"') DO SET "OSVer=2K8"
			
			::START /WAIT REGEDIT.EXE /E "%Temp%.\%%~n0.dat" "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion" 
			::  FOR /F "tokens=2 delims==" %%A IN ('TYPE "%Temp%.\%%~n0.dat" ^| FIND /I "ProductName" ^| Find /I "2008"') DO SET OSVer=2K8
		:2k8?-end

	:End-Find-Ver
	 
	
	:Start-FIND-SP
		CALL :Start-Echo "Service pack: %OSSP%"
		IF /I "%OSSP%"=="NOMATCH" IF /I "%TVer%" GEQ "5.1.2600" FOR /F "Tokens=5" %%R IN ('reg query "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion" /v CSDVersion 2^>^&1 ^| FIND /I "CSDVersion"') DO SET "OSSP=SP%%R"
	:End-FIND-SP
	CALL :Start-Echo "End Find SP"

::------------------------------------------------------------------------------------------------::
:: This section finds Whether the Operating system is 32bit or 64bit for use later in the script. ::
::------------------------------------------------------------------------------------------------::
	:Start-Find-OS-Architechture
		SET "OSbit=%PROCESSOR_ARCHITECTURE:~-2%"&IF "%PROCESSOR_ARCHITECTURE:~-2%"=="86" SET "OSbit=32"
	:End-Find-OS-Architechture
	
::------------------------------------------------------------------------------------------------::
:: This section is intended to determin if the OS is Desktop or Server for differing installs.    ::
::------------------------------------------------------------------------------------------------::
	:Start-Set-Desktop-OS-Runs
		SET DTOSRun=NO
		CALL :Start-Echo "Checking if Desktop OS"
		CALL :Start-Echo "DTOSRun = %DTOSRun%, OSVer = %OSVer%, TVer = %TVer%"
		CALL :Start-Echo "Set DTOSRun to GO Only if OSVer is a desktop OS"
		IF /I "%OSVer%"=="XP" SET "DTOSRun=GO"
		IF /I "%OSVer%"=="Vista" SET "DTOSRun=GO"
		IF /I "%OSVer%"=="7" SET "DTOSRun=GO"
		
	:End-Set-Desktop-OS-Runs
	
	CALL :Start-Echo "OS Version = %OSVer%"
	CALL :Start-Echo "OS SP Version = %OSSP%"
	CALL :Start-Echo "OS Architechture = %OSbit% bit"
	CALL :Start-Echo "OS Desktop (NO or GO) = %DTOSRun%"
	
:End-Find-OS-Info



:Start-Main

	:Start-DoCopy

		CALL :Start-Set-Date
		CALL :Start-Echo "Start-DoCopy ON %TDate% AT %TTime%"
		IF /I "%RoBoVista%" NEQ "GO" SET "RoBoVista=/ZB /E /MIR /LOG+:%IDL% /TEE /NP /R:2 /W:2"
		IF /I "%RoBoVista%"=="GO" SET "RoBoVista=/ZB /E /DCOPY:T /MIR /LOG+:%IDL% /TEE /NP /R:2 /W:2"
		REM Below is the For Loop, %%A is the term pulled from the set to pass to the command loop
		REM  The terms are located within the open and close parenthesis, and quotes allow spaces to be ignored
		REM  If Quotes were not used, the spaces would be seen as a delimiter just as commas are.
		FOR %%A IN (%SrcDirectories%) DO call :Start-DoCopy-Loop %%A "2"
		SET "SkipLoop=NO"
		FOR %%A IN (%SrcDirectories%) DO call :Start-DoCopy-Loop %%A "3"
		GOTO End-DoCopy-Loop
		:Start-DoCopy-Loop
			IF /I "%~1"=="NOMATCH" SET "SkipLoop=YES"
			IF /I "%SkipLoop%"=="YES" GOTO :EOF
			CALL :Start-Set-Date
			REM This is the command loop, everythign inside this loop will be preocessed as a separate batch file
			REM we use %~1 for the variable the Tilde says to strip quotes off the passed text XD
			CALL :Start-Echo "ON %TDate% AT %TTime% Start-%~1"
			SET "TURL=%SrcDrv%\%~1"
			SET "TDst=%DrvDst%%~2\%~1"
			Robocopy "%TURL%" "%TmpDstDrv%" * %RoBoVista%
			GOTO :EOF
		:End-DoCopy-Loop

	:End-DoCopy
	CALL :Start-Set-Date
	CALL :Start-Echo "End-DoCopy ON %TDate% AT %TTime%"

:End-Main
GOTO :End-Script


::------------------------------------------------------------------------------------------------::
:: This section is intended to determin if the OS is Desktop or Server for differing installs.    ::
::------------------------------------------------------------------------------------------------::
GOTO End-Subs
GOTO End-Script
:Start-Subs
	GOTO :End-Set-Date
	:Start-Set-Date
		SET TDate=%date:~10,4%-%date:~4,2%-%date:~7,2%
		FOR /F "Tokens=1-3 Delims=:" %%T IN ("%TIME: =0%") DO SET "TTime=%%T.%%U.%%V"
		
		GOTO :EOF
	:End-Set-Date
	GOTO End-Echo
	:Start-Echo
		Set TEcho=%~1
		ECHO %Preamble% %TEcho% --
		IF EXIST "%IDir%" ECHO %Preamble% %TEcho% -- >> %IDL%
		GOTO :EOF
	:End-Echo
	GOTO End-Find-DL-Run-From
	:Start-Find-DL-Run-From
		SET "TDrvFind=%~1"
		FOR %%A IN (A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z) DO call :Start-DL-Loop %%A
		GOTO :EOF
		:Start-DL-Loop
			If "%TDrvFind%" NEQ "NOMATCH" GOTO :EOF
			SET "TDrv=%1:"
			FOR /F %%a IN ('Dir /D "%TDrv%\" 2^>^&1 ^| FIND /I " Directory of " ^| Find /I "%TDrv%\"') DO SET "TDrvFind=%TDrv%"
			GOTO :EOF
		:End-DL-Loop
	:End-Find-DL-Run-From
	GOTO End-Create-Dir
	:Start-Create-Dir
		SET TDir=%~1
		:: CALL :Start-Echo "%TDir%"
		CALL :Start-Echo "Checking For %TDir%"
		IF EXIST "%TDir%" CALL :Start-Echo "%TDir% -- Found!"
		IF EXIST "%TDir%" GOTO :EOF
		CALL :Start-Echo "Creating %TDir%"		
		FOR /F "Tokens=*" %%D IN ('MD "%TDir%" 2^>^&1') DO SET "MDOutput=%%D"&IF /I "%D" NEQ "A subdirectory or file %TDir% already exists." CALL :Start-Echo "The Command 'MD %TDir%' Completed with Error: %%D"
		IF EXIST "%TDir%" CALL :Start-Echo "Created %TDir%"
		GOTO :EOF
	:End-Create-Dir

:End-Subs


:End-Script
	CALL :Start-Set-Date
	CALL :Start-Echo "!!--!! %ScriptName% Exiting ON %TDate% AT %TTime% !!--!!
	GOTO :EOF

Open in new window

0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 11

Expert Comment

by:Ben Personick
ID: 33435524
oh FYI you want to use MIR, it copies AND purges.
0
 
LVL 4

Expert Comment

by:ebooyens
ID: 33435675
Yeah, /MIR is the same as /e /purge
0
 
LVL 11

Expert Comment

by:Ben Personick
ID: 33435740
I mention that becausr my script already includes the /MIR option.
0
 

Author Comment

by:Sleestack90
ID: 33446105
So after looking at all of this...I am getting ready to test it. It looks like I might just be able to use the following:

robocopy d:\data u: /e /xo /r:3 /purge /log:%LogFile2Data% 2>&1

So this SHOULD copy the data from D: to U: with all the sub folder, while only getting the files that have changed....and if a file is open it will retry copying it 3 times. Then it will write the log file without appending.

****Question: with the purge option...what happens if someone deletes a file or folder in the source that you want to get back....but after you've used the /purge feature...it deletes this in the destination right? So a user deletes a folder by mistake last night, your backup runs, and all of a sudden you don't have that information anymore because it deletes the copy of the folder you had in backup until yesterday since it no long exsist in the source destination....?


SO maybe my whole problem isn't xcopy (although I understand everyone thinks I need to move to robocopy).......my problem SOUNDS like I just don't have enough space on one of my NAS drives because my backup script doesn't delete old files, so it continues to collect files that are probably no longer in use or any good.

I believe I need something that compares the dates on a deleted file and if the source has been gone for 30 days...it will purge the destination file as well. IS this possible with a time stamp or something? If so any code examples please?

0
 
LVL 11

Expert Comment

by:Ben Personick
ID: 33447019
Well first off you didn't use /ZB if you don't you aren't copying files through he network in a restartable form (the B will switch to normal 'backup' form if the first copy attempt fails in order to get soft-locked files)

secondly /XO is unnessessary, Robocopy ojnly copies changed files.

Thirdly if you want to copy all files and THEN delete files OLDER than N days

First you run the normal copy job without any /MIR or /Purge options

THEN you run the same job only you specify /MinAge:n and use the /NoCopy /Purge options.

that will tell robocopy only to delete files not in the source and older than X days.

Of course if you want to actually tell when a file was deleted then you can't use this option unles you DON'T Copy file time in step 1.  if you are really trying to run backups you might consider using NTBackup instead.

Onhe Final Note the 2>&1 redirection is pointless here as you are already logging both StdIn and StdOut by virtue of the /Log option..

Here are the commands:

Step1: Robocopy "D:\Data" "U:" * /COPY:DASO /E /ZB /Log+%LogFileData% R:3 W:2

Step2: Robocopy "D:\Data" "U:" * /NoCopy /Purge /MinAge:30 /S /ZB /Log+%LogFileData% /R:3 /W:2
0
 

Author Comment

by:Sleestack90
ID: 33448346
@QCubed: I am looking over this and trying take all of that in (being that I am new to scripting).

Will the code you provided still pass error levels back so when I email the results of the log file %errorlevel% to myself I will continue to provide me with that information?

0
 
LVL 11

Accepted Solution

by:
Ben Personick earned 500 total points
ID: 33449039
if you are trying to capture error levels in another part than you should be able to do so either the redirection.  Error level codes are separate information left over in the environment after a command executes.

  if you are trying to capture error levels into a log file to then parse the log file to look for them, then yet you will need to leave the 2>&1 in assuming ti worked for you that way before. 2>&1 says to take StdError and pipe the output to StandardOut instead..

However it was my understanding that Errorlevels were transient data set int he environment on completion of a command and not part of the StdError output.
0
 
LVL 4

Expert Comment

by:ebooyens
ID: 33455513
Sleestack, so the solution to your problem is using the 2>&1 to report error level codes or did you assign the solution to that comment by accident?  QCubed's comments are helpful indeed but I think you owe me at least a few points for pointing out that your not deleting files in the destination which seems to be the main issue here.  Thanks.
0

Featured Post

6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

Join & Write a Comment

The following is a collection of cases for strange behaviour when using advanced techniques in DOS batch files. You should have some basic experience in batch "programming", as I'm assuming some knowledge and not further explain the basics. For some…
This article was inspired by a question here at Experts Exchange (http://www.experts-exchange.com/Software/Photos_Graphics/Images_and_Photos/Q_28629170.html). The requirements stated in that question are (1) reduce the file size of a large number of…
This video discusses moving either the default database or any database to a new volume.
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now