Need help with Network Backup script for Windows 8/8.1 notebooks

Hi there,

I need someone with good batch file creation skills to create simple, logical and well documented backup script. I have some skills in batch file creation but luck of more advance knowledge and time put me on pressure to make this script work correctly as soon as possible.

No third party utilities, only what we already have (xcopy or robocopy) with Windows 8. We could use PowerShell, but I think we will ended up with more complex solutions compare to a simple batch file with few robocopy command lines. This backup script (shortcut) will be “pushed” onto users’ desktops with Active Directory Group Policy

Here is my backup script requirements:

1.      Check if the computer has “live” network connection to the home drive. If no network connection – quit script with the message “No Network Connection! Try again later…”
2.      Backup script should run without any problems even if the user is not a local admin.
3.      Backup (copy) only selected files (Microsoft Office 2013) from “My Documents” folder onto user’s network home drive “H:\MyBackup”
4.      Do not copy “big” files. Let’s say, if the PowerPoint file size is bigger than 30MB – skip it but tell user about why it wasn’t back up.  
5.      On the destinations (target) create the same folder structures as on Source
6.      Do not copy empty folders
7.      Backup (copy) only changed folders/files. Basically, if the folder/file hasn’t changed since the last backup time – do not copy this folder/file.
8.      Users don’t need to (and they don’t want to) see tons of messages during the backup run. However, some kind of progress status bar during the backup run would be nice to have.
9.      Very brief backup summary about how many files were backed up
10.      If there any error during backup (no network available, failed copy, big files detected, etc.) notify users about it.

Here is my batch file, please enhance it with more efficient command logic.
 
REM === Start of the script===
@echo off
REM === Setting script variables...
SET SOURCE="%userprofile%\My Documents"
SET DESTINATION= "h:\mydocs backup\"
echo.
echo ***********************************************************
echo.                                                                                           
echo  It might take few minutes for backup script to finish...
echo.                                                                                             
echo ***********************************************************
echo.
echo.
echo "Searching for new or updated Microsoft Office documents..."
echo.
robocopy %SOURCE% %DESTINATION% *.doc /S /DCOPY:DA /COPY:DAT /MT:4 /R:1 /W:5 /NP /NJS /XJ
robocopy %SOURCE% %DESTINATION% *.xls /S /DCOPY:DA /COPY:DAT /MT:4 /R:1 /W:5 /NP /NJS /XJ
robocopy %SOURCE% %DESTINATION% *.ppt /S /DCOPY:DA /COPY:DAT /MT:4 /R:1 /W:5 /NP /NJS /XJ
robocopy %SOURCE% %DESTINATION% *.one /S /DCOPY:DA /COPY:DAT /MT:4 /R:1 /W:5 /NP /NJS /XJ
robocopy %SOURCE% %DESTINATION% *.accdb /S /DCOPY:DA /COPY:DAT /MT:4 /R:1 /W:5 /NP /NJS /XJ
echo.
echo      All done
echo.
REM === End of the script===

!!! At the moment, even though I’m explicitly telling robocopy to copy only *.doc (*.xml, etc.) files…, robocopy is copping not just *.doc but also *.docX and *.docXX and *.docXXX, and so on
LVL 1
OlevoAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

oBdACommented:
Here's a first attempt.

1. Check if the computer has “live” network connection to the home drive.
Added.
2. Backup script should run without any problems even if the user is not a local admin.
Should be no issue, since only user files a re copied
3. Backup (copy) only selected files (Microsoft Office 2013)
Set the variable "ExtensionList" accordingly.
3a. robocopy is copping not just *.doc but also *.docX and *.docXX and *.docXXX, and so on
That's an issue with how Windows enumerates files by extension; you can't prevent that if you want to use robocopy.
You can test that with a simple "dir *.doc" command as well.
4. Do not copy “big” files.
Added; set the variable "MaxSize" accordingly.
5. On the destinations (target) create the same folder structures as on Source
robocopy does that by default.
6. Do not copy empty folders
Option /S
7. Backup (copy) only changed folders/files.
robocopy does that by default.
8. Users don’t need to (and they don’t want to) see tons of messages during the backup run. However, some kind of progress status bar during the backup run would be nice to have.
Default robocopy file log will be shown.
9. Very brief backup summary about how many files were backed up
Default robocopy job summary.
10. If there any error during backup (no network available, failed copy, big files detected, etc.) notify users about it.
robocopy will show errors. Files skipped because they were too big will be listed at the end.
@echo off
setlocal
REM === Start of the script===
REM === Setting script variables...
set Source=%UserProfile%\My Documents
set Target=H:\mydocs backup
set ExtensionList=*.doc *.xls *.ppt *.one *.accdb
set /a MaxSize = 1024 * 1024 * 30
set LogFile=%Temp%\MyDocsBackup.log

for %%a in ("%Target%") do set BackupDrive=%%~da
if not exist %BackupDrive%\ (
	color 4F
	echo.
	echo ***********************************************************
	echo.                                                                                           
	echo  Your backup drive '%BackupDrive%' is currently not available.
	echo  Please try again later.
	echo.                                                                                             
	echo ***********************************************************
	echo.
	pause
	exit /b 1
)

echo.
echo ***********************************************************
echo.                                                                                           
echo  It might take a few minutes for backup script to finish...
echo.                                                                                             
echo ***********************************************************
echo.
echo.
echo Searching for new or updated Microsoft Office documents ...
echo.
robocopy.exe "%Source%" "%Target%" %ExtensionList% /S /DCOPY:DA /COPY:DAT /R:1 /W:5 /NP /NJH /XJ /MAX:%MaxSize% /v /fp /bytes /tee /log:"%LogFile%" /L
echo.
echo      All done
echo.
set LargeFiles=0
for /f %%a in ('type "%LogFile%" ^| find.exe /i /c "  large"') do (set LargeFiles=%%a)
if %LargeFiles% gtr 0 (
	echo %LargeFiles% files were skipped because they were larger than %MaxSize% bytes:
	for /f "tokens=2,3 delims=	" %%a in ('type "%LogFile%" ^| find /i "  large"') do (
		echo '%%b': %%a bytes.
	)
)

REM === End of the script===

Open in new window


Edit: added list numbers.
pgm554Commented:
Why don't you just setup Windows file history?
Why reinvent the wheel?

https://www.winhelp.us/file-history-in-windows-8.html

https://technet.microsoft.com/en-us/magazine/dn448546.aspx
OlevoAuthor Commented:
to: pgm554
We don't want to back up "everything"..., only the Office files. How will you do that with Windows File History?
Determine the Perfect Price for Your IT Services

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden with our free interactive tool and use it to determine the right price for your IT services. Download your free eBook now!

OlevoAuthor Commented:
Thanks oBdA for your help.

I like the way you set one variable for all office files and also how you are testing network connections by simply checking if %BackupDrive% exist.

Now, here is the problem. After running your batch file I can see robocopy report that 15 files were copied but I don’t see any files inside the target folder?! Where the heck are they? Running script again…, 15 files have been copied (again) and NOTHING was skipped!!! A bit puzzled here…
pgm554Commented:
Read the doc,it can be customized if need be ,but it essentially covers all the users home directory files on the users device.

Do not expect File History to cover all your folders or whole drives - it backs up all your default and custom Libraries (including Public Folders), plus Contacts, Desktop and Favorites folders. Your only way to include other folders is to create your own custom Libraries.
oBdACommented:
Oh, sorry, forgot to mention that.
The robocopy command is currently in test mode, and it will only log what it would copy, without actually doing anything.
Remove the /L at the end of the robocopy command line in line 36 to run it for real.
OlevoAuthor Commented:
to oBdA: Oh, feel like a noob now..., didnt see /L switch ;-)
Removed /L switch, after running script I can see files in target folder now. Running scripts again, files have been copied (again) and NOTHING was skipped?! As I have said before, if the folder/file hasn’t changed since the last backup time – do not copy this folder/file.
OlevoAuthor Commented:
Tried to copy few large files (over 30MB). Script detects big files and skip them during the backup. This is exactly what I want, the only wish here is show size in MB instead of bytes. If this cannot be done easily – don’t worry about it.

If this can be easily done I would like to add one more requirements, actually two. Tell the user about how big their network backup folder at the moment (in GB) and when was the last successful backup done. I am guessing that robocopy should write its log file on to the network “MyBackup” folder. With the next backup run, script will read the date of the log file…
oBdACommented:
robocopy won't copy files that already exist in the target (if size and date are the same), unless you specifically add the /IS (Include Same) argument.
Files that already exist will be tagged with "Same" in the log (instead of New or Changed), and in the summary, the "Skipped" column should list how many were skipped.
If the files were indeed copied again - is the target server some type of NAS or a real Windows server? If the former, add /FFT to the robocopy options, which uses the old DOS file times with a granularity of 2 seconds.
This now uses the robocopy file sizes and reports on the last backup date.
Calculating the network drive size in this script is something I'd advise against if you want to keep the complexity low.
Users can just get this by opening the properties of their H drive, where they can always cancel if it takes too long.
@echo off
setlocal enabledelayedexpansion
REM === Start of the script===
REM === Setting script variables...
set Source=%UserProfile%\My Documents
set Target=H:\mydocs backup
set ExtensionList=*.doc *.xls *.ppt *.one *.accdb
set /a MaxSizeMB = 30

set DatFile=%Target%\__%~n0__.dat
set LogFile=%Temp%\MyDocsBackup.log

set /a MaxSize = MaxSizeMB * 1024 * 1024
for %%a in ("%Target%") do set BackupDrive=%%~da
if not exist %BackupDrive%\ (
	color 4F
	echo.
	echo ***********************************************************
	echo.                                                                                           
	echo  Your backup drive '%BackupDrive%' is currently not available.
	echo  Please try again later.
	echo.                                                                                             
	echo ***********************************************************
	echo.
	pause
	exit /b 1
)

echo.
echo ***********************************************************
echo.                                                                                           
echo  It might take a few minutes for backup script to finish...
echo.                                                                                             
echo ***********************************************************
echo.
echo.
echo Searching for new or updated Microsoft Office documents ...
if exist "%DatFile%" (
	for /f "delims=" %%a in ('type "%DatFile%"') do (echo Last backup date: %%a)
)
>"%DatFile%" echo %Date%, %Time%
echo.
robocopy.exe "%Source%" "%Target%" %ExtensionList% /S /DCOPY:DA /COPY:DAT /R:1 /W:5 /NP /NJH /XJ /MAX:%MaxSize% /v /fp /tee /log:"%LogFile%"
echo.
echo      All done
echo.
set LargeFiles=0
for /f %%a in ('type "%LogFile%" ^| find.exe /i /c "  large"') do (set LargeFiles=%%a)
if %LargeFiles% gtr 0 (
	echo %LargeFiles% files were skipped because they were larger than %MaxSizeMB% MB:
	for /f "tokens=2,3 delims=	" %%a in ('type "%LogFile%" ^| find /i "  large"') do (
		echo '%%b': %%a
	)
)

REM === End of the script===

Open in new window

OlevoAuthor Commented:
Thanks oBdA for your help.
As you can see from the picture below, robocopy skipped one (large) file but always copy the other two. Maybe I am missing something but would robocopy say that three files are skipped not only one?!
Robocopy
oBdACommented:
The other two are flagged as "New", so robocopy hasn't found them in the target. If robocopy would have found them, they would have been tagged as same, older, newer, or changed.
Maybe the /L is still in the command line?
OlevoAuthor Commented:
Hmm... I dont see /L switch anywhere
robocopy.exe "%Source%" "%Target%" %ExtensionList% /S /DCOPY:DA /COPY:DAT /R:1 /W:5 /NP /NJH /XJ /MAX:%MaxSize% /v /fp /tee /log:"%LogFile%"

Open in new window

oBdACommented:
Have you verified that the files in question are actually there after the copy?
OlevoAuthor Commented:
Source Target
oBdACommented:
Please remove the /njh from the robocopy command line, and run the script again.
Then enter notepad "%Temp%\MyDocsBackup.log" into the command prompt, use <Ctrl-A><Ctrl-C> to copy the contents of the log file to the clipboard, and paste it into a Code box here.
OlevoAuthor Commented:
-------------------------------------------------------------------------------
   ROBOCOPY     ::     Robust File Copy for Windows                              
-------------------------------------------------------------------------------

  Started : Tuesday, 9 June 2015 7:52:04 PM
   Source : C:\Source\
     Dest : D:\Target\

    Files : *.doc
	    *.xls
	    *.ppt
	    *.one
	    *.accdb
	    
  Options : /V /FP /TEE /S /DCOPY:DA /COPY:DAT /NP /XJ /MAX:31457280 /R:1 /W:5 

------------------------------------------------------------------------------

	                   3	C:\Source\
	         large		   3.6 g	C:\Source\This is very big file.doc
	    New File  		    7497	C:\Source\New Microsoft Excel Worksheet.xlsx
	    New File  		   11282	C:\Source\New Microsoft Word Document.docx

------------------------------------------------------------------------------

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :         1         0         0         0         0         0
   Files :         3         2         1         0         0         0
   Bytes :   3.631 g    18.3 k   3.631 g         0         0         0
   Times :   0:00:00   0:00:00                       0:00:00   0:00:00


   Speed :            18779000 Bytes/sec.
   Speed :            1074.542 MegaBytes/min.
   Ended : Tuesday, 9 June 2015 7:52:04 PM

Open in new window

oBdACommented:
I had already written "Try as I might, I can't reproduce this." when I did one last test and stumbled onto something.
Could it be that D is a network drive, probably on a different OS than the machine you're on?
Just compared a W2k12R2 and a Win8 machine; even though both claim to be "[Version 6.3.9600]", there are differences in how wildcards are handled.
The W2k12R2 machine does it as all previous Windows versions: *.doc will (as you already noticed) return "*.docx" etc. as well.
The Win8 machine behaves differently; *.doc will indeed only return "*.doc".
In the code block below, C: is local on W2k12R2, D: is a mapped drive on Windows 8.1. No file operations took place between the commands.
C:\>dir /b C:\Temp\Source\*.doc
New Microsoft Word Document.docx
Not quite so big file.doc

C:\>dir /b D:\Target\*.doc
File Not Found

C:\>dir /b D:\Target\*.doc?
New Microsoft Word Document.docx

Open in new window


In other words, the solution is to simply change the extension list and include all types:
set ExtensionList=*.doc? *.xls? *.ppt? *.one *.accdb

Open in new window

Here's a list of all extensions used by Office: http://en.wikipedia.org/wiki/List_of_Microsoft_Office_filename_extensions
OlevoAuthor Commented:
Wow, simply adding “?” in the file extension does the trick. Now, robocopy is started to skip old files.

Thanks oBdA for your help. Last request, I promise ;-)

Since there can be lots of files and folders to backup, command prompt window is overloaded with hundreds of lines. Are the any way not to display meaning less lines? If this is hard to do, could we move “last backup date…” to the end of the script? Ideal, I would love to have result similar to the screen below.
r
oBdACommented:
This now reports on the screen only on fresh or changed files (so files that were actually copied) as confirmation for the user, plus the summary.
Last backup date moved to the end of the output,
@echo off
setlocal enabledelayedexpansion
REM === Start of the script===
REM === Setting script variables...
set Source=C:\Temp\Source
set Target=C:\Temp\Target
set ExtensionList=*.doc? *.xls? *.ppt? *.ppt? *.one *.accdb?
set /a MaxSizeMB = 10

set DatFile=%Target%\__%~n0__.dat
set LogFile=%Temp%\MyDocsBackup.log

set /a MaxSize = MaxSizeMB * 1024 * 1024
for %%a in ("%Target%") do set BackupDrive=%%~da
if not exist %BackupDrive%\ (
	color 4F
	echo.
	echo ***********************************************************
	echo.                                                                                           
	echo  Your backup drive '%BackupDrive%' is currently not available.
	echo  Please try again later.
	echo.                                                                                             
	echo ***********************************************************
	echo.
	pause
	exit /b 1
)

echo.
echo ***********************************************************
echo.                                                                                           
echo  It might take a few minutes for backup script to finish...
echo.                                                                                             
echo ***********************************************************
echo.
echo.
echo Searching for new or updated Microsoft Office documents ...
echo.
set BackupDate=%Date%, %Time%
robocopy.exe "%Source%" "%Target%" %ExtensionList% /S /DCOPY:DA /COPY:DAT /R:1 /W:5 /NP /XJ /MAX:%MaxSize% /njh /ndl /v /fp /tee /log:"%LogFile%" | findstr.exe /i /v /c:"  same" /c:"  large"
echo.
echo      All done
echo.
set LargeFiles=0
for /f %%a in ('type "%LogFile%" ^| find.exe /i /c "  large"') do (set LargeFiles=%%a)
if %LargeFiles% gtr 0 (
	echo %LargeFiles% files skipped because they were larger than %MaxSizeMB% MB:
	for /f "tokens=2,3 delims=	" %%a in ('type "%LogFile%" ^| find /i "  large"') do (
		echo * '%%b': %%a
	)
)
echo.
if exist "%DatFile%" (
	for /f "delims=" %%a in ('type "%DatFile%"') do (echo Last backup date before this one: %%a)
)
>"%DatFile%" echo %BackupDate%

REM === End of the script===

Open in new window

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
OlevoAuthor Commented:
Thanks oBdA for your help, all good now
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Batch

From novice to tech pro — start learning today.