Solved

Robocopy - copying empty folders even when it is told not to

Posted on 2016-08-03
17
67 Views
Last Modified: 2016-08-15
Summary of question:
Is Robocopy working as expected, or should I be using different command line parameters to prevent empty folder creation?

Detail Issue:
Below is the RoboCopy Job File, which is saved using the /SAVE parameter.

Note the /S parameter is used and not the /E parameter.

When the command is run, via a VBScript which assembles the command line, it works without issue EXCEPT it is creating empty folders.  From what I can tell from the documentation and every site I have looked at on the Internet for this issue, RoboCopy should NOT be creating these empty folders..

From what I understand, if a folder does not contain one of the desired file types, it should be considered empty.
If I have 1,200 folders nested 30 levels deep and those folders contain files of type .FUBAR, it should NOT create empty destination folders of the same name, yet it is.

::
::

::
:: Source Directory :
::
	/SD:\\ComputerName\c$\	:: Source Directory.

::
:: Destination Directory :
::
	/DD:E:\Migrated Data\ccnllp\Username\ComputerName\Root\ByExtension\	:: Destination Directory.

::
:: Include These Files :
::
	/IF		:: Include Files matching these names
		*.log
		*.txt
		*.acl
		*.ade
		*.asd
		*.cnv
		*.doc
		*.dot
		*.grv
		*.h1q
		*.iaf
		*.maf
		*.mam
		*.maq
		*.mar
		*.mat
		*.maw
		*.mda
		*.mdb
		*.mde
		*.mdt
		*.mdw
		*.mpd
		*.mpp
		*.mpt
		*.mso
		*.oab
		*.obi
		*.oft
		*.olm
		*.one
		*.ops
		*.ost
		*.pip
		*.pot
		*.ppa
		*.pps
		*.ppt
		*.prf
		*.pst
		*.pub
		*.puz
		*.slk
		*.snp
		*.svd
		*.vdx
		*.vsd
		*.vss
		*.vst
		*.vsx
		*.vtx
		*.wbk
		*.wll
		*.xar
		*.xla
		*.xlb
		*.xlc
		*.xll
		*.xlm
		*.xls
		*.xlt
		*.xlw
		*.xsf
		*.xsn
		*.dic
		*.lex
		*.nk2
		*.pdf
		*.rtf
		*.thmx
		*.ccda
		*.ccdb
		*.ccdb
		*.ccdc
		*.ccde
		*.ccdp
		*.ccdr
		*.ccdt
		*.ccdu
		*.crtx
		*.docm
		*.docx
		*.dotm
		*.dotx
		*.epkg
		*.pa
		*.pmsg
		*.potm
		*.potx
		*.ppam
		*.ppsm
		*.ppsx
		*.pptm
		*.pptx
		*.sldm
		*.sldx
		*.vsdx
		*.xl
		*.xlam
		*.xlsb
		*.xlsm
		*.xlsx
		*.xltm
		*.xltx
		*.xslb
		*.db
		*.dbf
		*.ldb
		*.mdb
		*.sdf
		*.sqlite
		*.bmp
		*.emf
		*.gif
		*.ico
		*.jpg
		*.png
		*.tif
		*.wmf
		*.dat
		*.kml
		*.kmz
		*.xml
		*.wav
		*.wmdb
		*.wpl
		*.cfg
		*.config
		*.icc
		*.ini
		*.properties
		*.zip
		*.msg
	/XJD		:: eXclude Junction points for Directories.

::
:: Exclude These Directories :
::
	/XD		:: eXclude Directories matching these names
		temp
		akamai
		cache
		unified_cache_leveldb_leveldb2
		\\ComputerName\c$\Users\Username\WiredRed
		\\ComputerName\c$\Users\Username\Videos
		\\ComputerName\c$\Users\Username\Music
		\\ComputerName\c$\Users\Username\Contacts
		\\ComputerName\c$\Users\Username\Pictures
		\\ComputerName\c$\Users\Username\Downloads
		\\ComputerName\c$\Users\Username\Documents
		\\ComputerName\c$\Users\Username\Desktop
		\\ComputerName\c$\Users\Username\Appdata\Roaming\Zeon
		\\ComputerName\c$\Users\Username\Appdata\Roaming\Litera
		\\ComputerName\c$\Users\Username\Appdata\Local\Litera
		\\ComputerName\c$\Users\Username\Appdata\Local\Interwoven\QuickSearch\Config
		\\ComputerName\C$\dbmakerW64
		\\ComputerName\C$\epop
		\\ComputerName\C$\epopAdmin
		\\ComputerName\C$\epopremove
		\\ComputerName\C$\epopremove-verified
		\\ComputerName\C$\ePopServer
		\\ComputerName\C$\inetpub
		\\ComputerName\C$\KB3AIK_EN
		\\ComputerName\C$\Robocopy
		\\ComputerName\C$\robocopy.bak
		\\ComputerName\C$\Scripts
		\\ComputerName\C$\StaffData
		\\ComputerName\C$\treesize
		\\ComputerName\C$\Visio
		\\ComputerName\C$\WiredRed
		content.ie*

::
:: Exclude These Files :
::
	/XF		:: eXclude Files matching these names
		*.tmp
		*cache*
		*akamai*
		*.lock
::
:: Copy options :
::
	/S		:: copy Subdirectories, but not empty ones.
	/PF		:: check run hours on a Per File (not per pass) basis.
	/COPY:DAT	:: what to COPY for files (default is /COPY:DAT).
	/ZB		:: use restartable mode; if access denied use Backup mode.
	/EFSRAW		:: copy all encrypted files in EFS RAW mode.
::
:: Retry Options :
::
	/R:1		:: number of Retries on failed copies: default 1 million.
	/W:10		:: Wait time between retries: default is 30 seconds.
::
:: Logging Options :
::
	/V		:: produce Verbose output, showing skipped files.
	/X		:: report all eXtra files, not just those selected.
	/TS		:: include source file Time Stamps in the output.
	/FP		:: include Full Pathname of files in the output.
	/NP		:: No Progress - don't display percentage copied.
	/LOG+:C:\Robocopy\Username_ComputerName_YYYYMMDD_HHMMSS.log	:: output status to LOG file (append to existing log).

Open in new window


If Robocopy is, in fact, performing as expected, I presume I am going to need to run something like the following to recursively remove empty folders.  This is a step I would rather not take, though would write as a secondary housekeeping script due to time constraints on the primary script.

'Sub to recursively delete all empty folders. 
Sub RecursiveDeleteEmptyFolders(ByVal strDirectory)
	On Error Resume Next
	Dim objFolder, objSubFolder
	Set objFolder = objFSO.GetFolder(strDirectory)
 
	WScript.Echo "Checking Folder: " & strDirectory & ". Contains " & objFolder.Files.Count & " files and " & objFolder.SubFolders.Count & " folders."

	'If the RemoveDesktopIni or RemoveThumbsDB Flag is set to True then remove any files called Desktop.ini or thumbs.db
	If objFSO.FileExists(strDirectory & "\desktop.ini") Then 
		objFSO.DeleteFile(strDirectory & "\desktop.ini")
		If Err Then
			WScript.Echo "Error deleting:" & strDirectory & "\desktop.ini" & " - " & Err.Description
			intErrorCount = intErrorCount + 1
			Err.Clear
		End If
   	End If
	If objFSO.FileExists(strDirectory & "\thumbs.db") Then 
		objFSO.DeleteFile(strDirectory & "\thumbs.db")
			If Err Then 
				WScript.Echo "Error deleting:" & strDirectory & "\thumbs.db" & " - " & Err.Description
				intErrorCount = intErrorCount + 1
				Err.Clear
		End If
   	End If
	
	'Check if there are any subfolders, and if so go through each of those recursively deleting them
    If objFolder.SubFolders.Count > 0 Then
    	For Each objSubFolder in objFolder.SubFolders
			RecursiveDeleteEmptyFolders objSubFolder.Path
		Next
    End If

    'Now check if the folder contains any files.
    If objFolder.Files.Count = 0 And objFolder.SubFolders.Count = 0 Then
		
		'Check that the folder name does not begin with a tildeand if it does not then delete it
		If Left(objFolder.Name,1) <> "~" Then
			WScript.Echo "Deleting: " & objFolder.Path
			objFolder.Delete
			If Err Then
				WScript.Echo "Error deleting:" & objFolder.Name & " – " & Err.Description
				intErrorCount = intErrorCount + 1
				Err.Clear
			End If
		End If
    End If 
	
End Sub

Open in new window

0
Comment
Question by:WalkaboutTigger
  • 9
  • 6
  • 2
17 Comments
 
LVL 21

Expert Comment

by:yo_bee
ID: 41741435
Even though you have /IF set maybe there are hidden files in the empty directories.
0
 
LVL 15

Author Comment

by:WalkaboutTigger
ID: 41741449
Even though you have /IF set maybe there are hidden files in the empty directories.

I wish it were that easy.  I have done extensive analysis on the data and the vast majority of the folders have no hidden files AND have no files which meet the criteria being set to copy.

Now, another solution, if anyone is aware of it, would be a tool which does exactly what the Windows Easy Transfer Wizard or USMT does in a non-proprietary format, providing individual file/folder copying and registry settings export and copying (they'd just be REG files at that point).
0
 
LVL 21

Expert Comment

by:yo_bee
ID: 41741472
I never us the /ZB , /EFSRAW or /XJ.

I would remove all those and see if things change
0
 
LVL 15

Author Comment

by:WalkaboutTigger
ID: 41741544
The /XJ excludes copying/following file junctions - have tried removing this with no change in behavior.
The /EFSRAW copies encrypted files in RAW format - this is required because, if the user has encrypted files on their local hard drive, the server performing the backup would not be able to back these files up - have tried removing this with no change in behavior (except that encrypted files are not copied and Robocopy generates errors when the script runs against machines with encrypted files.
The /ZB uses restartable mode, and if that fails, uses Backup mode.  This increases the probability that files opened by the user at the time Robocopy runs actually get backed up/copied - have tried removing this with no change in behavior.
0
 
LVL 21

Expert Comment

by:yo_bee
ID: 41741546
For testing can you remove the /xd.  I just saw someone else have a similar issue and removing the /xd returned the desired results minus the excluded directories.
0
 
LVL 15

Author Comment

by:WalkaboutTigger
ID: 41741562
The issue with removing the /XD, which specifies which folders to exclude from the copy, is one of the folders not listed in the above RCJ file is C:\Users.  If this folder is included, the Robocopy job, on average, goes from 2 to 9 GB to 14 to 58 GB.  Granted the network data storage is not an issue, but the time required per workstation increases dramatically even for an incremental backup,

Is the /XD switch responsible for the behavior of Robocopy creating these empty folders?
0
 
LVL 21

Assisted Solution

by:yo_bee
yo_bee earned 250 total points
ID: 41741624
So the empty  folders being there are more beneficial.
0
 
LVL 4

Assisted Solution

by:Alexandre Michel
Alexandre Michel earned 250 total points
ID: 41742389
Now, another solution, if anyone is aware of it, would be a tool which does exactly what the Windows Easy Transfer Wizard or USMT does in a non-proprietary format, providing individual file/folder copying and registry settings export and copying (they'd just be REG files at that point).

Have a look at http://www.mirinsoft.com/index.php/blog/243-new-project-cloneapp-on-the-run and  at http://docs.cloneapp.mirinsoft.com. It backs up files & registry and you can make your own "Plug-ins"
0
Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 15

Author Comment

by:WalkaboutTigger
ID: 41742988
So the empty  folders being there are more beneficial.
Yes, as I have a script which deletes these empty folders.
I would prefer it if these were not created and my log wasn't filled with erroneous messages indicating new folder creation without file copy, but copying the excluded folders is NOT and option.
0
 
LVL 15

Author Comment

by:WalkaboutTigger
ID: 41742994
Alexandre, the advertisements on that website are offensive beyond reason.  I believe the site operators do this intentionally to garner undeserved click-through revenue.
0
 
LVL 4

Expert Comment

by:Alexandre Michel
ID: 41743537
WalkaboutTigger, sorry about that!
I did not see many adds when I visited the site as I use an ad blocker and a script blocker

Have a look at http://alternativeto.net/software/robocopy/. However, it will be difficult to get a non proprietary software that does everything that you want to do.
After putting so much effort in Robocopy, you might try something else just to find that it too is only 99% perfect

Have you looked into RSYNC (https://rsync.samba.org/ and https://download.samba.org/pub/rsync/rsync.html). It is very powerful ... but it might have the same shortfall as robocopy: copying a source folder that is not empty but whose content is completly excluded and thus whose destination is empty...

Here is a solution of sort: use your script above to copy from source to temporary destination. Use Robocopy /S to copy from temporary destination to final distination. Empty folders will be skipped! Your script to delete empty  folders is probably simpler...
0
 
LVL 15

Author Comment

by:WalkaboutTigger
ID: 41745039
As an experiment, I removed the /XD parameter.  In addition to copying an additional 56 GB of data, Robocopy still created over 70,000 empty folders.

The performance impact this is causing to the script is becoming unacceptable.  What is worse is that, when an incremental job is run to update any changes, Robocopy recreates all of the now-deleted empty folders.

This is mind-mindbogglingly stupid behavior on the part of Robocopy.  It should NOT be creating folders and then looking in the source folder to see if there is anything to copy.  It should be examining the source folder first and only creating a destination folder if it is supposed to be doing something.

The biggest challenge with using USMT or WETW is that users have historically saved data all over their local drives - we had one person saving files in \Windows\System32.  In the new environment, users are NOT local admins and have redirected folders and document management integration, but we have to scan their entire drive for possible data.
0
 
LVL 21

Expert Comment

by:yo_bee
ID: 41745043
This is crazy i agree. What version of RC you running?  

Also have you tried running your RC with direct switches rather than a job.  I know you have a very complex structure, but it is something worth trying
0
 
LVL 15

Author Comment

by:WalkaboutTigger
ID: 41745050
Robocopy version 5.1.10.1027
It doesn't matter if I use a CMD prompt, run it from a VBSCRIPT or launch it from Powershell, the results are the same.
0
 
LVL 21

Expert Comment

by:yo_bee
ID: 41745569
Are you 100% sure that there are no hidden system level files with in the sub-directories that you visibility know are empty?
0
 
LVL 15

Accepted Solution

by:
WalkaboutTigger earned 0 total points
ID: 41750752
I did want to close out this issue, post the corrected VBScript for deleting empty folders, and thank those of you who responded.

Based on all testing scenarios and the behavior Robocopy exhibits when performing file exclusion by type, there is no way to prevent empty folders from being created.
Option Explicit

If (WScript.Arguments.Count <> 1) Then
    WScript.Echo("Usage: cscript DeleteEmpties.vbs path")
    WScript.Quit(1)
End If

Dim strPath

strPath = WScript.Arguments(0)

Dim fso
Set fso = CreateObject("Scripting.FileSystemObject")
Dim objFolder
Set objFolder = fso.GetFolder(strPath)
DeleteEmptyFolders objFolder

Sub DeleteEmptyFolders(folder)
    Dim subfolder
    For Each subfolder in folder.SubFolders
        DeleteEmptyFolders subfolder
    Next
    If folder.SubFolders.Count = 0 And folder.Files.Count = 0 Then
        WScript.Echo folder.Path & " is empty"
        On Error Resume Next
        fso.DeleteFolder folder.Path
    End If
End Sub

Open in new window

Simplified version without comments

'==========================================================================
' The Option Explicit declaration requires all variables used in the script are defined
' via a DIM statement prior to their being used within the script.  This helps to prevent
' undesirable or difficult-to-troubleshoot problems from arising through misspellings or
' other typographical errors introduced inadvertantly by the script writer.
'==========================================================================
Option Explicit

'==========================================================================
' Determine if command line parameters were passed to the script and
If (WScript.Arguments.Count <> 1) Then
, if not, display usage information
    WScript.Echo("Usage: cscript DeleteEmpties.vbs path")
' and exit with a result code of 1
    WScript.Quit(1)
End If

' Define the string variable for the path to search for empty folders
Dim strPath

' set the strPath variable to the first parameter passed on the command line
strPath = WScript.Arguments(0)

' Define the variable to use for the Scripting File System Object
Dim fso
' Initialize the File System Object
Set fso = CreateObject("Scripting.FileSystemObject")
' Define the object variable for the current Folder
Dim objFolder
' Initialize the folder object
Set objFolder = fso.GetFolder(strPath)
' Call the DeleteEmptyFolders subroutine and pass it the Folder object
DeleteEmptyFolders objFolder

' Subroutine to do the heavy lifting of deleting nested folders
Sub DeleteEmptyFolders(folder)
    ' Define the subfolder variable used to enumerate the subfolders in the folder
    Dim subfolder
    ' Iterate through all of the subfolders
    For Each subfolder in folder.SubFolders
        ' Recursively call the DeleteEmptyFolders subroutine for each subfolder in turn
        DeleteEmptyFolders subfolder
    Next
    ' If the folder has no subfolders
    If folder.SubFolders.Count = 0 And folder.Files.Count = 0 Then
        ' Send a message to the console to indicate the folder is empty
        WScript.Echo folder.Path & " is empty"
        ' if there is an error (the folder is help open, insufficient permissions, etc) continue processing
        On Error Resume Next
        ' Attempt to delete the folder
        fso.DeleteFolder folder.Path
    ' since the IF block is multiline, the script engine needs to know when the IF block ends
    End If
' End of the subroutine
End Sub

Open in new window

Verbose comments in code

Generally I write some level of comments in my code.  Usually this is to ensure that, 6 months later, I can maintain the script.  It is also a courtesy to the next person coming along behind me who may need to understand and maintain the script.

As I am currently working in 12 different scripting and compiled languages, comments also help align my thoughts around the language I need to write in and the goal of the code.
0
 
LVL 15

Author Closing Comment

by:WalkaboutTigger
ID: 41756109
No other presented solution provided a complete answer.
I do appreciate yo_bee's and Alexabdre's efforts.
0

Featured Post

Do email signature updates give you a headache?

Do you feel like you are constantly making changes to email signatures? Are the images not formatting how you want them to? Want high-quality HTML signatures on all devices, including on mobiles and Macs? Then, let Exclaimer solve all your email signature problems today.

Join & Write a Comment

New Windows 7 Installations take days for Windows-Updates to show up and install. This can easily be fixed. I have finally decided to write an article because this seems to get asked several times a day lately. This Article and the Links apply to…
OfficeMate Freezes on login or does not load after login credentials are input.
This tutorial will give a an overview on how to deploy remote agents in Backup Exec 2012 to new servers. Click on the Backup Exec button in the upper left corner. From here, are global settings for the application such as connecting to a remote Back…
This tutorial will walk an individual through locating and launching the BEUtility application and how to execute it on the appropriate database. Log onto the server running the Backup Exec database. In a larger environment, this would generally be …

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now