Save hours in development time and avoid common mistakes by learning the best practices to use for JavaScript.
Robocopy - copying empty folders even when it is told not to
Summary of question:
Is Robocopy working as expected, or should I be using different command line parameters to prevent empty folder creation?
Detail Issue:
Below is the RoboCopy Job File, which is saved using the /SAVE parameter.
Note the /S parameter is used and not the /E parameter.
When the command is run, via a VBScript which assembles the command line, it works without issue EXCEPT it is creating empty folders. From what I can tell from the documentation and every site I have looked at on the Internet for this issue, RoboCopy should NOT be creating these empty folders..
From what I understand, if a folder does not contain one of the desired file types, it should be considered empty.
If I have 1,200 folders nested 30 levels deep and those folders contain files of type .FUBAR, it should NOT create empty destination folders of the same name, yet it is.
If Robocopy is, in fact, performing as expected, I presume I am going to need to run something like the following to recursively remove empty folders. This is a step I would rather not take, though would write as a secondary housekeeping script due to time constraints on the primary script.
Is Robocopy working as expected, or should I be using different command line parameters to prevent empty folder creation?
Detail Issue:
Below is the RoboCopy Job File, which is saved using the /SAVE parameter.
Note the /S parameter is used and not the /E parameter.
When the command is run, via a VBScript which assembles the command line, it works without issue EXCEPT it is creating empty folders. From what I can tell from the documentation and every site I have looked at on the Internet for this issue, RoboCopy should NOT be creating these empty folders..
From what I understand, if a folder does not contain one of the desired file types, it should be considered empty.
If I have 1,200 folders nested 30 levels deep and those folders contain files of type .FUBAR, it should NOT create empty destination folders of the same name, yet it is.
::
::
::
:: Source Directory :
::
/SD:\\ComputerName\c$\ :: Source Directory.
::
:: Destination Directory :
::
/DD:E:\Migrated Data\ccnllp\Username\ComputerName\Root\ByExtension\ :: Destination Directory.
::
:: Include These Files :
::
/IF :: Include Files matching these names
*.log
*.txt
*.acl
*.ade
*.asd
*.cnv
*.doc
*.dot
*.grv
*.h1q
*.iaf
*.maf
*.mam
*.maq
*.mar
*.mat
*.maw
*.mda
*.mdb
*.mde
*.mdt
*.mdw
*.mpd
*.mpp
*.mpt
*.mso
*.oab
*.obi
*.oft
*.olm
*.one
*.ops
*.ost
*.pip
*.pot
*.ppa
*.pps
*.ppt
*.prf
*.pst
*.pub
*.puz
*.slk
*.snp
*.svd
*.vdx
*.vsd
*.vss
*.vst
*.vsx
*.vtx
*.wbk
*.wll
*.xar
*.xla
*.xlb
*.xlc
*.xll
*.xlm
*.xls
*.xlt
*.xlw
*.xsf
*.xsn
*.dic
*.lex
*.nk2
*.pdf
*.rtf
*.thmx
*.ccda
*.ccdb
*.ccdb
*.ccdc
*.ccde
*.ccdp
*.ccdr
*.ccdt
*.ccdu
*.crtx
*.docm
*.docx
*.dotm
*.dotx
*.epkg
*.pa
*.pmsg
*.potm
*.potx
*.ppam
*.ppsm
*.ppsx
*.pptm
*.pptx
*.sldm
*.sldx
*.vsdx
*.xl
*.xlam
*.xlsb
*.xlsm
*.xlsx
*.xltm
*.xltx
*.xslb
*.db
*.dbf
*.ldb
*.mdb
*.sdf
*.sqlite
*.bmp
*.emf
*.gif
*.ico
*.jpg
*.png
*.tif
*.wmf
*.dat
*.kml
*.kmz
*.xml
*.wav
*.wmdb
*.wpl
*.cfg
*.config
*.icc
*.ini
*.properties
*.zip
*.msg
/XJD :: eXclude Junction points for Directories.
::
:: Exclude These Directories :
::
/XD :: eXclude Directories matching these names
temp
akamai
cache
unified_cache_leveldb_leveldb2
\\ComputerName\c$\Users\Username\WiredRed
\\ComputerName\c$\Users\Username\Videos
\\ComputerName\c$\Users\Username\Music
\\ComputerName\c$\Users\Username\Contacts
\\ComputerName\c$\Users\Username\Pictures
\\ComputerName\c$\Users\Username\Downloads
\\ComputerName\c$\Users\Username\Documents
\\ComputerName\c$\Users\Username\Desktop
\\ComputerName\c$\Users\Username\Appdata\Roaming\Zeon
\\ComputerName\c$\Users\Username\Appdata\Roaming\Litera
\\ComputerName\c$\Users\Username\Appdata\Local\Litera
\\ComputerName\c$\Users\Username\Appdata\Local\Interwoven\QuickSearch\Config
\\ComputerName\C$\dbmakerW64
\\ComputerName\C$\epop
\\ComputerName\C$\epopAdmin
\\ComputerName\C$\epopremove
\\ComputerName\C$\epopremove-verified
\\ComputerName\C$\ePopServer
\\ComputerName\C$\inetpub
\\ComputerName\C$\KB3AIK_EN
\\ComputerName\C$\Robocopy
\\ComputerName\C$\robocopy.bak
\\ComputerName\C$\Scripts
\\ComputerName\C$\StaffData
\\ComputerName\C$\treesize
\\ComputerName\C$\Visio
\\ComputerName\C$\WiredRed
content.ie*
::
:: Exclude These Files :
::
/XF :: eXclude Files matching these names
*.tmp
*cache*
*akamai*
*.lock
::
:: Copy options :
::
/S :: copy Subdirectories, but not empty ones.
/PF :: check run hours on a Per File (not per pass) basis.
/COPY:DAT :: what to COPY for files (default is /COPY:DAT).
/ZB :: use restartable mode; if access denied use Backup mode.
/EFSRAW :: copy all encrypted files in EFS RAW mode.
::
:: Retry Options :
::
/R:1 :: number of Retries on failed copies: default 1 million.
/W:10 :: Wait time between retries: default is 30 seconds.
::
:: Logging Options :
::
/V :: produce Verbose output, showing skipped files.
/X :: report all eXtra files, not just those selected.
/TS :: include source file Time Stamps in the output.
/FP :: include Full Pathname of files in the output.
/NP :: No Progress - don't display percentage copied.
/LOG+:C:\Robocopy\Username_ComputerName_YYYYMMDD_HHMMSS.log :: output status to LOG file (append to existing log).
If Robocopy is, in fact, performing as expected, I presume I am going to need to run something like the following to recursively remove empty folders. This is a step I would rather not take, though would write as a secondary housekeeping script due to time constraints on the primary script.
'Sub to recursively delete all empty folders.
Sub RecursiveDeleteEmptyFolders(ByVal strDirectory)
On Error Resume Next
Dim objFolder, objSubFolder
Set objFolder = objFSO.GetFolder(strDirectory)
WScript.Echo "Checking Folder: " & strDirectory & ". Contains " & objFolder.Files.Count & " files and " & objFolder.SubFolders.Count & " folders."
'If the RemoveDesktopIni or RemoveThumbsDB Flag is set to True then remove any files called Desktop.ini or thumbs.db
If objFSO.FileExists(strDirectory & "\desktop.ini") Then
objFSO.DeleteFile(strDirectory & "\desktop.ini")
If Err Then
WScript.Echo "Error deleting:" & strDirectory & "\desktop.ini" & " - " & Err.Description
intErrorCount = intErrorCount + 1
Err.Clear
End If
End If
If objFSO.FileExists(strDirectory & "\thumbs.db") Then
objFSO.DeleteFile(strDirectory & "\thumbs.db")
If Err Then
WScript.Echo "Error deleting:" & strDirectory & "\thumbs.db" & " - " & Err.Description
intErrorCount = intErrorCount + 1
Err.Clear
End If
End If
'Check if there are any subfolders, and if so go through each of those recursively deleting them
If objFolder.SubFolders.Count > 0 Then
For Each objSubFolder in objFolder.SubFolders
RecursiveDeleteEmptyFolders objSubFolder.Path
Next
End If
'Now check if the folder contains any files.
If objFolder.Files.Count = 0 And objFolder.SubFolders.Count = 0 Then
'Check that the folder name does not begin with a tildeand if it does not then delete it
If Left(objFolder.Name,1) <> "~" Then
WScript.Echo "Deleting: " & objFolder.Path
objFolder.Delete
If Err Then
WScript.Echo "Error deleting:" & objFolder.Name & " – " & Err.Description
intErrorCount = intErrorCount + 1
Err.Clear
End If
End If
End If
End Sub
Do more with
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Even though you have /IF set maybe there are hidden files in the empty directories.
I wish it were that easy. I have done extensive analysis on the data and the vast majority of the folders have no hidden files AND have no files which meet the criteria being set to copy.
Now, another solution, if anyone is aware of it, would be a tool which does exactly what the Windows Easy Transfer Wizard or USMT does in a non-proprietary format, providing individual file/folder copying and registry settings export and copying (they'd just be REG files at that point).
The /XJ excludes copying/following file junctions - have tried removing this with no change in behavior.
The /EFSRAW copies encrypted files in RAW format - this is required because, if the user has encrypted files on their local hard drive, the server performing the backup would not be able to back these files up - have tried removing this with no change in behavior (except that encrypted files are not copied and Robocopy generates errors when the script runs against machines with encrypted files.
The /ZB uses restartable mode, and if that fails, uses Backup mode. This increases the probability that files opened by the user at the time Robocopy runs actually get backed up/copied - have tried removing this with no change in behavior.
The /EFSRAW copies encrypted files in RAW format - this is required because, if the user has encrypted files on their local hard drive, the server performing the backup would not be able to back these files up - have tried removing this with no change in behavior (except that encrypted files are not copied and Robocopy generates errors when the script runs against machines with encrypted files.
The /ZB uses restartable mode, and if that fails, uses Backup mode. This increases the probability that files opened by the user at the time Robocopy runs actually get backed up/copied - have tried removing this with no change in behavior.
For testing can you remove the /xd. I just saw someone else have a similar issue and removing the /xd returned the desired results minus the excluded directories.
The issue with removing the /XD, which specifies which folders to exclude from the copy, is one of the folders not listed in the above RCJ file is C:\Users. If this folder is included, the Robocopy job, on average, goes from 2 to 9 GB to 14 to 58 GB. Granted the network data storage is not an issue, but the time required per workstation increases dramatically even for an incremental backup,
Is the /XD switch responsible for the behavior of Robocopy creating these empty folders?
Is the /XD switch responsible for the behavior of Robocopy creating these empty folders?
Now, another solution, if anyone is aware of it, would be a tool which does exactly what the Windows Easy Transfer Wizard or USMT does in a non-proprietary format, providing individual file/folder copying and registry settings export and copying (they'd just be REG files at that point).
Have a look at http://www.mirinsoft.com/index.php/blog/243-new-project-cloneapp-on-the-run and at http://docs.cloneapp.mirinsoft.com. It backs up files & registry and you can make your own "Plug-ins"
So the empty folders being there are more beneficial.Yes, as I have a script which deletes these empty folders.
I would prefer it if these were not created and my log wasn't filled with erroneous messages indicating new folder creation without file copy, but copying the excluded folders is NOT and option.
Alexandre, the advertisements on that website are offensive beyond reason. I believe the site operators do this intentionally to garner undeserved click-through revenue.
WalkaboutTigger, sorry about that!
I did not see many adds when I visited the site as I use an ad blocker and a script blocker
Have a look at http://alternativeto.net/software/robocopy/. However, it will be difficult to get a non proprietary software that does everything that you want to do.
After putting so much effort in Robocopy, you might try something else just to find that it too is only 99% perfect
Have you looked into RSYNC (https://rsync.samba.org/ and https://download.samba.org/pub/rsync/rsync.html). It is very powerful ... but it might have the same shortfall as robocopy: copying a source folder that is not empty but whose content is completly excluded and thus whose destination is empty...
Here is a solution of sort: use your script above to copy from source to temporary destination. Use Robocopy /S to copy from temporary destination to final distination. Empty folders will be skipped! Your script to delete empty folders is probably simpler...
I did not see many adds when I visited the site as I use an ad blocker and a script blocker
Have a look at http://alternativeto.net/software/robocopy/. However, it will be difficult to get a non proprietary software that does everything that you want to do.
After putting so much effort in Robocopy, you might try something else just to find that it too is only 99% perfect
Have you looked into RSYNC (https://rsync.samba.org/ and https://download.samba.org/pub/rsync/rsync.html). It is very powerful ... but it might have the same shortfall as robocopy: copying a source folder that is not empty but whose content is completly excluded and thus whose destination is empty...
Here is a solution of sort: use your script above to copy from source to temporary destination. Use Robocopy /S to copy from temporary destination to final distination. Empty folders will be skipped! Your script to delete empty folders is probably simpler...
As an experiment, I removed the /XD parameter. In addition to copying an additional 56 GB of data, Robocopy still created over 70,000 empty folders.
The performance impact this is causing to the script is becoming unacceptable. What is worse is that, when an incremental job is run to update any changes, Robocopy recreates all of the now-deleted empty folders.
This is mind-mindbogglingly stupid behavior on the part of Robocopy. It should NOT be creating folders and then looking in the source folder to see if there is anything to copy. It should be examining the source folder first and only creating a destination folder if it is supposed to be doing something.
The biggest challenge with using USMT or WETW is that users have historically saved data all over their local drives - we had one person saving files in \Windows\System32. In the new environment, users are NOT local admins and have redirected folders and document management integration, but we have to scan their entire drive for possible data.
The performance impact this is causing to the script is becoming unacceptable. What is worse is that, when an incremental job is run to update any changes, Robocopy recreates all of the now-deleted empty folders.
This is mind-mindbogglingly stupid behavior on the part of Robocopy. It should NOT be creating folders and then looking in the source folder to see if there is anything to copy. It should be examining the source folder first and only creating a destination folder if it is supposed to be doing something.
The biggest challenge with using USMT or WETW is that users have historically saved data all over their local drives - we had one person saving files in \Windows\System32. In the new environment, users are NOT local admins and have redirected folders and document management integration, but we have to scan their entire drive for possible data.
This is crazy i agree. What version of RC you running?
Also have you tried running your RC with direct switches rather than a job. I know you have a very complex structure, but it is something worth trying
Also have you tried running your RC with direct switches rather than a job. I know you have a very complex structure, but it is something worth trying
Robocopy version 5.1.10.1027
It doesn't matter if I use a CMD prompt, run it from a VBSCRIPT or launch it from Powershell, the results are the same.
It doesn't matter if I use a CMD prompt, run it from a VBSCRIPT or launch it from Powershell, the results are the same.
Are you 100% sure that there are no hidden system level files with in the sub-directories that you visibility know are empty?
I did want to close out this issue, post the corrected VBScript for deleting empty folders, and thank those of you who responded.
Based on all testing scenarios and the behavior Robocopy exhibits when performing file exclusion by type, there is no way to prevent empty folders from being created.
Generally I write some level of comments in my code. Usually this is to ensure that, 6 months later, I can maintain the script. It is also a courtesy to the next person coming along behind me who may need to understand and maintain the script.
As I am currently working in 12 different scripting and compiled languages, comments also help align my thoughts around the language I need to write in and the goal of the code.
Based on all testing scenarios and the behavior Robocopy exhibits when performing file exclusion by type, there is no way to prevent empty folders from being created.
Option Explicit
If (WScript.Arguments.Count <> 1) Then
WScript.Echo("Usage: cscript DeleteEmpties.vbs path")
WScript.Quit(1)
End If
Dim strPath
strPath = WScript.Arguments(0)
Dim fso
Set fso = CreateObject("Scripting.FileSystemObject")
Dim objFolder
Set objFolder = fso.GetFolder(strPath)
DeleteEmptyFolders objFolder
Sub DeleteEmptyFolders(folder)
Dim subfolder
For Each subfolder in folder.SubFolders
DeleteEmptyFolders subfolder
Next
If folder.SubFolders.Count = 0 And folder.Files.Count = 0 Then
WScript.Echo folder.Path & " is empty"
On Error Resume Next
fso.DeleteFolder folder.Path
End If
End Sub
'==========================================================================
' The Option Explicit declaration requires all variables used in the script are defined
' via a DIM statement prior to their being used within the script. This helps to prevent
' undesirable or difficult-to-troubleshoot problems from arising through misspellings or
' other typographical errors introduced inadvertantly by the script writer.
'==========================================================================
Option Explicit
'==========================================================================
' Determine if command line parameters were passed to the script and
If (WScript.Arguments.Count <> 1) Then
, if not, display usage information
WScript.Echo("Usage: cscript DeleteEmpties.vbs path")
' and exit with a result code of 1
WScript.Quit(1)
End If
' Define the string variable for the path to search for empty folders
Dim strPath
' set the strPath variable to the first parameter passed on the command line
strPath = WScript.Arguments(0)
' Define the variable to use for the Scripting File System Object
Dim fso
' Initialize the File System Object
Set fso = CreateObject("Scripting.FileSystemObject")
' Define the object variable for the current Folder
Dim objFolder
' Initialize the folder object
Set objFolder = fso.GetFolder(strPath)
' Call the DeleteEmptyFolders subroutine and pass it the Folder object
DeleteEmptyFolders objFolder
' Subroutine to do the heavy lifting of deleting nested folders
Sub DeleteEmptyFolders(folder)
' Define the subfolder variable used to enumerate the subfolders in the folder
Dim subfolder
' Iterate through all of the subfolders
For Each subfolder in folder.SubFolders
' Recursively call the DeleteEmptyFolders subroutine for each subfolder in turn
DeleteEmptyFolders subfolder
Next
' If the folder has no subfolders
If folder.SubFolders.Count = 0 And folder.Files.Count = 0 Then
' Send a message to the console to indicate the folder is empty
WScript.Echo folder.Path & " is empty"
' if there is an error (the folder is help open, insufficient permissions, etc) continue processing
On Error Resume Next
' Attempt to delete the folder
fso.DeleteFolder folder.Path
' since the IF block is multiline, the script engine needs to know when the IF block ends
End If
' End of the subroutine
End Sub
Generally I write some level of comments in my code. Usually this is to ensure that, 6 months later, I can maintain the script. It is also a courtesy to the next person coming along behind me who may need to understand and maintain the script.
As I am currently working in 12 different scripting and compiled languages, comments also help align my thoughts around the language I need to write in and the goal of the code.
Premium Content
You need an Expert Office subscription to comment.Start Free Trial