xxcopy batch file going wrong.
I have the following batch file that should check all client subfolders for matching files based on an "ClientFolders.txt" input file. The first six characters for each client folder must match the input file names for it to process. What I mean is determine if there is a folder name match (1st six characters of each client folder), and if so then copy the folder and all subfolders to the destination and then delete the original and all subfolders (move).
The first time I ran it, I was watching the folders scroll by as it processed. Then all of a sudden I saw many folders listed that were not part of the input list. So I stopped it. I discovered if the same client folder existed at the destination, then it would copy all the client folders listed within the same letter (A, B, C) it was processing into the destination folder that already existing and created all the client folders for that letter without copying any client files. So let's say client folder "AMEAUT" existed at the destination, it automatically starts copying all the client folder structure in letter "A" into the "AMEAUT" folder without copying each folder's files. It would create each client folder folder structure, but not copy or move the files at the destination "AMEAUT". Which is definitely not correct since its not maintaining the source and destination structures intact. I also need to mention the log file did not record all folders that were processed that should not have.
The second time I ran it, I checked the remaining six letter client names within the input file. I removed all client names that existed in the the destination to see if that was the only problem. I ran it, and it appeared to work fine processing/moving the first 10 or so client subfolders, but then it searched other unrelated client folders within the same client folder letter (A, B, C...) and removed all subfolders that were empty at the source, and did not copy the empty folders to the destination. This is a major problem since it is violating the integrity of all source client folders. The log file did not record anything pertaining to it deleting the empty folders either.
This is a mess!!!!
Your input is greatly appreciated.
The ClientFolders.txt input file sample appears as follows:
ABS200
ADVCUS
AEH&SI
ALLARD
AMEAUT
ANDEHD
ASHLDW
ASSOCI
BAKERC
BANGLE
SrcDir structure is as follows:
A
ABS200
ADVCUS
AEH&SI
ALLARD
AMEAUT
ANDEHD
ASHLDW
ASSOCI...
B
BAKERC
BANGLE...
C
D
E
Within each alphabet directory are many client subfolders. Each client folder's first six letters are there client code, i.e. ABS200, ADVCUS, AEH&SI...
The batch file logic:
The first time I ran it, I was watching the folders scroll by as it processed. Then all of a sudden I saw many folders listed that were not part of the input list. So I stopped it. I discovered if the same client folder existed at the destination, then it would copy all the client folders listed within the same letter (A, B, C) it was processing into the destination folder that already existing and created all the client folders for that letter without copying any client files. So let's say client folder "AMEAUT" existed at the destination, it automatically starts copying all the client folder structure in letter "A" into the "AMEAUT" folder without copying each folder's files. It would create each client folder folder structure, but not copy or move the files at the destination "AMEAUT". Which is definitely not correct since its not maintaining the source and destination structures intact. I also need to mention the log file did not record all folders that were processed that should not have.
The second time I ran it, I checked the remaining six letter client names within the input file. I removed all client names that existed in the the destination to see if that was the only problem. I ran it, and it appeared to work fine processing/moving the first 10 or so client subfolders, but then it searched other unrelated client folders within the same client folder letter (A, B, C...) and removed all subfolders that were empty at the source, and did not copy the empty folders to the destination. This is a major problem since it is violating the integrity of all source client folders. The log file did not record anything pertaining to it deleting the empty folders either.
This is a mess!!!!
Your input is greatly appreciated.
The ClientFolders.txt input file sample appears as follows:
ABS200
ADVCUS
AEH&SI
ALLARD
AMEAUT
ANDEHD
ASHLDW
ASSOCI
BAKERC
BANGLE
SrcDir structure is as follows:
A
ABS200
ADVCUS
AEH&SI
ALLARD
AMEAUT
ANDEHD
ASHLDW
ASSOCI...
B
BAKERC
BANGLE...
C
D
E
Within each alphabet directory are many client subfolders. Each client folder's first six letters are there client code, i.e. ABS200, ADVCUS, AEH&SI...
The batch file logic:
@echo off
setlocal enabledelayedexpansion
set FNLog=F:\xxcopy\29MAR2016TESTFolder\Logs\BizNas-2017-2.log
set SrcDir=\\biznas-01ss\Client_Data_Files
set TgtDir=\\biznas-01ss\Public_Archives\CDF
set FNClients=F:\xxcopy\29MAR2016TESTFolder\ClientFolders.txt
for /d %%D in ("%SrcDir%\*") do (
set ClientFullDir=%%D
set ClientDir=%%~nD
for /f %%a in (%FNClients%) do (
if exist "!ClientFullDir!\%%a*" (
for /d %%E in ("!ClientFullDir!\%%a*") do (
xxcopy "!ClientFullDir!\%%~nE" "%TgtDir%\!ClientDir!\%%~nE" /SC /S /E /VL /YY /V /JV /RC /TTA0 /TCA /H /TCC /TCW /I /F /oA:"%FNLog%" /oF3 /oE3
echo.
)
)
)
)
Lionel MM
is \\biznas-01ss\Public_Archi
ASKER
Both folders reside side-to-side, so \\biznas-01ss\Public_Archi
ASKER
I have been running several tests, and discovered a problem with the client folder name beyond the first six characters.
Example: Lets take name "ABS200", this is the first six characters of the top client folder name. There is more characters afterwards. If there is a period in the middle of the folder name such as "ABS200 - E. Make Holder", it will not copy and move the folder. It simply will create the folder (ABS200 - E) and subfolder structure at the destination with no files (notice it truncated all the text after the period). Also, oddly, it will also create all the folder structure for "all" the clients folders in same directory as the "ABS200", so that's all the "A" folders as well. All the client folders names are intact with "." as originally named, but all these folders have no files. This is good that they are empty for cleanup purposes.
So it will copy and move client folders that do not have a period in the top client folder name, and it will not copy and move folders with "." periods within the client folder name. I also tested it further by placing a "." in a client folder subfolder, and it had no effect. The problem appears to be top level client folder names with "." within it.
If this is all the problem, then how can the batch file get modified to avoid this issue?
Example: Lets take name "ABS200", this is the first six characters of the top client folder name. There is more characters afterwards. If there is a period in the middle of the folder name such as "ABS200 - E. Make Holder", it will not copy and move the folder. It simply will create the folder (ABS200 - E) and subfolder structure at the destination with no files (notice it truncated all the text after the period). Also, oddly, it will also create all the folder structure for "all" the clients folders in same directory as the "ABS200", so that's all the "A" folders as well. All the client folders names are intact with "." as originally named, but all these folders have no files. This is good that they are empty for cleanup purposes.
So it will copy and move client folders that do not have a period in the top client folder name, and it will not copy and move folders with "." periods within the client folder name. I also tested it further by placing a "." in a client folder subfolder, and it had no effect. The problem appears to be top level client folder names with "." within it.
If this is all the problem, then how can the batch file get modified to avoid this issue?
try this
if exist "!ClientFullDir!\%%a*.*" (
for /d %%E in ("!ClientFullDir!\%%a*.*")
if exist "!ClientFullDir!\%%a*.*" (
for /d %%E in ("!ClientFullDir!\%%a*.*")
ASKER
Please note, I have spot checked several client directory folder names and they appear to contain whole company names or person's names, such as:
ABC500 - ABS 500, Inc. - John D. Wood, Jr.
DAV123 - David S. Johnson III
SKIPER - Thomas T. Morgan, Sr. & Assoc., P.C.
....
They appear to have been using ".", ",", "&", "#", "$".... in the folder and subfolder names.
ABC500 - ABS 500, Inc. - John D. Wood, Jr.
DAV123 - David S. Johnson III
SKIPER - Thomas T. Morgan, Sr. & Assoc., P.C.
....
They appear to have been using ".", ",", "&", "#", "$".... in the folder and subfolder names.
I think you have found the problem -- many of these special characters will cause issues in trying to read existing folders and to recreate new ones.
I did a quick fix. Not sure if this is what you want.
Note: My root folder is c:\local\test\EE. Please change to your need.
It returns this:
Does that look like what you need?
Note: My root folder is c:\local\test\EE. Please change to your need.
@echo off
setlocal enabledelayedexpansion
set FNLog=c:\local\test\Logs\BizNas-2017-2.log
set SrcDir=c:\local\test\EE\Client_Data_Files
set TgtDir=c:\local\test\EE\Public_Archives\CDF
set FNClients=c:\local\test\EE\ClientFolders.txt
pushd "%SrcDir%"
for /f %%a in (%FNClients%) do (
if exist "!SrcDir!\%%a*" (
for /d %%E in ("!SrcDir!\%%a*") do (
echo xxcopy "!SrcDir!\%%~nE" "%TgtDir%\%%a\%%~nE" /SC /S /E /VL /YY /V /JV /RC /TTA0 /TCA /H /TCC /TCW /I /F /oA:"%FNLog%" /oF3 /oE3
echo.
)
)
)
)
popdIt returns this:
xxcopy "c:\local\test\EE\Client_Data_Files\ABC500 - ABS 500, Inc. - John D" "c:\local\test\EE\Public_Archives\CDF\ABC500\ABC500 - ABS 500, Inc. - John D" /SC /S /E /VL /YY /V /JV /RC /TTA0 /TCA /H /TCC /TCW /I /F /oA:"c:\local\test\Logs\BizNas-2017-2.log" /oF3 /oE3
xxcopy "c:\local\test\EE\Client_Data_Files\DAV123 - David S" "c:\local\test\EE\Public_Archives\CDF\DAV123\DAV123 - David S" /SC /S /E /VL /YY /V /JV /RC /TTA0 /TCA /H /TCC /TCW /I /F /oA:"c:\local\test\Logs\BizNas-2017-2.log" /oF3 /oE3
xxcopy "c:\local\test\EE\Client_Data_Files\SKIPER - Thomas T. Morgan, Sr. & Assoc., P" "c:\local\test\EE\Public_Archives\CDF\SKIPER\SKIPER - Thomas T. Morgan, Sr. & Assoc., P" /SC /S /E /VL /YY /V /JV /RC /TTA0 /TCA /H /TCC /TCW /I /F /oA:"c:\local\test\Logs\BizNas-2017-2.log" /oF3 /oE3Does that look like what you need?
ASKER
Not sure if it matters, but look at "DAV123 - David S" as displayed above "It returns this". In my above example the source was "DAV123 - David S. Johnson III". If you used that folder name as the source, and then it truncated the remainder of the folder name after "S.", then that's the problem we experienced. You may have not used exact same folder name as I provided, so that could be it.
The existing problem appears to do this with top level folder names that have periods "." anywhere in it, expect as the last character. Its strange that it creates a folder in \CDF\DAV123 - David S, and then subfolders are created with all the other top level folders using the exact same folder names and subfolder structure as the source with no files. Each of the top level folders that have no files have the correct top level folder names with periods and all "DAV123 - David S. Johnson III". To make matters worse, all the top level folders that are copied with all the folder structure and no files, all the empty subfolders that exist at the source are removed (deleted) as well. That's scary and unpredictable. When I get into the office Monday night I will try your revised logic. Thank you for the response.
The existing problem appears to do this with top level folder names that have periods "." anywhere in it, expect as the last character. Its strange that it creates a folder in \CDF\DAV123 - David S, and then subfolders are created with all the other top level folders using the exact same folder names and subfolder structure as the source with no files. Each of the top level folders that have no files have the correct top level folder names with periods and all "DAV123 - David S. Johnson III". To make matters worse, all the top level folders that are copied with all the folder structure and no files, all the empty subfolders that exist at the source are removed (deleted) as well. That's scary and unpredictable. When I get into the office Monday night I will try your revised logic. Thank you for the response.
For one of the "It returns this" lines, can you correct it to show what it should be instead?
ASKER
I just used example: DAV123 - David S. Johnson III
The first six characters are the client codes, and then hyphen with either the full client name or company name, i.e.
ABC111 - John H. Doe, Jr.
AJC567 - John Deer, P.C.
AYC253 - DCS-ALT & Enterprises, Inc.
AJE898 - Jane P. Hyle
....
So, if the input file has: ABC111, then it will create a folder at the destination as "ABC111 - John H", and then under that same folder it will create empty folders under this folder as follows: ABC111 - John H. Doe, Jr., AJC567 - John Deer, P.C., AYC253 - DCS-ALT & Enterprises, Inc., AJE898 - Jane P. Hyle... Also, the source ABC111 - John H. Doe, Jr. will not be copied to the destination as expected or removed from the source location.
As it copies all the client folders, it will delete any empty subfolder under each source client folder.
The first six characters are the client codes, and then hyphen with either the full client name or company name, i.e.
ABC111 - John H. Doe, Jr.
AJC567 - John Deer, P.C.
AYC253 - DCS-ALT & Enterprises, Inc.
AJE898 - Jane P. Hyle
....
So, if the input file has: ABC111, then it will create a folder at the destination as "ABC111 - John H", and then under that same folder it will create empty folders under this folder as follows: ABC111 - John H. Doe, Jr., AJC567 - John Deer, P.C., AYC253 - DCS-ALT & Enterprises, Inc., AJE898 - Jane P. Hyle... Also, the source ABC111 - John H. Doe, Jr. will not be copied to the destination as expected or removed from the source location.
As it copies all the client folders, it will delete any empty subfolder under each source client folder.
I meant, can you show me what my code should look like? The xxcopy lines. That would be better.
ASKER
xxcopy "c:\local\test\EE\Client_D
xxcopy "c:\local\test\EE\Client_D
xxcopy "c:\local\test\EE\Client_D
I appears the top level folder names were not right.
1st line has:
xxcopy "c:\local\test\EE\Client_D
And it should have been: "c:\local\test\EE\Client_D
2nd line has:
xxcopy "c:\local\test\EE\Client_D
And it should have been: "c:\local\test\EE\Client_D
3rd line has:
"c:\local\test\EE\Client_D
And it should have been: "c:\local\test\EE\Client_D
It appears both the source and destinations folder names match, but the source appeared to have truncated names than specified in my examples.
Also, please remember the source folder structure is as follows:
\Client_Data_Files\A\
\Client_Data_Files\B\
\Client_Data_Files\C\
\Client_Data_Files\D\
xxcopy "c:\local\test\EE\Client_D
xxcopy "c:\local\test\EE\Client_D
I appears the top level folder names were not right.
1st line has:
xxcopy "c:\local\test\EE\Client_D
And it should have been: "c:\local\test\EE\Client_D
2nd line has:
xxcopy "c:\local\test\EE\Client_D
And it should have been: "c:\local\test\EE\Client_D
3rd line has:
"c:\local\test\EE\Client_D
And it should have been: "c:\local\test\EE\Client_D
It appears both the source and destinations folder names match, but the source appeared to have truncated names than specified in my examples.
Also, please remember the source folder structure is as follows:
\Client_Data_Files\A\
\Client_Data_Files\B\
\Client_Data_Files\C\
\Client_Data_Files\D\
ASKER
I did not get a chance to test your revised script last night. I will attempt to do so tonight.
ASKER
I modified the paths, and when I ran it, I received error: The parameter is incorrect.
ASKER
I removed "echo" in front of xxcopy command, and the "echo." on the following line. When I run it, a DOS window opens and then immediately closes. Nothing runs.
The ECHO is meant to be used as a debugging tool.
1. Open a CMD window.
2. Run the .bat
3. Examine the output of the xxcopy command carefuly. Does it look correct?
1. Open a CMD window.
2. Run the .bat
3. Examine the output of the xxcopy command carefuly. Does it look correct?
ASKER
I placed the second "echo." back, and the xcopy command no longer has "echo" in front of it. Running the bat file via the command prompt made no difference. No log file is generated either, so there is nothing to review.
ASKER
I just put "echo" back in front of the xxcopy command line, and it did not error, but it does not run and no log file is generated.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I just tried it, and it does the same thing. It all has to do with top level folder names that contain one or more periods.
For instance:
DRMARL-Dr. Marlene H. Lecastor, O.D., P.C.
DOAPES-D.O.A. Pesteline Services, Inc.
DEBDAV-Debie Z. Davenport, D.D.S., P.C.
For instance:
DRMARL-Dr. Marlene H. Lecastor, O.D., P.C.
DOAPES-D.O.A. Pesteline Services, Inc.
DEBDAV-Debie Z. Davenport, D.D.S., P.C.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
I just tried it, and it does the same thing. It all has to do with top level folder names that contain one or more periods.Can you expand on this please, what did it do, what do you want it to do, etc?
~bp
ASKER
I ran the batch file with the pushd command, and a dos window opens and then immediately closes. No log file generated either.
I ran the following test.
I created these folders:
I created these folders:
md "B:\ee\EE29001020\Client_Data_Files\A\ABC111 - John H. Doe, Jr."
md "B:\ee\EE29001020\Client_Data_Files\A\ABC500 - ABS 500, Inc. - John D. Wood, Jr."
md "B:\ee\EE29001020\Client_Data_Files\A\AJC567 - John Deer, P.C."
md "B:\ee\EE29001020\Client_Data_Files\A\AJE898 - Jane P. Hyle"
md "B:\ee\EE29001020\Client_Data_Files\A\AYC253 - DCS-ALT & Enterprises, Inc."
md "B:\ee\EE29001020\Client_Data_Files\D\DAV123 - David S. Johnson III"
md "B:\ee\EE29001020\Client_Data_Files\S\SKIPER - Thomas T. Morgan, Sr. & Assoc., P.C."B:\ee\EE29001020\Client_Data_Files\S\SKIPER - Thomas T. Morgan, Sr. & Assoc., P.C
B:\ee\EE29001020\Client_Data_Files\A\ABC500 - ABS 500, Inc. - John D. Wood, Jr
B:\ee\EE29001020\Client_Data_Files\A\AJE898 - Jane P. Hyle
B:\ee\EE29001020\Client_Data_Files\A\AJC567 - John Deer, P.C
B:\ee\EE29001020\Client_Data_Files\A\ABC111 - John H. Doe, Jr
B:\ee\EE29001020\Client_Data_Files\A\AYC253 - DCS-ALT & Enterprises, Inc
B:\ee\EE29001020\Client_Data_Files\D\DAV123 - David S. Johnson IIIxxcopy "B:\ee\EE29001020\Client_Data_Files\S\SKIPER - Thomas T. Morgan, Sr. & Assoc., P.C" "B:\ee\EE29001020\Public_Archives\CDF\S\SKIPER - Thomas T. Morgan, Sr. & Assoc., P.C" /SC /S /E /VL /YY /V /JV /RC /TTA0 /TCA /H /TCC /TCW /I /F /oA:"B:\ee\EE29001020\BizNas-2017-2.log" /oF3 /oE3
xxcopy "B:\ee\EE29001020\Client_Data_Files\A\ABC111 - John H. Doe, Jr" "B:\ee\EE29001020\Public_Archives\CDF\A\ABC111 - John H. Doe, Jr" /SC /S /E /VL /YY /V /JV /RC /TTA0 /TCA /H /TCC /TCW /I /F /oA:"B:\ee\EE29001020\BizNas-2017-2.log" /oF3 /oE3
xxcopy "B:\ee\EE29001020\Client_Data_Files\A\ABC500 - ABS 500, Inc. - John D. Wood, Jr" "B:\ee\EE29001020\Public_Archives\CDF\A\ABC500 - ABS 500, Inc. - John D. Wood, Jr" /SC /S /E /VL /YY /V /JV /RC /TTA0 /TCA /H /TCC /TCW /I /F /oA:"B:\ee\EE29001020\BizNas-2017-2.log" /oF3 /oE3
xxcopy "B:\ee\EE29001020\Client_Data_Files\A\AJC567 - John Deer, P.C" "B:\ee\EE29001020\Public_Archives\CDF\A\AJC567 - John Deer, P.C" /SC /S /E /VL /YY /V /JV /RC /TTA0 /TCA /H /TCC /TCW /I /F /oA:"B:\ee\EE29001020\BizNas-2017-2.log" /oF3 /oE3
xxcopy "B:\ee\EE29001020\Client_Data_Files\A\AJE898 - Jane P. Hyle" "B:\ee\EE29001020\Public_Archives\CDF\A\AJE898 - Jane P. Hyle" /SC /S /E /VL /YY /V /JV /RC /TTA0 /TCA /H /TCC /TCW /I /F /oA:"B:\ee\EE29001020\BizNas-2017-2.log" /oF3 /oE3
xxcopy "B:\ee\EE29001020\Client_Data_Files\A\AYC253 - DCS-ALT & Enterprises, Inc" "B:\ee\EE29001020\Public_Archives\CDF\A\AYC253 - DCS-ALT & Enterprises, Inc" /SC /S /E /VL /YY /V /JV /RC /TTA0 /TCA /H /TCC /TCW /I /F /oA:"B:\ee\EE29001020\BizNas-2017-2.log" /oF3 /oE3
xxcopy "B:\ee\EE29001020\Client_Data_Files\D\DAV123 - David S. Johnson III" "B:\ee\EE29001020\Public_Archives\CDF\D\DAV123 - David S. Johnson III" /SC /S /E /VL /YY /V /JV /RC /TTA0 /TCA /H /TCC /TCW /I /F /oA:"B:\ee\EE29001020\BizNas-2017-2.log" /oF3 /oE3
Open a DOS command prompt window and run it there so you can see messages.
~bp
~bp
ASKER
I have a test folder "c:\local\test\EE\Client_D
c:\local\test\EE\Client_Da
c:\local\test\EE\Client_Da
c:\local\test\EE\Client_Da
When I run the batch file it leaves all the above listed folders, but removes any subfolders that are empty. If there are more folders listed, then it will remove all empty folders in those folders too. That's pretty disdurbing.
Then at the destination, it does the following:
it creates:
c:\local\test\EE\Client_Da
c:\local\test\EE\Client_Da
c:\local\test\EE\Client_Da
It copies the folder structures only, and no files within an subfolder. Within each client folder structure it creates at the destination it also replicates the structure of the other three client folders listed at the top level. If I have hundreds of client folder listed at the top it would do so for all of them if not stopped. It's really a mess.
c:\local\test\EE\Client_Da
c:\local\test\EE\Client_Da
c:\local\test\EE\Client_Da
When I run the batch file it leaves all the above listed folders, but removes any subfolders that are empty. If there are more folders listed, then it will remove all empty folders in those folders too. That's pretty disdurbing.
Then at the destination, it does the following:
it creates:
c:\local\test\EE\Client_Da
c:\local\test\EE\Client_Da
c:\local\test\EE\Client_Da
It copies the folder structures only, and no files within an subfolder. Within each client folder structure it creates at the destination it also replicates the structure of the other three client folders listed at the top level. If I have hundreds of client folder listed at the top it would do so for all of them if not stopped. It's really a mess.
Okay, I assumed you had sorted out the XXCOPY command line and options, sounds like that isn't the case. I'll look at those options next...
~bp
~bp
ASKER
I reviewed the xcopy options and they appear fine. You can double check them though. It does not appear to be an xxcopy issue. Also, my log file does not show the destination copied, or only lists the source path. Not sure why, but that's what it does.
ASKER
So when I review the xxcopy log, I see a section as follows:
Source base directory = "c:\local\test\EE\Client_D
Destination directory = "c:\local\test\EE\Client_D
File name template = "DOAPES-D.O.A"
Directories processed = 78
Total Data in Bytes = 0
Elapsed time in sec. = 0.219
Action speed (MB/min) = 0
Files copied = 0
log file appended = c:\local\test\Logs\BizNas-
Exit code = 100 (no files were found to copy)
Source base directory = "c:\local\test\EE\Client_D
Destination directory = "c:\local\test\EE\Client_D
File name template = "DOAPES-D.O.A"
Directories processed = 78
Total Data in Bytes = 0
Elapsed time in sec. = 0.219
Action speed (MB/min) = 0
Files copied = 0
log file appended = c:\local\test\Logs\BizNas-
Exit code = 100 (no files were found to copy)
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I just tested it out, and it made no difference in that any top level folder that contains multiple periods in the the name are not moved to the destination and just the folder structure is created at the destination without any files. The source folders all remain intact with the exception of all folders regardless of top folder name, it deleted any and all folders that are empty. Pretty scary. The log file does not reveal that either.