# Copy files from nested structure and rename by appending former directory name onto filename

I have thousands of files that are present in a series of directories like this:
SourceFolder
FolderA
1001
FileUniqueNameA.txt
1003
FileUniqueNameB.txt
FolderB
2001
FileUniqueNameC.txt
FiletobeIgnored.pdf
2004
FileUniqueNameD.txt
FiletobeIgnored.pdf

I'm looking to copy all of the txt files to a common directory like this:
CommonTargetFolder
1001_FileUniqueNameA.txt
1003_FileUniqueNameB.txt
2001_FileUniqueNameC.txt
2004_FileUniqueNameD.txt

Basic Points:
1) The directory name containing the files in question is unique, and we want to append that to the beginning of the filename as part of the copy...separated by an underscore.
2) We  can select by extension - we want only the txt files and nothing else.
3) The original directory and files needs to stay intact.
4) There are a lot of large files that will be ignored.
5) There are no repetitions of the terminal directory names anywhere in the SourceFolder - they are truly unique.  They start at 01 and go to roughly 1000000.
6) There are no repetitions of the file names (txt) anywhere in the SourceFolder - they are also truly unique.
7) However, the ignored files do have a generic name.

For now it would be acceptable to just do this as a single batch job, however...
Down the road I will want to be able to keep things up to date.  Perhaps set a script to run once a day from 'Scheduled Tasks' or the like, and move newly written files from the source into the target, such that the target stays up to date as the source receives new files.
Expecting ~20,000 text files.  Guaranteed to not bust 65,000 for a couple years.
Windows server 2003, 2008, or 2012.
Strong preference for a solution as a batch or an AutoIT script.

Thank you.
###### Who is Participating?

Commented:
Here's a small BAT that should do the job based on the structure you defined.  Save as a BAT, adjust the folder names, and then run as a test.

@echo off
setlocal

set BaseDir=c:\SourceFolder
set DestDir=c:\CommonTargetFolder

for /D %%A in ("%BaseDir%\*.*") do (
for /D %%B in ("%%~A\*.*") do (
for %%C in ("%%~B\*.txt") do (
copy "%%~C" "%DestDir%\%%~nB_%%~nxC"
)
)
)

~bp
0

System EngineerCommented:
This is not a solution, just my opinion: It may be beneficial to install cygwin and use bash scripts for such repeated jobs of this complexity.
Alternatively, share the folder over samba and mount it in a Linux box to use bash script there.
If you can accept either of these, I can help you write a bash script.
0

IT ConsultantCommented:
Here is a powershell solution:
Param (
[string]$inputPath = "\path\to\sourcefolder", [string]$outputPath = "\path\to\targetfolder"
)

$fileList = Get-ChildItem -Path$inputPath -Recurse:1 -File -Include "*.txt"
foreach ($i in$fileList) {
$newName =$i.FullName.split("\")
$newName =$newName[($newName.count - 2)..($newName.count - 1)]  ## get only the name and last dir
$newName =$newName -join("_")
$newNameWithPath =$outputPath + "\" + $newName$test = Test-Path $newNameWithPath ## check if the file already exists if ($test -like "False") {
Copy-Item $i -Destination$newNameWithPath
}
}


The script will
- extract all txt files from your source folder,
- merge the file name with the name of the last containing folder, using "_" as the separator
- test if a file with that name already exists in your target folder
- copy the file if not already there.

HTH,
Dan
0

IT ConsultancyCommented:
Only thing I would add is maybe the use of  xcopy /d instead of copy then a second run would only copy the new files as requested, sound sensible Bill?

Steve
0

Commented:
Steve,

Yes, I sort of forgot about the rerun in the future part that was mentioned after the requirements. It wasn't clear what this meant though, so I'd want more info on that to design the best solution. More details would be needed, including things like:

- If a file was copied previously should it just be skipped?
- Or if a file was already copied do we need to copy if "newer"?
- Do any files in the dest directory that no longer exist in the source directory need to be removed?
- Etc...

~bp
0

Author Commented:
Sorry for the slow reply (hit with the flu).
Bill,  This is the kind of thing that I'm looking for.  Thank you.
Steve, Thank you for bringing up the part about re-running in the future.

To answer some of the questions:
-If a file was copied previously, it should in fact be skipped.
-Files will never be removed from either directory.
-If a previously existing file was re-added to the source folder, it would be appropriate to re-copy it to the destination, overwriting the prior copy by default.  (Fortunately in this particular case, in the source folder, the 'date modified' happens to be the date and time written to the source folder, so the newer copy would have a newer date.)

Would it be possible to get the most recent date modified from the destination directory, and plug that into the /d in xcopy?
Ed
0

IT ConsultancyCommented:
Ed - xcopy with /d and no date will only copy if it is newer and not do anything otherwise, so something like this would do -- xcopy prompts whether a destination is a file or directory if it doesn't exist so could either do this in place of

copy "%%~C" "%DestDir%\%%~nB_%%~nxC"


echo f | xcopy /d "%%~C" "%DestDir%\%%~nB_%%~nxC" >NUL 2>&1


or

if exist "%DestDir%\%%~nB_%%~nxC" (
xcopy /d "%%~C" "%DestDir%\%%~nB_%%~nxC"
) ELSE (
copy "%%~C" "%DestDir%\%%~nB_%%~nxC"
)


i.e. if it doesn't exist then copy it, otherwise try xcopy /d to see if it has been updated

Steve
0

Commented:

~bp
0

Author Commented:
Thank you both.   Have it set up and running nicely.  Appreciate your help.

FYI, as it is set up, it will copy roughly 1000 files per minute (from and to the same array), and will evaluate at roughly 1500 files per minute if no copy is involved.

Steve,
I used the 'if exists' approach because the ' echo f | xcopy...' approach would occasionally and intermittently yield a "The process tried to write to a nonexistent pipe."  Most of the time it works fine.  When it fails it might return one or two errors out of a 500 file test batch.
0

Author Commented:
Also, for anyone else coming across this thread later, there is a third party solution (freeware) that could be viewed at http://www.bulkrenameutility.co.uk/Screenshots.php

They also have a command line version that supposedly does most of the functionality of the GUI version.

I've tried neither.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.