Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
Solved

# Copy files from nested structure and rename by appending former directory name onto filename

Posted on 2014-01-05
Medium Priority
546 Views
I have thousands of files that are present in a series of directories like this:
SourceFolder
FolderA
1001
FileUniqueNameA.txt
1003
FileUniqueNameB.txt
FolderB
2001
FileUniqueNameC.txt
FiletobeIgnored.pdf
2004
FileUniqueNameD.txt
FiletobeIgnored.pdf

I'm looking to copy all of the txt files to a common directory like this:
CommonTargetFolder
1001_FileUniqueNameA.txt
1003_FileUniqueNameB.txt
2001_FileUniqueNameC.txt
2004_FileUniqueNameD.txt

Basic Points:
1) The directory name containing the files in question is unique, and we want to append that to the beginning of the filename as part of the copy...separated by an underscore.
2) We  can select by extension - we want only the txt files and nothing else.
3) The original directory and files needs to stay intact.
4) There are a lot of large files that will be ignored.
5) There are no repetitions of the terminal directory names anywhere in the SourceFolder - they are truly unique.  They start at 01 and go to roughly 1000000.
6) There are no repetitions of the file names (txt) anywhere in the SourceFolder - they are also truly unique.
7) However, the ignored files do have a generic name.

For now it would be acceptable to just do this as a single batch job, however...
Down the road I will want to be able to keep things up to date.  Perhaps set a script to run once a day from 'Scheduled Tasks' or the like, and move newly written files from the source into the target, such that the target stays up to date as the source receives new files.
Expecting ~20,000 text files.  Guaranteed to not bust 65,000 for a couple years.
Windows server 2003, 2008, or 2012.
Strong preference for a solution as a batch or an AutoIT script.

Thank you.
0
Question by:nosliwde99
[X]
###### Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

• Help others & share knowledge
• Earn cash & points
• 3
• 3
• 2
• +2

LVL 8

Expert Comment

ID: 39758633
This is not a solution, just my opinion: It may be beneficial to install cygwin and use bash scripts for such repeated jobs of this complexity.
Alternatively, share the folder over samba and mount it in a Linux box to use bash script there.
If you can accept either of these, I can help you write a bash script.
0

LVL 35

Expert Comment

ID: 39758634
Here is a powershell solution:
Param (
[string]$inputPath = "\path\to\sourcefolder", [string]$outputPath = "\path\to\targetfolder"
)

$fileList = Get-ChildItem -Path$inputPath -Recurse:1 -File -Include "*.txt"
foreach ($i in$fileList) {
$newName =$i.FullName.split("\")
$newName =$newName[($newName.count - 2)..($newName.count - 1)]  ## get only the name and last dir
$newName =$newName -join("_")
$newNameWithPath =$outputPath + "\" + $newName$test = Test-Path $newNameWithPath ## check if the file already exists if ($test -like "False") {
Copy-Item $i -Destination$newNameWithPath
}
}


The script will
- extract all txt files from your source folder,
- merge the file name with the name of the last containing folder, using "_" as the separator
- test if a file with that name already exists in your target folder
- copy the file if not already there.

HTH,
Dan
0

LVL 58

Accepted Solution

Bill Prew earned 1600 total points
ID: 39759048
Here's a small BAT that should do the job based on the structure you defined.  Save as a BAT, adjust the folder names, and then run as a test.

@echo off
setlocal

set BaseDir=c:\SourceFolder
set DestDir=c:\CommonTargetFolder

for /D %%A in ("%BaseDir%\*.*") do (
for /D %%B in ("%%~A\*.*") do (
for %%C in ("%%~B\*.txt") do (
copy "%%~C" "%DestDir%\%%~nB_%%~nxC"
)
)
)

~bp
0

LVL 43

Expert Comment

ID: 39760565
Only thing I would add is maybe the use of  xcopy /d instead of copy then a second run would only copy the new files as requested, sound sensible Bill?

Steve
0

LVL 58

Expert Comment

ID: 39761176
Steve,

Yes, I sort of forgot about the rerun in the future part that was mentioned after the requirements. It wasn't clear what this meant though, so I'd want more info on that to design the best solution. More details would be needed, including things like:

- If a file was copied previously should it just be skipped?
- Or if a file was already copied do we need to copy if "newer"?
- Do any files in the dest directory that no longer exist in the source directory need to be removed?
- Etc...

~bp
0

Author Comment

ID: 39766958
Sorry for the slow reply (hit with the flu).
Bill,  This is the kind of thing that I'm looking for.  Thank you.
Steve, Thank you for bringing up the part about re-running in the future.

To answer some of the questions:
-If a file was copied previously, it should in fact be skipped.
-Files will never be removed from either directory.
-If a previously existing file was re-added to the source folder, it would be appropriate to re-copy it to the destination, overwriting the prior copy by default.  (Fortunately in this particular case, in the source folder, the 'date modified' happens to be the date and time written to the source folder, so the newer copy would have a newer date.)

Would it be possible to get the most recent date modified from the destination directory, and plug that into the /d in xcopy?
Ed
0

LVL 43

Assisted Solution

Steve Knight earned 400 total points
ID: 39767304
Ed - xcopy with /d and no date will only copy if it is newer and not do anything otherwise, so something like this would do -- xcopy prompts whether a destination is a file or directory if it doesn't exist so could either do this in place of

copy "%%~C" "%DestDir%\%%~nB_%%~nxC"


echo f | xcopy /d "%%~C" "%DestDir%\%%~nB_%%~nxC" >NUL 2>&1


or

if exist "%DestDir%\%%~nB_%%~nxC" (
xcopy /d "%%~C" "%DestDir%\%%~nB_%%~nxC"
) ELSE (
copy "%%~C" "%DestDir%\%%~nB_%%~nxC"
)


i.e. if it doesn't exist then copy it, otherwise try xcopy /d to see if it has been updated

Steve
0

LVL 58

Expert Comment

ID: 39774121

~bp
0

Author Closing Comment

ID: 39775252
Thank you both.   Have it set up and running nicely.  Appreciate your help.

FYI, as it is set up, it will copy roughly 1000 files per minute (from and to the same array), and will evaluate at roughly 1500 files per minute if no copy is involved.

Steve,
I used the 'if exists' approach because the ' echo f | xcopy...' approach would occasionally and intermittently yield a "The process tried to write to a nonexistent pipe."  Most of the time it works fine.  When it fails it might return one or two errors out of a 500 file test batch.
0

Author Comment

ID: 39775258
Also, for anyone else coming across this thread later, there is a third party solution (freeware) that could be viewed at http://www.bulkrenameutility.co.uk/Screenshots.php

They also have a command line version that supposedly does most of the functionality of the GUI version.

I've tried neither.
0

## Featured Post

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

You may have already been in the need to update a whole folder stucture using a script. Robocopy does it well and even provides a list of non-updated files in a log (if asked to). Generally those files that were locked by a user or a process by the …
With User Account Control (UAC) enabled in Windows 7, one needs to open an elevated Command Prompt in order to run scripts under administrative privileges. Although the elevated Command Prompt accomplishes the task, the question How to run as script…
This tutorial will teach you the special effect of super speed similar to the fictional character Wally West aka "The Flash" After Shake : http://www.videocopilot.net/presets/after_shake/ All lightning effects with instructions : http://www.mediaf…
In response to a need for security and privacy, and to continue fostering an environment members can turn to for support, solutions, and education, Experts Exchange has created anonymous question capabilities. This new feature is available to our Pr…
###### Suggested Courses
Course of the Month9 days, 7 hours left to enroll