Use powershell to read a text file containing filenames to move just those files from folder A to folder B.

powershell and Windows 7.

Hi,

I have a text file request.txt containing filenames such as:
18-111
18-222

Actual filenames:
18-111 member 91-A.pdf
18-111 MD 91-A.pdf
18-222 member 91-B.pdf
18-222 MD 91-B.pdf
18-333 member 91-C.pdf
18-333 MD 91-C.pdf

QUESTION: how do I use powershell to
1) loop through request.txt and do a partial match to identify certain .pdf files to move from folder A to a specific folder B that has the same name as the last 4 characters of the filename?  

RESULT:
18-111 member 91-A.pdf is moved to folder 91-A.
18-111 MD 91-A.pdf is moved to folder 91-A.
18-222 member 91-B.pdf is moved to folder 91-B.
18-222 MD 91-B.pdf is moved to folder 91-B.

2) Add today's timestamp to the end of the filename in the format YYYMMDD_HHMM in 24hour format.
RESULT: If today is 3/26/2019 1:21pm.
18-111 member 91-A 20190326_1321.pdf
18-111 MD 91-A 20190326_1321.pdf
18-222 member 91-B 20190326_1321.pdf
18-222 MD 91-B 20190326_1321.pdf
paultran00Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

oBdACommented:
This should work; it's in test mode and will only display what it would move where. Remove the -WhatIf at the end of line 24 to run it for real.
$source = "D:\Temp\Source"
$target = "D:\Temp\Target"
$filePatterns = Get-Content D:\temp\Source\files.txt
$dirPatterns = Get-ChildItem -Path $target -Directory
$timeStamp = Get-Date -Format 'yyyyMMdd_HHmm'
Get-ChildItem -Path $source -Filter *.pdf | ForEach-Object {
	Write-Host "Processing '$($_.Name)' ..."
	$copy = $false
	ForEach ($pattern in $filePatterns) {
		If ($_.Name -like "$($pattern)*") {
			$copy = $true
			Break
		}
	}
	If ($copy) {
		$target = $null
		ForEach ($pattern in $dirPatterns) {
			If ($_.BaseName -like "*$($pattern.Name)") {
				$target = $pattern
				Break
			}
		}
		If ($target) {
			Move-Item -Path $_.FullName -Destination "$($target.FullName)\$($_.BaseName) $($timeStamp)$($_.Extension)" -WhatIf
			Write-Host "... moved to '$($target.FullName)'!" -ForegroundColor Green
		} Else {
			Write-Host "... skipped, no matching target folder found!" -ForegroundColor Yellow
		}
	} Else {
		Write-Host "... skipped, no match in pattern file."
	}
}

Open in new window

paultran00Author Commented:
Wow you're fast.  I tried to run it in powershell but it's not working.

I created 2 folders in C;\temp\Source  and C;\temp\Target

I created C;\temp\source\files.txt and its contents look like:
18-036154960
18-036181365

Folder C;\temp\source has 18 .pdf files and 4 files match:
18-036154960 MD 91-1104421C.pdf
18-036154960 Member 91-1104421C.pdf
18-036181365 MD 91-1104421C.pdf
18-036181365 Member 91-1104421C.pdf


When I run it, the error message is:

PS H:\> N:\BATCH\DenialLetters\Get_CAS_files.ps1
Processing '18-036154960 MD 91-1104421C.pdf' ...
... skipped, no matching target folder found!
Processing '18-036154960 Member 91-1104421C.pdf' ...
... skipped, no matching target folder found!
Processing '18-036181365 MD 91-1104421C.pdf' ...
... skipped, no matching target folder found!
Processing '18-036181365 Member 91-1104421C.pdf' ...
... skipped, no matching target folder found!
Processing '18-036279556 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036279556 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036330631 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036330631 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036342089 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036342089 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036523443 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036523443 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547207 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547207 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547218 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547218 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036554129 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036554129 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.

PS H:\>
oBdACommented:
Then there's no folder "MD 91-1104421C" or "91-1104421C" or similar in C:\temp\target ...
CompTIA Network+

Prepare for the CompTIA Network+ exam by learning how to troubleshoot, configure, and manage both wired and wireless networks.

Learn More!
paultran00Author Commented:
In folder target, I created SUBfolder "91-1104421C" and it successfully moved 4 files there.

QUESTION:
1. where in the code holds the value "91-1104421C" for it to look for a folder of that name ?
2.  when I changed the command from Move-Item to Copy-Item, it doesn't work.  Are the syntax different?
3. how do I see the syntax for move-item and copy-item?

thank you.
oBdACommented:
1. Nowhere. It compares the end of the file name with the folder names in the target folder.
2. Works just fine with Copy-Item without any other changes. If you just moved the files back into the source folder, don't forget that they now have the date attached at the end, and won't find a matching target folder anymore.
3. Possibly the most important cmdlet: Get-Help
Get-Help Move-Item -ShowWindow
Get-Help Copy-Item -ShowWindow

Open in new window

paultran00Author Commented:
1. How does it determine what length of the end of the source filename to parse to figure out that "91-1104421C"  is the string that it will use to find a subfolder with that name?
oBdACommented:
In line 4, it reads all the folder items in the target:
$dirPatterns = Get-ChildItem -Path $target -Directory

In lines 17-22, it compares the name of the current file with each folder item's name, using a wildcard, not a static length:
If ($_.BaseName -like "*$($pattern.Name)")
paultran00Author Commented:
3. What if files.txt contained fileMickey and fileDonald but these files do not exist in the source folder; how would the code look to write to a log file that says: today's date and the message Cannot find test_file.

files.txt looks like this:
18-036154960
18-036181365
fileMickey
fileDonald

log file looks like:
20190326_1641 Cannot find fileMickey
20190326_1641 Cannot find fileDonald
--------------------------------

4. What if the target folder does NOT exist?  What would the code look like to move the files to a subfolder named UNKNOWN

Thank you.
oBdACommented: 
$source = "D:\Temp\Source"
$target = "D:\Temp\Target"
$targetUnknown = "$($target)\Unknown"
$fileList = "D:\Temp\Source\files.txt"
$logFile = "D:\Temp\Source\files.log"

$filePatterns = Get-Content -Path $fileList | Select-Object -Property @{n='Name'; e={$_}},  @{n='Found'; e={$false}}
$dirPatterns = Get-ChildItem -Path $target -Directory
$timeStamp = Get-Date -Format 'yyyyMMdd_HHmm'

Start-Transcript -Path $logFile
Get-ChildItem -Path $source -Filter *.pdf | ForEach-Object {
	Write-Host "Processing '$($_.Name)' ..."
	$copy = $false
	ForEach ($pattern in $filePatterns) {
		If ($_.Name -like "$($pattern.Name)*") {
			$copy = $true
			$pattern.Found = $true
			Break
		}
	}
	If ($copy) {
		$target = $null
		ForEach ($pattern in $dirPatterns) {
			If ($_.BaseName -like "*$($pattern.Name)") {
				$target = $pattern
				Break
			}
		}
		If ($target) {
			$destination = "$($target.FullName)\$($_.BaseName) $($timeStamp)$($_.Extension)"
		} Else {
			$destination = $targetUnknown
		}
		Move-Item -Path $_.FullName -Destination $destination  -WhatIf
		Write-Host "... moved to '$($destination)'." -ForegroundColor Green
	} Else {
		Write-Host "... skipped, no match in pattern file."
	}
}
$filePatterns | Where-Object {-not $_.Found} | ForEach-Object {
	Write-Host "$($timeStamp) No file matching '$($_.Name)' found!" -ForegroundColor Yellow
}
Stop-Transcript

Open in new window

paultran00Author Commented:
Thank you.

ERROR message:
Start-Transcript : This host does not support transcription.
At C:\Temp\Get_CAS_files_log.ps1:11 char:1
+ Start-Transcript -Path $logFile
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotImplemented: (:) [Start-Transcript], PSNotSupportedException
    + FullyQualifiedErrorId : NotSupported,Microsoft.PowerShell.Commands.StartTranscriptCommand



I am using Windows 7, PS version 4:

PS C:\Windows\System32\WindowsPowerShell\v1.0> $PSVersionTable.PSVersion

Major  Minor  Build  Revision
-----  -----  -----  --------
4      0      -1     -1
oBdACommented:
Start it in a normal PS console, not in the ISE.
paultran00Author Commented:
I ran it from the command prompt and ran without errors.  But it did not copy the files.

powershell.exe c:\temp\Get_CAS_files_log.ps1

The LOG file files.log did get created and populated with entries (I replaced private information in the log below with xxx):

**********************
Windows PowerShell transcript start
Start time: 20190327095303
Username  : xxx
Machine        :  xxx  (Microsoft Windows NT 6.1.7601 Service Pack 1)
**********************
Transcript started, output file is C:\Temp\Source\files.log
Processing '18-036154960 MD 91-1104421C.pdf' ...
What if: Performing the operation "Copy File" on target "Item: C:\Temp\Source\18-036154960 MD 91-1104421C.pdf Destination: C:\Temp\Target\91-1104421C\18-036154960 MD 91-1104421C 20190327_0953.pdf".
... moved to 'C:\Temp\Target\91-1104421C\18-036154960 MD 91-1104421C 20190327_0953.pdf'.
Processing '18-036154960 Member 91-1104421C.pdf' ...
What if: Performing the operation "Copy File" on target "Item: C:\Temp\Source\18-036154960 Member 91-1104421C.pdf Destination: C:\Temp\Target\91-1104421C\18-036154960 Member 91-1104421C 20190327_0953.pdf".
... moved to 'C:\Temp\Target\91-1104421C\18-036154960 Member 91-1104421C 20190327_0953.pdf'.
Processing '18-036181365 MD 91-1104421C.pdf' ...
What if: Performing the operation "Copy File" on target "Item: C:\Temp\Source\18-036181365 MD 91-1104421C.pdf Destination: C:\Temp\Target\91-1104421C\18-036181365 MD 91-1104421C 20190327_0953.pdf".
... moved to 'C:\Temp\Target\91-1104421C\18-036181365 MD 91-1104421C 20190327_0953.pdf'.
Processing '18-036181365 Member 91-1104421C.pdf' ...
What if: Performing the operation "Copy File" on target "Item: C:\Temp\Source\18-036181365 Member 91-1104421C.pdf Destination: C:\Temp\Target\91-1104421C\18-036181365 Member 91-1104421C 20190327_0953.pdf".
... moved to 'C:\Temp\Target\91-1104421C\18-036181365 Member 91-1104421C 20190327_0953.pdf'.
Processing '18-036279556 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036279556 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036330631 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036330631 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036342089 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036342089 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036523443 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036523443 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547207 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547207 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547218 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547218 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036554129 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036554129 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
20190327_0953 No file matching 'test_file' found!
**********************
Windows PowerShell transcript end
End time: 20190327095303
**********************
oBdACommented:
As before in test mode (-WhatIf in line 35)
That's why you see the
What if: Performing the operation "Copy File" ...
in the log and in the console; it means that the command isn't really executed, it only pretends to run,
paultran00Author Commented:
That worked after I removed the WHAT IF.

QUESTION:
4. The existing LOG file files.log appears to be overwritten each time.

What would the code look like if we wrote to a new LOG file that is created (with a datestamp at the end of the filename)?
oBdACommented:
You move the $timeStamp definition before the $logFile definition and add the time stamp to the $logFile definition.
$source = "D:\Temp\Source"
$target = "D:\Temp\Target"
$targetUnknown = "$($target)\Unknown"
$fileList = "D:\Temp\Source\files.txt"
$timeStamp = Get-Date -Format 'yyyyMMdd_HHmm'
$logFile = "D:\Temp\Source\files_$($timeStamp).log"

$filePatterns = Get-Content -Path $fileList | Select-Object -Property @{n='Name'; e={$_}},  @{n='Found'; e={$false}}
$dirPatterns = Get-ChildItem -Path $target -Directory

Start-Transcript -Path $logFile
Get-ChildItem -Path $source -Filter *.pdf | ForEach-Object {
	Write-Host "Processing '$($_.Name)' ..."
	$copy = $false
	ForEach ($pattern in $filePatterns) {
		If ($_.Name -like "$($pattern.Name)*") {
			$copy = $true
			$pattern.Found = $true
			Break
		}
	}
	If ($copy) {
		$target = $null
		ForEach ($pattern in $dirPatterns) {
			If ($_.BaseName -like "*$($pattern.Name)") {
				$target = $pattern
				Break
			}
		}
		If ($target) {
			$destination = "$($target.FullName)\$($_.BaseName) $($timeStamp)$($_.Extension)"
		} Else {
			$destination = $targetUnknown
		}
		Move-Item -Path $_.FullName -Destination $destination  -WhatIf
		Write-Host "... moved to '$($destination)'." -ForegroundColor Green
	} Else {
		Write-Host "... skipped, no match in pattern file."
	}
}
$filePatterns | Where-Object {-not $_.Found} | ForEach-Object {
	Write-Host "$($timeStamp) No file matching '$($_.Name)' found!" -ForegroundColor Yellow
}
Stop-Transcript

Open in new window

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
paultran00Author Commented:
Wow.  that worked!

Previously in QUESTION 3 when I was using 1 LOG file files.log and it was overwritten each time, if I had wanted to APPEND new entries into the same file, what would that code have looked like?
paultran00Author Commented:
QUESTION:
5. Let's say the user does not give me a files.txt, therefore I have to look at all the *.pdf files in the SOURCE folder and do these checks before I copy them to the TARGET folder :

EXAMPLE of *.pdf files in the SOURCE FOLDER:

18-111 member 91-A.pdf
18-111 MD 91-A.pdf

Check a SQL SERVER table tblFilesProcessed and the columns source_filename , processed (Y/N).
If the query (looks for source_filename=18-111 member 91-A.pdf,  Processed=Y) does NOT return any records, then:
5a. insert this record into the SQL SERVER table tblFilesProcessed .

18-111 member 91-A.pdf, Y
18-111 MD 91-A.pdf, Y

else insert:

18-111 member 91-A.pdf, N
18-111 MD 91-A.pdf, N


5b. If the query does NOT return any records, Copy only these files to the TARGET folder 91-A.

18-111 member 91-A.pdf
18-111 MD 91-A.pdf

5c. MOVE all files from the SOURCE folder to a subfolder ARCHIVE and add a datestamp to the end of the filename.
oBdACommented:
As to append to the log: remember the Single Most Important cmdlet?
Get-Help -ShowWindow Start-Transcript

Open in new window

will show that Start-Transcript supports an "-Append" argument.
As for your new question: I think the initial question has been exhaustively answered. Please don't keep tacking on questions. If you want a different approach in selecting the files to copy, then please accept the comment(s) that helped you, and open a new question in the respective TAs.
paultran00Author Commented:
Hi, I've created a new question to continue with SQL SERVER.

https://www.experts-exchange.com/questions/29140943/Powershell-and-SQL-Server.html#questionAdd

Thank you.
paultran00Author Commented:
Hi, what does this line below mean ,especially around the array: e={$_}}    ,   e={$false}}

$filePatterns = Get-Content -Path $fileList | Select-Object -Property @{n='Name'; e={$_}},  @{n='Found'; e={$false}}

$source = "C:\Temp\Source"
$target = "C:\Temp\Target"
$targetUnknown = "$($target)\Unknown"
$fileList = "C:\Temp\Source\files.txt"
#$logFile = "C:\Temp\Source\files.log"
$timeStamp = Get-Date -Format 'yyyyMMdd_HHmm'
$logFile = "C:\Temp\Source\files_$($timeStamp).log"

$filePatterns = Get-Content -Path $fileList | Select-Object -Property @{n='Name'; e={$_}},  @{n='Found'; e={$false}}
$dirPatterns = Get-ChildItem -Path $target -Directory

Start-Transcript -Path $logFile
Get-ChildItem -Path $source -Filter *.pdf | ForEach-Object {
	Write-Host "Processing '$($_.Name)' ..."
	$copy = $false
	ForEach ($pattern in $filePatterns) {
		If ($_.Name -like "$($pattern.Name)*") {
			$copy = $true
			$pattern.Found = $true
			Break
		}
	}
	If ($copy) {
		$target = $null
		ForEach ($pattern in $dirPatterns) {
			If ($_.BaseName -like "*$($pattern.Name)") {
				$target = $pattern
				Break
			}
		}
		If ($target) {
			$destination = "$($target.FullName)\$($_.BaseName) $($timeStamp)$($_.Extension)"
		} Else {
			$destination = $targetUnknown
		}
		Copy-Item -Path $_.FullName -Destination $destination  #-WhatIf
		Write-Host "... moved to '$($destination)'." -ForegroundColor Green
	} Else {
		Write-Host "... skipped, no match in pattern file."
	}
}
$filePatterns | Where-Object {-not $_.Found} | ForEach-Object {
	Write-Host "$($timeStamp) No file matching '$($_.Name)' found!" -ForegroundColor Yellow
}
Stop-Transcript

Open in new window

oBdACommented:
That are two hash tables:
@{n='Name'; e={$_}}
@{n='Found'; e={$false}}
These are called "calculated property", that is, a property that is created on the fly.
In this case, it uses the strings and creates objects with two properties (Name and Found) from then, so that the script can trick which patterns have been found.
Using PowerShell's Calculated Properties
https://mcpmag.com/articles/2017/01/19/using-powershell-calculated-properties.aspx
paultran00Author Commented:
Hi.  I created a new question if you could please help me (there's no SQL SERVER involved):

https://www.experts-exchange.com/questions/29141924/Powershell-copy-and-archive.html
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Powershell

From novice to tech pro — start learning today.