Use powershell to read a text file containing filenames to move just those files from folder A to folder B.

paultran00
paultran00 used Ask the Experts™
on
powershell and Windows 7.

Hi,

I have a text file request.txt containing filenames such as:
18-111
18-222

Actual filenames:
18-111 member 91-A.pdf
18-111 MD 91-A.pdf
18-222 member 91-B.pdf
18-222 MD 91-B.pdf
18-333 member 91-C.pdf
18-333 MD 91-C.pdf

QUESTION: how do I use powershell to
1) loop through request.txt and do a partial match to identify certain .pdf files to move from folder A to a specific folder B that has the same name as the last 4 characters of the filename?  

RESULT:
18-111 member 91-A.pdf is moved to folder 91-A.
18-111 MD 91-A.pdf is moved to folder 91-A.
18-222 member 91-B.pdf is moved to folder 91-B.
18-222 MD 91-B.pdf is moved to folder 91-B.

2) Add today's timestamp to the end of the filename in the format YYYMMDD_HHMM in 24hour format.
RESULT: If today is 3/26/2019 1:21pm.
18-111 member 91-A 20190326_1321.pdf
18-111 MD 91-A 20190326_1321.pdf
18-222 member 91-B 20190326_1321.pdf
18-222 MD 91-B 20190326_1321.pdf
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
View Solution Only
oBdA
Most Valuable Expert 2018
Distinguished Expert 2018

Commented:
This should work; it's in test mode and will only display what it would move where. Remove the -WhatIf at the end of line 24 to run it for real.
$source = "D:\Temp\Source"
$target = "D:\Temp\Target"
$filePatterns = Get-Content D:\temp\Source\files.txt
$dirPatterns = Get-ChildItem -Path $target -Directory
$timeStamp = Get-Date -Format 'yyyyMMdd_HHmm'
Get-ChildItem -Path $source -Filter *.pdf | ForEach-Object {
	Write-Host "Processing '$($_.Name)' ..."
	$copy = $false
	ForEach ($pattern in $filePatterns) {
		If ($_.Name -like "$($pattern)*") {
			$copy = $true
			Break
		}
	}
	If ($copy) {
		$target = $null
		ForEach ($pattern in $dirPatterns) {
			If ($_.BaseName -like "*$($pattern.Name)") {
				$target = $pattern
				Break
			}
		}
		If ($target) {
			Move-Item -Path $_.FullName -Destination "$($target.FullName)\$($_.BaseName) $($timeStamp)$($_.Extension)" -WhatIf
			Write-Host "... moved to '$($target.FullName)'!" -ForegroundColor Green
		} Else {
			Write-Host "... skipped, no matching target folder found!" -ForegroundColor Yellow
		}
	} Else {
		Write-Host "... skipped, no match in pattern file."
	}
}

Open in new window

paultran00

Author

Commented:
Wow you're fast.  I tried to run it in powershell but it's not working.

I created 2 folders in C;\temp\Source  and C;\temp\Target

I created C;\temp\source\files.txt and its contents look like:
18-036154960
18-036181365

Folder C;\temp\source has 18 .pdf files and 4 files match:
18-036154960 MD 91-1104421C.pdf
18-036154960 Member 91-1104421C.pdf
18-036181365 MD 91-1104421C.pdf
18-036181365 Member 91-1104421C.pdf


When I run it, the error message is:

PS H:\> N:\BATCH\DenialLetters\Get_CAS_files.ps1
Processing '18-036154960 MD 91-1104421C.pdf' ...
... skipped, no matching target folder found!
Processing '18-036154960 Member 91-1104421C.pdf' ...
... skipped, no matching target folder found!
Processing '18-036181365 MD 91-1104421C.pdf' ...
... skipped, no matching target folder found!
Processing '18-036181365 Member 91-1104421C.pdf' ...
... skipped, no matching target folder found!
Processing '18-036279556 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036279556 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036330631 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036330631 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036342089 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036342089 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036523443 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036523443 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547207 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547207 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547218 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547218 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036554129 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036554129 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.

PS H:\>
oBdA
Most Valuable Expert 2018
Distinguished Expert 2018

Commented:
Then there's no folder "MD 91-1104421C" or "91-1104421C" or similar in C:\temp\target ...
Learn SQL Server Core 2016

This course will introduce you to SQL Server Core 2016, as well as teach you about SSMS, data tools, installation, server configuration, using Management Studio, and writing and executing queries.

Learn More!
paultran00

Author

Commented:
In folder target, I created SUBfolder "91-1104421C" and it successfully moved 4 files there.

QUESTION:
1. where in the code holds the value "91-1104421C" for it to look for a folder of that name ?
2.  when I changed the command from Move-Item to Copy-Item, it doesn't work.  Are the syntax different?
3. how do I see the syntax for move-item and copy-item?

thank you.
oBdA
Most Valuable Expert 2018
Distinguished Expert 2018

Commented:
1. Nowhere. It compares the end of the file name with the folder names in the target folder.
2. Works just fine with Copy-Item without any other changes. If you just moved the files back into the source folder, don't forget that they now have the date attached at the end, and won't find a matching target folder anymore.
3. Possibly the most important cmdlet: Get-Help
Get-Help Move-Item -ShowWindow
Get-Help Copy-Item -ShowWindow

Open in new window

paultran00

Author

Commented:
1. How does it determine what length of the end of the source filename to parse to figure out that "91-1104421C"  is the string that it will use to find a subfolder with that name?
oBdA
Most Valuable Expert 2018
Distinguished Expert 2018

Commented:
In line 4, it reads all the folder items in the target:
$dirPatterns = Get-ChildItem -Path $target -Directory

In lines 17-22, it compares the name of the current file with each folder item's name, using a wildcard, not a static length:
If ($_.BaseName -like "*$($pattern.Name)")
paultran00

Author

Commented:
3. What if files.txt contained fileMickey and fileDonald but these files do not exist in the source folder; how would the code look to write to a log file that says: today's date and the message Cannot find test_file.

files.txt looks like this:
18-036154960
18-036181365
fileMickey
fileDonald

log file looks like:
20190326_1641 Cannot find fileMickey
20190326_1641 Cannot find fileDonald
--------------------------------

4. What if the target folder does NOT exist?  What would the code look like to move the files to a subfolder named UNKNOWN

Thank you.
oBdA
Most Valuable Expert 2018
Distinguished Expert 2018

Commented: 
$source = "D:\Temp\Source"
$target = "D:\Temp\Target"
$targetUnknown = "$($target)\Unknown"
$fileList = "D:\Temp\Source\files.txt"
$logFile = "D:\Temp\Source\files.log"

$filePatterns = Get-Content -Path $fileList | Select-Object -Property @{n='Name'; e={$_}},  @{n='Found'; e={$false}}
$dirPatterns = Get-ChildItem -Path $target -Directory
$timeStamp = Get-Date -Format 'yyyyMMdd_HHmm'

Start-Transcript -Path $logFile
Get-ChildItem -Path $source -Filter *.pdf | ForEach-Object {
	Write-Host "Processing '$($_.Name)' ..."
	$copy = $false
	ForEach ($pattern in $filePatterns) {
		If ($_.Name -like "$($pattern.Name)*") {
			$copy = $true
			$pattern.Found = $true
			Break
		}
	}
	If ($copy) {
		$target = $null
		ForEach ($pattern in $dirPatterns) {
			If ($_.BaseName -like "*$($pattern.Name)") {
				$target = $pattern
				Break
			}
		}
		If ($target) {
			$destination = "$($target.FullName)\$($_.BaseName) $($timeStamp)$($_.Extension)"
		} Else {
			$destination = $targetUnknown
		}
		Move-Item -Path $_.FullName -Destination $destination  -WhatIf
		Write-Host "... moved to '$($destination)'." -ForegroundColor Green
	} Else {
		Write-Host "... skipped, no match in pattern file."
	}
}
$filePatterns | Where-Object {-not $_.Found} | ForEach-Object {
	Write-Host "$($timeStamp) No file matching '$($_.Name)' found!" -ForegroundColor Yellow
}
Stop-Transcript

Open in new window

paultran00

Author

Commented:
Thank you.

ERROR message:
Start-Transcript : This host does not support transcription.
At C:\Temp\Get_CAS_files_log.ps1:11 char:1
+ Start-Transcript -Path $logFile
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotImplemented: (:) [Start-Transcript], PSNotSupportedException
    + FullyQualifiedErrorId : NotSupported,Microsoft.PowerShell.Commands.StartTranscriptCommand



I am using Windows 7, PS version 4:

PS C:\Windows\System32\WindowsPowerShell\v1.0> $PSVersionTable.PSVersion

Major  Minor  Build  Revision
-----  -----  -----  --------
4      0      -1     -1
oBdA
Most Valuable Expert 2018
Distinguished Expert 2018

Commented:
Start it in a normal PS console, not in the ISE.
paultran00

Author

Commented:
I ran it from the command prompt and ran without errors.  But it did not copy the files.

powershell.exe c:\temp\Get_CAS_files_log.ps1

The LOG file files.log did get created and populated with entries (I replaced private information in the log below with xxx):

**********************
Windows PowerShell transcript start
Start time: 20190327095303
Username  : xxx
Machine        :  xxx  (Microsoft Windows NT 6.1.7601 Service Pack 1)
**********************
Transcript started, output file is C:\Temp\Source\files.log
Processing '18-036154960 MD 91-1104421C.pdf' ...
What if: Performing the operation "Copy File" on target "Item: C:\Temp\Source\18-036154960 MD 91-1104421C.pdf Destination: C:\Temp\Target\91-1104421C\18-036154960 MD 91-1104421C 20190327_0953.pdf".
... moved to 'C:\Temp\Target\91-1104421C\18-036154960 MD 91-1104421C 20190327_0953.pdf'.
Processing '18-036154960 Member 91-1104421C.pdf' ...
What if: Performing the operation "Copy File" on target "Item: C:\Temp\Source\18-036154960 Member 91-1104421C.pdf Destination: C:\Temp\Target\91-1104421C\18-036154960 Member 91-1104421C 20190327_0953.pdf".
... moved to 'C:\Temp\Target\91-1104421C\18-036154960 Member 91-1104421C 20190327_0953.pdf'.
Processing '18-036181365 MD 91-1104421C.pdf' ...
What if: Performing the operation "Copy File" on target "Item: C:\Temp\Source\18-036181365 MD 91-1104421C.pdf Destination: C:\Temp\Target\91-1104421C\18-036181365 MD 91-1104421C 20190327_0953.pdf".
... moved to 'C:\Temp\Target\91-1104421C\18-036181365 MD 91-1104421C 20190327_0953.pdf'.
Processing '18-036181365 Member 91-1104421C.pdf' ...
What if: Performing the operation "Copy File" on target "Item: C:\Temp\Source\18-036181365 Member 91-1104421C.pdf Destination: C:\Temp\Target\91-1104421C\18-036181365 Member 91-1104421C 20190327_0953.pdf".
... moved to 'C:\Temp\Target\91-1104421C\18-036181365 Member 91-1104421C 20190327_0953.pdf'.
Processing '18-036279556 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036279556 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036330631 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036330631 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036342089 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036342089 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036523443 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036523443 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547207 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547207 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547218 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036547218 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036554129 MD 91-1104421C.pdf' ...
... skipped, no match in pattern file.
Processing '18-036554129 Member 91-1104421C.pdf' ...
... skipped, no match in pattern file.
20190327_0953 No file matching 'test_file' found!
**********************
Windows PowerShell transcript end
End time: 20190327095303
**********************
oBdA
Most Valuable Expert 2018
Distinguished Expert 2018

Commented:
As before in test mode (-WhatIf in line 35)
That's why you see the
What if: Performing the operation "Copy File" ...
in the log and in the console; it means that the command isn't really executed, it only pretends to run,
paultran00

Author

Commented:
That worked after I removed the WHAT IF.

QUESTION:
4. The existing LOG file files.log appears to be overwritten each time.

What would the code look like if we wrote to a new LOG file that is created (with a datestamp at the end of the filename)?
Most Valuable Expert 2018
Distinguished Expert 2018
Commented:
You move the $timeStamp definition before the $logFile definition and add the time stamp to the $logFile definition.
$source = "D:\Temp\Source"
$target = "D:\Temp\Target"
$targetUnknown = "$($target)\Unknown"
$fileList = "D:\Temp\Source\files.txt"
$timeStamp = Get-Date -Format 'yyyyMMdd_HHmm'
$logFile = "D:\Temp\Source\files_$($timeStamp).log"

$filePatterns = Get-Content -Path $fileList | Select-Object -Property @{n='Name'; e={$_}},  @{n='Found'; e={$false}}
$dirPatterns = Get-ChildItem -Path $target -Directory

Start-Transcript -Path $logFile
Get-ChildItem -Path $source -Filter *.pdf | ForEach-Object {
	Write-Host "Processing '$($_.Name)' ..."
	$copy = $false
	ForEach ($pattern in $filePatterns) {
		If ($_.Name -like "$($pattern.Name)*") {
			$copy = $true
			$pattern.Found = $true
			Break
		}
	}
	If ($copy) {
		$target = $null
		ForEach ($pattern in $dirPatterns) {
			If ($_.BaseName -like "*$($pattern.Name)") {
				$target = $pattern
				Break
			}
		}
		If ($target) {
			$destination = "$($target.FullName)\$($_.BaseName) $($timeStamp)$($_.Extension)"
		} Else {
			$destination = $targetUnknown
		}
		Move-Item -Path $_.FullName -Destination $destination  -WhatIf
		Write-Host "... moved to '$($destination)'." -ForegroundColor Green
	} Else {
		Write-Host "... skipped, no match in pattern file."
	}
}
$filePatterns | Where-Object {-not $_.Found} | ForEach-Object {
	Write-Host "$($timeStamp) No file matching '$($_.Name)' found!" -ForegroundColor Yellow
}
Stop-Transcript

Open in new window

paultran00

Author

Commented:
Wow.  that worked!

Previously in QUESTION 3 when I was using 1 LOG file files.log and it was overwritten each time, if I had wanted to APPEND new entries into the same file, what would that code have looked like?
paultran00

Author

Commented:
QUESTION:
5. Let's say the user does not give me a files.txt, therefore I have to look at all the *.pdf files in the SOURCE folder and do these checks before I copy them to the TARGET folder :

EXAMPLE of *.pdf files in the SOURCE FOLDER:

18-111 member 91-A.pdf
18-111 MD 91-A.pdf

Check a SQL SERVER table tblFilesProcessed and the columns source_filename , processed (Y/N).
If the query (looks for source_filename=18-111 member 91-A.pdf,  Processed=Y) does NOT return any records, then:
5a. insert this record into the SQL SERVER table tblFilesProcessed .

18-111 member 91-A.pdf, Y
18-111 MD 91-A.pdf, Y

else insert:

18-111 member 91-A.pdf, N
18-111 MD 91-A.pdf, N


5b. If the query does NOT return any records, Copy only these files to the TARGET folder 91-A.

18-111 member 91-A.pdf
18-111 MD 91-A.pdf

5c. MOVE all files from the SOURCE folder to a subfolder ARCHIVE and add a datestamp to the end of the filename.
oBdA
Most Valuable Expert 2018
Distinguished Expert 2018

Commented:
As to append to the log: remember the Single Most Important cmdlet?
Get-Help -ShowWindow Start-Transcript

Open in new window

will show that Start-Transcript supports an "-Append" argument.
As for your new question: I think the initial question has been exhaustively answered. Please don't keep tacking on questions. If you want a different approach in selecting the files to copy, then please accept the comment(s) that helped you, and open a new question in the respective TAs.
paultran00

Author

Commented:
Hi, I've created a new question to continue with SQL SERVER.

https://www.experts-exchange.com/questions/29140943/Powershell-and-SQL-Server.html#questionAdd

Thank you.
paultran00

Author

Commented:
Hi, what does this line below mean ,especially around the array: e={$_}}    ,   e={$false}}

$filePatterns = Get-Content -Path $fileList | Select-Object -Property @{n='Name'; e={$_}},  @{n='Found'; e={$false}}

$source = "C:\Temp\Source"
$target = "C:\Temp\Target"
$targetUnknown = "$($target)\Unknown"
$fileList = "C:\Temp\Source\files.txt"
#$logFile = "C:\Temp\Source\files.log"
$timeStamp = Get-Date -Format 'yyyyMMdd_HHmm'
$logFile = "C:\Temp\Source\files_$($timeStamp).log"

$filePatterns = Get-Content -Path $fileList | Select-Object -Property @{n='Name'; e={$_}},  @{n='Found'; e={$false}}
$dirPatterns = Get-ChildItem -Path $target -Directory

Start-Transcript -Path $logFile
Get-ChildItem -Path $source -Filter *.pdf | ForEach-Object {
	Write-Host "Processing '$($_.Name)' ..."
	$copy = $false
	ForEach ($pattern in $filePatterns) {
		If ($_.Name -like "$($pattern.Name)*") {
			$copy = $true
			$pattern.Found = $true
			Break
		}
	}
	If ($copy) {
		$target = $null
		ForEach ($pattern in $dirPatterns) {
			If ($_.BaseName -like "*$($pattern.Name)") {
				$target = $pattern
				Break
			}
		}
		If ($target) {
			$destination = "$($target.FullName)\$($_.BaseName) $($timeStamp)$($_.Extension)"
		} Else {
			$destination = $targetUnknown
		}
		Copy-Item -Path $_.FullName -Destination $destination  #-WhatIf
		Write-Host "... moved to '$($destination)'." -ForegroundColor Green
	} Else {
		Write-Host "... skipped, no match in pattern file."
	}
}
$filePatterns | Where-Object {-not $_.Found} | ForEach-Object {
	Write-Host "$($timeStamp) No file matching '$($_.Name)' found!" -ForegroundColor Yellow
}
Stop-Transcript

Open in new window

oBdA
Most Valuable Expert 2018
Distinguished Expert 2018

Commented:
That are two hash tables:
@{n='Name'; e={$_}}
@{n='Found'; e={$false}}
These are called "calculated property", that is, a property that is created on the fly.
In this case, it uses the strings and creates objects with two properties (Name and Found) from then, so that the script can trick which patterns have been found.
Using PowerShell's Calculated Properties
https://mcpmag.com/articles/2017/01/19/using-powershell-calculated-properties.aspx
paultran00

Author

Commented:
Hi.  I created a new question if you could please help me (there's no SQL SERVER involved):

https://www.experts-exchange.com/questions/29141924/Powershell-copy-and-archive.html

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial