Link to home
Start Free TrialLog in
Avatar of namerg
namergFlag for United States of America

asked on

How to split the name of the file according to month,day,year and its pattern?

I hope i do not get spanked.
It is something similar on this post:

https://www.experts-exchange.com/questions/28077350/Powershell-Project-4-File-Pattern-verification-50-Variable-and-Time-Stamps-100-variable.html

I have tasks that create the following files but we do manually check their existence. Would like to automate the file checking.

Task1
The following file (2013_03_27_021208-batch.xls) has YYYY_MM_DD_whatever. We can control the year, day and month but the rest can be anything and is time  stamped today's date at 2:00am.
To do: Need to check of the existence of that file from Monday through Friday

Task 2:
The are five files:YYYYMMDD_whatever_clm.txt, YYYYMMDD_whatever_proc.txt,YYYYMMDD_whatever_pym.txt,YYYYMMDD_whatever_pymt_li.txt,YYYYMMDD_whatever_cc.txt
20130327_14401_clm.txt
20130327_14401_proc.txt
20130327_14401_pymt.txt
20130327_14401_pymt_li.txt
20130325_14401_cc.txt
To Do: Need to check the existence of *_clm.txt,*_proc.txt,*_pymt.txt and *_pymt_li.txt from Monday to Friday. But, on Monday need to check the existence of the *_cc.txt

Task3
There is one file:W_UANETPX_27-MAR-2013.TXT
Constant: W_UANETPX_
Variable: DD-MONTH-YYYY
Constant: .TXT
To Do: Need to check the existence of the file every day according to the pattern and DD-MONTH-YYYY

Taks 4
There is one folder: cg322701
On folder cg322701 we have to make sure the following file exist:
  UPI_YYYYMMDD.txt exists. DD = Today's date - 1 day

Hopefully this gives me enough intelligence to automate my other tasks
Avatar of footech
footech
Flag of United States of America image
No spanking.  :)  But I will ask that you try to integrate the code bits in each case to what you have already.

Task 1:
My other scripts show how I put together a string which is a timestamp accurate down to the minute (not seconds).  You should be able to integrate that into the Where-Object check.  Since there's no way to know what the exact filename will be, you won't be able to list it out if it's missing.  In other words, you can't check if the file is missing, but you can check if a file exists which fits the pattern.
$now = Get-Date
$strDate = "{0:yyyy_MM_dd}" -f ($now)
gci $folder | Where {$_.name -match "^$strDate\w+" }

Open in new window


Task 2:
Don't know how important the date part of the filename is here.  If not at all, then a simple comparison that includes $_.name -match "(clm|proc|pymt(_li)?)\.txt" or -match "cc.txt" would be fine.  If the date part is important then something like this would be needed.
$strDate = "{0:yyyyMMdd}" -f ($now)
$strDateFri = "{0:yyyyMMdd}" -f ($now).AddDays(-3)
If (today is Monday)  #already posted to code to check whether today is Monday in previous script
{
  gci $folder | Where {$_.name -match "^$strDateFri_\w+?_cc\.txt$" }
}
Else
{
  gci $folder | Where {$_.name -match "^$strDate_\w+?_(clm|proc|pymt(_li)?)\.txt$" }
}

Open in new window


Task 3:
Here's a code bit to match the format of the date information.
$strDate = "{0:dd-MMM-yyyy}" -f ($now)
$fileName = "W_UANETPX_" + $strDate + ".txt"
gci $folder | Where {$_.name -eq $fileName}

Open in new window


Task 4:
Another code bit.
$strDate = "{0:yyyyMMdd}" -f ($now).AddDays(-1)
$fileName = "UPI_" + $strDate + ".txt"

Open in new window

Avatar of namerg
namerg
Flag of United States of America image

ASKER

Thank you, thank you Footech... I am sure, I will be able to stick the bits to the previous code you have helped me. I will let you know.
Avatar of namerg
namerg
Flag of United States of America image

ASKER

:( Hey footech I started to work on Task 3 that seems to be easy to understand but I have not been able to figure it out. I keep getting the email: The following files were not found:
W_UANETPX_.txt (See Attachment)
$strDate = "{0:dd-MMM-yyyy}" -f ($now)
$fileName = "W_UANETPX_" + $strDate + ".txt"
$folder = "\\jupiter\Shared\HPD\HPD\UA NET Files"

#$files = gci $folder | Where {$_.name -eq $fileName -and $fileName -eq $_.Name -and $_.LastWriteTime.ToString("ddMMMyyyyHHmm") -eq $timeStamp } | ForEach { $_.Name }
$files = gci $folder | Where {$_.name -eq $fileName -and $fileName -eq $_.Name}

If ( !($files) )
{
  $missing = $fileName
  Send-MailMessage -from "UANetFiles@domain.com" `
    -to "Me" `
    -subject "UANet Files missing" `
    -body "The following files were not found:`n$($missing -join "`n")" `
    -smtpServer "exchange.domain.com"
}
Else
{ "All files found" }

Open in new window

Although the file exists. Thanks a lot.
Capture.PNG
Avatar of footech
footech
Flag of United States of America image
Two things I see.
1) As you can see "W_UANETPX_.txt" is the file without the datetime string, which tells us that the $strDate variable is empty.  You need to have $now defined (like in the bit I posted for Task 1) before using it to define $strDate.  Either that or you could just use Get-Date directly when defining $strDate.  The reason I defined $now to be the value returned from Get-Date was in some cases we were needing to define multiple datetime strings, and I wanted them to all be based off the same call (like in Task 2).  Really wouldn't make a performance impact in this case, but if I can perform a single lookup instead of two, I'll do it.  And I just kept using $now in the other code bits for consistency.
2) In your Where-Object scriptblock, there's no point in having both conditions since they both do the same thing.
Avatar of namerg
namerg
Flag of United States of America image

ASKER

PERFECT. I got Task 3 Done.
Avatar of namerg
namerg
Flag of United States of America image

ASKER

Damnn... I am working on Task4. It works fine but there is another file involved in it. And always I get "The following files were not found: UPI_20130328.txt " eventhough both file  exist.
$now = Get-Date
$strDate = "{0:yyyyMMdd}" -f ($now).AddDays(-1)
$strDate2 = "{0:yyyyMMdd}" -f ($now)
$fileName = "UPI_" + $strDate + ".txt"
$fileName2 = "upiCollegue_Billing_dpa_" + $strDate2 + ".txt"
$folder = "\\jupiter\Shared\IS\OPS\US Bank\cg322701"
$files = gci $folder | Where {$_.name -eq $fileName -and $_.name -eq $fileName2}

If ( !($files) )
{
  $missing = $fileName
  Send-MailMessage -from "USBankFiles@domain.com" `
    -to "Me" `
    -subject "USBank Files missing" `
    -body "The following files were not found:`n$($missing -join "`n")" `
    -smtpServer "exchange.domain.com"
}
Else
{ "All files found" }

Open in new window

Avatar of footech
footech
Flag of United States of America image
Change your "-and" to "-or".  With -and the filename would have to meet both conditions.
Avatar of namerg
namerg
Flag of United States of America image

ASKER

I did but it says now "All files found" even though there is one file existing and not two which is the right thing. I am testing....
Avatar of footech
footech
Flag of United States of America image
In this case, you need to compare the list of files found to a list of expected files.  My post at http:#a39022076 is similar.  So, first you have to put together the list.  In that other post I was able to add the second variable to the first without mashing the strings together because the first one was an array.  See the difference when you run these commands at the console.
$a = "file1"
$b = "file2"
$a + $b     <---strings are concatenated
$a = "file1","file3"
$a + $b     <---element is added to the array of strings
So what you need to do is create a new array and add both filenames to it.
$fileNames += $fileName,$fileName2

Open in new window

Another method is shown in http:#a39005823.

(this part is optional)
You can now change your Where-Object condition to something like
{$fileNames -contains $_.Name}
With only two filenames to compare against, it's not a big difference, but imagine if you had bigger list, like in http:#a39008823

Now, with the list of names, you need to do the compare against the list of found files.  But to do this (with Compare-Object) requires that the items being compared are the same type.  Since $fileNames is list of strings, $files has to be a list of strings as well (if we're going to use it for the compare).  In your script above, $files contains FileSystemInfo objects instead of strings.  Do you see how in previous scripts I retrieved just the names (which are strings) of the files?  You'll need to do the same here.
And with that done, you can do the compare.
Avatar of namerg
namerg
Flag of United States of America image

ASKER

hmm yeahh i saw that..but damn...it is hard to see the logic or flow in powershell..
Now i get the email but listing existing files :(
$now = Get-Date
$strDate = "{0:yyyyMMdd}" -f ($now).AddDays(-1)
$strDate2 = "{0:yyyyMMdd}" -f ($now).AddDays(-1)
$fileName1 = "UPI_" + $strDate + ".txt"
$fileName2 = "upiCollegue_Billing_dpa_" + $strDate2 + ".txt"
$filenames += $filename1 + $fileName2
$folder = "\\jupiter\Shared\IS\OPS\US Bank\Archive"

$files = gci $folder | Where { !($_.PsIsContainer) -and (($fileName1 -contains $_.Name) -or ($fileName2 -contains $_.Name))} | ForEach { $_.Name }

If ( !($files) )
{
  $missing = $fileNames
}
Else
{
  $missing = Compare-Object $files $fileNames -passthru | Where { $_.SideIndicator -eq "=>" }
}
If ( $missing )
{
  Send-MailMessage -from "USBankFiles@example.com" `
    -to "Me" `
    -subject "USBank Files missing" `
    -body "The following files were not found:`n$($missing -join "`n")" `
    -smtpServer "exchange.domain.com"
}
Else
{ "All files found" }

Open in new window

Avatar of footech
footech
Flag of United States of America image
What do you mean you saw that?  If you were already at that point, let me know so I don't waste time explaining what you already know.  You're almost there...

The problem is with line 6.  See my previous post for more info on joining strings into an array.
Avatar of namerg
namerg
Flag of United States of America image

ASKER

Hmm, if you are talking about this 
$fileNames = "DICT_SDRIVE_BA.TXT","DICT_SDRIVE_CHG_COMMENT_CODES.TXT"

Open in new window

Yes,that is an array but those filenames are constant, they do not change. On the task I am working the filenames change.
Avatar of footech
footech
Flag of United States of America image
No, what I meant is right above.
$fileNames += $fileName1,$fileName2

Open in new window

Avatar of namerg
namerg
Flag of United States of America image

ASKER

PERFECT. Task 4 is done
Avatar of namerg
namerg
Flag of United States of America image

ASKER

hmm, found something odd on task 3. I get the email saying The following files were not found: W_UANETPX_01-Apr-2013.txt although in the share path there is W_UANETPX_1-APR-2013.TXT

$now = Get-Date
$strDate = "{0:dd-MMM-yyyy}" -f ($now)
$fileName = "W_UANETPX_" + $strDate + ".txt"
$folder = "\\jupiter\Shared\HPD\HPD\UA NET Files"
$files = gci $folder | Where {$_.name -eq $fileName}

If ( !($files) )
{
  $missing = $fileName
  Send-MailMessage -from "UANetFiles@domain.com" `
    -to "Me" `
    -subject "UANet Files missing" `
    -body "The following files were not found:`n$($missing -join "`n")" `
    -smtpServer "exhancge.domain.com"
}
Else
{ "All files found" }

Open in new window

Well, I guess not odd, i need to strip 01 from dd
Avatar of footech
footech
Flag of United States of America image
This link contains a table with all the specifiers for a custom date-time string.
http://technet.microsoft.com/en-us/library/ee692801.aspx

You'll just need to change $strDate so that it has the right format.
Avatar of namerg
namerg
Flag of United States of America image

ASKER

You got it.
Avatar of namerg
namerg
Flag of United States of America image

ASKER

Now, working on task 1...auch..this one seems to be painful..
Avatar of namerg
namerg
Flag of United States of America image

ASKER

Wow..Got Task 1 :)
$now = Get-Date
$strDate = "{0:yyyy_MM_dd}" -f ($now)
$folder = "\\jupiter\Shared\UPI\FC\Pathology"
$files = gci $folder | Where {$_.name -match "^$strDate\w+"}
If ( !($files.Name) )
{
  $missing = $strDate + "_WHATEVER-batch.xls"
  Send-MailMessage -from "PathologyTask" `
    -to "Me" `
    -subject "Pathology File missing" `
    -body "The following file was not found:`n$($missing -join "`n")" `
    -smtpServer "Exchange"
}
Else
{ "All files found" }

Open in new window

The following file was not found:
2013_04_01_WHATEVER-batch.xls
Avatar of namerg
namerg
Flag of United States of America image

ASKER

Got Task 2, although i do not know all the tricks of powershell, i made it work.
$now = Get-Date
$strDate = "{0:yyyyMMdd}" -f ($now)
$folder = "\\up3\ftp\Backup"
If ($now.DayOfWeek -eq "Monday")
{
    $fileName1 = "{0:yyyyMMdd}_14401_cc.txt" -f $now
    $fileName2 = "{0:yyyyMMdd}_14401_clm.txt" -f $now
    $fileName3 = "{0:yyyyMMdd}_14401_proc.txt" -f $now
    $fileName4 = "{0:yyyyMMdd}_14401_pymt.txt" -f $now
    $fileName5 = "{0:yyyyMMdd}_14401_pymt_li.txt" -f $now
    $filenames = $filename1, $fileName2, $fileName3, $fileName4, $fileName5
    $files = gci $folder | Where { !($_.PsIsContainer) -and (($fileName1 -contains $_.Name) -or ($fileName2 -contains $_.Name) -or ($fileName3 -contains $_.Name) -or ($fileName4 -contains $_.Name) -or ($fileName5 -contains $_.Name) )} | ForEach { $_.Name }
    If ( !($files) )
    {
        $missing = $fileNames
    }
    Else
    {
        $missing = Compare-Object $files $fileNames -passthru | Where { $_.SideIndicator -eq "=>" }
    }
    If ( $missing )
      {
        Send-MailMessage 
       }
    Else
    { "All files found" }
    
}
Else 
{
    $fileName2 = "{0:yyyyMMdd}_14401_clm.txt" -f $now
    $fileName3 = "{0:yyyyMMdd}_14401_proc.txt" -f $now
    $fileName4 = "{0:yyyyMMdd}_14401_pymt.txt" -f $now
    $fileName5 = "{0:yyyyMMdd}_14401_pymt_li.txt" -f $now
    $filenames = $fileName2, $fileName3, $fileName4, $fileName5
    $files = gci $folder | Where { !($_.PsIsContainer) -and (($fileName1 -contains $_.Name) -or ($fileName2 -contains $_.Name) -or ($fileName3 -contains $_.Name) -or ($fileName4 -contains $_.Name) -or ($fileName5 -contains $_.Name) )} | ForEach { $_.Name }
    If ( !($files) )
    {
        $missing = $fileNames
    }
    Else
    {
        $missing = Compare-Object $files $fileNames -passthru | Where { $_.SideIndicator -eq "=>" }
    }
    If ( $missing )
      {
        Send-MailMessage
       }
    Else
    { "All files found" }
}

Open in new window

ASKER CERTIFIED SOLUTION
Avatar of footech
footech
Flag of United States of America image
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of namerg
namerg
Flag of United States of America image

ASKER

Damn...14401 changes between 14400 but randomly...
Avatar of namerg
namerg
Flag of United States of America image

ASKER

Hmm, got the code that you posted on https://www.experts-exchange.com/questions/28080058/How-to-split-the-name-of-the-file-according-to-month-day-year-and-its-pattern.html?anchorAnswerId=39030015#a39030015 for Taks 2
$strDate = "{0:yyyyMMdd}" -f ($now)
$strDateFri = "{0:yyyyMMdd}" -f ($now).AddDays(-3)

$folder = "\\up3\ftp\Backup"
If ($now.DayOfWeek -eq "Monday")
{
    $files = gci $folder | Where {$_.name -match "^$strDateFri_\w+?_cc\.txt$" }
    If ( !($files) )   
      {
        Send-MailMessage -from "PMMCFiles" `
            -to "Me" `
            -subject "PMMC Files Missing" `
            -body "The following files were not found:`n$($missing -join "`n")" `
            -smtpServer "Exchange"
       }
     Else { "All files found" }    
}
Else 
{
    $files =   gci $folder | Where {$_.name -match "^$strDate_\w+?_(clm|proc|pymt(_li)?)\.txt$" }
    If ( !($files) )   
      {
        Send-MailMessage -from "PMMCFiles" `
            -to "Me" `
            -subject "PMMC Files Missing" `
            -body "The following files were not found:`n$($missing -join "`n")" `
            -smtpServer "Exchange"
       }
     Else { "All files found" }
}

Open in new window

But i get the following error: 
You cannot call a method on a null-valued expression.
At C:\scripts\FTP_Tasks\10_PMMC.ps1:2 char:47
+ $strDateFri = "{0:yyyyMMdd}" -f ($now).AddDays <<<< (-3)
    + CategoryInfo          : InvalidOperation: (AddDays:String) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

Open in new window

Avatar of footech
footech
Flag of United States of America image
Well, the error is simple.  Remember to define a variable (in this case $now) before using it elsewhere.

However, as soon as that's fixed you would see another issue.  $missing is not defined anywhere.  This is pretty much the same problem as Task 1 - it's pretty hard to give an exact list of which files are missing when we don't know their exact names.

Let's clarify what files should be present.
Tues-Fri - should find 4 files named
yyyyMMdd_1440x_clm.txt
yyyyMMdd_1440x_proc.txt
yyyyMMdd_1440x_pymt.txt
yyyyMMdd_1440x_pymt_li.txt
where yyyyMMdd is today's date, and x is either 0 or 1.
Monday - should have the 4 files mentioned above, plus
yyyyMMdd_1440x_cc.txt
where for this file only "yyyyMMdd" equals last Friday's date.

Is this correct?
Avatar of namerg
namerg
Flag of United States of America image

ASKER

Correct about Tues-Fri files
Correct about files for Monday bu yyyyMMdd equals to todays' date., ohh i see....
I added $now = get-date and commented Line 2 and replaced strDateFri on Line 7 for strDate.
Ran the script, I get "All files found" but i did removed a file from the share, ran the script..and still get "all files found"
Avatar of footech
footech
Flag of United States of America image
In the the last script you posted, if the search for files turns up any matches (even one), it will return "all files found". If ( !($files) ) pretty much means "If $files does not contain anything"

Given that we don't know what the exact file names will be (due to the 0, 1 variability), I'm thinking the best course might be to reverse the logic.  In other words, given a particular day we know how many files we should find, so if we find less than expected then send an email with file names that we DID find, and leave it to the intelligence of the email recipient to determine what exactly is missing.

Beyond that, you may build a check for each file such that a result is returned that says something like "a file with "_cc.txt" in the name was not found".
Avatar of namerg
namerg
Flag of United States of America image

ASKER

@footech: I am looking on this code https://www.experts-exchange.com/questions/28069211/Powershell-Project-File-Pattern-verification-and-Time-Stamps.html Task1 but I am having a hard time to understand or debug $_.LastWriteTime.ToString("MMddyyyyHHmm")
What does this guy $_ do ?

So, for the task that i am working, yes list all the files and if the count is 4 or 5 according to the "Date modified" value say something....
Avatar of footech
footech
Flag of United States of America image
$_.LastWriteTime.ToString("MMddyyyyHHmm") takes the last write time of a file, which is a DateTime type, and converts it to a string type using the format as specified in the parentheses.  In this case I specified "MMddyyyyHHmm" because we were only concerned with the accuracy down to the minute, ignoring seconds.

Try these two bits.  The first one for day equals Monday, and the second for otherwise.  Since we're ultimately trying to get a list of file names, remember to include the pipe to ForEach { $_.Name } at the end of the Get-ChildItem call.  Last, for the email body, instead of using $missing, use $files.
$files = gci $folder | Where {$_.name -match "^$strDate_\w+?_(clm|proc|pymt(_li)?|cc)\.txt$" } | ForEach { $_.Name }
If ($files.count -lt 5)
{ Send the email }

$files = gci $folder | Where {$_.name -match "^$strDate_\w+?_(clm|proc|pymt(_li)?)\.txt$" } | ForEach { $_.Name }
If ($files.count -lt 4 )
{ Send the email }

Open in new window

Avatar of namerg
namerg
Flag of United States of America image

ASKER

@footech: Sorry, I could not interpret this 
_\w+?_(clm|proc|pymt(_li)?)\.txt$

Open in new window

But I think i got it, I came up with this:
$now = Get-Date
$strDate = "{0:yyyyMMdd}" -f ($now)

$folder = "\\up3\ftp\Backup"
If ($now.DayOfWeek -eq "Monday")
{
    $files0 = gci $folder | Where-Object {$_.LastWriteTime.ToString("yyyyMMdd") -eq $strDate }
    If ($files0.count -lt 5)
      {
        $missing = $strDate + "#####_cc.txt " + $strDate + "_#####_proc.txt " + $strDate + "_#####_pymt.txt " + + $strDate + "_#####_pymt_li.txt " + $strDate + "#####_clm.txt"
        Send-MailMessage -from "PMMCFiles" `
            -to "Me" `
            -subject "PMMC Files Missing" `
             -body "PMMC files not found:`nCheck for $($missing -join "`n") at $folder" `
            -smtpServer "Exchange"
       }
     Else { "All files found" }    
}
Else 
{
    $files1 = gci $folder | Where-Object {$_.LastWriteTime.ToString("yyyyMMdd") -eq $strDate }
    If ($files1.count -lt 4 )
      {
        $missing = $strDate + "#####_clm.txt " + $strDate + "_#####_proc.txt " + $strDate + "_#####_pymt.txt " + + $strDate + "_#####_pymt_li.txt"
        Send-MailMessage -from "PMMCFiles" `
            -to "Me" `
            -subject "PMMC Files Missing" `
            -body "PMMC files not found:`nCheck for $($missing -join "`n") at $folder" `
            -smtpServer "Exchange"
       }
     Else { "All files found" }
}

Open in new window

This guy "$_" is like a pointer to either $files0 or $files1, right ?
Avatar of footech
footech
Flag of United States of America image
Here's a breakdown of the regular expression "^$strDate_\w+?_(clm|proc|pymt(_li)?|cc)\.txt$".
^ - beginning of line
$strDate - I'm sure you know
_ - underscore character
\w - shorthand for any word character, including letters, digits, and underscore
+ - means the previous is repeated 1 or more times
? - means that the + should match as few as possible (called a lazy modifier)
_ - underscore character
(clm|proc|pymt(_li)?|cc) - translates to "clm", or "proc", or "pymt", or "pymt_li", or "cc"
\. - dot character (period)
txt - exactly that
$ - end of line
"$_" is the current object moving through the pipeline.

One thing that I might be worried about is if there are other files in the folder you're searching that would have today's date it could mess up your results.  Since the email will include a list of generic file names that should be present, perhaps it would also be helpful to include a list of file names that were found.
Avatar of namerg
namerg
Flag of United States of America image

ASKER

hmmm :( See screenshot. I do not think I will have a problem. Although, you are the guru
Capture.PNG
Avatar of footech
footech
Flag of United States of America image
Doesn't look like it will be a problem.