Powershell search log file with special criteria

Dear Expert

I am using those param below to create a script. $File is a log file, I would like to scan it with some criteria:
Alert.log looks like this when opened: 2018-04-03 12:07:38,748 GMT*8*ALERT0020*5890*5000*bc4d6b78....
Im going to grab from $timehour in this case: 2018-04-03 11:07:38 line to current time, check if there is any $Pattern and $PatternSMAA or $Pattern and $PatternST or $Pattern and $PatternST or $Pattern and $PatternFFA in the line, There are several questions here that I don't know how to solve:

1. I would like the search criteria to grab only from the $timehour to the current runtime. Ignore others, now this is not file create time, this is the free text in the alert file.
2. I would like to when it found the match, it breaks the operation, like this: if ($File -eq $Pattern, $PatternSMAA) {$SMAAcheck=1, break} else{$SMAAcheck=0} then you break only for this search.
3. the script check each line if the line got $timehour range is correct and $Pattern and $PatternSMAA, $SMAAcheck=1, and break for operation $SMAAcheck, go next operation. if the line got $Pattern and $timehour range is correct and $PatternFFA, $FFAcheck =1 and break the operation $FFAcheck...

Regards

$timehour = (Get-Date).Addhours(-1).ToString("yyyy-MM-dd HH:mm:ss") 
$Pattern = "error"
$PatternSMAA = "UIFSMAAMbr"
$PatternLED = "UIFLEDMbr"
$PatternST = "UIFSTbr"
$PatternFFA = "UIFFFAMbr"
$File = "c:\ALERT.log"

if ($File -eq $Pattern, $PatternSMAA) {$SMAAcheck=1}
else{$SMAAcheck=0}
if ($File -eq $Pattern, $PatternST) {$STcheck=1}
else{$STcheck=0}
if ($File -eq $Pattern, $PatternLED) {$LEDcheck=1}
else{$LEDcheck=0}
if ($File -eq $Pattern, $PatternFFA){$FFAcheck=1}
else{$FFAcheck=0}

Open in new window

LVL 1
WeTiAsked:
Who is Participating?
 
David Johnson, CD, MVPOwnerCommented:
error will not match ERROR
With more checks
$timehour = (Get-Date).Addhours(-1).ToString('yyyy-MM-dd HH:mm:ss') 
$currenttime = (Get-Date).ToString('yyyy-MM-dd HH:mm:ss')
$PatternSMAA = 'UIFSMAAMbr'
$PatternLED = 'UIFLEDMbr'
$PatternST = 'UIFSTbr'
$PatternFFA = 'UIFFFAMbr'
$Pattern = 'ERROR'
$file = 'c:\test\ALERT.log'
foreach($line in Get-Content $file)
{
  $logtime = $line.substring(0,19)
  if ($logtime -ge $timehour) 
  {
    if ($line -match $Pattern)
    {
      if ($line -match $PatternSMAA)
      {
        Write-Output -InputObject ("$logtime match $PatternSMAA")
      }
      else 
      {
        if ($line -match $PatternLED)
        {
          Write-Output -InputObject ("$logtime match $PatternLED")
        }
        else 
        {
          if ($line -match $PatternST)
          {
            Write-Output -InputObject ("$logtime match $PatternST")
          }
          else 
          {
            if($line -match $PatternFFA)
            {
              Write-Output -InputObject ("$logtime match $PatternFFA")
            }
          }
          else{
            Write-Output -InputObject "$logtime No Match"
          }
        }
      }
    }
  }
  else 
  {
    Write-Debug -Message ('{0} Less than {1}' -f $logtime, $timehour)
  }
}
  

Open in new window

1
 
aikimarkCommented:
Please post a few lines of your file and a time value that should match the first field.
0
 
WeTiAuthor Commented:
Well it's just a java log. with datetime and UIFLEDMbr is Pattern and ERROR, for exemple then the grab should be 3rd line.

2018-04-03 14:14:53,611 GMT*8*EXCP0001*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BB75F68D568209DFCF94A0D81737E0B42*System*UIF-LED-Mbr-Work*UIFLEDMbr:01.01.01*a5023d02c28b17012997a363e86846d2*N*1
2018-04-03 14:14:53,611 GMT*8*EXCP0001*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BB75F68D568209DFCF94A0D81737E0B42*System*UIF-LED-Mbr-Work*UIFLEDMbr:01.01.01*a5023d02c28b17012997a363e86846d2*N*1
2018-04-03 14:14:53,627 GMT*8*PEGA0010*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BB75F68D568209DFCF94A0D81737E0B42*System*UIF-LED-Mbr-Work*UIFLEDMbr:01.01.01*a5023d02c28b17012997a363e86846d2*N*1 ERROR
0
Problems using Powershell and Active Directory?

Managing Active Directory does not always have to be complicated.  If you are spending more time trying instead of doing, then it's time to look at something else. For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why

 
aikimarkCommented:
thanks.
what is the time interval you need to match?
0
 
WeTiAuthor Commented:
I only need to match from 1 hour ago to current time. the text pattern of the beginning of the line, not file creation time.
0
 
aikimarkCommented:
$log=@()
$log+="2018-04-03 14:14:53,611 GMT*8*EXCP0001*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BB75F68D568209DFCF94A0D81737E0B42*System*UIF-LED-Mbr-Work*UIFLEDMbr:01.01.01*a5023d02c28b17012997a363e86846d2*N*1"
$log+="2018-04-03 14:14:53,611 GMT*8*EXCP0001*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BB75F68D568209DFCF94A0D81737E0B42*System*UIF-LED-Mbr-Work*UIFLEDMbr:01.01.01*a5023d02c28b17012997a363e86846d2*N*1"
$log+="2018-04-03 14:14:53,627 GMT*8*PEGA0010*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BB75F68D568209DFCF94A0D81737E0B42*System*UIF-LED-Mbr-Work*UIFLEDMbr:01.01.01*a5023d02c28b17012997a363e86846d2*N*1 ERROR"
#$log
$timehour = (Get-Date).Addhours(-1).ToString("yyyy-MM-dd HH:mm:ss") 
$Pattern = "error"
$PatternSMAA = "UIFSMAAMbr"
$PatternLED = "UIFLEDMbr"
$PatternST = "UIFSTbr"
$PatternFFA = "UIFFFAMbr"
$File = "c:\ALERT.log"

$log | %{
    if ($_ -match $Pattern -and $_ -match $PatternSMAA ) {$SMAAcheck=1}
    else{$SMAAcheck=0}
    
    if ($_ -match  $Pattern -and $_ -match $PatternST) {$STcheck=1}
    else{$STcheck=0}

    if ($_ -match  $Pattern -and $_ -match $PatternLED) {$LEDcheck=1}
    else{$LEDcheck=0}
    
    if ($_ -match  $Pattern -and $_ -match $PatternFFA) {$FFAcheck=1}
    else{$FFAcheck=0}
    
    write-host $SMAAcheck,$STcheck,$LEDcheck,$FFAcheck
    }

Open in new window

1
 
aikimarkCommented:
The script I posted does not do any time interval checking
0
 
aikimarkCommented:
This includes a time interval check
$log=@()
$log+="2018-04-03 14:14:53,611 GMT*8*EXCP0001*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BB75F68D568209DFCF94A0D81737E0B42*System*UIF-LED-Mbr-Work*UIFLEDMbr:01.01.01*a5023d02c28b17012997a363e86846d2*N*1"
$log+="2018-04-03 14:14:53,611 GMT*8*EXCP0001*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BB75F68D568209DFCF94A0D81737E0B42*System*UIF-LED-Mbr-Work*UIFLEDMbr:01.01.01*a5023d02c28b17012997a363e86846d2*N*1"
$log+="2018-04-03 14:14:53,627 GMT*8*PEGA0010*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BB75F68D568209DFCF94A0D81737E0B42*System*UIF-LED-Mbr-Work*UIFLEDMbr:01.01.01*a5023d02c28b17012997a363e86846d2*N*1 ERROR"
#$log
#$timehour = (Get-Date).Addhours(-1).ToString("yyyy-MM-dd HH:mm:ss") 
$timehour = (Get-Date).Addhours(-1)
$Pattern = "error"
$PatternSMAA = "UIFSMAAMbr"
$PatternLED = "UIFLEDMbr"
$PatternST = "UIFSTbr"
$PatternFFA = "UIFFFAMbr"
$File = "c:\ALERT.log"

$log | %{
    if ([datetime]($_ -split ",",2)[0] -ge $timehour -and $_ -match $Pattern) {

        $SMAAcheck = [int]($_ -match $PatternSMAA )
        
        $STcheck = [int]($_ -match $PatternST)

        $LEDcheck = [int]($_ -match $PatternLED)
        
        $FFAcheck = [int]($_ -match $PatternFFA)
        
        write-host $SMAAcheck,$STcheck,$LEDcheck,$FFAcheck
        }
    }

Open in new window

0
 
WeTiAuthor Commented:
Hi thanks alot, but now the $log variable are statics, the challenge here is, the alert.log is updating all the times, it's alive, now I need to compare the $timehour with $currenttime and if it's later then grab the criteria. Otherwise ignore, reason for this is the log files will grown bigger and lines will get more, if it already grabbed something from one hours ago, I don't want to grab it again....

$timehour = (Get-Date).Addhours(-1).ToString("yyyy-MM-dd HH:mm:ss") 
$currenttime = (Get-Date).ToString("yyyy-MM-dd HH:mm:ss")
$Pattern = "error"
$PatternSMAA = "UIFSMAAMbr"
$PatternLED = "UIFLEDMbr"
$PatternST = "UIFSTbr"
$PatternFFA = "UIFFFAMbr"
$file = "c:\ALERT.log"
foreach($line in Get-Content $file){
if ($line -match $Pattern -and $PatternSMAA -and $timehour -lt $currenttime){$SMAACheck=1}
Else {$SMAACheck = 0}
if ($line -match $Pattern -and $PatternFFA -and $timehour -lt $currenttime){$FFACheck=1}
Else {$FFACheck = 0}
if ($line -match $Pattern -and $PatternLED -and $timehour -lt $currenttime){$LEDCheck=1}
Else {$LEDCheck = 0}
if ($line -match $Pattern -and $PatternST -and $timehour -lt $currenttime){$STCheck=1}
Else {$STCheck = 0}
}
$SMAACheck
$FFACheck
$LEDCheck
$STCheck

Open in new window

0
 
aikimarkCommented:
please test my updated script
0
 
WeTiAuthor Commented:
I did, the result is nothing, this [datetime]($_ -split ",",2)[0] -ge $timehour I don't know what does it do? The log file got a text string as datetime like:  2018-04-03 14:14:53 and compare that with the currentdate time, and ignore other lines, grab the lines in the time range $timehour.
0
 
oBdACommented:
Try this; it includes the file information:
$time = (Get-Date).AddHours(-1)
$path = "\\server\logs"
$patternList = @{
	'UIFSMAAMbr' =	'SMAA'
	'UIFLEDMbr' =	'LED'
	'UIFSTbr' =		'ST'
	'UIFFFAMbr' =	'FFA'
}

$pattern = "(?<Date>\d\d\d\d-\d\d-\d\d\s\d\d:\d\d:\d\d),\d\d\d\sGMT.*\*(?<Type>$($patternList.Keys -join '|')):.*\sERROR\Z"
$result = Get-ChildItem -Path $Path -Filter *.* -Recurse -File |
	Select-String -Pattern $pattern |
	Select-Object -Property Path, @{n='Time'; e={[datetime]$_.Matches[0].Groups[1].Value}}, LineNumber, Line, @{n='Type'; e={$patternList[$_.Matches[0].Groups[2].Value]}}, @{n='Pattern'; e={$_.Matches[0].Groups[2].Value}} |
	Where-Object {$_.Time -gt $time}
$result

Open in new window

1
 
WeTiAuthor Commented:
Dear experts oBdA and aiki
Please listen me out...
What does this:

 [datetime]($_ -split ",",2)[0] -ge $timehour do?

And this: :.*\sERROR\Z" ? And this: (?<Date>\d\d\d\d-\d\d-\d\d\s\d\d:\d\d:\d\d),\d\d\d\sGMT.*\*(?
0
 
aikimarkCommented:
Did you run my script?  What was its output?

Did you incorporate my script into some other script that you haven't posted?  If so, please post the script you are testing.

Note: I produce an actual datetime value for the $timehour variable.  If you are using your value assignment statement, then you have failed to adequately transfer my code into yours.  My code commented your original statement.

[datetime]($_ -split ",",2)[0] -ge $timehour
It splits the log line at a comma character, producing two strings.  It references the first of the split results and casts the value as a datetime data type.  It then does a greater-than-or-equal-to comparison of the $timehour variable.
0
 
aikimarkCommented:
are you testing the version of my code that does the time interval check or the first one I posted?
0
 
WeTiAuthor Commented:
I did test your code Aiki, the result is nothing, no result.
0
 
WeTiAuthor Commented:
Aiki I don't know why you need:
$log+="2018-04-03 14:14:53,611 GMT*8*EXCP0001*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BB75F68D568209DFCF94A0D81737E0B42*System*UIF-LED-Mbr-Work*UIFLEDMbr:01.01.01*a5023d02c28b17012997a363e86846d2*N*1"
$log+="2018-04-03 14:14:53,611 GMT*8*EXCP0001*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BB75F68D568209DFCF94A0D81737E0B42*System*UIF-LED-Mbr-Work*UIFLEDMbr:01.01.01*a5023d02c28b17012997a363e86846d2*N*1"
$log+="2018-04-03 14:14:53,627 GMT*8*PEGA0010*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BB75F68D568209DFCF94A0D81737E0B42*System*UIF-LED-Mbr-Work*UIFLEDMbr:01.01.01*a5023d02c28b17012997a363e86846d2*N*1 ERROR"

Open in new window

That is only a content of the log, I don't need to know this. I just show you what is the log content.
0
 
oBdACommented:
"(?<Date>\d\d\d\d-\d\d-\d\d\s\d\d:\d\d:\d\d),\d\d\d\sGMT.*\*(?<Type>$($patternList.Keys -join '|')):.*\sERROR\Z"
is a regular expression. It makes sure that Select-String only matches lines that
(?<Date>\d\d\d\d-\d\d-\d\d\s\d\d:\d\d:\d\d),\d\d\d\sGMT - start with something like "2018-04-02 14:14:53,627 GMT" (\d is a single digit, \s a space)
.* - followed by whatever (. means any character, * means zero or more repetitions
\*(?<Type>$($patternList.Keys -join '|')): - followed by an asterisk, one of the list of words you're interested in, and a colon (\* is a literal asterisk, | is a regex "or")
.* - followed by whatever again,
\sERROR\Z - and ends with a space and ERROR (\z is the end of the string).
The "(?<Date>...)" and "(?<Type>...)" are named "capturing groups" that will store the respective match (the names can't actually be used with Select-String, but I like to keep them for clarity), referenced in "$_.Matches[0].Groups[x].Value".
@{n='Time'; e={[datetime]$_.Matches[0].Groups[1].Value}} will turn the first matched group (the DateTime string) into a real DateTime object, that will then be used in the next line to filter the output.
1
 
WeTiAuthor Commented:
Thanks man, now Where-Object {$_.Time -gt $time} what does it compare the $time with? The Pattern d/d/d... is the time in the log file. Does it compare with that value?
0
 
oBdACommented:
It compares the $time with the calculated "Time" property added in line 13; this property is taken from the matched DateTime group (the \d... part of the pattern) and cast to a DateTime object that allows for proper comparison.
0
 
aikimarkCommented:
Did you test my code as posted?

If you removed the initialization of the $log variable, then you did not test my code.
0
 
David Johnson, CD, MVPOwnerCommented:
You only want lines with ERRORS?
$timehour = (Get-Date).Addhours(-1).ToString("yyyy-MM-dd HH:mm:ss") 


$currenttime = (Get-Date).ToString("yyyy-MM-dd HH:mm:ss")
$Pattern = "ERROR"
$file = "c:\test\ALERT.log"
foreach($line in Get-Content $file){
  $logtime = $line.substring(0,19)
  if ($logtime -ge $timehour) {
    if ($line -match $pattern){
      $line
    }
    else {
      write-debug "No Match"
    }
  }
  else { 
        write-debug("{0} Less than {1}" -f $logtime,$timehour)
        }
  }

Open in new window


2018-04-04 13:14:53,611 GMT*8*EXCP0001*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BB75F68D568209DFCF94A0D81737E0B42*System*UIF-LED-Mbr-Work*UIFLEDMbr:01.01.01*a5023d02c28b17012997a363e86846d2*N*1
2018-04-04 13:14:53,611 GMT*8*EXCP0001*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BB75F68D568209DFCF94A0D81737E0B42*System*UIF-LED-Mbr-Work*UIFLEDMbr:01.01.01*a5023d02c28b17012997a363e86846d2*N*1
2018-04-04 1

Open in new window


PS G:\> . 'G:\Documents\WindowsPowerShell\Scripts\Untitled28.ps1' <# script is not saved yet #>
2018-04-04 13:14:53,627 GMT*8*PEGA0010*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BB75F68D568209DFCF94A0D81737E0B42*S
ystem*UIF-LED-Mbr-Work*UIFLEDMbr:01.01.01*a5023d02c28b17012997a363e86846d2*N*1 ERROR

PS G:\>
0
 
WeTiAuthor Commented:
I used David solution and its working great. Thanks. Im not sure how to assign assisted solution in here...
0
 
aikimarkCommented:
Im not sure how to assign assisted solution
Do you need help accepting multiple comments as the solution?
0
 
WeTiAuthor Commented:
Yes, If i click on Best solution, it will be that. No assisted solution.
0
 
oBdACommented:
There have been changes to the site (not so sure if for the better); check out the discussion here:
2 April, 2018 Release Notes
https://www.experts-exchange.com/discussions/220484/2-April-2018-Release-Notes.html?notificationFollowed=205945783#c2145408

Can't tell how current the following article is; as of now, it says "Last Updated: Apr 04, 2018 10:50AM PDT", while Craig Kehler's comment "The patch is live" at https://www.experts-exchange.com/discussions/220484/2-April-2018-Release-Notes.html?notificationFollowed=205945783#c2145432 is from 2018-04-11 04:26 PM:
How do I close my question?
http://support.experts-exchange.com/customer/portal/articles/2527982-how-do-i-close-my-question-
0
 
aikimarkCommented:
In the subsequent dialog, there should be a link to select more comments.  Clicking that takes you back to the question thread which shows assisted links
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.