Track log file with powershell and with criteria.

Dear expert

I opened a case before and I thought it worked, but it doesn't. I reopen this again... Some experts (oBdA etc) helped with the code below and I changed abit. The log file start with a timeline with format:  2018-04-08 22:18:07 row1, row2 new timeline 2018-04-08 23:28:07 a new line. I want to search for each line, if it got Pattern: 'UIFSMAAMbr''UIFLEDMbr''UIFSTbr''UIFFFAMbr' and got Pattern 'Error' IN THE SAME LINE, do something. The next thing is the time criteria only check the lines with timeline is less than 2018-04-08 23:28:07 ignore others, mean if timeline is 2018-04-08 22:22:02 ignore that line. Only greater should count in.

Anyone can help?
Thanks

$time = (Get-Date).Adddays(-3).addhours(-5).ToString('yyyy-MM-dd hh:mm:ss')
$path = 'D:\test\ALERT.log'
$patternlist = 'UIFSMAAMbr''UIFLEDMbr''UIFSTbr''UIFFFAMbr'

$pattern = "$time*\*(?<Type>$($patternList.Keys -join '|')):.*\sError case\Z"
$result = Get-ChildItem -Path $Path -Filter *.* -Recurse -File |
	Select-String -Pattern $pattern |
	Select-Object -Property Path, @{n='Time'; e={[datetime]$_.Matches[0].Groups[1].Value}}, LineNumber, Line, @{n='Type'; e={$patternList[$_.Matches[0].Groups[2].Value]}}, @{n='Pattern'; e={$_.Matches[0].Groups[2].Value}} |
	Where-Object {$_.Time -gt $time}
$result

Open in new window

LVL 1
WeTiAsked:
Who is Participating?
 
oBdACommented:
$result will now contain a list of the patterns from typeList that were found in the log.
$time = (Get-Date).AddDays(-3).AddHours(-5)
$path = 'D:\test\ALERT.log'
$typeList = 'UIFSMAAMbr', 'UIFLEDMbr', 'UIFSTbr', 'UIFFFAMbr'
$errorString = '0*****NA*NA*NA*NA*NA*NA*'

$typeList = $typeList | ForEach-Object {[regex]::Escape($_)}
$errorString = [regex]::Escape($errorString)
$pattern = "(?<Date>\d\d\d\d-\d\d-\d\d\s\d\d:\d\d:\d\d),\d\d\d\sGMT.*\*(?<Type>$($typeList -join '|')):.*$($errorString)"
$result = Get-ChildItem -Path $Path -Filter *.* -Recurse -File |
	Select-String -Pattern $pattern |
	Select-Object -Property Path, @{n='Time'; e={[datetime]$_.Matches[0].Groups[1].Value}}, LineNumber, Line, @{n='Pattern'; e={$_.Matches[0].Groups[2].Value}} |
	Where-Object {$_.Time -gt $time} |
	Select-Object -ExpandProperty Pattern -Unique
If ($result) {
	"Something dreadful happened since $($time)!"
	$result
}

Open in new window

0
 
oBdACommented:
This should do the trick, based on the line samples from the previous question. If not, provide actual sample lines, not a description.
Do not change the DateTime object created in line 1 to a string!
$time = (Get-Date)..AddDays(-3).AddHours(-5)
$path = 'D:\test\ALERT.log'
$patternlist = 'UIFSMAAMbr', 'UIFLEDMbr', 'UIFSTbr', 'UIFFFAMbr'

$pattern = "(?<Date>\d\d\d\d-\d\d-\d\d\s\d\d:\d\d:\d\d),\d\d\d\sGMT.*\*(?<Type>$($patternList -join '|')):.*\sERROR\Z"
Get-ChildItem -Path $Path -Filter *.* -Recurse -File |
	Select-String -Pattern $pattern |
	Select-Object -Property Path, @{n='Time'; e={[datetime]$_.Matches[0].Groups[1].Value}}, LineNumber, Line, @{n='Type'; e={$patternList[$_.Matches[0].Groups[2].Value]}}, @{n='Pattern'; e={$_.Matches[0].Groups[2].Value}} |
	Where-Object {$_.Time -gt $time}

Open in new window

1
 
WeTiAuthor Commented:
I changed the pattern to 0*****NA*NA*NA*NA*NA*NA its not working. Now I need to have ability to change pattern to what I want tho. It says pattern is not a valid regular exp? And I still want the if statement. I just want to know how the statement is build tho.
1
Problems using Powershell and Active Directory?

Managing Active Directory does not always have to be complicated.  If you are spending more time trying instead of doing, then it's time to look at something else. For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why

 
WeTiAuthor Commented:
And this time I don't need to know where the file is located and which line, I just need to know if the line is there with those criteria.
0
 
WeTiAuthor Commented:
I want to do like this way:
$timelimit = (Get-Date).Adddays(-4).addhours(-5).ToString('yyyy-MM-dd hh:mm:ss')
$PatternFFA = 'UIFFFAMbr'
$PatternLED = 'UIFLEDMbr'
$PatternSMAA = 'UIFSMAAMbr'
$PatternST = 'UIFSTbr'
$logtext = '0*****NA*NA*NA*NA*NA*NA'
$currenttime = (Get-Date).toString('yyyy-MM-dd hh:mm:ss')
$file = 'c:\ALERT.log'
$read = Get-Content $file
foreach ($line in $read) {
if ($timelimit -gt $currenttime){
if ($PatternFFA + $logtext) {
$resultFFA = 1 }
if ($PatternLED + $logtext) {
$resultLED = 1 }
if ($PatternSMAA + $logtext){
$resultSMAA = 1 }
if ($PatternST + $logtext) {
$resultST = 1 }
}
}
$resultFFA
$resultLED
$resultSMAA
$resultST

Open in new window

This is not working as I want too...
0
 
oBdACommented:
The line format you seem to be trying to process here is utterly different from the one in the original question, and the "ERROR" is not even in there anymore.
Again: if the line format has changed since the previous question, provide actual sample lines, not a description.
1
 
WeTiAuthor Commented:
OK here is the sample line:
2018-04-09 09:17:32,000 GMT*8*EXCP0001*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BDA3F88F5A6581D4568D76C6BF65F8EF2*System*UIF-SMAA-Mbr-Work*UIFSMAAMbr:01.01.01*c70e7722fab3f9c82d1bac5393be9f71*N*1*BDA3F88F5A6581D4568D76C6BF65F8EF2*307*Thread-3704 (HornetQ-client-global-threads-213754902)*STANDARD*com.pega.pegarules.session.internal.mgmt.Executable**UIF-FW-MbrFW-Work.CheckPrevMemTermAndUpdate*Rule-Obj-Activity:CheckPrevMemTermAndUpdate*UIF-FW-MBRFW-WORK STOPAGENTQUEUE #20170801T091459.106 GMT Step: 1 Circum: 0*****NA*NA*NA*NA*NA*NA*doActivity Rule-Obj-Activity:StopAgentQueue;Call StopAgentQueue;RULE-OBJ-ACTIVITY UIF-FW-MBRFW-WORK CHECKPREVMEMTERMANDUPDATE #20171120T093127

As you can see the UIFSMAAMBR is there, and 0*****NA*NA*NA*NA*NA*NA* is there. Line begin with 2018-04-09 09:17:32,000 GMT* that is live and changes when the time is going foward. if the script detect this it will send a alert to us. Now the alert part is covered but not script locate text part, reason why you need this timeline is, I don't want it to search for the pattern again, this script will be run in a hours scheduling.  for exemple: time 1300 run this script, and it detect the line and warn us, then 1400 run this script again, now this will ignore the 1300 alert and only scan for after 1300 to 1400...
0
 
oBdACommented:
What happened to the "ERROR" that you said needs to appear in the same line?
0
 
WeTiAuthor Commented:
Error is falling too far behind the text line, the error log is very long, that is just apart of it.
0
 
oBdACommented:
Then please provide a shortened sample of the ERROR string within its context.
0
 
WeTiAuthor Commented:
I can't show you the whole line but error is here:
Error: SenasteMedlemskap service call failed from NonMIRCOFTerminationCheck agent for ...
0
 
oBdACommented:
For a proper regular expression, the search patterns should be "anchored" to avoid unwanted matches, so I need the ERROR string within its context. You didn't post any context before the string.
Which characters may appear directly before "Error"?
Which characters may appear directly after "Error"; is there always a colon?
0
 
WeTiAuthor Commented:
That was why I wanted to change the pattern to 0*****NA*NA*NA*NA*NA*NA* instead of error, time date pattern is always at beginning then the pattern $PatternFFA = 'UIFFFAMbr' or $PatternLED = 'UIFLEDMbr' or $PatternSMAA = 'UIFSMAAMbr' or
$PatternST = 'UIFSTbr' will show after that, then the 0*****NA*NA*NA*NA*NA*NA*, when that happens the service of UIFFFAMbr or UIFLEDMbr or UIFSMAAMbr or UIFSTbr will go down, without any notice, we need a larm system that when logs shows line: with time inside the time range, with in the same line got pattern UIFSTbr or any others and have 0*****NA*NA*NA*NA*NA*NA*  then $larm = true or something.
0
 
oBdACommented:
I still don't get it.
Do you still need the Error in search pattern, or has that requirement been replaced by "0*****NA*NA*NA*NA*NA*NA*"?
0
 
WeTiAuthor Commented:
Because I noticed that Error, appears too many time at the end, and this "0*****NA*NA*NA*NA*NA*NA*"  only appears once, so this is better pattern to grab.
1
 
WeTiAuthor Commented:
oBdA thanks again, I need the search for the 'UIFSMAAMbr', 'UIFLEDMbr', 'UIFSTbr', 'UIFFFAMbr'  to be separate, because UIFSMAAMbr is a service and UIFLEDMbr is another service etc. If you find the UIFSMAAMbr with 0*****NA*NA*NA*NA*NA*NA* and inside the time frame, that means UIFSMAAMbr service is down, and if you find the UIFLEDMbr with 0*****NA*NA*NA*NA*NA*NA* and inside the time frame, that means UIFLEDMbr service is down, that is why i want a $resultUIFSMAAMbr = 1 and when it's 1 send email to alert. the send email part is done, I just want the $resultUIFSMAAMbr and $resultUIFLEDMbr in your script. Is that possible?
0
 
oBdACommented:
As I said: $result will contain each of these strings exactly one time if found inside the time frame.
You can basically take the list, join it, and send it:
If ($result) {
	$failedServices = "The following services failed: $($result -join ', ')"
}

Open in new window

0
 
WeTiAuthor Commented:
oBdA great help again. Thanks man. I feel that I have too much to learn about PS, and often when I don't really understand the code, I question it. Thank you for your patient.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.