Track log file with powershell and with criteria.

Dear expert

I opened a case before and I thought it worked, but it doesn't. I reopen this again... Some experts (oBdA etc) helped with the code below and I changed abit. The log file start with a timeline with format:  2018-04-08 22:18:07 row1, row2 new timeline 2018-04-08 23:28:07 a new line. I want to search for each line, if it got Pattern: 'UIFSMAAMbr''UIFLEDMbr''UIFSTbr''UIFFFAMbr' and got Pattern 'Error' IN THE SAME LINE, do something. The next thing is the time criteria only check the lines with timeline is less than 2018-04-08 23:28:07 ignore others, mean if timeline is 2018-04-08 22:22:02 ignore that line. Only greater should count in.

Anyone can help?
Thanks

$time = (Get-Date).Adddays(-3).addhours(-5).ToString('yyyy-MM-dd hh:mm:ss')
$path = 'D:\test\ALERT.log'
$patternlist = 'UIFSMAAMbr''UIFLEDMbr''UIFSTbr''UIFFFAMbr'

$pattern = "$time*\*(?<Type>$($patternList.Keys -join '|')):.*\sError case\Z"
$result = Get-ChildItem -Path $Path -Filter *.* -Recurse -File |
	Select-String -Pattern $pattern |
	Select-Object -Property Path, @{n='Time'; e={[datetime]$_.Matches[0].Groups[1].Value}}, LineNumber, Line, @{n='Type'; e={$patternList[$_.Matches[0].Groups[2].Value]}}, @{n='Pattern'; e={$_.Matches[0].Groups[2].Value}} |
	Where-Object {$_.Time -gt $time}
$result

Open in new window

LVL 1
WeTiAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

oBdACommented:
This should do the trick, based on the line samples from the previous question. If not, provide actual sample lines, not a description.
Do not change the DateTime object created in line 1 to a string!
$time = (Get-Date)..AddDays(-3).AddHours(-5)
$path = 'D:\test\ALERT.log'
$patternlist = 'UIFSMAAMbr', 'UIFLEDMbr', 'UIFSTbr', 'UIFFFAMbr'

$pattern = "(?<Date>\d\d\d\d-\d\d-\d\d\s\d\d:\d\d:\d\d),\d\d\d\sGMT.*\*(?<Type>$($patternList -join '|')):.*\sERROR\Z"
Get-ChildItem -Path $Path -Filter *.* -Recurse -File |
	Select-String -Pattern $pattern |
	Select-Object -Property Path, @{n='Time'; e={[datetime]$_.Matches[0].Groups[1].Value}}, LineNumber, Line, @{n='Type'; e={$patternList[$_.Matches[0].Groups[2].Value]}}, @{n='Pattern'; e={$_.Matches[0].Groups[2].Value}} |
	Where-Object {$_.Time -gt $time}

Open in new window

1
WeTiAuthor Commented:
I changed the pattern to 0*****NA*NA*NA*NA*NA*NA its not working. Now I need to have ability to change pattern to what I want tho. It says pattern is not a valid regular exp? And I still want the if statement. I just want to know how the statement is build tho.
1
WeTiAuthor Commented:
And this time I don't need to know where the file is located and which line, I just need to know if the line is there with those criteria.
0
Challenges in Government Cyber Security

Has cyber security been a challenge in your government organization? Are you looking to improve your government's network security? Learn more about how to improve your government organization's security by viewing our on-demand webinar!

WeTiAuthor Commented:
I want to do like this way:
$timelimit = (Get-Date).Adddays(-4).addhours(-5).ToString('yyyy-MM-dd hh:mm:ss')
$PatternFFA = 'UIFFFAMbr'
$PatternLED = 'UIFLEDMbr'
$PatternSMAA = 'UIFSMAAMbr'
$PatternST = 'UIFSTbr'
$logtext = '0*****NA*NA*NA*NA*NA*NA'
$currenttime = (Get-Date).toString('yyyy-MM-dd hh:mm:ss')
$file = 'c:\ALERT.log'
$read = Get-Content $file
foreach ($line in $read) {
if ($timelimit -gt $currenttime){
if ($PatternFFA + $logtext) {
$resultFFA = 1 }
if ($PatternLED + $logtext) {
$resultLED = 1 }
if ($PatternSMAA + $logtext){
$resultSMAA = 1 }
if ($PatternST + $logtext) {
$resultST = 1 }
}
}
$resultFFA
$resultLED
$resultSMAA
$resultST

Open in new window

This is not working as I want too...
0
oBdACommented:
The line format you seem to be trying to process here is utterly different from the one in the original question, and the "ERROR" is not even in there anymore.
Again: if the line format has changed since the previous question, provide actual sample lines, not a description.
1
WeTiAuthor Commented:
OK here is the sample line:
2018-04-09 09:17:32,000 GMT*8*EXCP0001*0*0*bc4d6b78f25828d4364b2b04e14be8de*NA*NA*BDA3F88F5A6581D4568D76C6BF65F8EF2*System*UIF-SMAA-Mbr-Work*UIFSMAAMbr:01.01.01*c70e7722fab3f9c82d1bac5393be9f71*N*1*BDA3F88F5A6581D4568D76C6BF65F8EF2*307*Thread-3704 (HornetQ-client-global-threads-213754902)*STANDARD*com.pega.pegarules.session.internal.mgmt.Executable**UIF-FW-MbrFW-Work.CheckPrevMemTermAndUpdate*Rule-Obj-Activity:CheckPrevMemTermAndUpdate*UIF-FW-MBRFW-WORK STOPAGENTQUEUE #20170801T091459.106 GMT Step: 1 Circum: 0*****NA*NA*NA*NA*NA*NA*doActivity Rule-Obj-Activity:StopAgentQueue;Call StopAgentQueue;RULE-OBJ-ACTIVITY UIF-FW-MBRFW-WORK CHECKPREVMEMTERMANDUPDATE #20171120T093127

As you can see the UIFSMAAMBR is there, and 0*****NA*NA*NA*NA*NA*NA* is there. Line begin with 2018-04-09 09:17:32,000 GMT* that is live and changes when the time is going foward. if the script detect this it will send a alert to us. Now the alert part is covered but not script locate text part, reason why you need this timeline is, I don't want it to search for the pattern again, this script will be run in a hours scheduling.  for exemple: time 1300 run this script, and it detect the line and warn us, then 1400 run this script again, now this will ignore the 1300 alert and only scan for after 1300 to 1400...
0
oBdACommented:
What happened to the "ERROR" that you said needs to appear in the same line?
0
WeTiAuthor Commented:
Error is falling too far behind the text line, the error log is very long, that is just apart of it.
0
oBdACommented:
Then please provide a shortened sample of the ERROR string within its context.
0
WeTiAuthor Commented:
I can't show you the whole line but error is here:
Error: SenasteMedlemskap service call failed from NonMIRCOFTerminationCheck agent for ...
0
oBdACommented:
For a proper regular expression, the search patterns should be "anchored" to avoid unwanted matches, so I need the ERROR string within its context. You didn't post any context before the string.
Which characters may appear directly before "Error"?
Which characters may appear directly after "Error"; is there always a colon?
0
WeTiAuthor Commented:
That was why I wanted to change the pattern to 0*****NA*NA*NA*NA*NA*NA* instead of error, time date pattern is always at beginning then the pattern $PatternFFA = 'UIFFFAMbr' or $PatternLED = 'UIFLEDMbr' or $PatternSMAA = 'UIFSMAAMbr' or
$PatternST = 'UIFSTbr' will show after that, then the 0*****NA*NA*NA*NA*NA*NA*, when that happens the service of UIFFFAMbr or UIFLEDMbr or UIFSMAAMbr or UIFSTbr will go down, without any notice, we need a larm system that when logs shows line: with time inside the time range, with in the same line got pattern UIFSTbr or any others and have 0*****NA*NA*NA*NA*NA*NA*  then $larm = true or something.
0
oBdACommented:
I still don't get it.
Do you still need the Error in search pattern, or has that requirement been replaced by "0*****NA*NA*NA*NA*NA*NA*"?
0
WeTiAuthor Commented:
Because I noticed that Error, appears too many time at the end, and this "0*****NA*NA*NA*NA*NA*NA*"  only appears once, so this is better pattern to grab.
1
oBdACommented:
$result will now contain a list of the patterns from typeList that were found in the log.
$time = (Get-Date).AddDays(-3).AddHours(-5)
$path = 'D:\test\ALERT.log'
$typeList = 'UIFSMAAMbr', 'UIFLEDMbr', 'UIFSTbr', 'UIFFFAMbr'
$errorString = '0*****NA*NA*NA*NA*NA*NA*'

$typeList = $typeList | ForEach-Object {[regex]::Escape($_)}
$errorString = [regex]::Escape($errorString)
$pattern = "(?<Date>\d\d\d\d-\d\d-\d\d\s\d\d:\d\d:\d\d),\d\d\d\sGMT.*\*(?<Type>$($typeList -join '|')):.*$($errorString)"
$result = Get-ChildItem -Path $Path -Filter *.* -Recurse -File |
	Select-String -Pattern $pattern |
	Select-Object -Property Path, @{n='Time'; e={[datetime]$_.Matches[0].Groups[1].Value}}, LineNumber, Line, @{n='Pattern'; e={$_.Matches[0].Groups[2].Value}} |
	Where-Object {$_.Time -gt $time} |
	Select-Object -ExpandProperty Pattern -Unique
If ($result) {
	"Something dreadful happened since $($time)!"
	$result
}

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
WeTiAuthor Commented:
oBdA thanks again, I need the search for the 'UIFSMAAMbr', 'UIFLEDMbr', 'UIFSTbr', 'UIFFFAMbr'  to be separate, because UIFSMAAMbr is a service and UIFLEDMbr is another service etc. If you find the UIFSMAAMbr with 0*****NA*NA*NA*NA*NA*NA* and inside the time frame, that means UIFSMAAMbr service is down, and if you find the UIFLEDMbr with 0*****NA*NA*NA*NA*NA*NA* and inside the time frame, that means UIFLEDMbr service is down, that is why i want a $resultUIFSMAAMbr = 1 and when it's 1 send email to alert. the send email part is done, I just want the $resultUIFSMAAMbr and $resultUIFLEDMbr in your script. Is that possible?
0
oBdACommented:
As I said: $result will contain each of these strings exactly one time if found inside the time frame.
You can basically take the list, join it, and send it:
If ($result) {
	$failedServices = "The following services failed: $($result -join ', ')"
}

Open in new window

0
WeTiAuthor Commented:
oBdA great help again. Thanks man. I feel that I have too much to learn about PS, and often when I don't really understand the code, I question it. Thank you for your patient.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Powershell

From novice to tech pro — start learning today.