event log digging with Get-WinEvent

Dear experts,

I need to read event logs from 30 servers, so I wrote a piece of script to dig it with Get-WinEvent
We have some old WS2008 and SBS2008/SBS2011

There are some known complications:
1, If you’re writing a PowerShell script to handle events from Vista or Server 2008, avoid the Get-WinEvent –FilterHashtable parameter

2, you have to use PowerShell v2.0 while Exchange 2007/2010 is installed, but PowerShell v2.0 do not support export-csv -append parameter

My script looks like:

$Server = [System.Net.Dns]::GetHostEntry([string]"localhost").HostName
$yesterday = (Get-Date) - (New-TimeSpan -Day 1)
$OutputFile = "log-$server.csv"

Get-WinEvent -ComputerName $server -logname security | Where-Object {$_.TimeCreated -ge $yesterday -and $_.LevelDisplayName -ne "Information"} | export-csv $OutputFile

The script is a CPU-intensive. On my server (new xeon, 8GB RAM) it reads system LOG (30 000 events, 10MB) 60 seconds and consume 50% of CPU. Security LOG (15 000 events, 10MB) even more then 12 minutes!

Now the questions:
is my script ok?
Do you know better way to dig event logs to .csv?

Kind Regards,
Jaroslav LatalMSPAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

In most cases I find that it's better anyway from a performance perspective if the script doesn't need to make use of the -append parameter of Export-CSV, so this isn't a big issue for me.

You might consider querying the info remotely from a Win7/2008R2+ machine so that you can make full use of the -filterhashtable parameter.  Or you could adapt what you have to use -FilterXPath.
Phil BossmanSenior Client Systems AdminstratorCommented:
When using Powershell, you should Filter as far to the left as possible, meaning that if possible filter before the pipe.

When I'm working with the event logs and want to optimize it for speed and repeat ability. I start with the Event Viewer.  Build the filter with the GUI, then click on the XML Tab to extract the XML filter.  Sometime I need to further edit the XML to tweak filter to allow for dynamic entries in the filter.

Here is a sample filter i made based on what you have defined.
PS C:\> $XMLFilter = @"
>> <QueryList>
>>   <Query Id="0" Path="Security">
>>     <Select Path="Security">*[System[(Level=1  or Level=2 or Level=3 or Level=5) and TimeCreated[timediff(@SystemTime) &lt;= 86400000]]]</Select>
>>   </Query>
>> </QueryList>
>> "@

Open in new window

This will show only informational alert over the past 24 hours.

Then use the -FilterXPAth parameter from the Get-WinEvent.  

You can see on my computer it took half the time just by filtering before the pipe
PS C:\> Measure-Command -Expression { Get-WinEvent -FilterXPath $XMLFilter -LogName Security }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 13
Milliseconds      : 14
Ticks             : 130146797
TotalDays         : 0.000150632866898148
TotalHours        : 0.00361518880555556
TotalMinutes      : 0.216911328333333
TotalSeconds      : 13.0146797
TotalMilliseconds : 13014.6797

PS C:\> Measure-Command -Expression { Get-WinEvent -LogName Security | Where-Object {$_.TimeCreated -ge $yesterday -and $_.LevelDisplayName -ne "Information"} }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 24
Milliseconds      : 681
Ticks             : 246817927
TotalDays         : 0.000285668896990741
TotalHours        : 0.00685605352777778
TotalMinutes      : 0.411363211666667
TotalSeconds      : 24.6817927
TotalMilliseconds : 24681.7927

Open in new window


Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
I should have mentioned that you can also use the -FilterXML parameter.
Make Network Traffic Fast and Furious with SD-WAN

Software-defined WAN (SD-WAN) is a technology that determines the most effective way to route traffic to and from datacenter sites. Register for the webinar today to learn how your business can benefit from SD-WAN!

Qlemo"Batchelor", Developer and EE Topic AdvisorCommented:
Use jobs to allow all servers to be processed simultanously. Their output can be collected, optionally sorted, and stored in one go or server-wise or ...

And of course the filter should always be executed at the source machine, as stated above.
Jaroslav LatalMSPAuthor Commented:
Hi, thanks for suggestions, I will dive into it and let you know in few day :)

Jaroslav LatalMSPAuthor Commented:
Sorry, I have not tested it yet...

Jaroslav LatalMSPAuthor Commented:
Sorry about late reponse, I agree with:
•PBossman's comment #a40656268 (168 points)
•footech's comment #a40656246 (166 points)
•Qlemo's comment #a40662940 (166 points)

Thanks for helping me.

It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.