Link to home
Create AccountLog in
Avatar of Carlo-Giuliani
Carlo-GiulianiFlag for Canada

asked on

Parse Astaro/Ulogd file using PowerShell?

I posted a question on this already, at https://www.experts-exchange.com/questions/27775849/A-way-to-import-parse-this-Astaro-firewall-log-format-ulogd-into-an-Excel-workbook.html?anchorAnswerId=38141499#a38141499 , and got a solution using Excel VBA.  

But I am wondering if there is an easy way to parse this stuff using PowerShell, and then spit it out in CSV format.
 
See the original question for full details.
ASKER CERTIFIED SOLUTION
Avatar of Qlemo
Qlemo
Flag of Germany image

Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
Avatar of Carlo-Giuliani

ASKER

Nice.   I figured the parsing could be done with regular expressions, but didn't see an easy way to build an object that would work with Export-CSV.

It took me a little time to understand how that "While ($Matches" loop works.  Very sneaky.

Thanks.
I didn't expect you to find out yourself how that works ;-). It took me a while myself to figure out a proper RegExp method, which even is dynamic.

In fact, the dynamic part does not work that well. Should the log entry format change within the logfile, the export will not contain the new "columns". But that is a restriction of CSV and similar formats, which usually only check for a certain amount of rows to determine the format.
Most PowerShell cmdlets will not wait for more than the first line; else they could not be used in non-blocking (= streaming) mode. Sort-Object and Group-Object are cmdlets which have to consider the whole stream, and so they wait until all content has been passed, while e.g. export-csv starts as soon as possible. That speeds up processing.

For future readers:
The while loop is using the conincidence that the remainder of the log line, containing key=value pairs, is collected as $matches[3], the 3rd matching expression in the RegExp. So both the starting expression and the loop expression need to match the "tail" to be processed as that 3rd pattern.