asked on Parse Astaro/Ulogd file using PowerShell?
I posted a question on this already, at https://www.experts-exchange.com/questions/27775849/A-way-to-import-parse-this-Astaro-firewall-log-format-ulogd-into-an-Excel-workbook.html?anchorAnswerId=38141499#a38141499 , and got a solution using Excel VBA.
But I am wondering if there is an easy way to parse this stuff using PowerShell, and then spit it out in CSV format.
See the original question for full details.
But I am wondering if there is an easy way to parse this stuff using PowerShell, and then spit it out in CSV format.
See the original question for full details.
PowershellSoftware Firewalls
Last Comment
Qlemo
ASKER CERTIFIED SOLUTION
THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
I didn't expect you to find out yourself how that works ;-). It took me a while myself to figure out a proper RegExp method, which even is dynamic.
In fact, the dynamic part does not work that well. Should the log entry format change within the logfile, the export will not contain the new "columns". But that is a restriction of CSV and similar formats, which usually only check for a certain amount of rows to determine the format.
Most PowerShell cmdlets will not wait for more than the first line; else they could not be used in non-blocking (= streaming) mode. Sort-Object and Group-Object are cmdlets which have to consider the whole stream, and so they wait until all content has been passed, while e.g. export-csv starts as soon as possible. That speeds up processing.
For future readers:
The while loop is using the conincidence that the remainder of the log line, containing key=value pairs, is collected as $matches[3], the 3rd matching expression in the RegExp. So both the starting expression and the loop expression need to match the "tail" to be processed as that 3rd pattern.
In fact, the dynamic part does not work that well. Should the log entry format change within the logfile, the export will not contain the new "columns". But that is a restriction of CSV and similar formats, which usually only check for a certain amount of rows to determine the format.
Most PowerShell cmdlets will not wait for more than the first line; else they could not be used in non-blocking (= streaming) mode. Sort-Object and Group-Object are cmdlets which have to consider the whole stream, and so they wait until all content has been passed, while e.g. export-csv starts as soon as possible. That speeds up processing.
For future readers:
The while loop is using the conincidence that the remainder of the log line, containing key=value pairs, is collected as $matches[3], the 3rd matching expression in the RegExp. So both the starting expression and the loop expression need to match the "tail" to be processed as that 3rd pattern.
It took me a little time to understand how that "While ($Matches" loop works. Very sneaky.
Thanks.