Link to home
Start Free TrialLog in
Avatar of Rich Rumble
Rich RumbleFlag for United States of America

asked on

Read xml file, and execute command

In question Q_28621867 I asked for a script to read from a log file an execute commands that queried and output the windows event logs to separate files. That works well. I'd like to take it further, and search each event log for the <EventRecordID> tag and the value in it. Using that value I'd like to execute a command similar to this:
wevtutil qe "Microsoft-Windows-Sysmon/Operational" /q:"*[System[Provider[@Name='Microsoft-Windows-Sysmon'] and EventRecordID > xyz ]]"
The bolded items above need to be replaced by their respective log names, and the number found in the <EventRecordID> tag.
Not all logs will contain data, and some that do have data may not have that tag, those should just be skipped.
PowerShell is out, but VBScript or Batch is fine, as long as wevturil is being called by the Vbs or Bat file.
The goal again to be that instead of writing the log over itself, have it look for the highest number in the event log's (in the EventRecordID tag), and then issue appended query results to the respective log files.
1st run: wevtutil qe "Microsoft-Windows-Sysmon/Operational"
2nd run: wevtutil qe "Microsoft-Windows-Sysmon/Operational" /q:"*[System[Provider[@Name='Microsoft-Windows-Sysmon'] and EventRecordID > 123456 ]]"
Again where 123456 was the highest number found in eventrecordID tag in the sysmon event log from the first run. All runs after CountGood >=1 should use the "second run" query.
-rich
Avatar of Bill Prew
Bill Prew

Made some headway on this tonight, more to come...

~bp
Avatar of Rich Rumble

ASKER

In the interest of saving space, and possibly the search times, each query should probably get it's own log file, and then you'd only have to look in the latest file for the highest ID.
Sysmon-1.xml (200Mb)
Sysmon-2.xml (10Mb)
Sysmon-3.xml (20Mb)
It may make sense to log the name of the latest query that completed...
Microsoft-Windows-Sysmon/Operational, 3, 3, Microsoft-Windows-Sysmon-3.xml
Where ...Sysmon-3.xml is the file that completed last. Then the next time the query is run, it will look in sysmon-3.xml for the highest eventnumberID, since it's guaranteed to have the highest.
-rich
ASKER CERTIFIED SOLUTION
Avatar of Bill Prew
Bill Prew

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
THIS.IS.AWESOME. I'm going to open yet another question, I'd like to send these log to a central location, and the initial upload is worrisome to the network and storage teams. I'd really like to compress the data before sending, and data that doesn't get sent have it queued to send. If you're interested, I'll post "part 3" in just a bit!
Thanks again, this is a fine piece of work!
-rich
Super work, and it's VBscript so it makes it a bit easier to get through our red-tape at work!
Glad that was helpful.  Think through the next piece and work up your ideas.  A few random thoughts that come to mind:

What frequency will this run on each PC?
What compression technology would work, ZIP, CAP, ...?
If you compress then you will probably need a separate file for each day or run of the extract.
What would prevent files from being sent to the server and require a queue manager?
What will these files ultimately be used for?  If they are just in separate files / ZIP files, it will be very hard to search them for specific items down the road.
~bp
Correct on all counts. They will likely run hourly, compression I'm messing with, CAB is better for these logs than Zip (compressed folder) http://blogs.technet.com/b/heyscriptingguy/archive/2006/07/06/how-can-i-create-a-compressed-folder-and-add-files-to-it.aspx but 7zip is best overall. 7zip isn't part of the normal build, however I have in the past converted binaries to base64 in Vbscript and then output them when needed. So it's possible to deliver an exe in a vbs that way. Your also correct about queuing, in the past we did a simple "did it send completely? if yes, delete the file that is over 7 days old". The beauty is "searchability" isn't even being asked for, this is more of a forensic item rather than what you'd expect with a SIEM or other central logging solution. In any case that's what zgrep is for :) We aren't going to top notch, more of a patchwork solution until a more proper solution can be stood up.
I'm going to experiment with a few things and I should have a new Q for you in 8-12 hours.
I'll cross post the link here. Thanks again!
-rich
Nice Work!