Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

convert syslog into snort format

Posted on 2009-05-20
10
Medium Priority
?
556 Views
Last Modified: 2013-11-18
I have an application with tons of log data in a standard comma separated format, but my SIEM doesn't support it.  I would like to export the logs to another server that automatically parses it into a snort log format then send it to the SIEM.  Anyone have any scripts that do this, or provide some useful information to help me get started?
0
Comment
Question by:clearacid
  • 4
  • 3
10 Comments
 
LVL 6

Author Comment

by:clearacid
ID: 24489750
The source systems are all different variants of linux and appliances.  My goal is to basically collect all those logs, parse them to snort format.  Does this help?  
0
 
LVL 6

Author Comment

by:clearacid
ID: 24489776
Here's an example of something I'm trying to accomplish.  We have a content filter - I would like to dump all the shadow logs from the application to a central server that reparses the information into a security information event manager.  The goal is to correlate web usage with anti-virus and other malicious acts.

Original Log

192.168.1.7,dbgroup,2009/05/28,00:32:35,GSTREAM,0,4,,http://*.VIDEO.MSN.COM/,http://edge1.catalog.video.msn.com/videoByTag.aspx?ps=10&tag=top news&vs=0&ff=8a&mk=us&ns=Fox Sports_Gallery&ind=1&&responseEncoding=rss&p=msntoolbar_Sports

Convert to:

2009/05/28,00:32:35,192.168.1.7,dbgroup,msg: http://*.VIDEO.MSN.COM/,http://edge1.catalog.video.msn.com/videoByTag.aspx?ps=10&tag=top news&vs=0&ff=8a&mk=us&ns=Fox Sports_Gallery&ind=1&&responseEncoding=rss&p=msntoolbar_Sports
0
 
LVL 6

Author Comment

by:clearacid
ID: 24500025
Thanks.
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
LVL 20

Expert Comment

by:Gns
ID: 24513723
So basically you'd like to parse a file with eight fields into a five field format, with some slight changes in order and content? Sounds easy enough:-)... Perl would be a good tool, or awk (or even sed)... Something like
-------
#!/usr/bin/perl

while($_=<STDIN>) {
   @line = split(',',$_);
  print $line[2],",",$line[3],",",$line[0],",",$line[1],",",$line[7],"\n"; # last might need amending...
}
-------
... put in a script and made executable ... See the code above as an example, I've not tested it (at all:-).
Say the script is "conv.pl", then
./conv.pl < /path/to/original/file > /path/to/converted/file
should do whatr you want.

Cheers
-- Glenn
0
 
LVL 20

Expert Comment

by:Gns
ID: 24513727
Oops, change the print line to:
  print $line[2],",",$line[3],",",$line[0],",",$line[1],",msg: ",$line[7],"\n"; # last might need amending...
... and see if that does it for you.

-- Glenn
0
 
LVL 6

Author Comment

by:clearacid
ID: 24514894
Thanks, I'll heck i out and report back.

Also - do you know a way to do this automatic ally - let's say I want all outgoing syslog from the server that is parsing the data to run the perl script that converts the syslog.
0
 
LVL 20

Accepted Solution

by:
Gns earned 2000 total points
ID: 24516160
I just noticed the "," in "http://*.VIDEO.MSN.COM/,http://edge1.catalog.video.msn.com/videoByTag.aspx?ps=10&tag=top news&vs=0&ff=8a&mk=us&ns=Fox Sports_Gallery&ind=1&&responseEncoding=rss&p=msntoolbar_Sports" ... might make the print need be:
print $line[2],",",$line[3],",",$line[0],",",$line[1],",msg: ",$line[8],",",$line[9]"\n";

... should do what you need (assuming the lines actually look like your example.

To automate, this all depends on how it really gets logged. If it really is through the syslog facility, you could amend that entry to pipe the logging through your perl script. Then (of course) that script would need print to a file rather than to stdout.

If you just need it periodically, it'd be trivial to set up a cron job for it all.

Cheers
0

Featured Post

 [eBook] Windows Nano Server

Download this FREE eBook and learn all you need to get started with Windows Nano Server, including deployment options, remote management
and troubleshooting tips and tricks

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Whether you’re a college noob or a soon-to-be pro, these tips are sure to help you in your journey to becoming a programming ninja and stand out from the crowd.
Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
This theoretical tutorial explains exceptions, reasons for exceptions, different categories of exception and exception hierarchy.
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
Suggested Courses
Course of the Month13 days, 2 hours left to enroll

972 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question