Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
?
Solved

convert syslog into snort format

Posted on 2009-05-20
10
Medium Priority
?
558 Views
Last Modified: 2013-11-18
I have an application with tons of log data in a standard comma separated format, but my SIEM doesn't support it.  I would like to export the logs to another server that automatically parses it into a snort log format then send it to the SIEM.  Anyone have any scripts that do this, or provide some useful information to help me get started?
0
Comment
Question by:clearacid
  • 4
  • 3
7 Comments
 
LVL 6

Author Comment

by:clearacid
ID: 24489750
The source systems are all different variants of linux and appliances.  My goal is to basically collect all those logs, parse them to snort format.  Does this help?  
0
 
LVL 6

Author Comment

by:clearacid
ID: 24489776
Here's an example of something I'm trying to accomplish.  We have a content filter - I would like to dump all the shadow logs from the application to a central server that reparses the information into a security information event manager.  The goal is to correlate web usage with anti-virus and other malicious acts.

Original Log

192.168.1.7,dbgroup,2009/05/28,00:32:35,GSTREAM,0,4,,http://*.VIDEO.MSN.COM/,http://edge1.catalog.video.msn.com/videoByTag.aspx?ps=10&tag=top news&vs=0&ff=8a&mk=us&ns=Fox Sports_Gallery&ind=1&&responseEncoding=rss&p=msntoolbar_Sports

Convert to:

2009/05/28,00:32:35,192.168.1.7,dbgroup,msg: http://*.VIDEO.MSN.COM/,http://edge1.catalog.video.msn.com/videoByTag.aspx?ps=10&tag=top news&vs=0&ff=8a&mk=us&ns=Fox Sports_Gallery&ind=1&&responseEncoding=rss&p=msntoolbar_Sports
0
 
LVL 6

Author Comment

by:clearacid
ID: 24500025
Thanks.
0
Managing Security Policy in a Changing Environment

The enterprise network environment is evolving rapidly as companies extend their physical data centers to embrace cloud computing and software-defined networking. This new reality means that the challenge of managing the security policy is much more dynamic and complex.

 
LVL 20

Expert Comment

by:Gns
ID: 24513723
So basically you'd like to parse a file with eight fields into a five field format, with some slight changes in order and content? Sounds easy enough:-)... Perl would be a good tool, or awk (or even sed)... Something like
-------
#!/usr/bin/perl

while($_=<STDIN>) {
   @line = split(',',$_);
  print $line[2],",",$line[3],",",$line[0],",",$line[1],",",$line[7],"\n"; # last might need amending...
}
-------
... put in a script and made executable ... See the code above as an example, I've not tested it (at all:-).
Say the script is "conv.pl", then
./conv.pl < /path/to/original/file > /path/to/converted/file
should do whatr you want.

Cheers
-- Glenn
0
 
LVL 20

Expert Comment

by:Gns
ID: 24513727
Oops, change the print line to:
  print $line[2],",",$line[3],",",$line[0],",",$line[1],",msg: ",$line[7],"\n"; # last might need amending...
... and see if that does it for you.

-- Glenn
0
 
LVL 6

Author Comment

by:clearacid
ID: 24514894
Thanks, I'll heck i out and report back.

Also - do you know a way to do this automatic ally - let's say I want all outgoing syslog from the server that is parsing the data to run the perl script that converts the syslog.
0
 
LVL 20

Accepted Solution

by:
Gns earned 2000 total points
ID: 24516160
I just noticed the "," in "http://*.VIDEO.MSN.COM/,http://edge1.catalog.video.msn.com/videoByTag.aspx?ps=10&tag=top news&vs=0&ff=8a&mk=us&ns=Fox Sports_Gallery&ind=1&&responseEncoding=rss&p=msntoolbar_Sports" ... might make the print need be:
print $line[2],",",$line[3],",",$line[0],",",$line[1],",msg: ",$line[8],",",$line[9]"\n";

... should do what you need (assuming the lines actually look like your example.

To automate, this all depends on how it really gets logged. If it really is through the syslog facility, you could amend that entry to pipe the logging through your perl script. Then (of course) that script would need print to a file rather than to stdout.

If you just need it periodically, it'd be trivial to set up a cron job for it all.

Cheers
0

Featured Post

Who's Defending Your Organization from Threats?

Protecting against advanced threats requires an IT dream team – a well-oiled machine of people and solutions working together to defend your organization. Download our resource kit today to learn more about the tools you need to build you IT Dream Team!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Fine Tune your automatic Updates for Ubuntu / Debian
I have written articles previously comparing SARDU and YUMI.  I also included a couple of lines about Easy2boot (easy2boot.com).  I have now been using, and enjoying easy2boot as my sole multiboot utility for some years and realize that it deserves …
The goal of the tutorial is to teach the user how to use functions in C++. The video will cover how to define functions, how to call functions and how to create functions prototypes. Microsoft Visual C++ 2010 Express will be used as a text editor an…
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…
Suggested Courses

572 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question