Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

convert syslog into snort format

Posted on 2009-05-20
10
544 Views
Last Modified: 2013-11-18
I have an application with tons of log data in a standard comma separated format, but my SIEM doesn't support it.  I would like to export the logs to another server that automatically parses it into a snort log format then send it to the SIEM.  Anyone have any scripts that do this, or provide some useful information to help me get started?
0
Comment
Question by:clearacid
  • 4
  • 3
10 Comments
 
LVL 6

Author Comment

by:clearacid
ID: 24489750
The source systems are all different variants of linux and appliances.  My goal is to basically collect all those logs, parse them to snort format.  Does this help?  
0
 
LVL 6

Author Comment

by:clearacid
ID: 24489776
Here's an example of something I'm trying to accomplish.  We have a content filter - I would like to dump all the shadow logs from the application to a central server that reparses the information into a security information event manager.  The goal is to correlate web usage with anti-virus and other malicious acts.

Original Log

192.168.1.7,dbgroup,2009/05/28,00:32:35,GSTREAM,0,4,,http://*.VIDEO.MSN.COM/,http://edge1.catalog.video.msn.com/videoByTag.aspx?ps=10&tag=top news&vs=0&ff=8a&mk=us&ns=Fox Sports_Gallery&ind=1&&responseEncoding=rss&p=msntoolbar_Sports

Convert to:

2009/05/28,00:32:35,192.168.1.7,dbgroup,msg: http://*.VIDEO.MSN.COM/,http://edge1.catalog.video.msn.com/videoByTag.aspx?ps=10&tag=top news&vs=0&ff=8a&mk=us&ns=Fox Sports_Gallery&ind=1&&responseEncoding=rss&p=msntoolbar_Sports
0
 
LVL 6

Author Comment

by:clearacid
ID: 24500025
Thanks.
0
Back Up Your Microsoft Windows Server®

Back up all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

 
LVL 20

Expert Comment

by:Gns
ID: 24513723
So basically you'd like to parse a file with eight fields into a five field format, with some slight changes in order and content? Sounds easy enough:-)... Perl would be a good tool, or awk (or even sed)... Something like
-------
#!/usr/bin/perl

while($_=<STDIN>) {
   @line = split(',',$_);
  print $line[2],",",$line[3],",",$line[0],",",$line[1],",",$line[7],"\n"; # last might need amending...
}
-------
... put in a script and made executable ... See the code above as an example, I've not tested it (at all:-).
Say the script is "conv.pl", then
./conv.pl < /path/to/original/file > /path/to/converted/file
should do whatr you want.

Cheers
-- Glenn
0
 
LVL 20

Expert Comment

by:Gns
ID: 24513727
Oops, change the print line to:
  print $line[2],",",$line[3],",",$line[0],",",$line[1],",msg: ",$line[7],"\n"; # last might need amending...
... and see if that does it for you.

-- Glenn
0
 
LVL 6

Author Comment

by:clearacid
ID: 24514894
Thanks, I'll heck i out and report back.

Also - do you know a way to do this automatic ally - let's say I want all outgoing syslog from the server that is parsing the data to run the perl script that converts the syslog.
0
 
LVL 20

Accepted Solution

by:
Gns earned 500 total points
ID: 24516160
I just noticed the "," in "http://*.VIDEO.MSN.COM/,http://edge1.catalog.video.msn.com/videoByTag.aspx?ps=10&tag=top news&vs=0&ff=8a&mk=us&ns=Fox Sports_Gallery&ind=1&&responseEncoding=rss&p=msntoolbar_Sports" ... might make the print need be:
print $line[2],",",$line[3],",",$line[0],",",$line[1],",msg: ",$line[8],",",$line[9]"\n";

... should do what you need (assuming the lines actually look like your example.

To automate, this all depends on how it really gets logged. If it really is through the syslog facility, you could amend that entry to pipe the logging through your perl script. Then (of course) that script would need print to a file rather than to stdout.

If you just need it periodically, it'd be trivial to set up a cron job for it all.

Cheers
0

Featured Post

Migrating Your Company's PCs

To keep pace with competitors, businesses must keep employees productive, and that means providing them with the latest technology. This document provides the tips and tricks you need to help you migrate an outdated PC fleet to new desktops, laptops, and tablets.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Rate limit for DNS queries 7 84
Bartender Integration Builder 3 44
CLI command keep running after close 7 36
How does PHP Storm display on Linux high resolution laptops? 1 37
Whether you’re a college noob or a soon-to-be pro, these tips are sure to help you in your journey to becoming a programming ninja and stand out from the crowd.
Fine Tune your automatic Updates for Ubuntu / Debian
This video teaches viewers about errors in exception handling.
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

837 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question