Solved

convert syslog into snort format

Posted on 2009-05-20
10
546 Views
Last Modified: 2013-11-18
I have an application with tons of log data in a standard comma separated format, but my SIEM doesn't support it.  I would like to export the logs to another server that automatically parses it into a snort log format then send it to the SIEM.  Anyone have any scripts that do this, or provide some useful information to help me get started?
0
Comment
Question by:clearacid
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
10 Comments
 
LVL 6

Author Comment

by:clearacid
ID: 24489750
The source systems are all different variants of linux and appliances.  My goal is to basically collect all those logs, parse them to snort format.  Does this help?  
0
 
LVL 6

Author Comment

by:clearacid
ID: 24489776
Here's an example of something I'm trying to accomplish.  We have a content filter - I would like to dump all the shadow logs from the application to a central server that reparses the information into a security information event manager.  The goal is to correlate web usage with anti-virus and other malicious acts.

Original Log

192.168.1.7,dbgroup,2009/05/28,00:32:35,GSTREAM,0,4,,http://*.VIDEO.MSN.COM/,http://edge1.catalog.video.msn.com/videoByTag.aspx?ps=10&tag=top news&vs=0&ff=8a&mk=us&ns=Fox Sports_Gallery&ind=1&&responseEncoding=rss&p=msntoolbar_Sports

Convert to:

2009/05/28,00:32:35,192.168.1.7,dbgroup,msg: http://*.VIDEO.MSN.COM/,http://edge1.catalog.video.msn.com/videoByTag.aspx?ps=10&tag=top news&vs=0&ff=8a&mk=us&ns=Fox Sports_Gallery&ind=1&&responseEncoding=rss&p=msntoolbar_Sports
0
 
LVL 6

Author Comment

by:clearacid
ID: 24500025
Thanks.
0
Use Case: Protecting a Hybrid Cloud Infrastructure

Microsoft Azure is rapidly becoming the norm in dynamic IT environments. This document describes the challenges that organizations face when protecting data in a hybrid cloud IT environment and presents a use case to demonstrate how Acronis Backup protects all data.

 
LVL 20

Expert Comment

by:Gns
ID: 24513723
So basically you'd like to parse a file with eight fields into a five field format, with some slight changes in order and content? Sounds easy enough:-)... Perl would be a good tool, or awk (or even sed)... Something like
-------
#!/usr/bin/perl

while($_=<STDIN>) {
   @line = split(',',$_);
  print $line[2],",",$line[3],",",$line[0],",",$line[1],",",$line[7],"\n"; # last might need amending...
}
-------
... put in a script and made executable ... See the code above as an example, I've not tested it (at all:-).
Say the script is "conv.pl", then
./conv.pl < /path/to/original/file > /path/to/converted/file
should do whatr you want.

Cheers
-- Glenn
0
 
LVL 20

Expert Comment

by:Gns
ID: 24513727
Oops, change the print line to:
  print $line[2],",",$line[3],",",$line[0],",",$line[1],",msg: ",$line[7],"\n"; # last might need amending...
... and see if that does it for you.

-- Glenn
0
 
LVL 6

Author Comment

by:clearacid
ID: 24514894
Thanks, I'll heck i out and report back.

Also - do you know a way to do this automatic ally - let's say I want all outgoing syslog from the server that is parsing the data to run the perl script that converts the syslog.
0
 
LVL 20

Accepted Solution

by:
Gns earned 500 total points
ID: 24516160
I just noticed the "," in "http://*.VIDEO.MSN.COM/,http://edge1.catalog.video.msn.com/videoByTag.aspx?ps=10&tag=top news&vs=0&ff=8a&mk=us&ns=Fox Sports_Gallery&ind=1&&responseEncoding=rss&p=msntoolbar_Sports" ... might make the print need be:
print $line[2],",",$line[3],",",$line[0],",",$line[1],",msg: ",$line[8],",",$line[9]"\n";

... should do what you need (assuming the lines actually look like your example.

To automate, this all depends on how it really gets logged. If it really is through the syslog facility, you could amend that entry to pipe the logging through your perl script. Then (of course) that script would need print to a file rather than to stdout.

If you just need it periodically, it'd be trivial to set up a cron job for it all.

Cheers
0

Featured Post

Migrating Your Company's PCs

To keep pace with competitors, businesses must keep employees productive, and that means providing them with the latest technology. This document provides the tips and tricks you need to help you migrate an outdated PC fleet to new desktops, laptops, and tablets.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
convert Systemjs to Webpack 3 83
NTPD Client Port Usage 12 65
simple bash script needed to unpack tar.gz 4 30
Adjust the codes 3 36
There is an easy way, in .NET, to centralize the treatment of all unexpected errors. First of all, instead of launching the application directly in a Form, you need first to write a Sub called Main, in a module. Then, set the Startup Object to th…
The purpose of this article is to fix the unknown display problem in Linux Mint operating system. After installing the OS if you see Display monitor is not recognized then we can install "MESA" utilities to fix this problem or we can install additio…
Viewers will learn how to properly install Eclipse with the necessary JDK, and will take a look at an introductory Java program. Download Eclipse installation zip file: Extract files from zip file: Download and install JDK 8: Open Eclipse and …
The viewer will learn how to clear a vector as well as how to detect empty vectors in C++.

726 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question