Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
?
Solved

How to modify Windows Servers events from multiline to a single line format

Posted on 2015-01-12
6
Medium Priority
?
81 Views
Last Modified: 2015-02-12
Hello,

We have about 900 Windows servers which are being indexed by our single logging system(splunk enterprise server). We are then forwarding these server logs in a standard syslog format to a 3rd party system. The 3rd party system perceives the logs in a multiline format. We need to convert them to single line because they do not support multiline.

Here is an event example from the 3rd party system:

    Dec 29 07:47:18 172.25.32.44 12/29/2014 02:47:17 AM
    Dec 29 07:47:18 172.25.32.44 LogName=Security
    Dec 29 07:47:18 172.25.32.44 SourceName=Microsoft Windows security auditing.
    Dec 29 07:47:18 172.25.32.44 EventCode=4689
    Dec 29 07:47:18 172.25.32.44 EventType=0
    Dec 29 07:47:18 172.25.32.44 Type=Information
    Dec 29 07:47:18 172.25.32.44 ComputerName=MYSERVER.dev.ad
    Dec 29 07:47:18 172.25.32.44 TaskCategory=Process Termination
    Dec 29 07:47:18 172.25.32.44 OpCode=Info
    Dec 29 07:47:18 172.25.32.44 RecordNumber=9663387
    Dec 29 07:47:18 172.25.32.44 Keywords=Audit Success
    Dec 29 07:47:18 172.25.32.44 Message=A process has exited.
    Dec 29 07:47:18 172.25.32.44 Subject:
    Dec 29 07:47:18 172.25.32.44 Security ID: NT AUTHORITY\LOCAL SERVICE
    Dec 29 07:47:18 172.25.32.44 Account Name: LOCAL SERVICE
    Dec 29 07:47:18 172.25.32.44 Account Domain: NT AUTHORITY
    Dec 29 07:47:18 172.25.32.44 Logon ID: 0x3e5
    Dec 29 07:47:18 172.25.32.44 Process Information:
    Dec 29 07:47:18 172.25.32.44 Process ID: 0xa84
    Dec 29 07:47:18 172.25.32.44 Process Name: D:\Program Files (x86)\Citrix\HealthMon\Tests\Citrix\RequestTicket.exe
    Dec 29 07:47:18 172.25.32.44 Exit Status: 0x0
     
     

How can we go about modifying the feed/events from a standard multiline format to a single line format?

I was told perhaps using regex to change the line breaks to some other delimiter could work.

Please advise as to how we can mod the multiline format to single line format?

--
Please advise if you need more details

Many thanks,

T
0
Comment
Question by:tobe1424
  • 4
  • 2
6 Comments
 
LVL 85

Expert Comment

by:ozo
ID: 40545590
What should the single line format look like?
0
 

Author Comment

by:tobe1424
ID: 40549266
It took a while for me to obtain the format from my team. Here is how it should look:

Dec 29 07:47:18 172.25.32.44 12/29/2014 02:47:17 AM LogName=Security; SourceName=Microsoft Windows security auditing.; EventCode=4689; EventType=0; Type=Information; ComputerName=MYSERVER.dev.ad; TaskCategory=Process Termination; OpCode=Info; RecordNumber=9663387; Keywords=Audit Success; Message=A process has exited.; Subject=; Security ID=NT AUTHORITY\LOCAL SERVICE; Account Name=LOCAL SERVICE; Account Domain=NT AUTHORITY; Logon ID=0x3e5; Process Information=; Process ID=0xa84; Process Name=D:\Program Files (x86)\Citrix\HealthMon\Tests\Citrix\RequestTicket.exe; Exit Status=0x0

--

T
0
 
LVL 85

Accepted Solution

by:
ozo earned 2000 total points
ID: 40550193
#!/usr/bin/perl -lna
BEGIN{$"="; "}
$d=join" ",splice @F,0,4;
if( $d ne $p && @d ){
    print "$d @d";
    @d=();
}
$p=$d;
push @d,(join" ",@F)=~s/:\s?/=/r;
END{print "$d @d"}
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 

Author Comment

by:tobe1424
ID: 40551372
Thanks so much ozo. I will test this.

cheers
0
 

Author Comment

by:tobe1424
ID: 40560065
newb q

is this a regular expression in perl ?
0
 

Author Comment

by:tobe1424
ID: 40574008
would i need to run this perl script at the indexer or on an intermediate server ?

from the indexer the windows event logs are convert/forwarded to a 3rd party appliance ( a log collector/aggregator) which the logs show with a date/time stamp on ever single line break. Even though we assume we are correctly configured to forward/convert the logs from wins to a syslog format.

t
0

Featured Post

Hire Technology Freelancers with Gigs

Work with freelancers specializing in everything from database administration to programming, who have proven themselves as experts in their field. Hire the best, collaborate easily, pay securely, and get projects done right.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article aims to explain the working of CircularLogArchiver. This tool was designed to solve the buildup of log file in cases where systems do not support circular logging or where circular logging is not enabled
Originally, this post was published on Monitis Blog, you can check it here . In business circles, we sometimes hear that today is the “age of the customer.” And so it is. Thanks to the enormous advances over the past few years in consumer techno…
In this fifth video of the Xpdf series, we discuss and demonstrate the PDFdetach utility, which is able to list and, more importantly, extract attachments that are embedded in PDF files. It does this via a command line interface, making it suitable …
Video by: Mark
This lesson goes over how to construct ordered and unordered lists and how to create hyperlinks.
Suggested Courses

572 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question