[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

How to Parse Message Field in Syslog Data

Posted on 2009-04-21
16
Medium Priority
?
445 Views
Last Modified: 2013-11-16
Does anyone know how to parse out the msg, src, dst, and proto data in the MsgText field in Syslog data stored in SQL Server?
0
Comment
Question by:ljacobs
  • 8
  • 8
16 Comments
 
LVL 41

Expert Comment

by:ralmada
ID: 24197699
0
 

Author Comment

by:ljacobs
ID: 24197737
I am not looking for a syslog server.  I am using Kiwi Syslog Server and saving data to a SQL Server database.  I want to parse the data in the MsgTxt field into individual data fields.
0
 
LVL 41

Expert Comment

by:ralmada
ID: 24197801
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:ljacobs
ID: 24197858
That link does not give me a way to parse the TxtMsg field into the individual components of the data field.
0
 
LVL 41

Expert Comment

by:ralmada
ID: 24197869
If not, provide some sample log entries so we can tell you how to display that in SQL.
 
0
 

Author Comment

by:ljacobs
ID: 24198213
id=firewall sn=0017C52C1D49 time="2009-04-20 22:17:23" fw=x.x.x.x pri=6 c=262144 m=98 msg="Connection Opened" n=0 src=x.x.x.x:2912:X0:SQL dst=x.x.x.x:8443:X1:hx-x-x-x.mntimn.dedicated.static.tds.net proto=tcp/8443

All of this type of data is found in the MsgTxt field.  I want the fw, msg, src, dst, and proto parsed out into individual fields within a new table.
0
 
LVL 41

Expert Comment

by:ralmada
ID: 24198413
Give this a try first, then you can add it into a new table like this
insert yournewtable
select .... from ....

select case when patindex('fw=', MsgTxt) <> 0 then
		substring(
				MsgTxt, 
				patindex('fw=', MsgTxt) + 3
				len(MsgTxt) - charindex(' ', right(MsgTxt, len(MsgTxt) - patindex('fw=', MsgTxt) + 3))
			)
	else '' END as fw,
	
	case when patindex('msg=', MsgTxt) <> 0 then
		substring(
				MsgTxt, 
				patindex('msg=', MsgTxt) + 4
				len(MsgTxt) - charindex('"', right(MsgTxt, len(MsgTxt) - patindex('msg=', MsgTxt) + 4)) - 1
			)
	else '' END as msg,
 
	case when patindex('src=', MsgTxt) <> 0 then
		substring(
				MsgTxt, 
				patindex('src=', MsgTxt) + 4
				len(MsgTxt) - charindex(' ', right(MsgTxt, len(MsgTxt) - patindex('src=', MsgTxt) + 4))
			)
	else '' END as src,
 
	case when patindex('dst=', MsgTxt) <> 0 then
		substring(
				MsgTxt, 
				patindex('dst=', MsgTxt) + 4
				len(MsgTxt) - charindex(' ', right(MsgTxt, len(MsgTxt) - patindex('dst=', MsgTxt) + 4))
			)
	else '' END as dst,
 
	case when patindex('proto=', MsgTxt) <> 0 then
		substring(
				MsgTxt, 
				patindex('proto=', MsgTxt) + 6
				len(MsgTxt) - charindex(' ', right(MsgTxt, len(MsgTxt) - patindex('proto=', MsgTxt) + 6))
			)
	else '' END as proto
from yourtable

Open in new window

0
 

Author Comment

by:ljacobs
ID: 24198517
Get an error that says incorrect syntax near len.
0
 
LVL 41

Expert Comment

by:ralmada
ID: 24198538
if proto is always the last part. just change line 33 to 39 of the above code with this:

	case when patindex('proto=', MsgTxt) <> 0 then
		right(msgTxt, patindex('proto=', reverse(msgTxt)) - 6)
	else '' END as proto

Open in new window

0
 
LVL 41

Expert Comment

by:ralmada
ID: 24198556
And you are right. missed some commas. (don't have a test environment here :)
select case when patindex('fw=', MsgTxt) <> 0 then
		substring(
				MsgTxt, 
				patindex('fw=', MsgTxt) + 3,
				len(MsgTxt) - charindex(' ', right(MsgTxt, len(MsgTxt) - patindex('fw=', MsgTxt) + 3))
			)
	else '' END as fw,
	
	case when patindex('msg=', MsgTxt) <> 0 then
		substring(
				MsgTxt, 
				patindex('msg=', MsgTxt) + 4,
				len(MsgTxt) - charindex('"', right(MsgTxt, len(MsgTxt) - patindex('msg=', MsgTxt) + 4)) - 1
			)
	else '' END as msg,
 
	case when patindex('src=', MsgTxt) <> 0 then
		substring(
				MsgTxt, 
				patindex('src=', MsgTxt) + 4,
				len(MsgTxt) - charindex(' ', right(MsgTxt, len(MsgTxt) - patindex('src=', MsgTxt) + 4))
			)
	else '' END as src,
 
	case when patindex('dst=', MsgTxt) <> 0 then
		substring(
				MsgTxt, 
				patindex('dst=', MsgTxt) + 4,
				len(MsgTxt) - charindex(' ', right(MsgTxt, len(MsgTxt) - patindex('dst=', MsgTxt) + 4))
			)
	else '' END as dst,
 
	case when patindex('proto=', MsgTxt) <> 0 then
		right(msgTxt, patindex('proto=', reverse(msgTxt)) - 6)
	else '' END as proto
 
from yourtable

Open in new window

0
 

Author Comment

by:ljacobs
ID: 24198600
A select grid is created but there is no data.
0
 
LVL 41

Expert Comment

by:ralmada
ID: 24198650
And now?
select case when patindex('%fw=%', MsgTxt) <> 0 then
		substring(
				MsgTxt, 
				patindex('%fw=%', MsgTxt) + 3,
				len(MsgTxt) - charindex(' ', right(MsgTxt, len(MsgTxt) - patindex('%fw=%', MsgTxt) + 3))
			)
	else '' END as fw,
	
	case when patindex('%msg=%', MsgTxt) <> 0 then
		substring(
				MsgTxt, 
				patindex('%msg=%', MsgTxt) + 4,
				len(MsgTxt) - charindex('"', right(MsgTxt, len(MsgTxt) - patindex('%msg=%', MsgTxt) + 4)) - 1
			)
	else '' END as msg,
 
	case when patindex('%src=%', MsgTxt) <> 0 then
		substring(
				MsgTxt, 
				patindex('%src=%', MsgTxt) + 4,
				len(MsgTxt) - charindex(' ', right(MsgTxt, len(MsgTxt) - patindex('%src=%', MsgTxt) + 4))
			)
	else '' END as src,
 
	case when patindex('%dst=%', MsgTxt) <> 0 then
		substring(
				MsgTxt, 
				patindex('%dst=%', MsgTxt) + 4,
				len(MsgTxt) - charindex(' ', right(MsgTxt, len(MsgTxt) - patindex('%dst=%', MsgTxt) + 4))
			)
	else '' END as dst,
 
	case when patindex('%proto=%', MsgTxt) <> 0 then
		right(msgTxt, patindex('%proto=%', reverse(msgTxt)) - 6)
	else '' END as proto
 
from yourtable

Open in new window

0
 

Author Comment

by:ljacobs
ID: 24200591
Getting this error now:

Msg 536, Level 16, State 2, Line 1
Invalid length parameter passed to the RIGHT function.
0
 

Author Comment

by:ljacobs
ID: 24200596
FYI

And I made a mistake.  MsgTxt should be MsgText.  I correct it before running the query.
0
 
LVL 41

Accepted Solution

by:
ralmada earned 1000 total points
ID: 24201172
This should work:
select case when patindex('%fw=%', MsgText) <> 0 then
		substring(
				MsgText, 
				patindex('%fw=%', MsgText) + 3,
				charindex(' ', right(MsgText, len(MsgText) - patindex('%fw=%', MsgText) - 2))
			)
	else '' END as fw,
	
	case when patindex('%msg=%', MsgText) <> 0 then
		substring(
				MsgText, 
				patindex('%msg=%', MsgText) + 5,
				charindex('"', right(MsgText, len(MsgText) - patindex('%msg=%', MsgText) -4)) - 1
			)
	else '' END as msg,
 
	case when patindex('%src=%', MsgText) <> 0 then
		substring(
				MsgText, 
				patindex('%src=%', MsgText) + 4,
				charindex(' ', right(MsgText, len(MsgText) - patindex('%src=%', MsgText) - 3))
			)
	else '' END as src,
 
	case when patindex('%dst=%', MsgText) <> 0 then
		substring(
				MsgText, 
				patindex('%dst=%', MsgText) + 4,
				charindex(' ', right(MsgText, len(MsgText) - patindex('%dst=%', MsgText) - 3))
			)
	else '' END as dst,
 
	case when patindex('%proto=%', MsgText) <> 0 then
		right(MsgText, patindex('%=otorp%', reverse(MsgText)) - 1)
	else '' END as proto
 
from yourtable

Open in new window

0
 

Author Closing Comment

by:ljacobs
ID: 31572867
Thank you.  It's great!
0

Featured Post

New Tabletop Appliances Blow Competitors Away!

WatchGuard’s new T15, T35 and T55 tabletop UTMs provide the highest-performing security inspection in their class, allowing users at small offices, home offices and distributed enterprises to experience blazing-fast Internet speeds without sacrificing enterprise-grade security.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Recently we ran in to an issue while running some SQL jobs where we were trying to process the cubes.  We got an error saying failure stating 'NT SERVICE\SQLSERVERAGENT does not have access to Analysis Services. So this is a way to automate that wit…
An introduction to the wonderful sport of Scam Baiting.  Learn how to help fight scammers by beating them at their own game. This great pass time helps the world, while providing an endless source of entertainment. Enjoy!
Viewers will learn how to use the UPDATE and DELETE statements to change or remove existing data from their tables. Make a table: Update a specific column given a specific row using the UPDATE statement: Remove a set of values using the DELETE s…
Established in 1997, Technology Architects has become one of the most reputable technology solutions companies in the country. TA have been providing businesses with cost effective state-of-the-art solutions and unparalleled service that is designed…

873 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question