Solved

Sort of lost it with a regexp and firewall log file.

Posted on 2002-04-11
13
164 Views
Last Modified: 2010-03-05
Hi.

Using the following regexp,

([1-2]),\[([0-9]{2}/[A-Z][a-z]{2}/[0-9]{4}) ([0-9]{2}:[0-9]{2}:[0-9]{2})\] Rule '(.*)': (Permitted|Blocked): (In|Out) (TCP|UDP|Other), (.*)->(.*):([0-9]{1,5}), Owner: (.*)

on the following 2 lines of a log file,

2,[18/Mar/2002 09:56:00] Rule '113 Attempting to be authorised': Permitted: In TCP, 217.33.99.2:57447->localhost:113, Owner: no owner

and

1,[18/Mar/2002 13:50:07] Rule 'Catch all other access and log them': Blocked: In TCP, cn708223-a.hershey1.pa.home.com [67.160.132.11:11304]->localhost:80, Owner: C:\WINNT\SYSTEM32\INETSRV\INETINFO.EXE


I get the following "bits",

2
18/Mar/2002
09:56:00
113 Attempting to be authorised
Permitted
In
TCP
217.33.99.2:57447
localhost
113
no owner

and

1
18/Mar/2002
13:50:07
Catch all other access and log them
Blocked
In
TCP
cn708223-a.hershey1.pa.home.com [67.160.132.11:11304]
localhost
80
C:\WINNT\SYSTEM32\INETSRV\INETINFO.EXE


What I want to do is on line 8 of the output

217.33.99.2:57447

and

cn708223-a.hershey1.pa.home.com [67.160.132.11:11304]

I want to split this out so I would get ...

<blank line>
217.33.99.2
57447

and

cn708223-a.hershey1.pa.home.com
67.160.132.11
11304

Just to make this clear, the exact "bits" I want are ...

2
18/Mar/2002
09:56:00
113 Attempting to be authorised
Permitted
In
TCP

217.33.99.2
57447
localhost
113
no owner

and

1
18/Mar/2002
13:50:07
Catch all other access and log them
Blocked
In
TCP
cn708223-a.hershey1.pa.home.com
67.160.132.11
11304
localhost
80
C:\WINNT\SYSTEM32\INETSRV\INETINFO.EXE



Whoever gets this working will get the points.

Regards,

Richard Quadling.

0
Comment
Question by:RQuadling
  • 7
  • 4
  • 2
13 Comments
 
LVL 16

Expert Comment

by:maneshr
ID: 6935146
RQuadling,

".Using the following regexp,..."

Can you pl. post a larger snippet of code around the regexp?

This will help me give you a more accurate answer, faster.
0
 
LVL 40

Author Comment

by:RQuadling
ID: 6936285
There is no code yet.

I am using an editor called EditPadPro to manually do a search and replace on the log file to extract the bits as you have seen.

I am trying to emulate the preg_split() function of PHP in Delphi, but as yet, I've not managed to get a regexp library for Delphi and I have not been able to get the correct regexp for this log file.

The reason for using the Perl forum is that regexp and Perl are very closely associated.

All I really want is the expression that can decode the lines I've supplied into the bits I've used.

I am a newbie with regexps and the documentation I've got does not explain about optional bits (substrings?).

Richard.
0
 
LVL 16

Expert Comment

by:maneshr
ID: 6937975
RQuadling ,

"..There is no code yet..."

So how did you get

([1-2]),\[([0-9]{2}/[A-Z][a-z]{2}/[0-9]{4}) ([0-9]{2}:[0-9]{2}:[0-9]{2})\] Rule '(.*)': (Permitted|Blocked):
                     (In|Out) (TCP|UDP|Other), (.*)->(.*):([0-9]{1,5}), Owner: (.*)

??

Did you use the above regexp in some Perl code? OR did you get it from web site or other code?

Pl. let me know.
0
 
LVL 10

Expert Comment

by:rj2
ID: 6938652
#!/usr/bin/perl
use strict;
#$_= "2,[18/Mar/2002 09:56:00] Rule '113 Attempting to be authorised': Permitted: In TCP, 217.33.99.2:57447->localhost:113,Owner: no owner";

$_="1,[18/Mar/2002 13:50:07] Rule 'Catch all other access and log them': Blocked: In TCP, cn708223-a.hershey1.pa.home.com[67.160.132.11:11304]->localhost:80, Owner: C:\WINNT\SYSTEM32\INETSRV\INETINFO.EXE";

/([1-2]),\[([0-9]{2}\/[A-Z][a-z]{2}\/[0-9]{4}) ([0-9]{2}:[0-9]{2}:[0-9]{2})\] Rule '(.*?)': (Permitted|Blocked): (In|Out) (TCP|UDP|Other), (.*?)->(.*?):([0-9]{1,5}), ?Owner: (.*)/;
my @result=($1,$2,$3,$4,$5,$6,$7,$8,'','',$9,$10,$11);
if($result[7] =~ /\[/) {
     $result[7] =~ /(.*?)\[([^:]*):(.*?)\]/;
     ($result[7],$result[8],$result[9])=($1,$2,$3);
} else {
     $result[7] =~ /(.*?):(.*)/;
     ($result[7],$result[8],$result[9])=('',$1,$2);
}          
print join("\n",@result);
0
 
LVL 10

Accepted Solution

by:
rj2 earned 200 total points
ID: 6939035
Here is version you can use as filter.
Regexp is one long line, remove any crlf inside it if line is broken when posted.

#!/usr/bin/perl
use strict;

while(<>) {
     /([1-2]),\[([0-9]{2}\/[A-Z][a-z]{2}\/[0-9]{4})\x20([0-9]{2}:[0-9]{2}:[0-9]{2})\]\x20Rule\x20'(.*?)':\x20(Permitted|Blocked):\x20(In|Out)\x20(TCP|UDP|Other),\x20(.*?)->(.*?):([0-9]{1,5}),\x20?Owner:\x20(.*)/;
     my @result=($1,$2,$3,$4,$5,$6,$7,$8,'','',$9,$10,$11);
     if($result[7] =~ /\[/) {
         $result[7] =~ /(.*?)\[([^:]*):(.*?)\]/;
         ($result[7],$result[8],$result[9])=($1,$2,$3);
     } else {
         $result[7] =~ /(.*?):(.*)/;
         ($result[7],$result[8],$result[9])=('',$1,$2);
     }          
     print join("\n",@result);
}    
0
 
LVL 40

Author Comment

by:RQuadling
ID: 6941386
So there is no way a SINGLE line conversion can be done?

Though I'm not using Perl, I can see the solution IF I have to use code, beyond a single line.

Thanks,

Regards,

Richard Quadling.
0
How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

 
LVL 40

Author Comment

by:RQuadling
ID: 6941387
To Maneshr,

I did it myself using EditPadPro and the help file supplied.

As I said, I am a beginner, so I assume I've made HUGE errors! Having said that, the regexp works upto a point, so I can't have got it THAT wrong!

Regards,

Richard.
0
 
LVL 40

Author Comment

by:RQuadling
ID: 6941469
I managed to get it into 1 line!!

([1-2]),\[(\d{2}/[A-Z][a-z]{2}/\d{4}) (\d{2}:\d{2}:\d{2})\] Rule '(.*?)': (Permitted|Blocked): (In|Out) (TCP|UDP|Other), (.* )?\[?(.*):(\d{1,5})\]?->(.*), Owner: (.*)


\1
Date : \2
Time : \3
Rule : \4
Access : \5
Direction : \6
Protocol : \7
Name : \8
Address : \9
Port : \10
Local address : \11
Owner : \12

Gives me ...

2
Date : 18/Mar/2002
Time : 09:56:00
Rule : 113 Attempting to be authorised
Access : Permitted
Direction : In
Protocol : TCP
Name : mx2.global.net.uk
Address : 195.147.246.224
Port : 56226
Local address : localhost:113
Owner : no owner

2
Date : 18/Mar/2002
Time : 10:10:48
Rule : 113 Attempting to be authorised
Access : Permitted
Direction : In
Protocol : TCP
Name :
Address : 217.33.99.2
Port : 64259
Local address : localhost:113
Owner : no owner

Which is about there!

I am having trouble with the local address as this is sometimes localhost:port and other times it is server [ip:port], which is more or less the same, but I don't really need to break this down as it will always be either localhost OR server ip and port.

Thanks for your help.

Richard.
0
 
LVL 40

Author Comment

by:RQuadling
ID: 6941541
A follow on question to this log file issue.

1,[03/Apr/2002 10:44:16] Rule 'Catch all other access and log them': Blocked: Out ICMP [11] Time Exceeded, localhost->213.122.36.72, Owner: Tcpip Kernel Driver
1,[05/Apr/2002 08:55:23] Rule 'Catch all other access and log them': Blocked: Out protocol [2], localhost->224.0.1.24, Owner: Tcpip Kernel Driver

The regexp ...

([1-2]),\[(\d{2}/[A-Z][a-z]{2}/\d{4}) (\d{2}:\d{2}:\d{2})\] Rule '(.*?)': (Permitted|Blocked): (In|Out) (TCP|UDP|Other), (.* )?\[?(.*):(\d{1,5})\]?->(.*), Owner: (.*)

does not handle the ICMP and protocol [x] protocols.

How do I do ...

Match TCP or
Match UDP or
Match ICMP \[\d{1,5}\] .* or
Match protocol \[\d{1,5}\]

and get whatever matches as the back-reference?

I tried ...

([1-2]),\[(\d{2}/[A-Z][a-z]{2}/\d{4}) (\d{2}:\d{2}:\d{2})\] Rule '(.*?)': (Permitted|Blocked): (In|Out) (TCP|UDP|(?:.*?)), (.* )?\[?(.*):(\d{1,5})\]?->(.*), Owner: (.*)

(TCP|UDP|(?:.*?)) being the important bit, but no-joy.

I've increased the points to 200.

Regards,

Richard Quadling.
0
 
LVL 10

Expert Comment

by:rj2
ID: 6942963
You said that "Whoever gets this working will get the points." . The script does that.
0
 
LVL 40

Author Comment

by:RQuadling
ID: 6944266
Sorry. I will ask another question!

Your script does what I wanted to that point.

I need to expand on it and that does deserve a new question.

Richard.
0
 
LVL 10

Expert Comment

by:rj2
ID: 6945835
ok, thanks.
Does regexp below do what you need? (Match TCP or Match UDP or ...) ?

/(TCP|UDP|ICMP \[\d{1,5}\]|protocol \[\d{1,5}\])/
0
 
LVL 40

Author Comment

by:RQuadling
ID: 6947145
Aha!

And If I want to get "ICMP [11] Timeout Something" then I can use

/(TCP|UDP|ICMP \[\d{1,5}\].*?|protocol \[\d{1,5}\])/

which works excellently!

It is strange. Every time I think I'm there I get another puzzle to solve.

Regards,

Richard.
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

I've just discovered very important differences between Windows an Unix formats in Perl,at least 5.xx.. MOST IMPORTANT: Use Unix file format while saving Your script. otherwise it will have ^M s or smth likely weird in the EOL, Then DO NOT use m…
Checking the Alert Log in AWS RDS Oracle can be a pain through their user interface.  I made a script to download the Alert Log, look for errors, and email me the trace files.  In this article I'll describe what I did and share my script.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

760 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now