Solved

Sort of lost it with a regexp and firewall log file.

Posted on 2002-04-11
13
168 Views
Last Modified: 2010-03-05
Hi.

Using the following regexp,

([1-2]),\[([0-9]{2}/[A-Z][a-z]{2}/[0-9]{4}) ([0-9]{2}:[0-9]{2}:[0-9]{2})\] Rule '(.*)': (Permitted|Blocked): (In|Out) (TCP|UDP|Other), (.*)->(.*):([0-9]{1,5}), Owner: (.*)

on the following 2 lines of a log file,

2,[18/Mar/2002 09:56:00] Rule '113 Attempting to be authorised': Permitted: In TCP, 217.33.99.2:57447->localhost:113, Owner: no owner

and

1,[18/Mar/2002 13:50:07] Rule 'Catch all other access and log them': Blocked: In TCP, cn708223-a.hershey1.pa.home.com [67.160.132.11:11304]->localhost:80, Owner: C:\WINNT\SYSTEM32\INETSRV\INETINFO.EXE


I get the following "bits",

2
18/Mar/2002
09:56:00
113 Attempting to be authorised
Permitted
In
TCP
217.33.99.2:57447
localhost
113
no owner

and

1
18/Mar/2002
13:50:07
Catch all other access and log them
Blocked
In
TCP
cn708223-a.hershey1.pa.home.com [67.160.132.11:11304]
localhost
80
C:\WINNT\SYSTEM32\INETSRV\INETINFO.EXE


What I want to do is on line 8 of the output

217.33.99.2:57447

and

cn708223-a.hershey1.pa.home.com [67.160.132.11:11304]

I want to split this out so I would get ...

<blank line>
217.33.99.2
57447

and

cn708223-a.hershey1.pa.home.com
67.160.132.11
11304

Just to make this clear, the exact "bits" I want are ...

2
18/Mar/2002
09:56:00
113 Attempting to be authorised
Permitted
In
TCP

217.33.99.2
57447
localhost
113
no owner

and

1
18/Mar/2002
13:50:07
Catch all other access and log them
Blocked
In
TCP
cn708223-a.hershey1.pa.home.com
67.160.132.11
11304
localhost
80
C:\WINNT\SYSTEM32\INETSRV\INETINFO.EXE



Whoever gets this working will get the points.

Regards,

Richard Quadling.

0
Comment
Question by:Richard Quadling
  • 7
  • 4
  • 2
13 Comments
 
LVL 16

Expert Comment

by:maneshr
ID: 6935146
RQuadling,

".Using the following regexp,..."

Can you pl. post a larger snippet of code around the regexp?

This will help me give you a more accurate answer, faster.
0
 
LVL 40

Author Comment

by:Richard Quadling
ID: 6936285
There is no code yet.

I am using an editor called EditPadPro to manually do a search and replace on the log file to extract the bits as you have seen.

I am trying to emulate the preg_split() function of PHP in Delphi, but as yet, I've not managed to get a regexp library for Delphi and I have not been able to get the correct regexp for this log file.

The reason for using the Perl forum is that regexp and Perl are very closely associated.

All I really want is the expression that can decode the lines I've supplied into the bits I've used.

I am a newbie with regexps and the documentation I've got does not explain about optional bits (substrings?).

Richard.
0
 
LVL 16

Expert Comment

by:maneshr
ID: 6937975
RQuadling ,

"..There is no code yet..."

So how did you get

([1-2]),\[([0-9]{2}/[A-Z][a-z]{2}/[0-9]{4}) ([0-9]{2}:[0-9]{2}:[0-9]{2})\] Rule '(.*)': (Permitted|Blocked):
                     (In|Out) (TCP|UDP|Other), (.*)->(.*):([0-9]{1,5}), Owner: (.*)

??

Did you use the above regexp in some Perl code? OR did you get it from web site or other code?

Pl. let me know.
0
Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
LVL 10

Expert Comment

by:rj2
ID: 6938652
#!/usr/bin/perl
use strict;
#$_= "2,[18/Mar/2002 09:56:00] Rule '113 Attempting to be authorised': Permitted: In TCP, 217.33.99.2:57447->localhost:113,Owner: no owner";

$_="1,[18/Mar/2002 13:50:07] Rule 'Catch all other access and log them': Blocked: In TCP, cn708223-a.hershey1.pa.home.com[67.160.132.11:11304]->localhost:80, Owner: C:\WINNT\SYSTEM32\INETSRV\INETINFO.EXE";

/([1-2]),\[([0-9]{2}\/[A-Z][a-z]{2}\/[0-9]{4}) ([0-9]{2}:[0-9]{2}:[0-9]{2})\] Rule '(.*?)': (Permitted|Blocked): (In|Out) (TCP|UDP|Other), (.*?)->(.*?):([0-9]{1,5}), ?Owner: (.*)/;
my @result=($1,$2,$3,$4,$5,$6,$7,$8,'','',$9,$10,$11);
if($result[7] =~ /\[/) {
     $result[7] =~ /(.*?)\[([^:]*):(.*?)\]/;
     ($result[7],$result[8],$result[9])=($1,$2,$3);
} else {
     $result[7] =~ /(.*?):(.*)/;
     ($result[7],$result[8],$result[9])=('',$1,$2);
}          
print join("\n",@result);
0
 
LVL 10

Accepted Solution

by:
rj2 earned 200 total points
ID: 6939035
Here is version you can use as filter.
Regexp is one long line, remove any crlf inside it if line is broken when posted.

#!/usr/bin/perl
use strict;

while(<>) {
     /([1-2]),\[([0-9]{2}\/[A-Z][a-z]{2}\/[0-9]{4})\x20([0-9]{2}:[0-9]{2}:[0-9]{2})\]\x20Rule\x20'(.*?)':\x20(Permitted|Blocked):\x20(In|Out)\x20(TCP|UDP|Other),\x20(.*?)->(.*?):([0-9]{1,5}),\x20?Owner:\x20(.*)/;
     my @result=($1,$2,$3,$4,$5,$6,$7,$8,'','',$9,$10,$11);
     if($result[7] =~ /\[/) {
         $result[7] =~ /(.*?)\[([^:]*):(.*?)\]/;
         ($result[7],$result[8],$result[9])=($1,$2,$3);
     } else {
         $result[7] =~ /(.*?):(.*)/;
         ($result[7],$result[8],$result[9])=('',$1,$2);
     }          
     print join("\n",@result);
}    
0
 
LVL 40

Author Comment

by:Richard Quadling
ID: 6941386
So there is no way a SINGLE line conversion can be done?

Though I'm not using Perl, I can see the solution IF I have to use code, beyond a single line.

Thanks,

Regards,

Richard Quadling.
0
 
LVL 40

Author Comment

by:Richard Quadling
ID: 6941387
To Maneshr,

I did it myself using EditPadPro and the help file supplied.

As I said, I am a beginner, so I assume I've made HUGE errors! Having said that, the regexp works upto a point, so I can't have got it THAT wrong!

Regards,

Richard.
0
 
LVL 40

Author Comment

by:Richard Quadling
ID: 6941469
I managed to get it into 1 line!!

([1-2]),\[(\d{2}/[A-Z][a-z]{2}/\d{4}) (\d{2}:\d{2}:\d{2})\] Rule '(.*?)': (Permitted|Blocked): (In|Out) (TCP|UDP|Other), (.* )?\[?(.*):(\d{1,5})\]?->(.*), Owner: (.*)


\1
Date : \2
Time : \3
Rule : \4
Access : \5
Direction : \6
Protocol : \7
Name : \8
Address : \9
Port : \10
Local address : \11
Owner : \12

Gives me ...

2
Date : 18/Mar/2002
Time : 09:56:00
Rule : 113 Attempting to be authorised
Access : Permitted
Direction : In
Protocol : TCP
Name : mx2.global.net.uk
Address : 195.147.246.224
Port : 56226
Local address : localhost:113
Owner : no owner

2
Date : 18/Mar/2002
Time : 10:10:48
Rule : 113 Attempting to be authorised
Access : Permitted
Direction : In
Protocol : TCP
Name :
Address : 217.33.99.2
Port : 64259
Local address : localhost:113
Owner : no owner

Which is about there!

I am having trouble with the local address as this is sometimes localhost:port and other times it is server [ip:port], which is more or less the same, but I don't really need to break this down as it will always be either localhost OR server ip and port.

Thanks for your help.

Richard.
0
 
LVL 40

Author Comment

by:Richard Quadling
ID: 6941541
A follow on question to this log file issue.

1,[03/Apr/2002 10:44:16] Rule 'Catch all other access and log them': Blocked: Out ICMP [11] Time Exceeded, localhost->213.122.36.72, Owner: Tcpip Kernel Driver
1,[05/Apr/2002 08:55:23] Rule 'Catch all other access and log them': Blocked: Out protocol [2], localhost->224.0.1.24, Owner: Tcpip Kernel Driver

The regexp ...

([1-2]),\[(\d{2}/[A-Z][a-z]{2}/\d{4}) (\d{2}:\d{2}:\d{2})\] Rule '(.*?)': (Permitted|Blocked): (In|Out) (TCP|UDP|Other), (.* )?\[?(.*):(\d{1,5})\]?->(.*), Owner: (.*)

does not handle the ICMP and protocol [x] protocols.

How do I do ...

Match TCP or
Match UDP or
Match ICMP \[\d{1,5}\] .* or
Match protocol \[\d{1,5}\]

and get whatever matches as the back-reference?

I tried ...

([1-2]),\[(\d{2}/[A-Z][a-z]{2}/\d{4}) (\d{2}:\d{2}:\d{2})\] Rule '(.*?)': (Permitted|Blocked): (In|Out) (TCP|UDP|(?:.*?)), (.* )?\[?(.*):(\d{1,5})\]?->(.*), Owner: (.*)

(TCP|UDP|(?:.*?)) being the important bit, but no-joy.

I've increased the points to 200.

Regards,

Richard Quadling.
0
 
LVL 10

Expert Comment

by:rj2
ID: 6942963
You said that "Whoever gets this working will get the points." . The script does that.
0
 
LVL 40

Author Comment

by:Richard Quadling
ID: 6944266
Sorry. I will ask another question!

Your script does what I wanted to that point.

I need to expand on it and that does deserve a new question.

Richard.
0
 
LVL 10

Expert Comment

by:rj2
ID: 6945835
ok, thanks.
Does regexp below do what you need? (Match TCP or Match UDP or ...) ?

/(TCP|UDP|ICMP \[\d{1,5}\]|protocol \[\d{1,5}\])/
0
 
LVL 40

Author Comment

by:Richard Quadling
ID: 6947145
Aha!

And If I want to get "ICMP [11] Timeout Something" then I can use

/(TCP|UDP|ICMP \[\d{1,5}\].*?|protocol \[\d{1,5}\])/

which works excellently!

It is strange. Every time I think I'm there I get another puzzle to solve.

Regards,

Richard.
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A year or so back I was asked to have a play with MongoDB; within half an hour I had downloaded (http://www.mongodb.org/downloads),  installed and started the daemon, and had a console window open. After an hour or two of playing at the command …
There are many situations when we need to display the data in sorted order. For example: Student details by name or by rank or by total marks etc. If you are working on data driven based projects then you will use sorting techniques very frequently.…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

861 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question