Solved

Sort of lost it with a regexp and firewall log file.

Posted on 2002-04-11
13
171 Views
Last Modified: 2010-03-05
Hi.

Using the following regexp,

([1-2]),\[([0-9]{2}/[A-Z][a-z]{2}/[0-9]{4}) ([0-9]{2}:[0-9]{2}:[0-9]{2})\] Rule '(.*)': (Permitted|Blocked): (In|Out) (TCP|UDP|Other), (.*)->(.*):([0-9]{1,5}), Owner: (.*)

on the following 2 lines of a log file,

2,[18/Mar/2002 09:56:00] Rule '113 Attempting to be authorised': Permitted: In TCP, 217.33.99.2:57447->localhost:113, Owner: no owner

and

1,[18/Mar/2002 13:50:07] Rule 'Catch all other access and log them': Blocked: In TCP, cn708223-a.hershey1.pa.home.com [67.160.132.11:11304]->localhost:80, Owner: C:\WINNT\SYSTEM32\INETSRV\INETINFO.EXE


I get the following "bits",

2
18/Mar/2002
09:56:00
113 Attempting to be authorised
Permitted
In
TCP
217.33.99.2:57447
localhost
113
no owner

and

1
18/Mar/2002
13:50:07
Catch all other access and log them
Blocked
In
TCP
cn708223-a.hershey1.pa.home.com [67.160.132.11:11304]
localhost
80
C:\WINNT\SYSTEM32\INETSRV\INETINFO.EXE


What I want to do is on line 8 of the output

217.33.99.2:57447

and

cn708223-a.hershey1.pa.home.com [67.160.132.11:11304]

I want to split this out so I would get ...

<blank line>
217.33.99.2
57447

and

cn708223-a.hershey1.pa.home.com
67.160.132.11
11304

Just to make this clear, the exact "bits" I want are ...

2
18/Mar/2002
09:56:00
113 Attempting to be authorised
Permitted
In
TCP

217.33.99.2
57447
localhost
113
no owner

and

1
18/Mar/2002
13:50:07
Catch all other access and log them
Blocked
In
TCP
cn708223-a.hershey1.pa.home.com
67.160.132.11
11304
localhost
80
C:\WINNT\SYSTEM32\INETSRV\INETINFO.EXE



Whoever gets this working will get the points.

Regards,

Richard Quadling.

0
Comment
Question by:Richard Quadling
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 7
  • 4
  • 2
13 Comments
 
LVL 16

Expert Comment

by:maneshr
ID: 6935146
RQuadling,

".Using the following regexp,..."

Can you pl. post a larger snippet of code around the regexp?

This will help me give you a more accurate answer, faster.
0
 
LVL 40

Author Comment

by:Richard Quadling
ID: 6936285
There is no code yet.

I am using an editor called EditPadPro to manually do a search and replace on the log file to extract the bits as you have seen.

I am trying to emulate the preg_split() function of PHP in Delphi, but as yet, I've not managed to get a regexp library for Delphi and I have not been able to get the correct regexp for this log file.

The reason for using the Perl forum is that regexp and Perl are very closely associated.

All I really want is the expression that can decode the lines I've supplied into the bits I've used.

I am a newbie with regexps and the documentation I've got does not explain about optional bits (substrings?).

Richard.
0
 
LVL 16

Expert Comment

by:maneshr
ID: 6937975
RQuadling ,

"..There is no code yet..."

So how did you get

([1-2]),\[([0-9]{2}/[A-Z][a-z]{2}/[0-9]{4}) ([0-9]{2}:[0-9]{2}:[0-9]{2})\] Rule '(.*)': (Permitted|Blocked):
                     (In|Out) (TCP|UDP|Other), (.*)->(.*):([0-9]{1,5}), Owner: (.*)

??

Did you use the above regexp in some Perl code? OR did you get it from web site or other code?

Pl. let me know.
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 10

Expert Comment

by:rj2
ID: 6938652
#!/usr/bin/perl
use strict;
#$_= "2,[18/Mar/2002 09:56:00] Rule '113 Attempting to be authorised': Permitted: In TCP, 217.33.99.2:57447->localhost:113,Owner: no owner";

$_="1,[18/Mar/2002 13:50:07] Rule 'Catch all other access and log them': Blocked: In TCP, cn708223-a.hershey1.pa.home.com[67.160.132.11:11304]->localhost:80, Owner: C:\WINNT\SYSTEM32\INETSRV\INETINFO.EXE";

/([1-2]),\[([0-9]{2}\/[A-Z][a-z]{2}\/[0-9]{4}) ([0-9]{2}:[0-9]{2}:[0-9]{2})\] Rule '(.*?)': (Permitted|Blocked): (In|Out) (TCP|UDP|Other), (.*?)->(.*?):([0-9]{1,5}), ?Owner: (.*)/;
my @result=($1,$2,$3,$4,$5,$6,$7,$8,'','',$9,$10,$11);
if($result[7] =~ /\[/) {
     $result[7] =~ /(.*?)\[([^:]*):(.*?)\]/;
     ($result[7],$result[8],$result[9])=($1,$2,$3);
} else {
     $result[7] =~ /(.*?):(.*)/;
     ($result[7],$result[8],$result[9])=('',$1,$2);
}          
print join("\n",@result);
0
 
LVL 10

Accepted Solution

by:
rj2 earned 200 total points
ID: 6939035
Here is version you can use as filter.
Regexp is one long line, remove any crlf inside it if line is broken when posted.

#!/usr/bin/perl
use strict;

while(<>) {
     /([1-2]),\[([0-9]{2}\/[A-Z][a-z]{2}\/[0-9]{4})\x20([0-9]{2}:[0-9]{2}:[0-9]{2})\]\x20Rule\x20'(.*?)':\x20(Permitted|Blocked):\x20(In|Out)\x20(TCP|UDP|Other),\x20(.*?)->(.*?):([0-9]{1,5}),\x20?Owner:\x20(.*)/;
     my @result=($1,$2,$3,$4,$5,$6,$7,$8,'','',$9,$10,$11);
     if($result[7] =~ /\[/) {
         $result[7] =~ /(.*?)\[([^:]*):(.*?)\]/;
         ($result[7],$result[8],$result[9])=($1,$2,$3);
     } else {
         $result[7] =~ /(.*?):(.*)/;
         ($result[7],$result[8],$result[9])=('',$1,$2);
     }          
     print join("\n",@result);
}    
0
 
LVL 40

Author Comment

by:Richard Quadling
ID: 6941386
So there is no way a SINGLE line conversion can be done?

Though I'm not using Perl, I can see the solution IF I have to use code, beyond a single line.

Thanks,

Regards,

Richard Quadling.
0
 
LVL 40

Author Comment

by:Richard Quadling
ID: 6941387
To Maneshr,

I did it myself using EditPadPro and the help file supplied.

As I said, I am a beginner, so I assume I've made HUGE errors! Having said that, the regexp works upto a point, so I can't have got it THAT wrong!

Regards,

Richard.
0
 
LVL 40

Author Comment

by:Richard Quadling
ID: 6941469
I managed to get it into 1 line!!

([1-2]),\[(\d{2}/[A-Z][a-z]{2}/\d{4}) (\d{2}:\d{2}:\d{2})\] Rule '(.*?)': (Permitted|Blocked): (In|Out) (TCP|UDP|Other), (.* )?\[?(.*):(\d{1,5})\]?->(.*), Owner: (.*)


\1
Date : \2
Time : \3
Rule : \4
Access : \5
Direction : \6
Protocol : \7
Name : \8
Address : \9
Port : \10
Local address : \11
Owner : \12

Gives me ...

2
Date : 18/Mar/2002
Time : 09:56:00
Rule : 113 Attempting to be authorised
Access : Permitted
Direction : In
Protocol : TCP
Name : mx2.global.net.uk
Address : 195.147.246.224
Port : 56226
Local address : localhost:113
Owner : no owner

2
Date : 18/Mar/2002
Time : 10:10:48
Rule : 113 Attempting to be authorised
Access : Permitted
Direction : In
Protocol : TCP
Name :
Address : 217.33.99.2
Port : 64259
Local address : localhost:113
Owner : no owner

Which is about there!

I am having trouble with the local address as this is sometimes localhost:port and other times it is server [ip:port], which is more or less the same, but I don't really need to break this down as it will always be either localhost OR server ip and port.

Thanks for your help.

Richard.
0
 
LVL 40

Author Comment

by:Richard Quadling
ID: 6941541
A follow on question to this log file issue.

1,[03/Apr/2002 10:44:16] Rule 'Catch all other access and log them': Blocked: Out ICMP [11] Time Exceeded, localhost->213.122.36.72, Owner: Tcpip Kernel Driver
1,[05/Apr/2002 08:55:23] Rule 'Catch all other access and log them': Blocked: Out protocol [2], localhost->224.0.1.24, Owner: Tcpip Kernel Driver

The regexp ...

([1-2]),\[(\d{2}/[A-Z][a-z]{2}/\d{4}) (\d{2}:\d{2}:\d{2})\] Rule '(.*?)': (Permitted|Blocked): (In|Out) (TCP|UDP|Other), (.* )?\[?(.*):(\d{1,5})\]?->(.*), Owner: (.*)

does not handle the ICMP and protocol [x] protocols.

How do I do ...

Match TCP or
Match UDP or
Match ICMP \[\d{1,5}\] .* or
Match protocol \[\d{1,5}\]

and get whatever matches as the back-reference?

I tried ...

([1-2]),\[(\d{2}/[A-Z][a-z]{2}/\d{4}) (\d{2}:\d{2}:\d{2})\] Rule '(.*?)': (Permitted|Blocked): (In|Out) (TCP|UDP|(?:.*?)), (.* )?\[?(.*):(\d{1,5})\]?->(.*), Owner: (.*)

(TCP|UDP|(?:.*?)) being the important bit, but no-joy.

I've increased the points to 200.

Regards,

Richard Quadling.
0
 
LVL 10

Expert Comment

by:rj2
ID: 6942963
You said that "Whoever gets this working will get the points." . The script does that.
0
 
LVL 40

Author Comment

by:Richard Quadling
ID: 6944266
Sorry. I will ask another question!

Your script does what I wanted to that point.

I need to expand on it and that does deserve a new question.

Richard.
0
 
LVL 10

Expert Comment

by:rj2
ID: 6945835
ok, thanks.
Does regexp below do what you need? (Match TCP or Match UDP or ...) ?

/(TCP|UDP|ICMP \[\d{1,5}\]|protocol \[\d{1,5}\])/
0
 
LVL 40

Author Comment

by:Richard Quadling
ID: 6947145
Aha!

And If I want to get "ICMP [11] Timeout Something" then I can use

/(TCP|UDP|ICMP \[\d{1,5}\].*?|protocol \[\d{1,5}\])/

which works excellently!

It is strange. Every time I think I'm there I get another puzzle to solve.

Regards,

Richard.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
hard perl script 16 166
rename outfile before writing 2 77
Perl Script - Remove column of data based on column value 2 59
Perl script to delete older files 6 99
I have been pestered over the years to produce and distribute regular data extracts, and often the request have explicitly requested the data be emailed as an Excel attachement; specifically Excel, as it appears: CSV files confuse (no Red or Green h…
A year or so back I was asked to have a play with MongoDB; within half an hour I had downloaded (http://www.mongodb.org/downloads),  installed and started the daemon, and had a console window open. After an hour or two of playing at the command …
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

739 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question