?
Solved

[123.234.435.234] err <-- Extract Numbers from between brackets

Posted on 2006-06-23
9
Medium Priority
?
332 Views
Last Modified: 2012-05-05
Hello All;

  I need to go through A LOT of Mail Server Log Files.

This is what the string will look like:

20060623 055059 127.0.0.1       SMTPD (002700E8) [210.86.39.32] ERR my-domain.com invalid user
(Of course the my-domain.com, is not Our Domain, just changed it for here.

And I need a way to extract out the BAD IP Address.
Which as you can see is located in the 5th Column   [210.86.39.32]
I need to then put the Bad IP Address's into another TMemo.

Our log files are FULL of Bad IP Address, The code must be very robust.

Thanks all;
Carrzkiss
0
Comment
Question by:Wayne Barron
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
9 Comments
 
LVL 26

Expert Comment

by:Russell Libby
ID: 16973171
Wayne,

You could use the following to extract the ip addr from each line. It does simple verification by checking the string for the following:

- starts with [
- contains numbers
- ends with ]
- has 3 "." chars embedded in the numeric string

Not that it can't be fooled, eg [3..2.22], but it should be fine for your log parsing. If you load your log file into a TStrings or TStringList, then you can just iterate each line and pass it to the function. If no IP addresses are matched in the (line) string, then a blank string will be returned, otherwise the ip address is returned. if you have questions, let me know.

Regards,
Russell

function ExtractIPAddr(Value: String): String;
var  lpszStart:     PChar;
     lpszEnd:       PChar;
     lpszCheck:     PChar;
     dwCount:       Integer;
begin

  // Set default result
  SetLength(result, 0);

  // Check length
  if (Length(Value) > 0) then
  begin
     // Scan for [
     lpszStart:=StrScan(Pointer(Value), '[');
     // Check result
     while Assigned(lpszStart) do
     begin
        // Push past the [
        Inc(lpszStart);
        // Scan for ]
        lpszEnd:=StrScan(lpszStart, ']');
        // Check result
        if Assigned(lpszEnd) then
        begin
           // Set check string
           lpszCheck:=lpszStart;
           // Set counter for .
           dwCount:=0;
           // Check embedded string
           while (lpszCheck < lpszEnd) do
           begin
              // Check for .
              if (lpszCheck^ = '.') then
                 // Increment the counter
                 Inc(dwCount)
              // Check for numerics
              else if not(lpszCheck^ in ['0'..'9']) then
                 // Break, not a numeric
                 break;
              Inc(lpszCheck);
           end;
           // Check last char for numeric
           if (dwCount = 3) and (lpszCheck^ = ']') then
           begin
              // Found result
              SetString(result, lpszStart, lpszEnd-lpszStart);
              // Done processing
              break;
           end;
           // Scan again
           lpszStart:=StrScan(lpszEnd, '[');
        end
        else
           // Can't match
           lpszStart:=nil;
     end;
  end;

end;

  // Example
  ShowMessage(ExtractIPAddr('20060623 055059 127.0.0.1     SMTPD (002700E8) [210.86.39.32] ERR my-domain.com invalid user'));



0
 
LVL 31

Author Comment

by:Wayne Barron
ID: 16973223
Hello Russell;

How would I implement grabbing the IP Address's from between the [] ERR
And listing them in the 2ndMemo.
And not adding in ""Duplicate"" entries?
As the files will have sections where the Entries are going to multiple
invalid users.
(Which is a complete pain, gives us names that we do not want to use :)  )

Thanks
Wayne
0
 
LVL 26

Accepted Solution

by:
Russell Libby earned 1200 total points
ID: 16973272
Example provided, source first, dfm follows:

Russell

---

unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
  StdCtrls;

type
  TForm1 = class(TForm)
    Button1: TButton;
    Memo1: TMemo;
    Memo2: TMemo;
    procedure Button1Click(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation
{$R *.DFM}

function ExtractIPAddr(Value: String): String;
var  lpszStart:     PChar;
     lpszEnd:       PChar;
     lpszCheck:     PChar;
     dwCount:       Integer;
begin

  // Set default result
  SetLength(result, 0);

  // Check length
  if (Length(Value) > 0) then
  begin
     // Scan for [
     lpszStart:=StrScan(Pointer(Value), '[');
     // Check result
     while Assigned(lpszStart) do
     begin
        // Push past the [
        Inc(lpszStart);
        // Scan for ]
        lpszEnd:=StrScan(lpszStart, ']');
        // Check result
        if Assigned(lpszEnd) then
        begin
           // Set check string
           lpszCheck:=lpszStart;
           // Set counter for .
           dwCount:=0;
           // Check embedded string
           while (lpszCheck < lpszEnd) do
           begin
              // Check for .
              if (lpszCheck^ = '.') then
                 // Increment the counter
                 Inc(dwCount)
              // Check for numerics
              else if not(lpszCheck^ in ['0'..'9']) then
                 // Break, not a numeric
                 break;
              Inc(lpszCheck);
           end;
           // Check last char for numeric
           if (dwCount = 3) and (lpszCheck^ = ']') then
           begin
              // Found result
              SetString(result, lpszStart, lpszEnd-lpszStart);
              // Done processing
              break;
           end;
           // Scan again
           lpszStart:=StrScan(lpszEnd, '[');
        end
        else
           // Can't match
           lpszStart:=nil;
     end;
  end;

end;

procedure TForm1.Button1Click(Sender: TObject);
var  listAddrs:     TStringList;
     szIp:          String;
     dwIndex:       Integer;
begin

  Memo1.Lines.BeginUpdate;
  try
     listAddrs:=TStringList.Create;
     try
        listAddrs.Sorted:=True;
        listAddrs.Duplicates:=dupIgnore;
        for dwIndex:=0 to Pred(Memo1.Lines.Count) do
        begin
           szIp:=ExtractIPAddr(Memo1.Lines[dwIndex]);
           if (Length(szIp) > 0) then listAddrs.Add(szIp);
        end;
        Memo2.Lines.Assign(listAddrs);
     finally
        listAddrs.Free;
     end;
  finally
     Memo1.Lines.EndUpdate;
  end;

end;

end.

---

object Form1: TForm1
  Left = 231
  Top = 127
  Width = 724
  Height = 366
  Caption = 'Form1'
  Color = clBtnFace
  Font.Charset = DEFAULT_CHARSET
  Font.Color = clWindowText
  Font.Height = -11
  Font.Name = 'MS Sans Serif'
  Font.Style = []
  OldCreateOrder = False
  PixelsPerInch = 96
  TextHeight = 13
  object Button1: TButton
    Left = 8
    Top = 8
    Width = 75
    Height = 25
    Caption = 'Process'
    TabOrder = 0
    OnClick = Button1Click
  end
  object Memo1: TMemo
    Left = 8
    Top = 40
    Width = 465
    Height = 285
    Lines.Strings = (
      'Hello All;'
      ''
      'I need to go through A LOT of Mail Server Log Files.'
      ''
      'This is what the string will look like:'
      ''
     
        '20060623 055059 127.0.0.1 SMTPD (002700E8) [210.86.39.32] ERR my' +
        '-domain.com invalid '
      'user '
     
        '(Of course the my-domain.com, is not Our Domain, just changed it' +
        ' for here.'
      ''
      'And I need a way to extract out the BAD IP Address.'
      'Which as you can see is located in the 5th Column [210.86.39.32]'
      'I need to then put the Bad IP Address'#39's into another TMemo.'
      ''
     
        'Our log files are FULL of Bad IP Address, The code must be very ' +
        'robust.'
      ''
      'Thanks all;'
      'Carrzkiss'
      'Hello All;'
      ''
      'I need to go through A LOT of Mail Server Log Files.'
      ''
      'This is what the string will look like:'
      ''
     
        '20060623 055059 127.0.0.1 SMTPD (002700E8) [210.86.39.32] ERR my' +
        '-domain.com invalid '
      'user '
     
        '(Of course the my-domain.com, is not Our Domain, just changed it' +
        ' for here.'
      ''
      'And I need a way to extract out the BAD IP Address.'
      'Which as you can see is located in the 5th Column [210.86.39.32]'
      'I need to then put the Bad IP Address'#39's into another TMemo.'
      ''
     
        'Our log files are FULL of Bad IP Address, The code must be very ' +
        'robust.'
      ''
      'Thanks all;'
      'Carrzkiss'
      'Hello All;'
      ''
      'I need to go through A LOT of Mail Server Log Files.'
      ''
      'This is what the string will look like:'
      ''
     
        '20060623 055059 127.0.0.1 SMTPD (002700E8) [210.86.39.32] ERR my' +
        '-domain.com invalid '
      'user '
     
        '(Of course the my-domain.com, is not Our Domain, just changed it' +
        ' for here.'
      ''
      'And I need a way to extract out the BAD IP Address.'
      'Which as you can see is located in the 5th Column [210.86.39.32]'
      'I need to then put the Bad IP Address'#39's into another TMemo.'
      ''
     
        'Our log files are FULL of Bad IP Address, The code must be very ' +
        'robust.'
      ''
      'Thanks all;'
      'Carrzkiss'
      'Hello All;'
      ''
      'I need to go through A LOT of Mail Server Log Files.'
      ''
      'This is what the string will look like:'
      ''
     
        '20060623 055059 127.0.0.1 SMTPD (002700E8) [210.86.39.32] ERR my' +
        '-domain.com invalid '
      'user '
     
        '(Of course the my-domain.com, is not Our Domain, just changed it' +
        ' for here.'
      ''
      'And I need a way to extract out the BAD IP Address.'
      'Which as you can see is located in the 5th Column [210.86.34.32]'
      'I need to then put the Bad IP Address'#39's into another TMemo.'
      ''
     
        'Our log files are FULL of Bad IP Address, The code must be very ' +
        'robust.'
      ''
      'Thanks all;'
      'Carrzkiss')
    ScrollBars = ssVertical
    TabOrder = 1
  end
  object Memo2: TMemo
    Left = 480
    Top = 40
    Width = 225
    Height = 285
    ScrollBars = ssVertical
    TabOrder = 2
  end
end
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 31

Author Comment

by:Wayne Barron
ID: 16973335
That is sweet.
Works like a charm, and list the IP Address in Numerical Order. Which is real cool.

Thank you once again. The code is easy to understand as well.
-------------
After see the way this code works, and how it arranges the IP Address in Numerical Order.
Got me to thinking about a site that I am doing for "Banned IP Address's"
The list is not "Numerically Ordered"

Do you think that you could do something like this?

Here is a segment of the HTML Code.

<tr><th rowspan=1>12.109.221.148  &nbsp;</td><th rowspan=1>&nbsp;255.255.255.255 </td><th rowspan=1><a href="http://www.dnsstuff.com/tools/whois.ch?ip=12.109.221.148"target="_blank">AT&T WorldNet Services</a></td><td><a href="http://att.com/worldnet/" target="_blank">AT&T</a></td></tr>

(Yes, you got it, we get in a LOT of spam)

Basically, you would need to look at the beginning of the line here:   <tr><th rowspan=1>12.109.221.148
And then loop through the entire list and arrange all the Numbers, "Without" corrupting the HTML Line String(s).
As that is what scares me about it, is "If" the code is not done correctly, then it could off-set the HTML Code
And cause it to split into other lines.

The Code above, is 1-Single Line
There is no double lines in the entire site, this is to help keep everything clean and neat for us to work in.

If this is something that you can do, let me know, and I will be more then happy to open another question
And point you to it.

Going to up the points here as well from 125 - 300
Thank you once again Russell.

Wayne
0
 
LVL 26

Expert Comment

by:Russell Libby
ID: 16973627

Probably best to switch to a regular expression parser at this point; and one that I am familiar with (having written it) that includes source can be download from:

http://users.adelphia.net/~rllibby/downloads/regexprex.zip
 
Then the following routine can be applied, just select the desired pattern to use:

uses
  ..., RegExprEx;

// Patterns for both the original and new question
const
  // Pattern for original question
  OldPattern = '\[(:0\d+\.\d+\.\d+.\d+)\] ERR';
  // Pattern for new question
  NewPattern = '<tr><th rowspan=1>(:0\d+\.\d+\.\d+\.\d+)!(</tr>)*</tr>';

var  listAddrs:     TStringList;
begin
 
  Memo1.Lines.BeginUpdate;
  try
     listAddrs:=TStringList.Create;
     try
        listAddrs.Sorted:=True;
        listAddrs.Duplicates:=dupIgnore;
        with TRegExpr.CreatePattern(NewPattern) do
        begin
           try
              Source:=PChar(Memo1.Lines.Text);
              if MatchFirst then
              begin
                 repeat
                    listAddrs.Add(BackReference[0]);
                 until not(MatchNext);
              end;
           finally
              Free;
           end;
        end;
        Memo2.Lines.Assign(listAddrs);
     finally
        listAddrs.Free;
     end;
  finally
     Memo1.Lines.EndUpdate;
  end;
 
end;
0
 
LVL 31

Author Comment

by:Wayne Barron
ID: 16973692
Hello Russell;

This is not quite what I had in mind.
I need to sort through the HTML Code, and Re-arrange (Organize) The IP Address's in Order
From Least to Greatest.
Example

234.54.56.129
125.34.57.123
241.45.23.12
24.345.54.92
221.23.14.15

In the above lines, they are not Ordered. I Would like (But is not a real big deal) to have them ordered
So that the ISP's will all be together, and the IP Address's will be much better to look through, as they are in Order.
So it would look something like this:

24.345.54.92
125.34.57.123
221.23.14.15
234.54.56.129
241.45.23.12
0
 
LVL 31

Author Comment

by:Wayne Barron
ID: 16973696
Sorry, It will still have the HTML Code wrapping it.
The HTML Code has to stay in place, and the [Lines] moved around.
Like a Virtual Washing Machine.
That moves the lines around until they are all Ordered in a Numbered Sequense.

Thank You
Wayne
0
 
LVL 26

Expert Comment

by:Russell Libby
ID: 16973702
So you are talking about modifying the html text so that the <tr> ... </tr> elements are ordered by the IP address specified within them?

Russell
0
 
LVL 31

Author Comment

by:Wayne Barron
ID: 16973711
Correct again, Mr Russell.
0

Featured Post

On Demand Webinar - Networking for the Cloud Era

This webinar discusses:
-Common barriers companies experience when moving to the cloud
-How SD-WAN changes the way we look at networks
-Best practices customers should employ moving forward with cloud migration
-What happens behind the scenes of SteelConnect’s one-click button

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A lot of questions regard threads in Delphi.   One of the more specific questions is how to show progress of the thread.   Updating a progressbar from inside a thread is a mistake. A solution to this would be to send a synchronized message to the…
This article explains how to create forms/units independent of other forms/units object names in a delphi project. Have you ever created a form for user input in a Delphi project and then had the need to have that same form in a other Delphi proj…
NetCrunch network monitor is a highly extensive platform for network monitoring and alert generation. In this video you'll see a live demo of NetCrunch with most notable features explained in a walk-through manner. You'll also get to know the philos…
In this video, Percona Solution Engineer Rick Golba discuss how (and why) you implement high availability in a database environment. To discuss how Percona Consulting can help with your design and architecture needs for your database and infrastr…
Suggested Courses
Course of the Month8 days, 18 hours left to enroll

764 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question