• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 361
  • Last Modified:

[123.234.435.234] err <-- Extract Numbers from between brackets

Hello All;

  I need to go through A LOT of Mail Server Log Files.

This is what the string will look like:

20060623 055059 127.0.0.1       SMTPD (002700E8) [210.86.39.32] ERR my-domain.com invalid user
(Of course the my-domain.com, is not Our Domain, just changed it for here.

And I need a way to extract out the BAD IP Address.
Which as you can see is located in the 5th Column   [210.86.39.32]
I need to then put the Bad IP Address's into another TMemo.

Our log files are FULL of Bad IP Address, The code must be very robust.

Thanks all;
Carrzkiss
0
Wayne Barron
Asked:
Wayne Barron
  • 5
  • 4
1 Solution
 
Russell LibbySoftware Engineer, Advisory Commented:
Wayne,

You could use the following to extract the ip addr from each line. It does simple verification by checking the string for the following:

- starts with [
- contains numbers
- ends with ]
- has 3 "." chars embedded in the numeric string

Not that it can't be fooled, eg [3..2.22], but it should be fine for your log parsing. If you load your log file into a TStrings or TStringList, then you can just iterate each line and pass it to the function. If no IP addresses are matched in the (line) string, then a blank string will be returned, otherwise the ip address is returned. if you have questions, let me know.

Regards,
Russell

function ExtractIPAddr(Value: String): String;
var  lpszStart:     PChar;
     lpszEnd:       PChar;
     lpszCheck:     PChar;
     dwCount:       Integer;
begin

  // Set default result
  SetLength(result, 0);

  // Check length
  if (Length(Value) > 0) then
  begin
     // Scan for [
     lpszStart:=StrScan(Pointer(Value), '[');
     // Check result
     while Assigned(lpszStart) do
     begin
        // Push past the [
        Inc(lpszStart);
        // Scan for ]
        lpszEnd:=StrScan(lpszStart, ']');
        // Check result
        if Assigned(lpszEnd) then
        begin
           // Set check string
           lpszCheck:=lpszStart;
           // Set counter for .
           dwCount:=0;
           // Check embedded string
           while (lpszCheck < lpszEnd) do
           begin
              // Check for .
              if (lpszCheck^ = '.') then
                 // Increment the counter
                 Inc(dwCount)
              // Check for numerics
              else if not(lpszCheck^ in ['0'..'9']) then
                 // Break, not a numeric
                 break;
              Inc(lpszCheck);
           end;
           // Check last char for numeric
           if (dwCount = 3) and (lpszCheck^ = ']') then
           begin
              // Found result
              SetString(result, lpszStart, lpszEnd-lpszStart);
              // Done processing
              break;
           end;
           // Scan again
           lpszStart:=StrScan(lpszEnd, '[');
        end
        else
           // Can't match
           lpszStart:=nil;
     end;
  end;

end;

  // Example
  ShowMessage(ExtractIPAddr('20060623 055059 127.0.0.1     SMTPD (002700E8) [210.86.39.32] ERR my-domain.com invalid user'));



0
 
Wayne BarronAuthor, Web DeveloperAuthor Commented:
Hello Russell;

How would I implement grabbing the IP Address's from between the [] ERR
And listing them in the 2ndMemo.
And not adding in ""Duplicate"" entries?
As the files will have sections where the Entries are going to multiple
invalid users.
(Which is a complete pain, gives us names that we do not want to use :)  )

Thanks
Wayne
0
 
Russell LibbySoftware Engineer, Advisory Commented:
Example provided, source first, dfm follows:

Russell

---

unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
  StdCtrls;

type
  TForm1 = class(TForm)
    Button1: TButton;
    Memo1: TMemo;
    Memo2: TMemo;
    procedure Button1Click(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation
{$R *.DFM}

function ExtractIPAddr(Value: String): String;
var  lpszStart:     PChar;
     lpszEnd:       PChar;
     lpszCheck:     PChar;
     dwCount:       Integer;
begin

  // Set default result
  SetLength(result, 0);

  // Check length
  if (Length(Value) > 0) then
  begin
     // Scan for [
     lpszStart:=StrScan(Pointer(Value), '[');
     // Check result
     while Assigned(lpszStart) do
     begin
        // Push past the [
        Inc(lpszStart);
        // Scan for ]
        lpszEnd:=StrScan(lpszStart, ']');
        // Check result
        if Assigned(lpszEnd) then
        begin
           // Set check string
           lpszCheck:=lpszStart;
           // Set counter for .
           dwCount:=0;
           // Check embedded string
           while (lpszCheck < lpszEnd) do
           begin
              // Check for .
              if (lpszCheck^ = '.') then
                 // Increment the counter
                 Inc(dwCount)
              // Check for numerics
              else if not(lpszCheck^ in ['0'..'9']) then
                 // Break, not a numeric
                 break;
              Inc(lpszCheck);
           end;
           // Check last char for numeric
           if (dwCount = 3) and (lpszCheck^ = ']') then
           begin
              // Found result
              SetString(result, lpszStart, lpszEnd-lpszStart);
              // Done processing
              break;
           end;
           // Scan again
           lpszStart:=StrScan(lpszEnd, '[');
        end
        else
           // Can't match
           lpszStart:=nil;
     end;
  end;

end;

procedure TForm1.Button1Click(Sender: TObject);
var  listAddrs:     TStringList;
     szIp:          String;
     dwIndex:       Integer;
begin

  Memo1.Lines.BeginUpdate;
  try
     listAddrs:=TStringList.Create;
     try
        listAddrs.Sorted:=True;
        listAddrs.Duplicates:=dupIgnore;
        for dwIndex:=0 to Pred(Memo1.Lines.Count) do
        begin
           szIp:=ExtractIPAddr(Memo1.Lines[dwIndex]);
           if (Length(szIp) > 0) then listAddrs.Add(szIp);
        end;
        Memo2.Lines.Assign(listAddrs);
     finally
        listAddrs.Free;
     end;
  finally
     Memo1.Lines.EndUpdate;
  end;

end;

end.

---

object Form1: TForm1
  Left = 231
  Top = 127
  Width = 724
  Height = 366
  Caption = 'Form1'
  Color = clBtnFace
  Font.Charset = DEFAULT_CHARSET
  Font.Color = clWindowText
  Font.Height = -11
  Font.Name = 'MS Sans Serif'
  Font.Style = []
  OldCreateOrder = False
  PixelsPerInch = 96
  TextHeight = 13
  object Button1: TButton
    Left = 8
    Top = 8
    Width = 75
    Height = 25
    Caption = 'Process'
    TabOrder = 0
    OnClick = Button1Click
  end
  object Memo1: TMemo
    Left = 8
    Top = 40
    Width = 465
    Height = 285
    Lines.Strings = (
      'Hello All;'
      ''
      'I need to go through A LOT of Mail Server Log Files.'
      ''
      'This is what the string will look like:'
      ''
     
        '20060623 055059 127.0.0.1 SMTPD (002700E8) [210.86.39.32] ERR my' +
        '-domain.com invalid '
      'user '
     
        '(Of course the my-domain.com, is not Our Domain, just changed it' +
        ' for here.'
      ''
      'And I need a way to extract out the BAD IP Address.'
      'Which as you can see is located in the 5th Column [210.86.39.32]'
      'I need to then put the Bad IP Address'#39's into another TMemo.'
      ''
     
        'Our log files are FULL of Bad IP Address, The code must be very ' +
        'robust.'
      ''
      'Thanks all;'
      'Carrzkiss'
      'Hello All;'
      ''
      'I need to go through A LOT of Mail Server Log Files.'
      ''
      'This is what the string will look like:'
      ''
     
        '20060623 055059 127.0.0.1 SMTPD (002700E8) [210.86.39.32] ERR my' +
        '-domain.com invalid '
      'user '
     
        '(Of course the my-domain.com, is not Our Domain, just changed it' +
        ' for here.'
      ''
      'And I need a way to extract out the BAD IP Address.'
      'Which as you can see is located in the 5th Column [210.86.39.32]'
      'I need to then put the Bad IP Address'#39's into another TMemo.'
      ''
     
        'Our log files are FULL of Bad IP Address, The code must be very ' +
        'robust.'
      ''
      'Thanks all;'
      'Carrzkiss'
      'Hello All;'
      ''
      'I need to go through A LOT of Mail Server Log Files.'
      ''
      'This is what the string will look like:'
      ''
     
        '20060623 055059 127.0.0.1 SMTPD (002700E8) [210.86.39.32] ERR my' +
        '-domain.com invalid '
      'user '
     
        '(Of course the my-domain.com, is not Our Domain, just changed it' +
        ' for here.'
      ''
      'And I need a way to extract out the BAD IP Address.'
      'Which as you can see is located in the 5th Column [210.86.39.32]'
      'I need to then put the Bad IP Address'#39's into another TMemo.'
      ''
     
        'Our log files are FULL of Bad IP Address, The code must be very ' +
        'robust.'
      ''
      'Thanks all;'
      'Carrzkiss'
      'Hello All;'
      ''
      'I need to go through A LOT of Mail Server Log Files.'
      ''
      'This is what the string will look like:'
      ''
     
        '20060623 055059 127.0.0.1 SMTPD (002700E8) [210.86.39.32] ERR my' +
        '-domain.com invalid '
      'user '
     
        '(Of course the my-domain.com, is not Our Domain, just changed it' +
        ' for here.'
      ''
      'And I need a way to extract out the BAD IP Address.'
      'Which as you can see is located in the 5th Column [210.86.34.32]'
      'I need to then put the Bad IP Address'#39's into another TMemo.'
      ''
     
        'Our log files are FULL of Bad IP Address, The code must be very ' +
        'robust.'
      ''
      'Thanks all;'
      'Carrzkiss')
    ScrollBars = ssVertical
    TabOrder = 1
  end
  object Memo2: TMemo
    Left = 480
    Top = 40
    Width = 225
    Height = 285
    ScrollBars = ssVertical
    TabOrder = 2
  end
end
0
Cloud Class® Course: Microsoft Azure 2017

Azure has a changed a lot since it was originally introduce by adding new services and features. Do you know everything you need to about Azure? This course will teach you about the Azure App Service, monitoring and application insights, DevOps, and Team Services.

 
Wayne BarronAuthor, Web DeveloperAuthor Commented:
That is sweet.
Works like a charm, and list the IP Address in Numerical Order. Which is real cool.

Thank you once again. The code is easy to understand as well.
-------------
After see the way this code works, and how it arranges the IP Address in Numerical Order.
Got me to thinking about a site that I am doing for "Banned IP Address's"
The list is not "Numerically Ordered"

Do you think that you could do something like this?

Here is a segment of the HTML Code.

<tr><th rowspan=1>12.109.221.148  &nbsp;</td><th rowspan=1>&nbsp;255.255.255.255 </td><th rowspan=1><a href="http://www.dnsstuff.com/tools/whois.ch?ip=12.109.221.148"target="_blank">AT&T WorldNet Services</a></td><td><a href="http://att.com/worldnet/" target="_blank">AT&T</a></td></tr>

(Yes, you got it, we get in a LOT of spam)

Basically, you would need to look at the beginning of the line here:   <tr><th rowspan=1>12.109.221.148
And then loop through the entire list and arrange all the Numbers, "Without" corrupting the HTML Line String(s).
As that is what scares me about it, is "If" the code is not done correctly, then it could off-set the HTML Code
And cause it to split into other lines.

The Code above, is 1-Single Line
There is no double lines in the entire site, this is to help keep everything clean and neat for us to work in.

If this is something that you can do, let me know, and I will be more then happy to open another question
And point you to it.

Going to up the points here as well from 125 - 300
Thank you once again Russell.

Wayne
0
 
Russell LibbySoftware Engineer, Advisory Commented:

Probably best to switch to a regular expression parser at this point; and one that I am familiar with (having written it) that includes source can be download from:

http://users.adelphia.net/~rllibby/downloads/regexprex.zip
 
Then the following routine can be applied, just select the desired pattern to use:

uses
  ..., RegExprEx;

// Patterns for both the original and new question
const
  // Pattern for original question
  OldPattern = '\[(:0\d+\.\d+\.\d+.\d+)\] ERR';
  // Pattern for new question
  NewPattern = '<tr><th rowspan=1>(:0\d+\.\d+\.\d+\.\d+)!(</tr>)*</tr>';

var  listAddrs:     TStringList;
begin
 
  Memo1.Lines.BeginUpdate;
  try
     listAddrs:=TStringList.Create;
     try
        listAddrs.Sorted:=True;
        listAddrs.Duplicates:=dupIgnore;
        with TRegExpr.CreatePattern(NewPattern) do
        begin
           try
              Source:=PChar(Memo1.Lines.Text);
              if MatchFirst then
              begin
                 repeat
                    listAddrs.Add(BackReference[0]);
                 until not(MatchNext);
              end;
           finally
              Free;
           end;
        end;
        Memo2.Lines.Assign(listAddrs);
     finally
        listAddrs.Free;
     end;
  finally
     Memo1.Lines.EndUpdate;
  end;
 
end;
0
 
Wayne BarronAuthor, Web DeveloperAuthor Commented:
Hello Russell;

This is not quite what I had in mind.
I need to sort through the HTML Code, and Re-arrange (Organize) The IP Address's in Order
From Least to Greatest.
Example

234.54.56.129
125.34.57.123
241.45.23.12
24.345.54.92
221.23.14.15

In the above lines, they are not Ordered. I Would like (But is not a real big deal) to have them ordered
So that the ISP's will all be together, and the IP Address's will be much better to look through, as they are in Order.
So it would look something like this:

24.345.54.92
125.34.57.123
221.23.14.15
234.54.56.129
241.45.23.12
0
 
Wayne BarronAuthor, Web DeveloperAuthor Commented:
Sorry, It will still have the HTML Code wrapping it.
The HTML Code has to stay in place, and the [Lines] moved around.
Like a Virtual Washing Machine.
That moves the lines around until they are all Ordered in a Numbered Sequense.

Thank You
Wayne
0
 
Russell LibbySoftware Engineer, Advisory Commented:
So you are talking about modifying the html text so that the <tr> ... </tr> elements are ordered by the IP address specified within them?

Russell
0
 
Wayne BarronAuthor, Web DeveloperAuthor Commented:
Correct again, Mr Russell.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Cloud Class® Course: CompTIA Healthcare IT Tech

This course will help prep you to earn the CompTIA Healthcare IT Technician certification showing that you have the knowledge and skills needed to succeed in installing, managing, and troubleshooting IT systems in medical and clinical settings.

  • 5
  • 4
Tackle projects and never again get stuck behind a technical roadblock.
Join Now