Find a IP address from txt files

Hey Guys,

I have some IIS log files, and would like to extract all the ip addresses from those, excluding 1 IP, which is the servers IP.

I am using agent ransack, but can't find a way to do this..
I am not even sure what is the best category of this question.

Please help.
Thanks
LVL 1
ShivtekAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Patrick MatthewsCommented:
The following is VBScript:


Dim fso, tsIn, tsOut, RegX, Mats, dic, ExcludeIP, wholething, Counter, arr

Set fso = CreateObject("Scripting.FileSystemObject")
Set tsIn = fso.OpenTextFile("c:\log.txt")
Set tsOut = fso.CreateTextFile("c:\IPs.txt")
Set RegX = New RegExp
Set dic = CreateObject("Scripting.Dictionary")

ExcludeIP = "1.2.3.4"

wholething = tsIn.ReadAll
tsIn.Close
Set tsIn = Nothing

With RegX
    .Pattern = "\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}"
    .Global = True
    Set Mats = .Execute
    For Counter = 1 To Mats.Count
        If Mats(Counter - 1) <> ExcludeIP Then
            If Not dic.Exists(Mats(Counter - 1)) Then
                dic.Add Mats(Counter - 1), Mats(Counter - 1)
            End If
        End If
    Next
    Set Mats = Nothing
End With
Set RegX = Nothing

arr = dic.Keys
For Counter = 0 To UBound(arr)
    tsOut.WriteLine dic.Item(arr(counter))
Next
tsOut.Close
Set tsOut = Nothing
Set fso = Nothing
Set dic = Nothing

MsgBox "Done"

Open in new window

0
wilcoxonCommented:
It's pretty easy to do in perl...
use strict;
use warnings;

my $byte = '(?:\d\d?|1\d\d|2[0-4]\d|25[0-5])';
my $ip = "^(?:$byte\.){3}$byte$";

# IP address to exclude
my $exclude = '123.123.123.123';

# global to store IPs
my %ips;

# arguments are the names of the log files - loop over them
foreach my $fil (@ARGV) {
    process($fil);
}

# do whatever you want with the IPs - this just prints them out
foreach my $ip (sort keys %ips) {
    print "saw $ip $ips{$ip} times\n";
}

sub process {
    my ($fil) = @_;
    open IN, '<', $fil or die "could not read $fil: $!";
    while (<IN>) {
        chomp;
        # while to pick up multiples on one line (if possible)
        while (s{((?:$byte\.){3}$byte)\b}{}) {
            $ips{$1}++;
        }
    }
    close IN;
}

Open in new window

0
wilcoxonCommented:
Oops.  I set $exclude but then didn't check it.

Change line 30 to:

$ips{$1}++ if ($1 ne $exclude);
0
Cloud Class® Course: Python 3 Fundamentals

This course will teach participants about installing and configuring Python, syntax, importing, statements, types, strings, booleans, files, lists, tuples, comprehensions, functions, and classes.

ShivtekAuthor Commented:
Hi Thank you.

But how do I run it so it looks in all files in a folder
0
Patrick MatthewsCommented:
Trying again...



Dim fso, fld, fil, tsIn, tsOut, RegX, Mats, dic, ExcludeIP, wholething, Counter, arr

Set fso = CreateObject("Scripting.FileSystemObject")
Set tsOut = fso.CreateTextFile("c:\IPs.txt")
Set RegX = New RegExp
Set dic = CreateObject("Scripting.Dictionary")

ExcludeIP = "1.2.3.4"

With RegX
    .Pattern = "\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}"
    .Global = True
End With

Set fld = fso.GetFolder("c:\Logs")
For Each fil In fld.Files
    If UCase(Right(fil.Name, 4)) = ".TXT" Then
        Set tsIn = fso.OpenTextFile(fil.Path)
        wholething = tsIn.ReadAll
        tsIn.Close
        Set Mats = RegX.Execute(wholething)
        For Counter = 1 To Mats.Count
            If Mats(Counter - 1) <> ExcludeIP Then
                If Not dic.Exists(Mats(Counter - 1)) Then
                    dic.Add Mats(Counter - 1), Mats(Counter - 1)
                End If
            End If
        Next
    End If
Next

Set Mats = Nothing
Set tsIn = Nothing
Set RegX = Nothing

arr = dic.Keys
For Counter = 0 To UBound(arr)
    tsOut.WriteLine dic.Item(arr(counter))
Next
tsOut.Close
Set tsOut = Nothing
Set fso = Nothing
Set dic = Nothing

MsgBox "Done"

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
ShivtekAuthor Commented:
How would I run this script though, I am not sure how VB scripts are run
0
ShivtekAuthor Commented:
I saved it as a vbs file, and changed the location of the folder...but it exports a black txt file though.

Also the files we are searching are .log files.
0
ShivtekAuthor Commented:
blank txt file I mean
0
Patrick MatthewsCommented:
You'll need to update:

ExcludeIP = "1.2.3.4"

to whatever the "exclude IP" value is, and update:

    If UCase(Right(fil.Name, 4)) = ".TXT" Then

to reflect the .log extension:

    If UCase(Right(fil.Name, 4)) = ".LOG" Then
0
ShivtekAuthor Commented:
.LOG / .TXT wasn't uppercase when I changed to .log.

Thank You.
It worked though!!

Just wondering now, how can I pick all the unique IP's from that exported list.
0
ShivtekAuthor Commented:
Used Excel for finding unique records.!!
0
wilcoxonCommented:
The perl script I put above already gives the unique IPs and a count of the times they show up in the logs.

Here's a modified version of the perl that incorporates my minor bug/fix and to go through all files in the current directory named *.log (with any capitalization of .log).

Also, curious if there's a reason you didn't comment on the perl solution at all?
use strict;
use warnings;

my $byte = '(?:\d\d?|1\d\d|2[0-4]\d|25[0-5])';

# IP address to exclude
my $exclude = '123.123.123.123';

# global to store IPs
my %ips;

# changed to loop over all logfiles in current dir
opendir DIR, '.' or die "could not read dir: $!";
my @files = grep /\.log$/i, readdir DIR;
closedir DIR;
foreach my $fil (@files) {
    process($fil);
}

# do whatever you want with the IPs - this just prints them out
foreach my $ip (sort keys %ips) {
    print "saw $ip $ips{$ip} times\n";
}

sub process {
    my ($fil) = @_;
    open IN, '<', $fil or die "could not read $fil: $!";
    while (<IN>) {
        chomp;
        # while to pick up multiples on one line (if possible)
        while (s{((?:$byte\.){3}$byte)\b}{}) {
            $ips{$1}++ if ($1 ne $exclude);
        }
    }
    close IN;
}

Open in new window

0
ShivtekAuthor Commented:
I actually didn't know how would I run this script.
0
wilcoxonCommented:
Save the file as scriptname.pl and then you just run it as:

perl scriptname.pl

or simply double-click on scriptname.pl if the filetype is registered on your machine.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Batch

From novice to tech pro — start learning today.