Solved

Perl hash help with parsing apache logs

Posted on 2011-03-05
5
463 Views
Last Modified: 2012-05-11
Hello,
I am reading an apache file and parsing the data.
I need to display number of accesses per hostname, number of accesses, and
a percentage of the total accesses that each host accounted for as follows.

I am having a problem calculating the percentage ot total access.

Thanks in advance

   Hits   %-age    Resource
 -----      -----        -----
     7       1            h10.163.23.98.static.ip.windstream.net
     6       1            ip98-179-8-48.om.om.cox.net
     4       1            ip98-168-193-160.om.om.cox.net
     3       1            ip68-110-22-151.om.om.cox.net
 
#reading in file
my ($file) = @ARGV;
open (LOG, $file);
 
 my ($host,$date,$method,$urls,$httpver,$status,$size,$referrer,$agent);
 
#hash for hits
my %Hits;

while ( my $line=<LOG>) {
   ($host,$date,$method,$urls,$httpver,$status,$size,$referrer,$agent) = $line =~
          m/^(\S+) - - \[(\S+ [\-|\+]\d{4})\] "(\S+) (\S+) ([^"]+)" (\d{3}) (\d+|-) "(.*?)" "([^"]+)"$/;

 #Counting number of hits per host, &Hnames is a subroutine that calls $host and does a reverse dns lookup 
$Hits{&Hnames}++

}
 
 
#------------------------------
       print "=" x 78,"\n";
       print "HOSTNAMES\n";
       print "=" x 78,"\n";
       printf "%6s %4s %s\n", "Hits", "%-age", "Recourse";
       printf "%6s %4s %s\n", "-----", "-----","-----";

# Sorting on hits high -> low
foreach my $key ( sort { $Hits{ $b } <=> $Hits{ $a } } (keys %Hits) ) {
     
        my $num += $Hits{$key};
        my $perc = $Hits{$key}/$num;
      
        printf "%6d %4d %5s\n", $Hits{ $key }, $perc, $key;
    
}

Open in new window

0
Comment
Question by:fac66
  • 3
5 Comments
 
LVL 26

Accepted Solution

by:
wilcoxon earned 500 total points
ID: 35044677
This should do what you want.  If not, let me know where you are seeing an issue...

The problem is that you need the total number of hits prior to looping through the keys to do the output.
#reading in file
my ($file) = @ARGV;
open (LOG, $file);
 
 my ($host,$date,$method,$urls,$httpver,$status,$size,$referrer,$agent);
 
#hash for hits
my (%Hits, $ttl);

while ( my $line=<LOG>) {
   ($host,$date,$method,$urls,$httpver,$status,$size,$referrer,$agent) = $line =~
          m/^(\S+) - - \[(\S+ [\-|\+]\d{4})\] "(\S+) (\S+) ([^"]+)" (\d{3}) (\d+|-) "(.*?)" "([^"]+)"$/;

 #Counting number of hits per host, &Hnames is a subroutine that calls $host and does a reverse dns lookup 
$Hits{&Hnames}++
$ttl++;

}
 
 
#------------------------------
       print "=" x 78,"\n";
       print "HOSTNAMES\n";
       print "=" x 78,"\n";
       printf "%6s %4s %s\n", "Hits", "%-age", "Recourse";
       printf "%6s %4s %s\n", "-----", "-----","-----";

# Sorting on hits high -> low
foreach my $key ( sort { $Hits{ $b } <=> $Hits{ $a } } (keys %Hits) ) {
     
        my $perc = $Hits{$key}/$ttl;
      
        printf "%6d %4d %5s\n", $Hits{ $key }, $perc, $key;
    
}

Open in new window

0
 

Author Comment

by:fac66
ID: 35044817
Thanks for the response!

This is what i get..

Hits   %-age   Resource
 -----   -----     -----
     7     0      h10.163.23.98.static.ip.windstream.net
     6     0      ip98-179-8-48.om.om.cox.net
     4     0      ip98-168-193-160.om.om.cox.net
0
 

Author Comment

by:fac66
ID: 35044850
Actually it does work:

=============================================================================
  Hits   %-age    Resource
 -----   -----    -----
     7   0.69   h10.163.23.98.static.ip.windstream.net
     6   0.59   ip98-179-8-48.om.om.cox.net
     4   0.40   ip98-168-193-160.om.om.cox.net
     3   0.30   ip68-110-22-151.om.om.cox.net

I had to chage the printf to reflect floating point.

printf "%6d %4.2f %5s\n", $Hits{ $key }, $perc, $key;

Open in new window

0
 

Author Comment

by:fac66
ID: 35044868

One other question please..

Is there a way I can get the same results but sorting alphabetically?
I would have to sort by the value rather than key.
What I have tried results in losing the hits count.
# Sorting on hits high -> low
foreach my $key ( sort { $Hits{ $b } <=> $Hits{ $a } } (keys %Hits) ) {     
        my $perc = $Hits{$key}/$ttl;     
        printf "%6d %4d %5s\n", $Hits{ $key }, $perc, $key;    
}

Open in new window

0
 
LVL 84

Expert Comment

by:ozo
ID: 35045231
foreach my $key ( sort keys %Hits ) {
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

On Microsoft Windows, if  when you click or type the name of a .pl file, you get an error "is not recognized as an internal or external command, operable program or batch file", then this means you do not have the .pl file extension associated with …
Whatever be the reason, if you are working on web development side,  you will need day-today validation codes like email validation, date validation , IP address validation, phone validation on any of the edit page or say at the time of registration…
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

920 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now