We help IT Professionals succeed at work.
Get Started

Calculate Statistics From Log file

uluttrell asked
Last Modified: 2013-11-15
This is not a homework assignment.  

I have log files for multiple servers that are kept in a repository.  The log file's filename format is application.servername.timestampinYYYYMMDD.log.

Each of the log files in the directory has the following format:

YYYYMMDD Time-TimeZone    Servername Application PID;  MeasuredStatistic          (XX/YY)  Value for TimeStamp
20030817 000216010-0400 servername application 4567;MeasuredStatisticOne(12/18) 5217
20030817 000216110-0400 servername application 4567;MeasuredStatisticTwo(12/14) 419276
20030817 000216110-0400 servername application 4567;MeasuredStatisticThree(12/31) 57912
20030817 000216110-0400 servername application 4567;MeasuredStatisticFour(12/12) 72
20030817 000216110-0400 servername application 4567;MeasuredStatisticFive(12/13) 1451718

The log files roll over at midnight.  Each statistic is measured at a random interval.

I am trying to write a perl script that will do the following:
For Each server
*determine the number of times that a given statistic appears on a day for a server.  Use that number to calculate the average for the MeasuredStatistic for the day.
* sum the days to determine the totals and averages for the week for a server.
* determine the average for all statics for all servers and export to a csv file to be used in an Excel spreadsheet.
* determine totals for all servers and export to a csv file to be used in an Excel spreadsheet.

I have written the following code, but it is not producing the desired results.
=====Begin code.pl
#! /usr/bin/perl

%module_count = ();
%module_sum = ();

while (<>) {
       next if (/^\s*$/);

       my ($date, $time, $host, $server, $pid, $metric, $value) = split(/\s/);

       $module_sum{$module} += $percent;

foreach $module (sort keys %module_count) {
       printf "%s %dx average is %d%%\n",
               $module_sum{$module} / $module_count{$module};
=====End code.pl

How would I script this properly in perl?

Watch Question
This problem has been solved!
Unlock 2 Answers and 16 Comments.
See Answers
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE