Solved

Parsing logs

Posted on 2003-11-18
4
272 Views
Last Modified: 2006-11-17
I'm in the middle of making a PHP script that will parse FTP logs and then add the unique connects, etc into my MySQL DB.

Let's say I have a FTP_logs directory, and the files are labeled as such (also look in subfolders to find these files)...

11072003.log
11082003.log
11092003.log

How would I tell PHP to loop through all those logs, and once PHP has opened the file, find a string like the following:

L 11/16/2003 - 12:07:26: "USERNAME<164><USERNAME_VALIDATING><>" connected, address "209.128.68.226:2767"

Get the address...in my case the IP address of the remote connection, and store it into $connectors_ip, in this case, 209.128.68.205, not the 2767. Also get the USERNAME and store it into $connectors_username.

Then, look for the following through the file,

L 11/16/2003 - 12:07:34: <USERID_0:0:347764><>" USERID validated

Where I need to just get the value after USERID_, in this case 0:0:347764, store it into $connectors_USERID.

Thanks in advance.
0
Comment
Question by:drakkarnoir
  • 2
4 Comments
 
LVL 4

Expert Comment

by:Fendrin
ID: 9781554
It seems to me that you need to use regular expressions. Here is a basic algorithm:

1) Open the file (fopen)
2) get the contents as a string (fread)
3) close the file (fclose)
4) get the first string (ereg or preg_match)
5) get the IP address from the first string (ereg or preg_match)
6) get the Username from the first string (ereg or preg_match)
7) get the second string (ereg or preg_match)
8) get the userid from the second string (ereg or preg_match)

repeat this for each file.
0
 

Author Comment

by:drakkarnoir
ID: 9782182
But how would I tell PHP to go through each file? Even in the sub-directories?

Something like the following: ?

while(dir(EOF();)) ?
0
 
LVL 4

Accepted Solution

by:
bljak earned 500 total points
ID: 9785798
No, you would list all files in directory add them to some kind of array, and loop through array, something like this

function file_list($dir,$extension)
{
  if(is_dir($dir))
  {
    $fd = @opendir($dir);
    while(($part = @readdir($fd)))
    {
      clearstatcache();
      if ($part != "." && $part != ".." && !is_dir($dir."/".$part) && eregi("\.".$extension."$",$part))
      {
        $files[] = $part;
      }
    }
  if($fd)
  {
    closedir($fd);
  }
  if(is_array($files))
  {
    asort($files);
    return $files;
  } else {
    return false;
  }
  } else {
    return false;
  }
}

This should return array, so if you do

$logs = file_list("/my/path", "log");

you should have $log array with all your file names in it, and then you can do what you want
as for the subdirectories, you could add check somewhere in function after if check
elseif($part != "." && $part != ".." && is_dir($dir."/".$part))
{
  file_list($dir."/".$part,$extension)
}

so long
bljak
0
 

Author Comment

by:drakkarnoir
ID: 9806899
Erm so how would I tell PHP to loop through all the files in just a particular directory and parsing their information...?
0

Featured Post

Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

Join & Write a Comment

Deprecated and Headed for the Dustbin By now, you have probably heard that some PHP features, while convenient, can also cause PHP security problems.  This article discusses one of those, called register_globals.  It is a thing you do not want.  …
Foreword (July, 2015) Since I first wrote this article, years ago, a great many more people have begun using the internet.  They are coming online from every part of the globe, learning, reading, shopping and spending money at an ever-increasing ra…
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now