Search in Files PHP

purpleoak
purpleoak used Ask the Experts™
on
I have a large amount of mail server logs which i need to be able to search through. These logs can be up to 25MB a day.

What i would like to be able to do is enter a search string I.E the from address and search through the file and output to screen the entire line that the string occurs on. I would like to do this is PHP.

With Regards

Ashley Hay-Ellis
Purple Oak
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
the following script reads the file line by line and searches the text stored in the $search variable in each line. if a metch is found then the line is displayes on the screen

hope it's what you need


<?php
 
$search="String to search";
 
$handle = @fopen("/tmp/inputfile.txt", "r");
if ($handle) {
    while (!feof($handle)) {
        $buffer = fgets($handle, 4096);
        
	if (substr_count($text, $search)>0 ) echo "<br>".$buffer;
 
    }
    fclose($handle);
}
?> 

Open in new window

you can use the set_time_limit(300); in line 9 if you get an error that the script executed more than 30 seconds witch is the default time for a script to run.
Commented:
I agree racmail, but you need to change $text to $buffer line 10.
Rowby Goren Makes an Impact on Screen and Online

Learn about longtime user Rowby Goren and his great contributions to the site. We explore his method for posing questions that are likely to yield a solution, and take a look at how his career transformed from a Hollywood writer to a website entrepreneur.

Try this
$search = 'Search String';
 
foreach (glob("*.txt") as $filename)
{
  $lines = file($filename)
  foreach ($lines as $line)
    if (substr_count($line, $search) > 0 ) echo "<br>".$line;
}

Open in new window

Commented:
If you have performance issues with very big files, you can also use fgrep under linux, like this :


$ret = 0;
$output = array();
exec("fgrep \"$query\" /home/www/test.log",$output,$code);
print_r($output);

Open in new window

Commented:
Graceful_Penguin> it will probably fail because php will try to load the entire file into memory.
@flob9 Yeah you are correct,
new version:

$search = 'Search String';
 
foreach (glob("*.txt") as $filename)
{
  $handle = @fopen($filename, "r");
  if ($handle) {
    while (!feof($handle)) {
        $buffer = fgets($handle, 4096);
	if (substr_count($buffer , $search)>0 ) echo "<br>".$buffer;
    }
    fclose($handle);
}

Open in new window

@flob9: thanks for that point - yes in line 10 it suld be $buffer

and yes for that reason i used the line by line reading of the file - with big files (over 500mb ) i had isues with memory web trying to save a backup with php

Author

Commented:
Exactly what I need. Works fast as well with all the log files cheers

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial