asked on

perl parse webpage

i am trying to count the number of occurances of a the following sentence in the source code of a webpage.



my script looks like this but it just runs and runs, i have to cancel it.

use LWP::Simple;
my $content = get('http://www.website');
my $count= 0;
$count++ while $content =~/\s+changed logic:noMatch to logic:notEqual to prevent similar symbols from displaying together\s/;
print $count;

Adam314

How long do you give it? It might be taking a while to get the webpage.

Add a few print statements so you can see where it hangs... What output do you get from this?

use LWP::Simple;
$|=1;
print "Getting page...\n";
my $content = get('http://www.website');
my $count= 0;
print "Got page: " . length($content) . " bytes\n";
 
print "Checking for line...\n";
$count++ while $content =~/\s+changed logic:noMatch to logic:notEqual to prevent similar symbols from displaying together\s/;
print "count=$count\n";

Open in new window

mcgilljd

ASKER

Getting page...
Got page: 2582373 bytes
Checking for line...

then it hangs

the line should occur about 2100 times

ASKER CERTIFIED SOLUTION

Adam314

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

mcgilljd

ASKER

i changed code alittle to this.

Now it just counts infinetly if the search string is found.

Maybe i need to split content into lines?

print "Checking for line...\n";
while ($content =~/\s+changed logic:noMatch to logic:notEqual to prevent similar symbols from displaying together\s/)
		{
		$count++ ;
			
		print "count=$count\n";
		}

Open in new window

Adam314

The /g should cause it to count properly. If the message is on multiple lines, you might need /s also.