asked on

grabbing HTML content of a webpage

does anybody have the script for grabbing or fetching (whatever you call it) the HTML content of a webpage?
i need it urgently!!!
thanks a lot guys

maneshr

mikarei,

"..g or fetching (whatever you call it) the HTML content of a webpage? i need it urgently!!!.."

Here is a script that will work for you, provide you have all the Perl modules installed & you are not connecting from behind the firewall.

#!/usr/local/bin/perl

$|++;
use CGI::Carp qw(fatalsToBrowser);
use CGI qw(:cgi);
$in = new CGI;

use LWP::UserAgent;
$ua = LWP::UserAgent->new;
use HTTP::Request::Common;

&head('ClinicsToGo');
&blah;
&footer;
exit;

sub head {
print $in->header('text/html');
}

sub blah {
$url = 'http://www.hotmail.com';
$response = $ua->request(GET $url);
$html = $response->content;
print $html;
}

sub footer {
print <<Footer;

</body>
</html>
Footer
}

mikarei

ASKER

so, i just copy what you gave me and execute it?

mikarei

ASKER

so, i just copy what you gave me and execute it?

mikarei

ASKER

maneshr ..

i've tried ur script ..
it's not working :(

mikarei

ASKER

maneshr ..

i've tried ur script ..
it's not working :(

ASKER CERTIFIED SOLUTION

allym

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

ahoffmann

lynx -dump <your URL>
wget <your URL>

maneshr

mikarei,

"..i've tried ur script ..
it's not working :( .."

What exactly does/does not happen? Do you get an error? If so, what is the exact error that you get?

What OS are you on? What web server are you using? Is there a URL where i can see this script?

How did you run this script (as a CGI program, or from the command line)?

Please provide as much more detail as you can.

This will help you get a more accurate answer, faster.

Netminder

Netminder

Admin notified of user neglect. Force/accepted by

Netminder
Community Support Moderator
Experts Exchange

mbormann

ahoffman:

Thanks for your help I used wget for windows to retrieve a web page.

ahoffmann

mbormann,
so wget was a solution to your question?
then please tell support (Netminder) @ EE about the correction (even allym's solution might work as well).

Netminder

ahoffmann,

Okay, so I'm not perfect... <grin>

Points for you at https://www.experts-exchange.com/jsp/qShow.jsp?ta=perl&qid=20307277

Netminder
CS Moderator