asked on

LWP - Get a url which is redirected

How do I fetch a document if the url redirects to another site for authentication and then redirects back to it?

In this case, how do I sign in onto a redirected website, and fetch the document of the provided link when authorization succeeded and brought back to the target link?

Thanks,
CJ

sosolala

Depence on the autentication methode

you are asking for a very technical answer

but your question doesn;t contain any technical methode

but If you can do it in a browser you can script it

Most probally you have to perseve the cookie.
in you script...
Gtz

SegFault

If the method is using cookies, you can use the CookieJar module, which plugs into LWP.

I had just this situation, however, the CookieJar was using version 2.0 of cookies and the site wanted version 1.0, so I had to break out etherreal and sniff the packets. It was a hack job, but it worked.

Take a look at the documentation for implementing a cookiejar.

If the site is not using cookies, we'd need to know specifically what authentication mechanism is being used. If you can provide some amplifying information, it would help.

seg

sosolala

Below a sample script that logs in to yahoo
to login to a second site...add under the coresponding
$req2 that looks like the $req pointing to a second site
$res2 that looks like the $res contains then the second html

this way you can login to 2 sites..and use the same cookie.
now sometimes you have a box that that pops up requesting the login and passwd...

Then there is a different autentication methode..needed
used let me know and I post it :-) hope this will answer you question.!

#!/usr/bin/perl

my login = "loginname";
my passwd = "passwrd";

use LWP::UserAgent;
use HTTP::Cookies;

$ua = LWP::UserAgent->new;
$ua->cookie_jar(HTTP::Cookies->new(file => "lwpcookies.txt",
autosave => 1));

$ua->agent("$0/0.1 " . $ua->agent);
# $ua->agent("Mozilla/8.0") # pretend we are very capable browser

$req = HTTP::Request->new(GET => 'http://login.yahoo.com/config/login?login=$login&passwd=$passwd&.src=quote&.done=http://finance.yahoo.com');
$req->header('Accept' => 'text/html');

# send request
$res = $ua->request($req);

# check the outcome
if ($res->is_success) {
#print $res->content;
} else {
print "Error: " . $res->status_line . "\n";
}

CaitlinJ

ASKER

so what if you have a pop up box requesting a login and password?

and it's an https site as well (using SSL authentication)

Thanks for all the tips, but I still have problem logging onto the site...

CJ

ASKER CERTIFIED SOLUTION

sosolala

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

jmcg

Nothing has happened on this question in over 7 months. It's time for cleanup!

My recommendation, which I will post in the Cleanup topic area, is to
accept answer by sosolala.

Please post any comments here within the next seven days.

PLEASE DO NOT ACCEPT THIS COMMENT AS AN ANSWER!

jmcg
EE Cleanup Volunteer