?
Solved

LWP - Get a url which is redirected

Posted on 2003-03-16
7
Medium Priority
?
192 Views
Last Modified: 2010-03-05
How do I fetch a document if the url redirects to another site for authentication and then redirects back to it?

In this case, how do I sign in onto a redirected website, and fetch the document of the provided link when authorization succeeded and brought back to the target link?

Thanks,
CJ
0
Comment
Question by:CaitlinJ
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
7 Comments
 
LVL 1

Expert Comment

by:sosolala
ID: 8151232
Depence on the autentication methode

you are asking for a very technical answer

but your question doesn;t contain any technical methode

but If you can do it in a browser you can script it


Most probally you have to perseve the cookie.
in you script...
Gtz
0
 
LVL 2

Expert Comment

by:SegFault
ID: 8152142
If the method is using cookies, you can use the CookieJar module, which plugs into LWP.

I had just this situation, however, the CookieJar was using version 2.0 of cookies and the site wanted version 1.0, so I had to break out etherreal and sniff the packets.  It was a hack job, but it worked.

Take a look at the documentation for implementing a cookiejar.

If the site is not using cookies, we'd need to know specifically what authentication mechanism is being used.  If you can provide some amplifying information, it would help.

seg
0
 
LVL 1

Expert Comment

by:sosolala
ID: 8152717
Below a sample script that logs in to yahoo
to login to a second site...add under the coresponding  
$req2 that looks like the $req pointing to a second site
$res2 that looks like the $res contains then the second html

this way you can login to 2 sites..and use the same cookie.
now sometimes you have a box that that pops up requesting the login and passwd...

Then there is a different autentication methode..needed
used let me know and I post it :-) hope this will answer you question.!

#!/usr/bin/perl

my login = "loginname";
my passwd = "passwrd";

use LWP::UserAgent;
  use HTTP::Cookies;

  $ua = LWP::UserAgent->new;
  $ua->cookie_jar(HTTP::Cookies->new(file => "lwpcookies.txt",
                                     autosave => 1));

  $ua->agent("$0/0.1 " . $ua->agent);
  # $ua->agent("Mozilla/8.0") # pretend we are very capable browser

  $req = HTTP::Request->new(GET => 'http://login.yahoo.com/config/login?login=$login&passwd=$passwd&.src=quote&.done=http://finance.yahoo.com');
  $req->header('Accept' => 'text/html');

  # send request
  $res = $ua->request($req);

  # check the outcome
  if ($res->is_success) {
     #print $res->content;
  } else {
     print "Error: " . $res->status_line . "\n";
  }

0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:CaitlinJ
ID: 8156710
so what if you have a pop up box requesting a login and password?

and it's an https site as well (using SSL authentication)

Thanks for all the tips, but I still have problem logging onto the site...

CJ
0
 
LVL 1

Accepted Solution

by:
sosolala earned 1000 total points
ID: 8158319
Find out the url that opens in the popupbox but if you mean a https popupbox you could use something like:


sub check_order {
        #check for an open order  (critical!!)
        my $req = new HTTP::Request 'POST','https://investment.datek.com/dummy.htm';
        $req->content_type('application/x-www-form-urlencoded');
        my $chx = "REPORT  ";
        my $action = "order";
        $req->content("ropen&name=$action&userid=$efusername&__chx=$chx");
        $req->authorization_basic($username,$passwd);
        $ua->agent("Mozilla/4.73 ");
        my $res = $ua->request($req);
        unless ($res->code eq "200") { return "URLERROR $res->code";  }
        my @stuff = split "\n", $res->as_string;

        #extract needed data:
0
 
LVL 20

Expert Comment

by:jmcg
ID: 9699439
Nothing has happened on this question in over 7 months. It's time for cleanup!

My recommendation, which I will post in the Cleanup topic area, is to
accept answer by sosolala.

Please post any comments here within the next seven days.

PLEASE DO NOT ACCEPT THIS COMMENT AS AN ANSWER!

jmcg
EE Cleanup Volunteer
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Email validation in proper way is  very important validation required in any web pages. This code is self explainable except that Regular Expression which I used for pattern matching. I originally published as a thread on my website : http://www…
A year or so back I was asked to have a play with MongoDB; within half an hour I had downloaded (http://www.mongodb.org/downloads),  installed and started the daemon, and had a console window open. After an hour or two of playing at the command …
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Six Sigma Control Plans
Suggested Courses

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question