Solved

perl script to view https page that is password protected

Posted on 2007-11-14
16
789 Views
Last Modified: 2010-05-19
I have this script from adam314 to view a https page but the page requires a userid and password, which i have.  How can I change the script to insert userid and password?


use Net::SSLeay qw(get_https);

($page) = get_https('www.bacus.pt', 443, '/');
die "No page\n" unless $page;

open(OUT,">page.txt") or die "output: $!\n";
print OUT $page;
close(OUT);
0
Comment
Question by:mcgilljd
  • 7
  • 5
  • 4
16 Comments
 
LVL 39

Expert Comment

by:Adam314
ID: 20281310
How does it ask for a username/password when you go to the page with a browser?
0
 

Author Comment

by:mcgilljd
ID: 20282354
yes when I go to page with browser it asks for user name, password
0
 
LVL 39

Expert Comment

by:Adam314
ID: 20282476
Do you get a web page asking for user/pass?  Or a window pop-up asking for user/pass?
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 48

Accepted Solution

by:
Tintin earned 300 total points
ID: 20282613
Assuming it uses basic authentication:

use LWP::UserAgent;
$ua = LWP::UserAgent->new;
$req = HTTP::Request->new(GET => 'https://www.bacus.pt/');
$req->authorization_basic('username', 'password');

open FILE, ">page.txt" or die "Can not open page.txt $!\n";
print FILE $ua->request($req)->as_string;
close FILE;
0
 

Author Comment

by:mcgilljd
ID: 20283521
it is a popup window
0
 
LVL 48

Expert Comment

by:Tintin
ID: 20283611
That means it uses basic authentication.
0
 

Author Comment

by:mcgilljd
ID: 20283735
Tintin, I tried to use your code.
It doesn't give an error, but it doesn't print to page.txt either

use LWP::UserAgent;
my $urlhome ="https://website.com/WebUI/Servlet/Reviewer?axn=review";
$ua = LWP::UserAgent->new;
$req = HTTP::Request->new(GET => $urlhome) or die "no page\n";
$req->authorization_basic('myuserID', 'mypassword');

open FILE, ">C:\Desktop\prices.txt" or die "Can not open page.txt $!\n";
print FILE $ua->request($req)->as_string;
close FILE;


0
 
LVL 48

Assisted Solution

by:Tintin
Tintin earned 300 total points
ID: 20284103
Try

use LWP::UserAgent;
my $urlhome ="https://website.com/WebUI/Servlet/Reviewer?axn=review";
$ua = LWP::UserAgent->new;
$req = HTTP::Request->new(GET => $urlhome) or die "no page\n";
$req->authorization_basic('myuserID', 'mypassword');
$res = $ua->request($req);

die $res->status_line unless $res->is_success;

open FILE, ">C:/Desktop/prices.txt" or die "Can not open prices.txt $!\n";
print FILE $res->decoded_content;
close FILE;
0
 
LVL 39

Assisted Solution

by:Adam314
Adam314 earned 200 total points
ID: 20284390
A small change to Tintin's code:

use LWP::UserAgent;
my $urlhome ="https://website.com/WebUI/Servlet/Reviewer?axn=review";
$ua = LWP::UserAgent->new;
$us->credentials( $netloc, $realm, $uname, $pass );   #NOTE This line is new
$req = HTTP::Request->new(GET => $urlhome) or die "no page\n";
$res = $ua->request($req);

die $res->status_line unless $res->is_success;

open FILE, ">C:/Desktop/prices.txt" or die "Can not open prices.txt $!\n";
print FILE $res->decoded_content;
close FILE;

## Another option is to override the get_basic_credentials function.  For simple stuff, this isn't needed.
0
 

Author Comment

by:mcgilljd
ID: 20284391
C:\>Perl  C:\PerlScripts\login3.pl
501 Protocol scheme 'https' is not supported (Crypt::SSLeay not installed) at C:
\PerlScripts\interactivelogin3.pl line 11.

I will install Crypt::SSLeay and try again later

0
 

Author Comment

by:mcgilljd
ID: 20284459
Ok i installed crypt::SSLeay and added line
use Crypt::SSLeay ;

Now I get
401 Unauthorized at line 12

But if I use browser to go to page it works.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 20284524
Possibly the site you are trying to connect to requires cookies or looks at the user agent string.

Try adding

$ua->agent("Mozilla/4.0");

and see if it makes a difference.
0
 
LVL 39

Expert Comment

by:Adam314
ID: 20284556
You shouldn't need to "use" Crypt::SSLeay - it will be used by the other modules if needed.

It sounds as if you've provided incorrect username and/or password (and/or netloc and/or realm if using the ->credentials function)

Are you sure you have these correct?  It is likely case sensitive.
0
 

Author Comment

by:mcgilljd
ID: 20284557
added

$ua->agent("Mozilla/4.0");

no change, i will have to try later
0
 

Author Comment

by:mcgilljd
ID: 20289785
I tried using credentials method here is my script

use LWP::UserAgent;
my $urlhome ="https://website.com/WebUI/Servlet/Reviewer?axn=review";
my $userid ="MyID";
my $password ="P@ssword";
my $netloc ="111.222.333.444:80";
my $realm ="REALM";
my $outfile ="C:\Documents and Settings\Desktop\interactiveprices.txt";
$ua = LWP::UserAgent->new;
$ua->credentials( $netloc, $realm, $userid, $password );  
$req = HTTP::Request->new(GET => $urlhome) or die "no page\n";

$res = $ua->request($req);
die $res->status_line unless $res->is_success;

open (OUTFILE, ">$outfile") or die "Can not open prices.txt $!\n";
print OUTFILE $res->decoded_content;
close OUTFILE;


I still get 401 unauthorized
my password and username are correct but I am not sure exactly what to use for "realm"
0
 
LVL 39

Assisted Solution

by:Adam314
Adam314 earned 200 total points
ID: 20290183
If the password contains the @ symbol (or another special character), it needs to be escaped in the $password:
my $password = "P\@ssword";
Otherwise, it will be interpreted as the @ssword variable.

Is real really 'REALM'?
0

Featured Post

Free Tool: Postgres Monitoring System

A PHP and Perl based system to collect and display usage statistics from PostgreSQL databases.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
perl script help 12 105
Perl strange behaviour 5 74
Need to install DBD::MySQL on a MAC without a local MySQL server for Perl 4 75
Convert grep lines to perl 6 75
There are many situations when we need to display the data in sorted order. For example: Student details by name or by rank or by total marks etc. If you are working on data driven based projects then you will use sorting techniques very frequently.…
In the distant past (last year) I hacked together a little toy that would allow a couple of Manager types to query, preview, and extract data from a number of MongoDB instances, to their tool of choice: Excel (http://dilbert.com/strips/comic/2007-08…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

685 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question