?
Solved

perl script to view https page that is password protected

Posted on 2007-11-14
16
Medium Priority
?
802 Views
Last Modified: 2010-05-19
I have this script from adam314 to view a https page but the page requires a userid and password, which i have.  How can I change the script to insert userid and password?


use Net::SSLeay qw(get_https);

($page) = get_https('www.bacus.pt', 443, '/');
die "No page\n" unless $page;

open(OUT,">page.txt") or die "output: $!\n";
print OUT $page;
close(OUT);
0
Comment
Question by:mcgilljd
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 7
  • 5
  • 4
16 Comments
 
LVL 39

Expert Comment

by:Adam314
ID: 20281310
How does it ask for a username/password when you go to the page with a browser?
0
 

Author Comment

by:mcgilljd
ID: 20282354
yes when I go to page with browser it asks for user name, password
0
 
LVL 39

Expert Comment

by:Adam314
ID: 20282476
Do you get a web page asking for user/pass?  Or a window pop-up asking for user/pass?
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
LVL 48

Accepted Solution

by:
Tintin earned 1200 total points
ID: 20282613
Assuming it uses basic authentication:

use LWP::UserAgent;
$ua = LWP::UserAgent->new;
$req = HTTP::Request->new(GET => 'https://www.bacus.pt/');
$req->authorization_basic('username', 'password');

open FILE, ">page.txt" or die "Can not open page.txt $!\n";
print FILE $ua->request($req)->as_string;
close FILE;
0
 

Author Comment

by:mcgilljd
ID: 20283521
it is a popup window
0
 
LVL 48

Expert Comment

by:Tintin
ID: 20283611
That means it uses basic authentication.
0
 

Author Comment

by:mcgilljd
ID: 20283735
Tintin, I tried to use your code.
It doesn't give an error, but it doesn't print to page.txt either

use LWP::UserAgent;
my $urlhome ="https://website.com/WebUI/Servlet/Reviewer?axn=review";
$ua = LWP::UserAgent->new;
$req = HTTP::Request->new(GET => $urlhome) or die "no page\n";
$req->authorization_basic('myuserID', 'mypassword');

open FILE, ">C:\Desktop\prices.txt" or die "Can not open page.txt $!\n";
print FILE $ua->request($req)->as_string;
close FILE;


0
 
LVL 48

Assisted Solution

by:Tintin
Tintin earned 1200 total points
ID: 20284103
Try

use LWP::UserAgent;
my $urlhome ="https://website.com/WebUI/Servlet/Reviewer?axn=review";
$ua = LWP::UserAgent->new;
$req = HTTP::Request->new(GET => $urlhome) or die "no page\n";
$req->authorization_basic('myuserID', 'mypassword');
$res = $ua->request($req);

die $res->status_line unless $res->is_success;

open FILE, ">C:/Desktop/prices.txt" or die "Can not open prices.txt $!\n";
print FILE $res->decoded_content;
close FILE;
0
 
LVL 39

Assisted Solution

by:Adam314
Adam314 earned 800 total points
ID: 20284390
A small change to Tintin's code:

use LWP::UserAgent;
my $urlhome ="https://website.com/WebUI/Servlet/Reviewer?axn=review";
$ua = LWP::UserAgent->new;
$us->credentials( $netloc, $realm, $uname, $pass );   #NOTE This line is new
$req = HTTP::Request->new(GET => $urlhome) or die "no page\n";
$res = $ua->request($req);

die $res->status_line unless $res->is_success;

open FILE, ">C:/Desktop/prices.txt" or die "Can not open prices.txt $!\n";
print FILE $res->decoded_content;
close FILE;

## Another option is to override the get_basic_credentials function.  For simple stuff, this isn't needed.
0
 

Author Comment

by:mcgilljd
ID: 20284391
C:\>Perl  C:\PerlScripts\login3.pl
501 Protocol scheme 'https' is not supported (Crypt::SSLeay not installed) at C:
\PerlScripts\interactivelogin3.pl line 11.

I will install Crypt::SSLeay and try again later

0
 

Author Comment

by:mcgilljd
ID: 20284459
Ok i installed crypt::SSLeay and added line
use Crypt::SSLeay ;

Now I get
401 Unauthorized at line 12

But if I use browser to go to page it works.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 20284524
Possibly the site you are trying to connect to requires cookies or looks at the user agent string.

Try adding

$ua->agent("Mozilla/4.0");

and see if it makes a difference.
0
 
LVL 39

Expert Comment

by:Adam314
ID: 20284556
You shouldn't need to "use" Crypt::SSLeay - it will be used by the other modules if needed.

It sounds as if you've provided incorrect username and/or password (and/or netloc and/or realm if using the ->credentials function)

Are you sure you have these correct?  It is likely case sensitive.
0
 

Author Comment

by:mcgilljd
ID: 20284557
added

$ua->agent("Mozilla/4.0");

no change, i will have to try later
0
 

Author Comment

by:mcgilljd
ID: 20289785
I tried using credentials method here is my script

use LWP::UserAgent;
my $urlhome ="https://website.com/WebUI/Servlet/Reviewer?axn=review";
my $userid ="MyID";
my $password ="P@ssword";
my $netloc ="111.222.333.444:80";
my $realm ="REALM";
my $outfile ="C:\Documents and Settings\Desktop\interactiveprices.txt";
$ua = LWP::UserAgent->new;
$ua->credentials( $netloc, $realm, $userid, $password );  
$req = HTTP::Request->new(GET => $urlhome) or die "no page\n";

$res = $ua->request($req);
die $res->status_line unless $res->is_success;

open (OUTFILE, ">$outfile") or die "Can not open prices.txt $!\n";
print OUTFILE $res->decoded_content;
close OUTFILE;


I still get 401 unauthorized
my password and username are correct but I am not sure exactly what to use for "realm"
0
 
LVL 39

Assisted Solution

by:Adam314
Adam314 earned 800 total points
ID: 20290183
If the password contains the @ symbol (or another special character), it needs to be escaped in the $password:
my $password = "P\@ssword";
Otherwise, it will be interpreted as the @ssword variable.

Is real really 'REALM'?
0

Featured Post

On Demand Webinar: Networking for the Cloud Era

Did you know SD-WANs can improve network connectivity? Check out this webinar to learn how an SD-WAN simplified, one-click tool can help you migrate and manage data in the cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Email validation in proper way is  very important validation required in any web pages. This code is self explainable except that Regular Expression which I used for pattern matching. I originally published as a thread on my website : http://www…
There are many situations when we need to display the data in sorted order. For example: Student details by name or by rank or by total marks etc. If you are working on data driven based projects then you will use sorting techniques very frequently.…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Six Sigma Control Plans
Suggested Courses

777 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question