[Last Call] Learn how to a build a cloud-first strategyRegister Now


logging on to secure site with cookies using Mechanize

Posted on 2007-10-11
Medium Priority
Last Modified: 2012-06-21
I am trying to log on to a secure site but I keep getting "No cookie information was received - modify your Internet browser preferences to accept cookies." from the site after a successful login.  

Here is my current code:
#!/usr/bin/perl -w

  use WWW::Mechanize;
  use WWW::Mechanize::Frames;
  use HTTP::Cookies;

    my $mech = WWW::Mechanize::Frames->new( autocheck => 1);
        die $mech->response->status_line unless $mech->success;
        form_number => 1,
        fields      => { username => 'myusername', password => 'mypassword'},
        die unless ($mech->success);

    @frames = $mech->get_frames();
    open(OUTFILE, ">frame1.htm");
    print OUTFILE $frames[0]->content;
    open(OUTFILE, ">frame2.htm");
    print OUTFILE $frames[1]->content;
Question by:Rock_Lobster

Author Comment

ID: 20066288
If no one has any thoughts on mechanize frames does anyone have any experience with LWP::UserAgent::FramesReady ? From the description it sounds like it may do what I need but I couldnt find any good code examples.  I think one of the issues is that there is javascript in the parent frame that interacts with the others that doesnt allow them to be called individually so I need something that will call the page in its entirety after I submit the form and allowing me to collect information on the resulting page.

Expert Comment

ID: 20068677
Try this to store cookies in a file:

my $mech = WWW::Mechanize::Frames->new( autocheck => 1);
my $cookiefile = 'cookies.txt';
my $cookie_jar = HTTP::Cookies->new(File => $cookiefile, autosave => 1);

Author Comment

ID: 20069523
I tried that with same result. The only thing in the cookie.txt file after I ran it was "#LWP-Cookies-1.0"
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!


Expert Comment

ID: 20071260
Here's a complete example I found at: http://www.perlmonks.org/?node_id=493645

#!/usr/bin/perl -w

use strict;
use warnings;
use Crypt::SSLeay;

use LWP::UserAgent;
use WWW::Mechanize;
use HTTP::Cookies;
use HTTP::Request;

##The first two lines of code are used because
##I was sending my results to a web browser.

use CGI::Carp qw/ fatalsToBrowser /;

print "Content-type: text/html\n\n";

my $user = 'MyUser';
my $pass = 'MyPass';
my $base_address = 'https://wwws.stocksRus.com';

my $output = '';
my $dv_data = '';
my $webpage = '';

my $url = 'https://wwws.stocksRus.com/cgi-bin/LogIn';

my $agent = WWW::Mechanize->new( autocheck => 1);

# Set up cookie jar

##Go through login to get appropriate cookies
##so that I can then move onto
##the following pages with my cookie jar properly loaded.
    die $agent->response->status_line unless $agent->success;
$output = $agent->content;

$agent->form_name( 'LoginFormName' );

    USERID => $user,
    PASSWORD => $pass

$agent -> submit();  

###END Login
##I now have the appropriate cookies and may continue.

#On the main page there is some javascript that codes for
#the pull down menus. I could not access these using
#WWW::Mechanize and one of them had the link I needed.
#To solve, I logged into the webpage in a normal browser
#and found the url of the page that I wanted and plugged
#it in below.
##Accessing this page directly does not work as there
##is some Authorization procedure that requires going
##through login first.

    die $agent->response->status_line unless $agent->success;

#This page has a form that holds a symbol that I can
#submit on to get data on that symbol.
$agent -> field    ('ticker','AMGN');

$agent -> submit();

#At first the webpage wouldn't come up in my web browser
#(which is where the output from this run was sent),
#because I didn't have the base URL. All the addresses
#were similar to "/cgi-bin/menu/equity".
#I substitute the base URL in below.
$webpage =  $agent -> content;

$webpage =~ s!(=['"])/!$1$base_address/!g;
#Prints out my webpage of interest.
print $webpage;        

LVL 17

Expert Comment

ID: 20072902
I think your problem is that you set $mech up as a WWW::Mechanize::Frames object, when cookie_jar is expecting a WWW::Mechanize object.  You have a use WWW::Mechanize statement in your code, but you're not using this module.

Author Comment

ID: 20079145
That may be the issue but the site has frames and I need content from both to navigate so how would I do that without Mechanize Frames?

Author Comment

ID: 20082166
I decided to just use Win32::IEAutomation for this specific application but I am running into one issue with the dialog box that pops up saying if you want to display the nonsecure items on the page.

I tried

my $clicker = Win32::IEAutomation::WinClicker->new( warnings => 1) ;
$clicker->push_button_yes("Security Information");

but I couldnt get either one to click yes to load the page.

Accepted Solution

Rock_Lobster earned 0 total points
ID: 20186695
Working fine now if anyone comes accross the same issue, just had to set the no wait flag to push the Security Information alert

Expert Comment

ID: 20215095
Closed, 500 points refunded.
Community Support Moderator

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this post we will learn how to make Android Gesture Tutorial and give different functionality whenever a user Touch or Scroll android screen.
If you are a mobile app developer and especially develop hybrid mobile apps then these 4 mistakes you must avoid for hybrid app development to be the more genuine app developer.
In this fifth video of the Xpdf series, we discuss and demonstrate the PDFdetach utility, which is able to list and, more importantly, extract attachments that are embedded in PDF files. It does this via a command line interface, making it suitable …
Screencast - Getting to Know the Pipeline

831 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question