logging on to secure site with cookies using Mechanize

Posted on 2007-10-11
Last Modified: 2012-06-21
I am trying to log on to a secure site but I keep getting "No cookie information was received - modify your Internet browser preferences to accept cookies." from the site after a successful login.  

Here is my current code:
#!/usr/bin/perl -w

  use WWW::Mechanize;
  use WWW::Mechanize::Frames;
  use HTTP::Cookies;

    my $mech = WWW::Mechanize::Frames->new( autocheck => 1);
        die $mech->response->status_line unless $mech->success;
        form_number => 1,
        fields      => { username => 'myusername', password => 'mypassword'},
        die unless ($mech->success);

    @frames = $mech->get_frames();
    open(OUTFILE, ">frame1.htm");
    print OUTFILE $frames[0]->content;
    open(OUTFILE, ">frame2.htm");
    print OUTFILE $frames[1]->content;
Question by:Rock_Lobster

    Author Comment

    If no one has any thoughts on mechanize frames does anyone have any experience with LWP::UserAgent::FramesReady ? From the description it sounds like it may do what I need but I couldnt find any good code examples.  I think one of the issues is that there is javascript in the parent frame that interacts with the others that doesnt allow them to be called individually so I need something that will call the page in its entirety after I submit the form and allowing me to collect information on the resulting page.
    LVL 7

    Expert Comment

    Try this to store cookies in a file:

    my $mech = WWW::Mechanize::Frames->new( autocheck => 1);
    my $cookiefile = 'cookies.txt';
    my $cookie_jar = HTTP::Cookies->new(File => $cookiefile, autosave => 1);

    Author Comment

    I tried that with same result. The only thing in the cookie.txt file after I ran it was "#LWP-Cookies-1.0"
    LVL 7

    Expert Comment

    Here's a complete example I found at:

    #!/usr/bin/perl -w

    use strict;
    use warnings;
    use Crypt::SSLeay;

    use LWP::UserAgent;
    use WWW::Mechanize;
    use HTTP::Cookies;
    use HTTP::Request;

    ##The first two lines of code are used because
    ##I was sending my results to a web browser.

    use CGI::Carp qw/ fatalsToBrowser /;

    print "Content-type: text/html\n\n";

    my $user = 'MyUser';
    my $pass = 'MyPass';
    my $base_address = '';

    my $output = '';
    my $dv_data = '';
    my $webpage = '';

    my $url = '';

    my $agent = WWW::Mechanize->new( autocheck => 1);

    # Set up cookie jar

    ##Go through login to get appropriate cookies
    ##so that I can then move onto
    ##the following pages with my cookie jar properly loaded.
        die $agent->response->status_line unless $agent->success;
    $output = $agent->content;

    $agent->form_name( 'LoginFormName' );

        USERID => $user,
        PASSWORD => $pass

    $agent -> submit();  

    ###END Login
    ##I now have the appropriate cookies and may continue.

    #On the main page there is some javascript that codes for
    #the pull down menus. I could not access these using
    #WWW::Mechanize and one of them had the link I needed.
    #To solve, I logged into the webpage in a normal browser
    #and found the url of the page that I wanted and plugged
    #it in below.
    ##Accessing this page directly does not work as there
    ##is some Authorization procedure that requires going
    ##through login first.

        die $agent->response->status_line unless $agent->success;

    #This page has a form that holds a symbol that I can
    #submit on to get data on that symbol.
    $agent -> field    ('ticker','AMGN');

    $agent -> submit();

    #At first the webpage wouldn't come up in my web browser
    #(which is where the output from this run was sent),
    #because I didn't have the base URL. All the addresses
    #were similar to "/cgi-bin/menu/equity".
    #I substitute the base URL in below.
    $webpage =  $agent -> content;

    $webpage =~ s!(=['"])/!$1$base_address/!g;
    #Prints out my webpage of interest.
    print $webpage;        

    LVL 17

    Expert Comment

    I think your problem is that you set $mech up as a WWW::Mechanize::Frames object, when cookie_jar is expecting a WWW::Mechanize object.  You have a use WWW::Mechanize statement in your code, but you're not using this module.

    Author Comment

    That may be the issue but the site has frames and I need content from both to navigate so how would I do that without Mechanize Frames?

    Author Comment

    I decided to just use Win32::IEAutomation for this specific application but I am running into one issue with the dialog box that pops up saying if you want to display the nonsecure items on the page.

    I tried

    my $clicker = Win32::IEAutomation::WinClicker->new( warnings => 1) ;
    $clicker->push_button_yes("Security Information");

    but I couldnt get either one to click yes to load the page.

    Accepted Solution

    Working fine now if anyone comes accross the same issue, just had to set the no wait flag to push the Security Information alert
    LVL 1

    Expert Comment

    Closed, 500 points refunded.
    Community Support Moderator

    Featured Post

    6 Surprising Benefits of Threat Intelligence

    All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

    Join & Write a Comment

    Go is an acronym of golang, is a programming language developed Google in 2007. Go is a new language that is mostly in the C family, with significant input from Pascal/Modula/Oberon family. Hence Go arisen as low-level language with fast compilation…
    Does the idea of dealing with bits scare or confuse you? Does it seem like a waste of time in an age where we all have terabytes of storage? If so, you're missing out on one of the core tools in every professional programmer's toolbox. Learn how to …
    In this fourth video of the Xpdf series, we discuss and demonstrate the PDFinfo utility, which retrieves the contents of a PDF's Info Dictionary, as well as some other information, including the page count. We show how to isolate the page count in a…
    In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

    754 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    19 Experts available now in Live!

    Get 1:1 Help Now