Link to home
Start Free TrialLog in
Avatar of kerrston
kerrston

asked on

LWP::UserAgent cookie problem

i get a error where it says does not seem to contain cookies


i am trying to do a search on 50 results instead of the 15 msn does it on. when i change the resuts to 50 it saves it as a cookie and not in the url line. So how would i send a request to get the source code for resluts of 1-50 not just 1-15

here is the cookie that it saves and the co=50 is the number of results



msnsrchnosp=0&co=50&RS=1search.msn.com/085208755229619988313043364829546562*


#!/usr/local/bin/perl -w

use LWP::UserAgent;




@keyword=( "map", "maps", "usa+maps", "county+maps", "us+regional+maps", "us+regional+map", "us+map", "us+maps", "state+maps", "map+of+the+us", "zip+code+maps", "city+maps", "census+tracts", "custom+maps", "town+map", "zip+code+map", "radius+map", "radius+maps");


#market map part
print  "*******  MARKET MAPS!!!!!!!   *******  \n \n";
open FILE, ">>key.txt";
print FILE "*******  msn!!!!!!!   *******  \n \n";
print FILE "*******  MARKET MAPS!!!!!!!   *******  \n \n";
close FILE;

#1-50
foreach $element (@keyword)
{
  $count=0;

  print $element;
  print "*******  MARKET MAPS!!!!!!!   *******  \n \n";
  $cookie_jar = HTTP::Cookies->new(file => 'tsearch.txt');
  $user_agent=new LWP::UserAgent;
  $request= new HTTP::Request('GET',

"http://search.msn.com/results.asp?q=$element&origq=market%2Bmaps&RS=CHECKED&FORM=SMCRT&v=1&cfg=SMCINITIAL&nosp=0&thr=");
 
 
  $cookie_jar->add_cookie_header($request);
   
  $user_agent->agent('Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.1) Gecko/20020826');
  $request->header('Accept' => 'image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-excel, application/msword,

application/vnd.ms-powerpoint, */*');
  $request->header('Accept-Language' => 'en;q=1.0');
  $cookie_jar->extract_cookies($response);
 
  $user_agent->cookie_jar($cookie_jar);
 
 $textd=$cookie_jar->as_string();
print $textd;
 
  $response=$user_agent->request($request);                    
  $content=$response->{_content};  
 

 while($content=~ /<span class="clsResultURL"><i>(.*?)<\/i>/sg)
  {
    $count=$count+1;
    if($1=~ /market.*?maps.*?\.com/s)
    {

      open FILE, ">>key.txt";
      print FILE "keyword: $element \n";
      print FILE "rank: $count \n";
      close FILE;

      last;
    }  
  }


 
 
}
Avatar of kerrston
kerrston

ASKER

test
ASKER CERTIFIED SOLUTION
Avatar of jmcg
jmcg
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Oops, cut'n'paste'n'get-distracted....

$cookie_jar = HTTP::Cookies->new();
$cookie_jar->save('tsearch.txt');
i did the project another way already bit longer progarm to run but works all the same
Well if you solved it yourself, there's no need to waste your points. Drop a request in the Community Support forum and request that my answer be unaccepted and your points refunded. I think they can do that for you.
Actually, a question once answered cannot be unanswered.  I can and will refund the points for the question.  I will also change the grade from a C to an A so it isn't a bad mark on the recipient.

Future PAQ purchasers:
This question remains unanswered so you may or may not obtain any value from the responses.  Since it cost you nothing to view, all should be well.

SpideyMod
Community Support Moderator @Experts Exchange