kerrston
asked on
LWP::UserAgent cookie problem
i get a error where it says does not seem to contain cookies
i am trying to do a search on 50 results instead of the 15 msn does it on. when i change the resuts to 50 it saves it as a cookie and not in the url line. So how would i send a request to get the source code for resluts of 1-50 not just 1-15
here is the cookie that it saves and the co=50 is the number of results
msnsrchnosp=0&co=50&RS=1se arch.msn.c om/0852087 5522961998 8313043364 829546562*
#!/usr/local/bin/perl -w
use LWP::UserAgent;
@keyword=( "map", "maps", "usa+maps", "county+maps", "us+regional+maps", "us+regional+map", "us+map", "us+maps", "state+maps", "map+of+the+us", "zip+code+maps", "city+maps", "census+tracts", "custom+maps", "town+map", "zip+code+map", "radius+map", "radius+maps");
#market map part
print "******* MARKET MAPS!!!!!!! ******* \n \n";
open FILE, ">>key.txt";
print FILE "******* msn!!!!!!! ******* \n \n";
print FILE "******* MARKET MAPS!!!!!!! ******* \n \n";
close FILE;
#1-50
foreach $element (@keyword)
{
$count=0;
print $element;
print "******* MARKET MAPS!!!!!!! ******* \n \n";
$cookie_jar = HTTP::Cookies->new(file => 'tsearch.txt');
$user_agent=new LWP::UserAgent;
$request= new HTTP::Request('GET',
"http://search.msn.com/results.asp?q=$element&origq=market%2Bmaps&RS=CHECKED&FORM=SMCRT&v=1&cfg=SMCINITIAL&nosp=0&thr=");
$cookie_jar->add_cookie_he ader($requ est);
$user_agent->agent('Mozill a/5.0 (X11; U; Linux i686; en-US; rv:1.1) Gecko/20020826');
$request->header('Accept' => 'image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-excel, application/msword,
application/vnd.ms-powerpo int, */*');
$request->header('Accept-L anguage' => 'en;q=1.0');
$cookie_jar->extract_cooki es($respon se);
$user_agent->cookie_jar($c ookie_jar) ;
$textd=$cookie_jar->as_str ing();
print $textd;
$response=$user_agent->req uest($requ est);
$content=$response->{_cont ent};
while($content=~ /<span class="clsResultURL"><i>(. *?)<\/i>/s g)
{
$count=$count+1;
if($1=~ /market.*?maps.*?\.com/s)
{
open FILE, ">>key.txt";
print FILE "keyword: $element \n";
print FILE "rank: $count \n";
close FILE;
last;
}
}
}
i am trying to do a search on 50 results instead of the 15 msn does it on. when i change the resuts to 50 it saves it as a cookie and not in the url line. So how would i send a request to get the source code for resluts of 1-50 not just 1-15
here is the cookie that it saves and the co=50 is the number of results
msnsrchnosp=0&co=50&RS=1se
#!/usr/local/bin/perl -w
use LWP::UserAgent;
@keyword=( "map", "maps", "usa+maps", "county+maps", "us+regional+maps", "us+regional+map", "us+map", "us+maps", "state+maps", "map+of+the+us", "zip+code+maps", "city+maps", "census+tracts", "custom+maps", "town+map", "zip+code+map", "radius+map", "radius+maps");
#market map part
print "******* MARKET MAPS!!!!!!! ******* \n \n";
open FILE, ">>key.txt";
print FILE "******* msn!!!!!!! ******* \n \n";
print FILE "******* MARKET MAPS!!!!!!! ******* \n \n";
close FILE;
#1-50
foreach $element (@keyword)
{
$count=0;
print $element;
print "******* MARKET MAPS!!!!!!! ******* \n \n";
$cookie_jar = HTTP::Cookies->new(file => 'tsearch.txt');
$user_agent=new LWP::UserAgent;
$request= new HTTP::Request('GET',
"http://search.msn.com/results.asp?q=$element&origq=market%2Bmaps&RS=CHECKED&FORM=SMCRT&v=1&cfg=SMCINITIAL&nosp=0&thr=");
$cookie_jar->add_cookie_he
$user_agent->agent('Mozill
$request->header('Accept' => 'image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-excel, application/msword,
application/vnd.ms-powerpo
$request->header('Accept-L
$cookie_jar->extract_cooki
$user_agent->cookie_jar($c
$textd=$cookie_jar->as_str
print $textd;
$response=$user_agent->req
$content=$response->{_cont
while($content=~ /<span class="clsResultURL"><i>(.
{
$count=$count+1;
if($1=~ /market.*?maps.*?\.com/s)
{
open FILE, ">>key.txt";
print FILE "keyword: $element \n";
print FILE "rank: $count \n";
close FILE;
last;
}
}
}
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Oops, cut'n'paste'n'get-distract ed....
$cookie_jar = HTTP::Cookies->new();
$cookie_jar->save('tsearch .txt');
$cookie_jar = HTTP::Cookies->new();
$cookie_jar->save('tsearch
ASKER
i did the project another way already bit longer progarm to run but works all the same
Well if you solved it yourself, there's no need to waste your points. Drop a request in the Community Support forum and request that my answer be unaccepted and your points refunded. I think they can do that for you.
Actually, a question once answered cannot be unanswered. I can and will refund the points for the question. I will also change the grade from a C to an A so it isn't a bad mark on the recipient.
Future PAQ purchasers:
This question remains unanswered so you may or may not obtain any value from the responses. Since it cost you nothing to view, all should be well.
SpideyMod
Community Support Moderator @Experts Exchange
Future PAQ purchasers:
This question remains unanswered so you may or may not obtain any value from the responses. Since it cost you nothing to view, all should be well.
SpideyMod
Community Support Moderator @Experts Exchange
ASKER