Solved

Please double-check my cURL script- why isn't this working?

Posted on 2008-10-06
2
352 Views
Last Modified: 2012-05-05
Please see my function below:

I am attempting to connect to a page like this:

http://yellowpages.com.au//onlineSolution_moreInfo.do;jsessionid=1FB1ABFBFD3CD9BB5BF919A3AADDEEEC.12?iblId=3543593&iblName=Artizan+Cabinets&listingPosition=2&originatingPage=ListingPage&authToken=11cd37700be|d509da26cd3e1412d3070a9514558873&pageNumber=3&st=cs&z=100004

Page opens fine in any browser, but when I use my script, the same page comes up as "The page you requested has been static for too long and has run out of time."

I have already tried a few USERAGENTs, IE6, FF, and even Googlebot. I get the same with all of them.

$myurl = the page where I grab each entry url (used to spoof the referrer)

I have even enabled cookies.

Can you think of any reason this is happening? Any help you can offer is appreciated.


function file_get_curl($url) {
	global $myurl, $cookiefile;
	$curl_handle=curl_init();
	curl_setopt($curl_handle,CURLOPT_URL,$url);
	curl_setopt($curl_handle,CURLOPT_CONNECTTIMEOUT,60);
	curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,1);
	curl_setopt($curl_handle,CURLOPT_AUTOREFERER, false);
    curl_setopt($curl_handle,CURLOPT_REFERER, $myurl);	
	curl_setopt($curl_handle,CURLOPT_COOKIEFILE, $cookiefile);
	curl_setopt($curl_handle,CURLOPT_COOKIEJAR, $cookiefile);
	curl_setopt($curl_handle,CURLOPT_USERAGENT, "Googlebot/2.1 (+http://www.googlebot.com/bot.html)");
	$data = curl_exec($curl_handle);
	curl_close($curl_handle);
	return $data;
}

Open in new window

0
Comment
Question by:jpschreibman
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
2 Comments
 
LVL 10

Accepted Solution

by:
kukno earned 500 total points
ID: 22653663
>Can you think of any reason this is happening? Any help you can offer is appreciated.

it's not a curl problem. The session in the server might have timed out.

There are two session "id's" in your URL.

         jsessionid=1FB1ABFBFD3CD9BB5BF919A3AADDEEEC
         authToken=11cd37700be|d509da26cd3e1412d3070a9514558873

Without any further information about the internals of the page, it's hard to decide what's going wrong. However, I suspect a session timeout in the web application.

What are you trying to do? There are better web automation tools than curl ;-)

Regards
Kurt
0
 
LVL 10

Expert Comment

by:kukno
ID: 22653692
0

Featured Post

Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When crafting your “Why Us” page, there are a plethora of pitfalls to avoid. Follow these five tips, and you’ll be well on your way to creating an effective page.
This article was originally published on Monitis Blog, you can check it here . Today it’s fairly well known that high-performing websites and applications bring in more visitors, higher SEO, and ultimately more sales. By the same token, downtime…
The viewer will learn the basics of jQuery including how to code hide show and toggles. Reference your jQuery libraries: (CODE) Include your new external js/jQuery file: (CODE) Write your first lines of code to setup your site for jQuery…
Learn how to create flexible layouts using relative units in CSS.  New relative units added in CSS3 include vw(viewports width), vh(viewports height), vmin(minimum of viewports height and width), and vmax (maximum of viewports height and width).

739 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question