Solved

Please double-check my cURL script- why isn't this working?

Posted on 2008-10-06
2
349 Views
Last Modified: 2012-05-05
Please see my function below:

I am attempting to connect to a page like this:

http://yellowpages.com.au//onlineSolution_moreInfo.do;jsessionid=1FB1ABFBFD3CD9BB5BF919A3AADDEEEC.12?iblId=3543593&iblName=Artizan+Cabinets&listingPosition=2&originatingPage=ListingPage&authToken=11cd37700be|d509da26cd3e1412d3070a9514558873&pageNumber=3&st=cs&z=100004

Page opens fine in any browser, but when I use my script, the same page comes up as "The page you requested has been static for too long and has run out of time."

I have already tried a few USERAGENTs, IE6, FF, and even Googlebot. I get the same with all of them.

$myurl = the page where I grab each entry url (used to spoof the referrer)

I have even enabled cookies.

Can you think of any reason this is happening? Any help you can offer is appreciated.


function file_get_curl($url) {

	global $myurl, $cookiefile;

	$curl_handle=curl_init();

	curl_setopt($curl_handle,CURLOPT_URL,$url);

	curl_setopt($curl_handle,CURLOPT_CONNECTTIMEOUT,60);

	curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,1);

	curl_setopt($curl_handle,CURLOPT_AUTOREFERER, false);

    curl_setopt($curl_handle,CURLOPT_REFERER, $myurl);	

	curl_setopt($curl_handle,CURLOPT_COOKIEFILE, $cookiefile);

	curl_setopt($curl_handle,CURLOPT_COOKIEJAR, $cookiefile);

	curl_setopt($curl_handle,CURLOPT_USERAGENT, "Googlebot/2.1 (+http://www.googlebot.com/bot.html)");

	$data = curl_exec($curl_handle);

	curl_close($curl_handle);

	return $data;

}

Open in new window

0
Comment
Question by:jpschreibman
  • 2
2 Comments
 
LVL 10

Accepted Solution

by:
kukno earned 500 total points
ID: 22653663
>Can you think of any reason this is happening? Any help you can offer is appreciated.

it's not a curl problem. The session in the server might have timed out.

There are two session "id's" in your URL.

         jsessionid=1FB1ABFBFD3CD9BB5BF919A3AADDEEEC
         authToken=11cd37700be|d509da26cd3e1412d3070a9514558873

Without any further information about the internals of the page, it's hard to decide what's going wrong. However, I suspect a session timeout in the web application.

What are you trying to do? There are better web automation tools than curl ;-)

Regards
Kurt
0
 
LVL 10

Expert Comment

by:kukno
ID: 22653692
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Password hashing is better than message digests or encryption, and you should be using it instead of message digests or encryption.  Find out why and how in this article, which supplements the original article on PHP Client Registration, Login, Logo…
This article discusses four methods for overlaying images in a container on a web page
Any person in technology especially those working for big companies should at least know about the basics of web accessibility. Believe it or not there are even laws in place that require businesses to provide such means for the disabled and aging p…
Learn how to create flexible layouts using relative units in CSS.  New relative units added in CSS3 include vw(viewports width), vh(viewports height), vmin(minimum of viewports height and width), and vmax (maximum of viewports height and width).

895 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now