Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Please double-check my cURL script- why isn't this working?

Posted on 2008-10-06
2
Medium Priority
?
373 Views
Last Modified: 2012-05-05
Please see my function below:

I am attempting to connect to a page like this:

http://yellowpages.com.au//onlineSolution_moreInfo.do;jsessionid=1FB1ABFBFD3CD9BB5BF919A3AADDEEEC.12?iblId=3543593&iblName=Artizan+Cabinets&listingPosition=2&originatingPage=ListingPage&authToken=11cd37700be|d509da26cd3e1412d3070a9514558873&pageNumber=3&st=cs&z=100004

Page opens fine in any browser, but when I use my script, the same page comes up as "The page you requested has been static for too long and has run out of time."

I have already tried a few USERAGENTs, IE6, FF, and even Googlebot. I get the same with all of them.

$myurl = the page where I grab each entry url (used to spoof the referrer)

I have even enabled cookies.

Can you think of any reason this is happening? Any help you can offer is appreciated.


function file_get_curl($url) {
	global $myurl, $cookiefile;
	$curl_handle=curl_init();
	curl_setopt($curl_handle,CURLOPT_URL,$url);
	curl_setopt($curl_handle,CURLOPT_CONNECTTIMEOUT,60);
	curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,1);
	curl_setopt($curl_handle,CURLOPT_AUTOREFERER, false);
    curl_setopt($curl_handle,CURLOPT_REFERER, $myurl);	
	curl_setopt($curl_handle,CURLOPT_COOKIEFILE, $cookiefile);
	curl_setopt($curl_handle,CURLOPT_COOKIEJAR, $cookiefile);
	curl_setopt($curl_handle,CURLOPT_USERAGENT, "Googlebot/2.1 (+http://www.googlebot.com/bot.html)");
	$data = curl_exec($curl_handle);
	curl_close($curl_handle);
	return $data;
}

Open in new window

0
Comment
Question by:jpschreibman
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
2 Comments
 
LVL 10

Accepted Solution

by:
kukno earned 2000 total points
ID: 22653663
>Can you think of any reason this is happening? Any help you can offer is appreciated.

it's not a curl problem. The session in the server might have timed out.

There are two session "id's" in your URL.

         jsessionid=1FB1ABFBFD3CD9BB5BF919A3AADDEEEC
         authToken=11cd37700be|d509da26cd3e1412d3070a9514558873

Without any further information about the internals of the page, it's hard to decide what's going wrong. However, I suspect a session timeout in the web application.

What are you trying to do? There are better web automation tools than curl ;-)

Regards
Kurt
0
 
LVL 10

Expert Comment

by:kukno
ID: 22653692
0

Featured Post

Simplify Your Workload with One Tool

How do you combat today’s intelligent hacker while managing multiple domains and platforms? By simplifying your workload with one tool. With Lunarpages hosting through Plesk Onyx, you can:

Automate SSL generation and installation with two clicks
Experience total server control

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Originally, this post was published on Monitis Blog, you can check it here . It goes without saying that technology has transformed society and the very nature of how we live, work, and communicate in ways that would’ve been incomprehensible 5 ye…
Although a lot of people devote their energy toward marketing for specific industries, there are some basic principles that can be applied to any sector imaginable. We’ll look at four steps to take and examine how those steps were put into action fo…
HTML5 has deprecated a few of the older ways of showing media as well as offering up a new way to create games and animations. Audio, video, and canvas are just a few of the adjustments made between XHTML and HTML5. As we learned in our last micr…
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …
Suggested Courses

618 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question