Solved

curl problem with some HTTPS sites

Posted on 2014-03-10
11
1,293 Views
Last Modified: 2014-03-26
I have an odd problem with the code below (that I got from another question here).  It works so far on 'http' sites and on most 'https' sites.  But not on one of my own sites and a client site I need it to connect to.  On those two sites I get a blank screen.  I re-arranged the original code to at least print out the url so I could see that it ran.
<?php
// url
$url = 'https://www.dibsiam.com/';
echo $url;

// Create a curl handle
$ch = curl_init($url);

$options = array(
        CURLOPT_FOLLOWLOCATION => true,         // follow redirects
        CURLOPT_ENCODING       => "",           // handle all encodings
        CURLOPT_USERAGENT      => "Mozilla/5.0",     // who am i
        CURLOPT_SSL_VERIFYHOST => 0,            // don't verify ssl
        CURLOPT_SSL_VERIFYPEER => true,        //
        CURLOPT_VERBOSE        => 1                //
    );
curl_setopt_array($ch,$options);

// Execute
curl_exec($ch);

// Check if any error occurred
if(!curl_errno($ch)) {
	$info = curl_getinfo($ch);

	echo 'Took ' . $info['total_time'] . ' seconds to send a request to ' . $info['url'];
	echo '<pre>';
	print_r($info);
	echo '</pre>';

}

// Close handle
curl_close($ch);
?>

Open in new window

0
Comment
Question by:Dave Baldwin
  • 6
  • 4
11 Comments
 
LVL 83

Author Comment

by:Dave Baldwin
ID: 39919057
Ok, here's version .9394959 of this code that appears to work.  However, I had to turn off SSL checking and set the SSL version.  I would prefer to have one that works with the checking turned back on.
<?php
// url
$url = 'https://www.dibsiam.com/';
echo $url.'<br>';

// Create a curl handle
$ch = curl_init($url);

$options = array(
        CURLOPT_FOLLOWLOCATION => true,         // follow redirects
        CURLOPT_ENCODING       => "",           // handle all encodings
        CURLOPT_USERAGENT      => "Mozilla/5.0",     // who am i
        CURLOPT_SSL_VERIFYHOST => 0,            // don't verify ssl
        CURLOPT_SSL_VERIFYPEER => false,        //
	CURLOPT_SSLVERSION     => 3,
        CURLOPT_VERBOSE        => 1                //
    );
curl_setopt_array($ch,$options);

// Execute
curl_exec($ch);

$err     = curl_errno($ch);
$errmsg  = curl_error($ch) ;
echo "Error # $err : Error message $errmsg";
$info = curl_getinfo($ch);
echo '<pre>';
print_r($info);
echo '</pre>';

//}

// Close handle
curl_close($ch);
?>

Open in new window

0
 
LVL 58

Expert Comment

by:Gary
ID: 39919117
No error message?
Do you get any headers?
Is it possible that curl is using outdated CA Roots? Might explain most sites work but some don't
http://curl.haxx.se/docs/caextract.html
0
 
LVL 83

Author Comment

by:Dave Baldwin
ID: 39919167
If I turn on the SSL verify, the error message shows in the second code.  And it did say there was a problem with the certificate.
Error # 60 : Error message SSL certificate problem, verify that the CA cert is OK. Details: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

I downloaded the 'pem' file but it's not clear where I should put it to use it.  The 'cURL' directory had newer versions of the dll's so I copied them over to the PHP directory.  Doesn't change anything.  I also tried this code on a newer version of PHP and it didn't change the problem.
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 58

Expert Comment

by:Gary
ID: 39919186
Well its definitely an issuing authority problem.
What are you running on ? Centos, Ubuntu? The path maybe different, I use Centos.
Sample update code from SSH (make a backup of whats there to start with just in case...)

curl http://curl.haxx.se/ca/cacert.pem -o /etc/pki/tls/certs/ca-bundle.crt

Hold off on that command, what distro are you using?
0
 
LVL 83

Author Comment

by:Dave Baldwin
ID: 39919223
It turns out that my even newer versions of PHP for Windows have a newer version of cURL, 7.24.0, that works fine.  The versions that are not working have 7.21.0 so I guess it's time (again) to upgrade PHP on this machine.

I have to write a command line program that does uploads and downloads from 4 different sites and I want to do it over SSL/TLS of course.  I'll end up sticking it in a batch file so the office people can do it with just a double click and very little thought.  I couldn't find a PHP to EXE compiler that would do this.
0
 
LVL 58

Expert Comment

by:Gary
ID: 39919243
cURL version wouldn't matter as its to do with the CA certs it is using and cURL doesn't come with them anymore so updating wouldn't update the roots.
Possible options, if you want to manually add the CA root.
http://curl.haxx.se/docs/sslcerts.html
0
 
LVL 83

Author Comment

by:Dave Baldwin
ID: 39919266
The methods implemented in the newer versions of cURL for PHP may be what's making the difference.  But in any case, I now have two ways of using cURL in PHP.  Note that 'cURL' in PHP is independent of the regular cURL program.
0
 
LVL 58

Expert Comment

by:Gary
ID: 39919278
Maybe so...
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39920234
Dave, I think the "encoding" is permissive, not restrictive.  This script worked for me on the Twitter SSL address.  See line 48 and 55-58
http://www.iconoun.com/demo/curl_get_example.php

<?php // /demo/curl_get_example.php
error_reporting(E_ALL);


// DEMONSTRATE THE BASICS OF CURL
// SOMETHING LIKE demo/curl_get_example.php?url=http://twitter.com


// YOU COULD HAVE SOMETHING LIKE THIS
$url = isset($_GET["url"]) ? $_GET["url"] : 'http://twitter.com';

// BUT SINCE IT IS ON MY SERVER, I HAVE HARD-CODED THIS
$url = 'https://twitter.com/RayPaseur';

// TRY THE REMOTE WEB SERVICE
$htm = my_curl($url);

// SHOW THE WORK PRODUCT OR BARK OUT ERROR MESSAGES
echo "<pre>";
echo PHP_EOL . '<strong>' . $url . '</strong>' . PHP_EOL;
echo PHP_EOL . htmlentities($htm);
echo PHP_EOL;


// A FUNCTION TO RUN A CURL-GET CLIENT CALL TO A FOREIGN SERVER
function my_curl
( $url
, $timeout=5
, $error_report=TRUE
)
{
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE THIS BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // ADDED FOR DAVE BALDWIN
    curl_setopt( $curl, CURLOPT_SSL_VERIFYHOST, FALSE  );
    curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE  );
	curl_setopt( $curl, CURLOPT_SSLVERSION,     3      );
    curl_setopt( $curl, CURLOPT_VERBOSE,        TRUE   );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

HTH, ~Ray
0
 
LVL 83

Accepted Solution

by:
Dave Baldwin earned 0 total points
ID: 39921169
Thanks, I'll give it a try later.  There were only two sites that gave me a problem with the code I posted.  All the rest (so far) worked fine and it appears to be a difference in the version of 'libcurl' that PHP is using.
0
 
LVL 83

Author Closing Comment

by:Dave Baldwin
ID: 39955451
After testing, I found that I was only having this problem on PHP 5.2.17 on Windows.  PHP 5.3 and 5.4 on Windows worked fine.  Thanks for your help but clearly PHP 5.2 won't be receiving any updates anymore.  The 'solution' is just to use a more recent version that works right.
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Build an array called $myWeek which will hold the array elements Today, Yesterday and then builds up the rest of the week by the name of the day going back 1 week.   (CODE) (CODE) Then you just need to pass your date to the function. If i…
I found this questions asking how to do this in many different forums, so I will describe here how to implement a solution using PHP and AJAX. The logical flow for the problem should be: Write an event handler for the first drop down box to get …
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.
Learn how to create flexible layouts using relative units in CSS.  New relative units added in CSS3 include vw(viewports width), vh(viewports height), vmin(minimum of viewports height and width), and vmax (maximum of viewports height and width).

679 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question