Solved

curl problem with some HTTPS sites

Posted on 2014-03-10
11
1,394 Views
Last Modified: 2014-03-26
I have an odd problem with the code below (that I got from another question here).  It works so far on 'http' sites and on most 'https' sites.  But not on one of my own sites and a client site I need it to connect to.  On those two sites I get a blank screen.  I re-arranged the original code to at least print out the url so I could see that it ran.
<?php
// url
$url = 'https://www.dibsiam.com/';
echo $url;

// Create a curl handle
$ch = curl_init($url);

$options = array(
        CURLOPT_FOLLOWLOCATION => true,         // follow redirects
        CURLOPT_ENCODING       => "",           // handle all encodings
        CURLOPT_USERAGENT      => "Mozilla/5.0",     // who am i
        CURLOPT_SSL_VERIFYHOST => 0,            // don't verify ssl
        CURLOPT_SSL_VERIFYPEER => true,        //
        CURLOPT_VERBOSE        => 1                //
    );
curl_setopt_array($ch,$options);

// Execute
curl_exec($ch);

// Check if any error occurred
if(!curl_errno($ch)) {
	$info = curl_getinfo($ch);

	echo 'Took ' . $info['total_time'] . ' seconds to send a request to ' . $info['url'];
	echo '<pre>';
	print_r($info);
	echo '</pre>';

}

// Close handle
curl_close($ch);
?>

Open in new window

0
Comment
Question by:Dave Baldwin
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 4
11 Comments
 
LVL 83

Author Comment

by:Dave Baldwin
ID: 39919057
Ok, here's version .9394959 of this code that appears to work.  However, I had to turn off SSL checking and set the SSL version.  I would prefer to have one that works with the checking turned back on.
<?php
// url
$url = 'https://www.dibsiam.com/';
echo $url.'<br>';

// Create a curl handle
$ch = curl_init($url);

$options = array(
        CURLOPT_FOLLOWLOCATION => true,         // follow redirects
        CURLOPT_ENCODING       => "",           // handle all encodings
        CURLOPT_USERAGENT      => "Mozilla/5.0",     // who am i
        CURLOPT_SSL_VERIFYHOST => 0,            // don't verify ssl
        CURLOPT_SSL_VERIFYPEER => false,        //
	CURLOPT_SSLVERSION     => 3,
        CURLOPT_VERBOSE        => 1                //
    );
curl_setopt_array($ch,$options);

// Execute
curl_exec($ch);

$err     = curl_errno($ch);
$errmsg  = curl_error($ch) ;
echo "Error # $err : Error message $errmsg";
$info = curl_getinfo($ch);
echo '<pre>';
print_r($info);
echo '</pre>';

//}

// Close handle
curl_close($ch);
?>

Open in new window

0
 
LVL 58

Expert Comment

by:Gary
ID: 39919117
No error message?
Do you get any headers?
Is it possible that curl is using outdated CA Roots? Might explain most sites work but some don't
http://curl.haxx.se/docs/caextract.html
0
 
LVL 83

Author Comment

by:Dave Baldwin
ID: 39919167
If I turn on the SSL verify, the error message shows in the second code.  And it did say there was a problem with the certificate.
Error # 60 : Error message SSL certificate problem, verify that the CA cert is OK. Details: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

I downloaded the 'pem' file but it's not clear where I should put it to use it.  The 'cURL' directory had newer versions of the dll's so I copied them over to the PHP directory.  Doesn't change anything.  I also tried this code on a newer version of PHP and it didn't change the problem.
0
Creating Instructional Tutorials  

For Any Use & On Any Platform

Contextual Guidance at the moment of need helps your employees/users adopt software o& achieve even the most complex tasks instantly. Boost knowledge retention, software adoption & employee engagement with easy solution.

 
LVL 58

Expert Comment

by:Gary
ID: 39919186
Well its definitely an issuing authority problem.
What are you running on ? Centos, Ubuntu? The path maybe different, I use Centos.
Sample update code from SSH (make a backup of whats there to start with just in case...)

curl http://curl.haxx.se/ca/cacert.pem -o /etc/pki/tls/certs/ca-bundle.crt

Hold off on that command, what distro are you using?
0
 
LVL 83

Author Comment

by:Dave Baldwin
ID: 39919223
It turns out that my even newer versions of PHP for Windows have a newer version of cURL, 7.24.0, that works fine.  The versions that are not working have 7.21.0 so I guess it's time (again) to upgrade PHP on this machine.

I have to write a command line program that does uploads and downloads from 4 different sites and I want to do it over SSL/TLS of course.  I'll end up sticking it in a batch file so the office people can do it with just a double click and very little thought.  I couldn't find a PHP to EXE compiler that would do this.
0
 
LVL 58

Expert Comment

by:Gary
ID: 39919243
cURL version wouldn't matter as its to do with the CA certs it is using and cURL doesn't come with them anymore so updating wouldn't update the roots.
Possible options, if you want to manually add the CA root.
http://curl.haxx.se/docs/sslcerts.html
0
 
LVL 83

Author Comment

by:Dave Baldwin
ID: 39919266
The methods implemented in the newer versions of cURL for PHP may be what's making the difference.  But in any case, I now have two ways of using cURL in PHP.  Note that 'cURL' in PHP is independent of the regular cURL program.
0
 
LVL 58

Expert Comment

by:Gary
ID: 39919278
Maybe so...
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39920234
Dave, I think the "encoding" is permissive, not restrictive.  This script worked for me on the Twitter SSL address.  See line 48 and 55-58
http://www.iconoun.com/demo/curl_get_example.php

<?php // /demo/curl_get_example.php
error_reporting(E_ALL);


// DEMONSTRATE THE BASICS OF CURL
// SOMETHING LIKE demo/curl_get_example.php?url=http://twitter.com


// YOU COULD HAVE SOMETHING LIKE THIS
$url = isset($_GET["url"]) ? $_GET["url"] : 'http://twitter.com';

// BUT SINCE IT IS ON MY SERVER, I HAVE HARD-CODED THIS
$url = 'https://twitter.com/RayPaseur';

// TRY THE REMOTE WEB SERVICE
$htm = my_curl($url);

// SHOW THE WORK PRODUCT OR BARK OUT ERROR MESSAGES
echo "<pre>";
echo PHP_EOL . '<strong>' . $url . '</strong>' . PHP_EOL;
echo PHP_EOL . htmlentities($htm);
echo PHP_EOL;


// A FUNCTION TO RUN A CURL-GET CLIENT CALL TO A FOREIGN SERVER
function my_curl
( $url
, $timeout=5
, $error_report=TRUE
)
{
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE THIS BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // ADDED FOR DAVE BALDWIN
    curl_setopt( $curl, CURLOPT_SSL_VERIFYHOST, FALSE  );
    curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE  );
	curl_setopt( $curl, CURLOPT_SSLVERSION,     3      );
    curl_setopt( $curl, CURLOPT_VERBOSE,        TRUE   );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

HTH, ~Ray
0
 
LVL 83

Accepted Solution

by:
Dave Baldwin earned 0 total points
ID: 39921169
Thanks, I'll give it a try later.  There were only two sites that gave me a problem with the code I posted.  All the rest (so far) worked fine and it appears to be a difference in the version of 'libcurl' that PHP is using.
0
 
LVL 83

Author Closing Comment

by:Dave Baldwin
ID: 39955451
After testing, I found that I was only having this problem on PHP 5.2.17 on Windows.  PHP 5.3 and 5.4 on Windows worked fine.  Thanks for your help but clearly PHP 5.2 won't be receiving any updates anymore.  The 'solution' is just to use a more recent version that works right.
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Build an array called $myWeek which will hold the array elements Today, Yesterday and then builds up the rest of the week by the name of the day going back 1 week.   (CODE) (CODE) Then you just need to pass your date to the function. If i…
Originally, this post was published on Monitis Blog, you can check it here . In business circles, we sometimes hear that today is the “age of the customer.” And so it is. Thanks to the enormous advances over the past few years in consumer techno…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Video by: Mark
This lesson goes over how to construct ordered and unordered lists and how to create hyperlinks.

617 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question