Solved

curl problem with some HTTPS sites

Posted on 2014-03-10
11
1,327 Views
Last Modified: 2014-03-26
I have an odd problem with the code below (that I got from another question here).  It works so far on 'http' sites and on most 'https' sites.  But not on one of my own sites and a client site I need it to connect to.  On those two sites I get a blank screen.  I re-arranged the original code to at least print out the url so I could see that it ran.
<?php
// url
$url = 'https://www.dibsiam.com/';
echo $url;

// Create a curl handle
$ch = curl_init($url);

$options = array(
        CURLOPT_FOLLOWLOCATION => true,         // follow redirects
        CURLOPT_ENCODING       => "",           // handle all encodings
        CURLOPT_USERAGENT      => "Mozilla/5.0",     // who am i
        CURLOPT_SSL_VERIFYHOST => 0,            // don't verify ssl
        CURLOPT_SSL_VERIFYPEER => true,        //
        CURLOPT_VERBOSE        => 1                //
    );
curl_setopt_array($ch,$options);

// Execute
curl_exec($ch);

// Check if any error occurred
if(!curl_errno($ch)) {
	$info = curl_getinfo($ch);

	echo 'Took ' . $info['total_time'] . ' seconds to send a request to ' . $info['url'];
	echo '<pre>';
	print_r($info);
	echo '</pre>';

}

// Close handle
curl_close($ch);
?>

Open in new window

0
Comment
Question by:Dave Baldwin
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 4
11 Comments
 
LVL 83

Author Comment

by:Dave Baldwin
ID: 39919057
Ok, here's version .9394959 of this code that appears to work.  However, I had to turn off SSL checking and set the SSL version.  I would prefer to have one that works with the checking turned back on.
<?php
// url
$url = 'https://www.dibsiam.com/';
echo $url.'<br>';

// Create a curl handle
$ch = curl_init($url);

$options = array(
        CURLOPT_FOLLOWLOCATION => true,         // follow redirects
        CURLOPT_ENCODING       => "",           // handle all encodings
        CURLOPT_USERAGENT      => "Mozilla/5.0",     // who am i
        CURLOPT_SSL_VERIFYHOST => 0,            // don't verify ssl
        CURLOPT_SSL_VERIFYPEER => false,        //
	CURLOPT_SSLVERSION     => 3,
        CURLOPT_VERBOSE        => 1                //
    );
curl_setopt_array($ch,$options);

// Execute
curl_exec($ch);

$err     = curl_errno($ch);
$errmsg  = curl_error($ch) ;
echo "Error # $err : Error message $errmsg";
$info = curl_getinfo($ch);
echo '<pre>';
print_r($info);
echo '</pre>';

//}

// Close handle
curl_close($ch);
?>

Open in new window

0
 
LVL 58

Expert Comment

by:Gary
ID: 39919117
No error message?
Do you get any headers?
Is it possible that curl is using outdated CA Roots? Might explain most sites work but some don't
http://curl.haxx.se/docs/caextract.html
0
 
LVL 83

Author Comment

by:Dave Baldwin
ID: 39919167
If I turn on the SSL verify, the error message shows in the second code.  And it did say there was a problem with the certificate.
Error # 60 : Error message SSL certificate problem, verify that the CA cert is OK. Details: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

I downloaded the 'pem' file but it's not clear where I should put it to use it.  The 'cURL' directory had newer versions of the dll's so I copied them over to the PHP directory.  Doesn't change anything.  I also tried this code on a newer version of PHP and it didn't change the problem.
0
PeopleSoft Has Never Been Easier

PeopleSoft Adoption Made Smooth & Simple!

On-The-Job Training Is made Intuitive & Easy With WalkMe's On-Screen Guidance Tool.  Claim Your Free WalkMe Account Now

 
LVL 58

Expert Comment

by:Gary
ID: 39919186
Well its definitely an issuing authority problem.
What are you running on ? Centos, Ubuntu? The path maybe different, I use Centos.
Sample update code from SSH (make a backup of whats there to start with just in case...)

curl http://curl.haxx.se/ca/cacert.pem -o /etc/pki/tls/certs/ca-bundle.crt

Hold off on that command, what distro are you using?
0
 
LVL 83

Author Comment

by:Dave Baldwin
ID: 39919223
It turns out that my even newer versions of PHP for Windows have a newer version of cURL, 7.24.0, that works fine.  The versions that are not working have 7.21.0 so I guess it's time (again) to upgrade PHP on this machine.

I have to write a command line program that does uploads and downloads from 4 different sites and I want to do it over SSL/TLS of course.  I'll end up sticking it in a batch file so the office people can do it with just a double click and very little thought.  I couldn't find a PHP to EXE compiler that would do this.
0
 
LVL 58

Expert Comment

by:Gary
ID: 39919243
cURL version wouldn't matter as its to do with the CA certs it is using and cURL doesn't come with them anymore so updating wouldn't update the roots.
Possible options, if you want to manually add the CA root.
http://curl.haxx.se/docs/sslcerts.html
0
 
LVL 83

Author Comment

by:Dave Baldwin
ID: 39919266
The methods implemented in the newer versions of cURL for PHP may be what's making the difference.  But in any case, I now have two ways of using cURL in PHP.  Note that 'cURL' in PHP is independent of the regular cURL program.
0
 
LVL 58

Expert Comment

by:Gary
ID: 39919278
Maybe so...
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39920234
Dave, I think the "encoding" is permissive, not restrictive.  This script worked for me on the Twitter SSL address.  See line 48 and 55-58
http://www.iconoun.com/demo/curl_get_example.php

<?php // /demo/curl_get_example.php
error_reporting(E_ALL);


// DEMONSTRATE THE BASICS OF CURL
// SOMETHING LIKE demo/curl_get_example.php?url=http://twitter.com


// YOU COULD HAVE SOMETHING LIKE THIS
$url = isset($_GET["url"]) ? $_GET["url"] : 'http://twitter.com';

// BUT SINCE IT IS ON MY SERVER, I HAVE HARD-CODED THIS
$url = 'https://twitter.com/RayPaseur';

// TRY THE REMOTE WEB SERVICE
$htm = my_curl($url);

// SHOW THE WORK PRODUCT OR BARK OUT ERROR MESSAGES
echo "<pre>";
echo PHP_EOL . '<strong>' . $url . '</strong>' . PHP_EOL;
echo PHP_EOL . htmlentities($htm);
echo PHP_EOL;


// A FUNCTION TO RUN A CURL-GET CLIENT CALL TO A FOREIGN SERVER
function my_curl
( $url
, $timeout=5
, $error_report=TRUE
)
{
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE THIS BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // ADDED FOR DAVE BALDWIN
    curl_setopt( $curl, CURLOPT_SSL_VERIFYHOST, FALSE  );
    curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE  );
	curl_setopt( $curl, CURLOPT_SSLVERSION,     3      );
    curl_setopt( $curl, CURLOPT_VERBOSE,        TRUE   );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

HTH, ~Ray
0
 
LVL 83

Accepted Solution

by:
Dave Baldwin earned 0 total points
ID: 39921169
Thanks, I'll give it a try later.  There were only two sites that gave me a problem with the code I posted.  All the rest (so far) worked fine and it appears to be a difference in the version of 'libcurl' that PHP is using.
0
 
LVL 83

Author Closing Comment

by:Dave Baldwin
ID: 39955451
After testing, I found that I was only having this problem on PHP 5.2.17 on Windows.  PHP 5.3 and 5.4 on Windows worked fine.  Thanks for your help but clearly PHP 5.2 won't be receiving any updates anymore.  The 'solution' is just to use a more recent version that works right.
0

Featured Post

Salesforce Made Easy to Use

On-screen guidance at the moment of need enables you & your employees to focus on the core, you can now boost your adoption rates swiftly and simply with one easy tool.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction Knockoutjs (Knockout) is a JavaScript framework (Model View ViewModel or MVVM framework).   The main ideology behind Knockout is to control from JavaScript how a page looks whilst creating an engaging user experience in the least …
SASS allows you to treat your CSS code in a more OOP way. Let's have a look on how you can structure your code in order for it to be easily maintained and reused.
Viewers will learn about if statements in Java and their use The if statement: The condition required to create an if statement: Variations of if statements: An example using if statements:
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

710 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question