Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

curl problem with some HTTPS sites

Posted on 2014-03-10
11
Medium Priority
?
1,454 Views
Last Modified: 2014-03-26
I have an odd problem with the code below (that I got from another question here).  It works so far on 'http' sites and on most 'https' sites.  But not on one of my own sites and a client site I need it to connect to.  On those two sites I get a blank screen.  I re-arranged the original code to at least print out the url so I could see that it ran.
<?php
// url
$url = 'https://www.dibsiam.com/';
echo $url;

// Create a curl handle
$ch = curl_init($url);

$options = array(
        CURLOPT_FOLLOWLOCATION => true,         // follow redirects
        CURLOPT_ENCODING       => "",           // handle all encodings
        CURLOPT_USERAGENT      => "Mozilla/5.0",     // who am i
        CURLOPT_SSL_VERIFYHOST => 0,            // don't verify ssl
        CURLOPT_SSL_VERIFYPEER => true,        //
        CURLOPT_VERBOSE        => 1                //
    );
curl_setopt_array($ch,$options);

// Execute
curl_exec($ch);

// Check if any error occurred
if(!curl_errno($ch)) {
	$info = curl_getinfo($ch);

	echo 'Took ' . $info['total_time'] . ' seconds to send a request to ' . $info['url'];
	echo '<pre>';
	print_r($info);
	echo '</pre>';

}

// Close handle
curl_close($ch);
?>

Open in new window

0
Comment
Question by:Dave Baldwin
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 4
11 Comments
 
LVL 84

Author Comment

by:Dave Baldwin
ID: 39919057
Ok, here's version .9394959 of this code that appears to work.  However, I had to turn off SSL checking and set the SSL version.  I would prefer to have one that works with the checking turned back on.
<?php
// url
$url = 'https://www.dibsiam.com/';
echo $url.'<br>';

// Create a curl handle
$ch = curl_init($url);

$options = array(
        CURLOPT_FOLLOWLOCATION => true,         // follow redirects
        CURLOPT_ENCODING       => "",           // handle all encodings
        CURLOPT_USERAGENT      => "Mozilla/5.0",     // who am i
        CURLOPT_SSL_VERIFYHOST => 0,            // don't verify ssl
        CURLOPT_SSL_VERIFYPEER => false,        //
	CURLOPT_SSLVERSION     => 3,
        CURLOPT_VERBOSE        => 1                //
    );
curl_setopt_array($ch,$options);

// Execute
curl_exec($ch);

$err     = curl_errno($ch);
$errmsg  = curl_error($ch) ;
echo "Error # $err : Error message $errmsg";
$info = curl_getinfo($ch);
echo '<pre>';
print_r($info);
echo '</pre>';

//}

// Close handle
curl_close($ch);
?>

Open in new window

0
 
LVL 58

Expert Comment

by:Gary
ID: 39919117
No error message?
Do you get any headers?
Is it possible that curl is using outdated CA Roots? Might explain most sites work but some don't
http://curl.haxx.se/docs/caextract.html
0
 
LVL 84

Author Comment

by:Dave Baldwin
ID: 39919167
If I turn on the SSL verify, the error message shows in the second code.  And it did say there was a problem with the certificate.
Error # 60 : Error message SSL certificate problem, verify that the CA cert is OK. Details: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

I downloaded the 'pem' file but it's not clear where I should put it to use it.  The 'cURL' directory had newer versions of the dll's so I copied them over to the PHP directory.  Doesn't change anything.  I also tried this code on a newer version of PHP and it didn't change the problem.
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
LVL 58

Expert Comment

by:Gary
ID: 39919186
Well its definitely an issuing authority problem.
What are you running on ? Centos, Ubuntu? The path maybe different, I use Centos.
Sample update code from SSH (make a backup of whats there to start with just in case...)

curl http://curl.haxx.se/ca/cacert.pem -o /etc/pki/tls/certs/ca-bundle.crt

Hold off on that command, what distro are you using?
0
 
LVL 84

Author Comment

by:Dave Baldwin
ID: 39919223
It turns out that my even newer versions of PHP for Windows have a newer version of cURL, 7.24.0, that works fine.  The versions that are not working have 7.21.0 so I guess it's time (again) to upgrade PHP on this machine.

I have to write a command line program that does uploads and downloads from 4 different sites and I want to do it over SSL/TLS of course.  I'll end up sticking it in a batch file so the office people can do it with just a double click and very little thought.  I couldn't find a PHP to EXE compiler that would do this.
0
 
LVL 58

Expert Comment

by:Gary
ID: 39919243
cURL version wouldn't matter as its to do with the CA certs it is using and cURL doesn't come with them anymore so updating wouldn't update the roots.
Possible options, if you want to manually add the CA root.
http://curl.haxx.se/docs/sslcerts.html
0
 
LVL 84

Author Comment

by:Dave Baldwin
ID: 39919266
The methods implemented in the newer versions of cURL for PHP may be what's making the difference.  But in any case, I now have two ways of using cURL in PHP.  Note that 'cURL' in PHP is independent of the regular cURL program.
0
 
LVL 58

Expert Comment

by:Gary
ID: 39919278
Maybe so...
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39920234
Dave, I think the "encoding" is permissive, not restrictive.  This script worked for me on the Twitter SSL address.  See line 48 and 55-58
http://www.iconoun.com/demo/curl_get_example.php

<?php // /demo/curl_get_example.php
error_reporting(E_ALL);


// DEMONSTRATE THE BASICS OF CURL
// SOMETHING LIKE demo/curl_get_example.php?url=http://twitter.com


// YOU COULD HAVE SOMETHING LIKE THIS
$url = isset($_GET["url"]) ? $_GET["url"] : 'http://twitter.com';

// BUT SINCE IT IS ON MY SERVER, I HAVE HARD-CODED THIS
$url = 'https://twitter.com/RayPaseur';

// TRY THE REMOTE WEB SERVICE
$htm = my_curl($url);

// SHOW THE WORK PRODUCT OR BARK OUT ERROR MESSAGES
echo "<pre>";
echo PHP_EOL . '<strong>' . $url . '</strong>' . PHP_EOL;
echo PHP_EOL . htmlentities($htm);
echo PHP_EOL;


// A FUNCTION TO RUN A CURL-GET CLIENT CALL TO A FOREIGN SERVER
function my_curl
( $url
, $timeout=5
, $error_report=TRUE
)
{
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE THIS BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // ADDED FOR DAVE BALDWIN
    curl_setopt( $curl, CURLOPT_SSL_VERIFYHOST, FALSE  );
    curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE  );
	curl_setopt( $curl, CURLOPT_SSLVERSION,     3      );
    curl_setopt( $curl, CURLOPT_VERBOSE,        TRUE   );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

HTH, ~Ray
0
 
LVL 84

Accepted Solution

by:
Dave Baldwin earned 0 total points
ID: 39921169
Thanks, I'll give it a try later.  There were only two sites that gave me a problem with the code I posted.  All the rest (so far) worked fine and it appears to be a difference in the version of 'libcurl' that PHP is using.
0
 
LVL 84

Author Closing Comment

by:Dave Baldwin
ID: 39955451
After testing, I found that I was only having this problem on PHP 5.2.17 on Windows.  PHP 5.3 and 5.4 on Windows worked fine.  Thanks for your help but clearly PHP 5.2 won't be receiving any updates anymore.  The 'solution' is just to use a more recent version that works right.
0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Preface This is the third article about the EE Collaborative Login Project. A Better Website Login System (http://www.experts-exchange.com/A_2902.html) introduces the Login System and shows how to implement a login page. The EE Collaborative Logi…
3 proven steps to speed up Magento powered sites. The article focus is on optimizing time to first byte (TTFB), full page caching and configuring server for optimal performance.
The viewer will learn the benefit of using external CSS files and the relationship between class and ID selectors. Create your external css file by saving it as style.css then set up your style tags: (CODE) Reference the nav tag and set your prop…
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

715 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question