Failure to verify host certificate with cURL call

Some back-story: I was running a server with Ubuntu 10.04 and used `apt-get upgrade` to move to 12.04.  The upgrade was successful, but I had a custom ppa in sources (ondrej/PHP5 for PHP 5.5).  The custom ppa also upgraded Apache from 2.2 to 2.4, which was undesirable.  It ended up being a painful mess to revert Apache/PHP back to 2.2/5.3.10.  That is complete, with everything appearing to be running OK.

Concurrently, I have started seeing a failure in an e-commerce site on the server.  For credit card processing, the application uses PHP's cURL library to call Authorize.net's AIM gateway.  Some investigation found that cURL was returning a validation error on the peer certificate:
[07-Aug-2014 04:21:03 UTC] -ELOG:1- (blindsusa:ddpjj4a23o9hk8sh1ka10fb265) curl exec err=60:SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

Open in new window

I tested with openssl s_client, and saw something very similar:
#> openssl s_client -connect secure.authorize.net:443 -verify 3
verify depth is 3
CONNECTED(00000003)
depth=2 C = US, O = "Entrust, Inc.", OU = www.entrust.net/CPS is incorporated by reference, OU = "(c) 2006 Entrust, Inc.", CN = Entrust Root Certification Authority
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=2 C = US, O = "Entrust, Inc.", OU = www.entrust.net/CPS is incorporated by reference, OU = "(c) 2006 Entrust, Inc.", CN = Entrust Root Certification Authority
verify error:num=27:certificate not trusted
verify return:1
depth=1 C = US, O = "Entrust, Inc.", OU = www.entrust.net/rpa is incorporated by reference, OU = "(c) 2009 Entrust, Inc.", CN = Entrust Certification Authority - L1E
verify return:1
depth=0 C = US, ST = California, L = Mountain View, 1.3.6.1.4.1.311.60.2.1.3 = US, 1.3.6.1.4.1.311.60.2.1.2 = Delaware, O = Cybersource Corporation, businessCategory = Private Organization, serialNumber = 2838921 + CN = secure.authorize.net
verify return:1
---
Certificate chain
 0 s:/C=US/ST=California/L=Mountain View/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=Delaware/O=Cybersource Corporation/businessCategory=Private Organization/serialNumber=2838921/CN=secure.authorize.net
   i:/C=US/O=Entrust, Inc./OU=www.entrust.net/rpa is incorporated by reference/OU=(c) 2009 Entrust, Inc./CN=Entrust Certification Authority - L1E
 1 s:/C=US/O=Entrust, Inc./OU=www.entrust.net/rpa is incorporated by reference/OU=(c) 2009 Entrust, Inc./CN=Entrust Certification Authority - L1E
   i:/C=US/O=Entrust, Inc./OU=www.entrust.net/CPS is incorporated by reference/OU=(c) 2006 Entrust, Inc./CN=Entrust Root Certification Authority
 2 s:/C=US/O=Entrust, Inc./OU=www.entrust.net/CPS is incorporated by reference/OU=(c) 2006 Entrust, Inc./CN=Entrust Root Certification Authority
   i:/C=US/O=Entrust.net/OU=www.entrust.net/CPS incorp. by ref. (limits liab.)/OU=(c) 1999 Entrust.net Limited/CN=Entrust.net Secure Server Certification Authority

Open in new window

Looking in /etc/ssl/certs, I can see what look to be the appropriate certificates.  Using -CApath or -CAfile on openssl showed no change.  I even tried downloading the current certs from Entrust's site, with no change.

Any ideas for how I can resolve this issue?  The short-term solution was to set CURL_SSL_VERIFYPEER to false, but I dislike.  How can I properly validate Authorize.net's certificate chain?
LVL 51
Steve BinkAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

gr8gonzoConsultantCommented:
Can you show us the cURL options in the code (you can mask anything sensitive)? Usually, defining the CA bundle does the trick, so if it's reading the bundle and still failing, then you might need to see if the bundle actually has the correct certificate authority inside it. You can always download an updated ca-bundle.crt from cURL's website, too. It's just named as cacert.pem, but it's the bundle that is processed automatically from Mozilla:

http://curl.haxx.se/ca/cacert.pem
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Steve BinkAuthor Commented:
The entire routine is here:
$ch = curl_init("https://secure.authorize.net/gateway/transact.dll");  
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
$buffer = curl_exec($ch);
curl_close($ch);

Open in new window

The $fields variable is key1=value1&key2=value2 POST string.  The commented line was put in to bypass this issue in the short term.

Downloading the new certificate bundle worked, but I'm curious to know why.  In my attempts yesterday, I updated the ca-certificates package through apt-get.  I even downloaded the appropriate certs from the issuer, though I can't be certain they were the correct ones or that I installed their hashes correctly.  How should I keep this updated in the future?
0
gr8gonzoConsultantCommented:
Normally, you don't have to update the bundle regularly, but if you really wanted to, you could set up a cron job to download that file once a month or something. It doesn't change that frequently.

Package managers like apt-get or yum will download pre-compiled packages that might still contain older components. So a package manager will always sacrifice the latest versions of things for ease of use.

If you do set up a cron job to download the file, don't download directly over top of the old file. Download it to a temporary file first and do some sanity checks (make sure the file hasn't drastically changed size, like doubled in size, or shrunk to a third, etc...) before swapping them out.

I wouldn't even bother updating the CA bundle automatically unless you have some situation where cURL is going to consistently encounter new domains and certificates. Really, the CA bundle is there to help YOU authenticate remote sources' certificates, so it is up to you to determine if you really need the latest bundle or not.

I know Entrust and Verisign both had some updates in the past year or two, so it's not a shock that the default CA bundle wouldn't have the latest updates that would validate a chain for a recently-issued certificate.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
SSL / HTTPS

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.