sendmail TLS not working right

Can you post your sendmail.mc?

include(`../m4/cf.m4')
VERSIONID(`default setup for Slackware Linux')dnl
OSTYPE(`linux')dnl
DOMAIN(generic)dnl
define(`confSMTP_LOGIN_MSG', `mail.ohprs.org Service ready; $b')dnl
define(`confPRIVACY_FLAGS', `authwarnings,novrfy,noexpn,restrictqrun')dnl
define(`confTO_IDENT', `0')dnl
define(`confBAD_RCPT_THROTTLE',`1')dnl
define(`confCONNECTION_RATE_THROTTLE',`3')dnl
define(`confDEAD_LETTER_DROP',`/dev/null')dnl
define(`confDOUBLE_BOUNCE_ADDRESS',`nobody')dnl
define(`confDF_BUFFER_SIZE',`16384')dnl
define(`confXF_BUFFER_SIZE',`16384')dnl
define(`confSUPER_SAFE',`true')dnl
define(`confCHECKPOINT_INTERVAL',`10')dnl
FEATURE(`use_cw_file')dnl
FEATURE(`use_ct_file')dnl
FEATURE(`mailertable',`hash -o /etc/mail/mailertable.db')dnl
FEATURE(`virtusertable',`hash -o /etc/mail/virtusertable.db')dnl
FEATURE(`access_db', `hash -T<TMPF> /etc/mail/access')dnl
FEATURE(`lookupdotdomain')dnl
FEATURE(`blacklist_recipients')dnl
FEATURE(`dnsbl',`bl.spamcop.net')dnl
FEATURE(`local_procmail',`',`procmail -t -Y -a $h -d $u')dnl
FEATURE(`redirect')dnl
define(`confCACERT_PATH',`/etc/ssl/certs/')dnl
define(`confCACERT',`/etc/ssl/certs/Go_Daddy_Root_Certificate_Authority_-_G2.pem')dnl
define(`confSERVER_CERT',`/etc/ssl/OHPRS/GoDaddy/Apache/c5fe0cc8242d6030.crt')dnl
define(`confSERVER_KEY',`/etc/ssl/OHPRS/GoDaddy/mail.ohprs.org.key')dnl
define(`confCLIENT_CERT',`/etc/ssl/OHPRS/GoDaddy/Apache/c5fe0cc8242d6030.crt')dnl
define(`confCLIENT_KEY',`/etc/ssl/OHPRS/GoDaddy/mail.ohprs.org.key')dnl
INPUT_MAIL_FILTER(`spamassassin', `S=local:/var/run/spamass.sock, F=, T=C:15m;S:4m;R:4m;E:10m')dnl
define(`confMILTER_MACROS_CONNECT',`t, b, j, _, {daemon_name}, {if_name}, {if_addr}')dnl
define(`confMILTER_MACROS_HELO',`s, {tls_version}, {cipher}, {cipher_bits}, {cert_subject}, {cert_issuer}')dnl
define(`confMILTER_MACROS_ENVRCPT',`r, v, Z')dnl
INPUT_MAIL_FILTER(`milter-bcc',`S=local:/var/run/milter-bcc.sock, F=, T=C:15m;S:4m;R:4m;E:10m')dnl
MASQUERADE_AS(`ohprs.org')dnl
MASQUERADE_DOMAIN(`ohprs.org')dnl
FEATURE(`allmasquerade')dnl
FEATURE(`masquerade_envelope')dnl
FEATURE(`always_add_domain')dnl
EXPOSED_USER(`root')dnl
LOCAL_DOMAIN(`localhost.localdomain')dnl
MAILER(local)dnl
MAILER(smtp)dnl
MAILER(procmail)dnl

Select allOpen in new window

I would expect that if you define your cert path as /etc/ssl/certs, that all data would fall somewhere under that directory.

You really should use the [distributed, updated] ca-bundle.crt for your CACERT.

When I do cert and key files, I create them as ".pem".

I don't see your auth configuration (this example allows plain and encrypted auth):

define(`confAUTH_OPTIONS', `A')dnl

TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl

Jan Springer:

I would expect that if you define your cert path as /etc/ssl/certs, that all data would fall somewhere under that directory.

That's where all the pre-installed/distributed CA Root files are. I didn't want to mess with that directory. I should think absolute paths would override that -- at they do seem to.

You really should use the [distributed, updated] ca-bundle.crt for your CACERT.

I got the following files from GoDaddy:

gd_bundle.crt
4111d11611d96.crt

So, you're saying I should use gd_bundle.crt as my confCACERT? I'l ltry that. Unfortunately I didn't see any instruction at GoDaddy for configuring sendmail and the tech help wasn't much.

When I do cert and key files, I create them as ".pem".

I followed GoDaddy's instructions on how to create the key and CSR <common name>,key is what they said, so that's what I did. The cert file is from them.

I don't see your auth configuration (this example allows plain and encrypted auth):

No, I don't have those. OK, I'll stick both of those lines in my .mc and post back with results.

Were their directions for sendmail or for apache (or some other web server)?

You shouldn't have to get a copy of the GoDaddy Ca data, it should already be in your bundle.  The exception is if you need an intermediate cert and that doesn't go in the CA bundle.

Nobody uses validated certificate chain for mail servers because nobody checks them (for interoperability)

Whoa! That didn't work out! Here's new applicable lines in the .mc

define(`confCACERT_PATH',`/etc/ssl/OHPRS/')dnl
define(`confCACERT',`/etc/ssl/OHPRS/GoDaddy/Apache/gd_bundle.crt')dnl
define(`confSERVER_CERT',`/etc/ssl/OHPRS/GoDaddy/Apache/c5fe0cc8242d6030.crt')dnl
define(`confSERVER_KEY',`/etc/ssl/OHPRS/GoDaddy/mail.ohprs.org.key')dnl
define(`confCLIENT_CERT',`/etc/ssl/OHPRS/GoDaddy/Apache/c5fe0cc8242d6030.crt')dnl
define(`confCLIENT_KEY',`/etc/ssl/OHPRS/GoDaddy/mail.ohprs.org.key')dnl
define(`confAUTH_OPTIONS', `A')dnl
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl

Select allOpen in new window

And I got:

Feb 24 16:12:10 mail sm-mta[12723]: starting daemon (8.14.9): SMTP+queueing@00:25:00
Feb 24 16:12:10 mail sm-msp-queue[12726]: starting daemon (8.14.9): queueing@00:25:00
Feb 24 16:12:11 mail sm-mta[12725]: STARTTLS=client, error: connect failed=-1, reason=tlsv1 alert decode error, SSL_error=1, errno=0, retry=-1
Feb 24 16:12:11 mail sm-mta[12725]: ruleset=tls_server, arg1=SOFTWARE, relay=mail3.usbank.com, reject=403 4.7.0 TLS handshake failed.
Feb 24 16:12:12 mail sm-mta[12725]: STARTTLS=client, error: connect failed=-1, reason=tlsv1 alert decode error, SSL_error=1, errno=0, retry=-1
Feb 24 16:12:12 mail sm-mta[12725]: ruleset=tls_server, arg1=SOFTWARE, relay=mail1.usbank.com, reject=403 4.7.0 TLS handshake failed.
Feb 24 16:12:12 mail sm-mta[12725]: STARTTLS=client, error: connect failed=-1, reason=tlsv1 alert decode error, SSL_error=1, errno=0, retry=-1
Feb 24 16:12:12 mail sm-mta[12725]: ruleset=tls_server, arg1=SOFTWARE, relay=mail2.usbank.com, reject=403 4.7.0 TLS handshake failed.
Feb 24 16:12:13 mail sm-mta[12725]: STARTTLS=client, error: connect failed=-1, reason=tlsv1 alert decode error, SSL_error=1, errno=0, retry=-1
Feb 24 16:12:13 mail sm-mta[12725]: ruleset=tls_server, arg1=SOFTWARE, relay=mail4.usbank.com, reject=403 4.7.0 TLS handshake failed.
Feb 24 16:12:13 mail sm-mta[12725]: t1OKuVOA016594: to=<investorservices@usbank.com>, ctladdr=<bpatterson@ohprs.org> (3000044/100), delay=00:15:42, xdelay=00:00:03, mailer=esmtp, pri=559186, relay=mail4.usbank.com. [170.135.72.73], dsn=4.0.0, stat=Deferred: 403 4.7.0 TLS handshake failed.
Feb 24 16:12:14 mail sm-mta[12725]: STARTTLS=client, error: connect failed=-1, reason=tlsv1 alert decode error, SSL_error=1, errno=0, retry=-1
Feb 24 16:12:14 mail sm-mta[12725]: ruleset=tls_server, arg1=SOFTWARE, relay=mail02.paycor.com, reject=403 4.7.0 TLS handshake failed.
Feb 24 16:12:14 mail sm-mta[12725]: t1OEnKLO016503: to=<PKeer@paycor.com>, delay=06:22:54, xdelay=00:00:01, mailer=esmtp, pri=1746724, relay=mail02.paycor.com. [69.61.226.160], dsn=4.0.0, stat=Deferred: 403 4.7.0 TLS handshake failed.
Feb 24 16:12:14 mail sm-mta[12725]: t1NLugQa013458: to=<PKeer@paycor.com>, delay=23:15:31, xdelay=00:00:00, mailer=esmtp, pri=5436649, relay=mail02.paycor.com., dsn=4.0.0, stat=Deferred

Select allOpen in new window

I commented out the confAUTH_OPTIONS and TRUST_AUTH_MECH lines (should have tried one thing at a time) and still got:

Feb 24 16:14:48 mail sm-mta[16286]: starting daemon (8.14.9): SMTP+queueing@00:25:00
Feb 24 16:14:48 mail sm-msp-queue[16289]: starting daemon (8.14.9): queueing@00:25:00
Feb 24 16:14:53 mail sm-mta[16288]: STARTTLS=client, error: connect failed=-1, reason=tlsv1 alert decode error, SSL_error=1, errno=0, retry=-1
Feb 24 16:14:53 mail sm-mta[16288]: ruleset=tls_server, arg1=SOFTWARE, relay=mail3.usbank.com, reject=403 4.7.0 TLS handshake failed.
Feb 24 16:14:55 mail sm-mta[16288]: STARTTLS=client, error: connect failed=-1, reason=tlsv1 alert decode error, SSL_error=1, errno=0, retry=-1
Feb 24 16:14:55 mail sm-mta[16288]: ruleset=tls_server, arg1=SOFTWARE, relay=mail2.usbank.com, reject=403 4.7.0 TLS handshake failed.
Feb 24 16:14:56 mail sm-mta[16288]: STARTTLS=client, error: connect failed=-1, reason=tlsv1 alert decode error, SSL_error=1, errno=0, retry=-1
Feb 24 16:14:56 mail sm-mta[16288]: ruleset=tls_server, arg1=SOFTWARE, relay=mail4.usbank.com, reject=403 4.7.0 TLS handshake failed.

Select allOpen in new window

So, changing to using the gd_bundle.crt in the confCCERT definintion didn't help. For the moment, I've put sendmail.cf back to the old version.

Thoughts?

Jan Springer:

Were their directions for sendmail or for apache (or some other web server)?

GoDaddy has instructions for creating CSRs for: Exchange Server 2010, Exchange Server 2007, NGINX, IIS 7, TOmcat 4.x/5.x/6.x, IIS 8, Mac OS X 10.7-10.9, Parallels Plesk Panel, F5 BIG-IP, Exchange Server 2013, IIS 5 & 6, MAC 0S X Server 10.6, MAC OS X Server 10.5, CPanel/WebHost Manager, Apache 2.x

Nothing in the list for sendmail TLS, so I picked Apache. When I went to download the cert, GoDaddy asked for type of server. My choices are: Apache, Exchange, IIS, Mac OS X, Tomcat, Other. I picked Apache, then downloaded again as Other. Both downloads contained exactly the same files: gd_bundle.crt and  4111d11611d96.crt, and they both had the same md5sums on the respective files.

You shouldn't have to get a copy of the GoDaddy Ca data, it should already be in your bundle.  The exception is if you need an intermediate cert and that doesn't go in the CA bundle.

Not sure I follow what do you mean by "copy of the GoDaddy Ca data"?

gheist:

Nobody uses validated certificate chain for mail servers because nobody checks them (for interoperability)

You guys need to bring in down a few notches for me. This is my 2nd time installing SSL cert for TLS and I probably didn't do it right the first time!

have you tried it with ".pem" configuration and put the auth stuff above the cert stuff.

OK, for the heck of it, I downloaded from GoDaddy for servers Exchange and Mac OS X. Both had the c5fe0cc8242d6030.crt file. Neither had the gd_bundle.crt.

The Exchange download had gd_iis_intermediates.p7b

The Mac OS X download had: gd_intermediate.crt

Is this useful information?

Jan Springer:

have you tried it with ".pem" configuration ...

What is the .pem configuration?

if you read the directions for sendmail tls, they are very clear in configuring the crt and key files in pem format.

https://support.eapps.com/index.php?/Knowledgebase/Article/View/115/67/user-guide---securing-email-with-ssltls

Jan Springer:

if you read the directions for sendmail tls, they are very clear in configuring the crt and key files in pem format.

Not sure this is helping me. I don't have a /etc/pki/tls/certs, nor a makefile for sendmail.pem that I know of.

This is precisely the command the GoDaddy tech gave me to create the CSR and keyfile:

openssl req -new -newkey rsa:2048 -nodes -keyout mail.ohprs.org.key -out mail.ohprs.org.csr

Is it your opinion that he was wrong? If so, I'll connect back with GoDaddy and re-gen the key. I can do that without tech help. What do you think the CSR and key generation commands should be?

in my opinion, you should probably follow the sendmail directions and create pem files.

Jan Springer:

in my opinion, you should probably follow the sendmail directions and create pem files.

What are the "sendmail instructions"?

According to my searches, .key and .crt are PEM formats, just different extension names for recognition by e.g. Windows.

http://serverfault.com/questions/9708/what-is-a-pem-file-and-how-does-it-differ-from-other-openssl-generated-key-file

Per example at http://www.slackwiki.com/Sendmail_TLS_SASL_SMTP-AUTH, tried:

openssl genrsa 2048 > smtp.key.pem
 openssl req -new -key smtp.key.pem > newreq.csr

GoDaddy says "You entered an invalid CSR. Please try again."

I posted the link in a couple of posts above.  try not using a registered cert, create a snake oil cert first and get it working.  noone looks at the cert for mail.

Jan Springer:

I posted the link in a couple of posts above ...

As I responded, that site was for Centos OS 4/5, and Fedora. Perhaps I didn't mention this earlier, but I'm using Slackware 14.1. I don't have a /etc/pki/tls/certs directory or the custom makefile specified by your link.

I did follow a Slakware howto on this at http://www.slackwiki.com/Sendmail_TLS_SASL_SMTP-AUTH, but as mentioned in my last post GoDaddy refused the resulting CSR saying, " "You entered an invalid CSR. Please try again."

I have a help request in to GoDaddy on this as they seem to have CSR creation and installation instructions for MTA's I've never heard of (NGINX?), but not Sendmail.

I will do as you suggest and try using CAcert.org, which I've used before ... I'll post back results

Man, I am just soooo confused! Latest /etc/mail/maillog messages:

Feb 24 23:18:49 mail sm-mta[3169]: STARTTLS=server, error: SSL_CTX_use_certificate_file(/etc/ssl/certs/OHPRS/CACERT/mail.ohprs.org.crt) failed
Feb 24 23:18:50 mail sm-mta[3171]: STARTTLS=client, error: SSL_CTX_use_PrivateKey_file(/etc/ssl/certs/OHPRS/CACERT/mail.ohprs.org.crt) failed

Select allOpen in new window

This is the keyfile I just created and .crt file I just downloaded from CAcert.com.

Files exist:

$ ls -l /etc/ssl/certs/OHPRS/CACERT/
total 16
-rw------- 1 root root 2569 2015-02-24 23:10 CAroot.cer
-rw------- 1 root root 1074 2015-02-24 23:08 mail.ohprs.org.crt
-rw------- 1 root root 1074 2015-02-24 22:03 newreq.csr
-rw------- 1 root root 1675 2015-02-24 22:02 smtp.key.pem

Select allOpen in new window

Relevant .mc file settings:

TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confCACERT_PATH',`/etc/ssl/certs/')dnl
define(`confCACERT',`/etc/ssl/certs/OHPRS/CACERT/CAroot.cer')dnl
define(`confSERVER_CERT',`/etc/ssl/certs/OHPRS/CACERT/mail.ohprs.org.crt')dnl
define(`confSERVER_KEY',`/etc/ssl/certs/OHPRS/CACERT/smtp.key.pem')dnl
define(`confCLIENT_CERT',`/etc/ssl/certs/OHPRS/CACERT/mail.ohprs.org.crt')dnl
define(`confCLIENT_KEY',`/etc/ssl/certs/OHPRS/CACERT/mail.ohprs.org.crt')dnl
define(`confAUTH_OPTIONS', `A')dnl

confCACERT is cacert.org's Class 1 PKI Key, PEM format.

I've done this before with CAcert and it worked ... I'm going to try to scrouge up other instructions

Last try for the evening (man, I thought this would be a snap!)

I regen'd everything according to some old instructions I had:

$ penssl genrsa -out privkey.pem 4096
$ openssl req -new -key privkey.pem -out cert.csr

Select allOpen in new window

Pasted CSR at cacert.org, got cert and pasted to cert.pem in path. New .mc settings:

TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN')dnl
define(`confCACERT_PATH',`/etc/ssl/certs/OHPRS/CACERT2')dnl
define(`confCACERT',`/etc/ssl/certs/OHPRS/CACERT2/CAroot.cer')dnl
define(`confSERVER_CERT',`/etc/ssl/certs/OHPRS/CACERT2/cert.pem')dnl
define(`confSERVER_KEY',`/etc/ssl/certs/OHPRS/CACERT2/privkey.pem')dnl
define(`confCLIENT_CERT',`/etc/ssl/certs/OHPRS/CACERT2/cert.pem')dnl
define(`confCLIENT_KEY',`/etc/ssl/certs/OHPRS/CACERT2/privkey.pem')dnl
define(`confAUTH_OPTIONS', `A')dnl

Select allOpen in new window

I no longer have the "_file(...) failed" error, but still no improvement otherwise:

Feb 24 23:37:54 mail sm-mta[7530]: STARTTLS=client, error: connect failed=-1, reason=tlsv1 alert decode error, SSL_error=1, errno=0, retry=-1
Feb 24 23:37:54 mail sm-mta[7530]: ruleset=tls_server, arg1=SOFTWARE, relay=mail1.usbank.com, reject=403 4.7.0 TLS handshake failed.
Feb 24 23:37:54 mail sm-mta[7530]: STARTTLS=client, error: connect failed=-1, reason=tlsv1 alert decode error, SSL_error=1, errno=0, retry=-1

Feb 24 23:37:56 mail sm-mta[7530]: STARTTLS=client, error: connect failed=-1, reason=tlsv1 alert decode error, SSL_error=1, errno=0, retry=-1
Feb 24 23:37:56 mail sm-mta[7530]: ruleset=tls_server, arg1=SOFTWARE, relay=mail02.paycor.com, reject=403 4.7.0 TLS handshake failed.
Feb 24 23:37:56 mail sm-mta[7530]: t1OEnKLO016503: to=<PKeer@paycor.com>, delay=13:48:36, xdelay=00:00:01, mailer=esmtp, pri=3906724, relay=mail02.paycor.com. [69.61.226.160], dsn=4.0.0, stat=Deferred: 403 4.7.0 TLS handshake failed.
Feb 24 23:37:56 mail sm-mta[7530]: t1NLugQa013458: to=<PKeer@paycor.com>, delay=1+06:41:13, xdelay=00:00:00, mailer=esmtp, pri=7596649, relay=mail02.paycor.com., dsn=4.0.0, stat=Deferred

Select allOpen in new window

Note, this site http://mail-index.netbsd.org/tech-userlevel/2014/06/17/msg008594.html, says:

After upgrading OpenSSL to 1.0.1g and 1.0.1h, sendmail started
producing this error when sending messages to some sites:

Jun 17 05:47:47 merteuil sendmail[14089]: STARTTLS=client, error: connect
failed=-1, reason=tlsv1 alert decode error, SSL_error=1, errno=0, retry=-1

After some investigation, it seems that the TLS padding extension, which
was introduced in OpenSSL 1.0.1g, is the culprit. There are a few workarounds:

(1) Force SSLv3, which cannot use the option. This does not require any
code change but is not very appealing.

Possible issue with my OpenSSL version? OpenSSL 1.0.1i 6 Aug 2014

Note that on my sites that work: OpenSSL 0.9.8r 8 Feb 2011

Yeah, this "tlsv1 alert decode error" is all over the web. Most are blaming it on clients not being able to hand tslv1 "padding". For testing, how can I force TLSv1/SSLv3, what would be the .mc statement for ${tls_version}?

Mark,

One thing to use is: openssl s_client -connect mail.hprs.local:25  -starttls -smtp
openssl s_client -connect mail.hprs.local:sslport -SMTP

Some of the earlier errors pointed to the certificate not including the signer chain I.e. .crt file needs to include the certificate chain from godaddy.

These days the function/additional settings are required when generating a CSR indicating the certificate includes mail exchange function.

Another error I saw in your earlier post with external domain and a tls error that suggest your configuration for your sendmail when it is sending email out to use TLS a with the remote which not sure you want to use that.

Arnold:

One thing to use is: openssl s_client -connect mail.hprs.local:25  -starttls -smtp

I did try that, but couldn't make much sense of the ouput. Here is one run using -ssl3:

$ openssl s_client -starttls smtp -ssl3 -connect mail1.usbank.com:25
CONNECTED(00000003)
139767667656384:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure:s3_pkt.c:1275:SSL alert number 40
139767667656384:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:598:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 127 bytes and written 0 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : SSLv3
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1424906204
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---

Select allOpen in new window

And here is one without the -ssl3

$ openssl s_client -starttls smtp -connect mail1.usbank.com:25
CONNECTED(00000003)
depth=2 C = US, O = "Entrust, Inc.", OU = See www.entrust.net/legal-terms, OU = "(c) 2009 Entrust, Inc. - for authorized use only", CN = Entrust Root Certification Authority - G2
verify error:num=19:self signed certificate in certificate chain
verify return:0
---
Certificate chain
 0 s:/C=US/ST=Minnesota/L=Richfield/O=US Bank/OU=ISS/CN=MAIL1.USBANK.COM
   i:/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2012 Entrust, Inc. - for authorized use only/CN=Entrust Certification Authority - L1K
 1 s:/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2012 Entrust, Inc. - for authorized use only/CN=Entrust Certification Authority - L1K
   i:/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2009 Entrust, Inc. - for authorized use only/CN=Entrust Root Certification Authority - G2
 2 s:/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2009 Entrust, Inc. - for authorized use only/CN=Entrust Root Certification Authority - G2
   i:/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2009 Entrust, Inc. - for authorized use only/CN=Entrust Root Certification Authority - G2
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFxTCCBK2gAwIBAgIEUNKF7zANBgkqhkiG9w0BAQsFADCBujELMAkGA1UEBhMC
VVMxFjAUBgNVBAoTDUVudHJ1c3QsIEluYy4xKDAmBgNVBAsTH1NlZSB3d3cuZW50
cnVzdC5uZXQvbGVnYWwtdGVybXMxOTA3BgNVBAsTMChjKSAyMDEyIEVudHJ1c3Qs
IEluYy4gLSBmb3IgYXV0aG9yaXplZCB1c2Ugb25seTEuMCwGA1UEAxMlRW50cnVz
dCBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eSAtIEwxSzAeFw0xNDExMjExNDM2NDda
Fw0xNzExMjAxNzU3NTJaMHAxCzAJBgNVBAYTAlVTMRIwEAYDVQQIEwlNaW5uZXNv
dGExEjAQBgNVBAcTCVJpY2hmaWVsZDEQMA4GA1UEChMHVVMgQmFuazEMMAoGA1UE
CxMDSVNTMRkwFwYDVQQDExBNQUlMMS5VU0JBTksuQ09NMIIBIjANBgkqhkiG9w0B
AQEFAAOCAQ8AMIIBCgKCAQEAgBj+V9PqtKV6c2tq0g4XPA2ZYN7SU2aEhCqYw8I7
E8tKx0GS4baXfUiS/0z8b2ZX4/n53pWQ6ymr4i/pRDBEKKt7Ai34UZoWiBHj+2x2
Ex/+dTer1lSh5h7j4yC7badwbL/KQUFzJtfJLG2Jpoeei2857gUL9qNtwsfI5vC6
ZNDf0XfEZxUoDyY3/wZe298D00xl+WeSfnzee3flMVpPkqlo7RZa4in7PbE9oWXq
yW+45+Wu1wh5W3XBXfndrT0U2a+76MQM/eMLPCZ0hABJOr24ueVTxHCSrSBQyWw5
+OmUqrsr5Y8lKbHWVY5ZB9eQRV/dQc90Lexer5ufQlpdrwIDAQABo4ICGjCCAhYw
CwYDVR0PBAQDAgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAzBgNV
HR8ELDAqMCigJqAkhiJodHRwOi8vY3JsLmVudHJ1c3QubmV0L2xldmVsMWsuY3Js
MEsGA1UdIAREMEIwNgYKYIZIAYb6bAoBBTAoMCYGCCsGAQUFBwIBFhpodHRwOi8v
d3d3LmVudHJ1c3QubmV0L3JwYTAIBgZngQwBAgIwaAYIKwYBBQUHAQEEXDBaMCMG
CCsGAQUFBzABhhdodHRwOi8vb2NzcC5lbnRydXN0Lm5ldDAzBggrBgEFBQcwAoYn
aHR0cDovL2FpYS5lbnRydXN0Lm5ldC9sMWstY2hhaW4yNTYuY2VyMIGwBgNVHREE
gagwgaWCEE1BSUwxLlVTQkFOSy5DT02CEG1haWwyLnVzYmFuay5jb22CEG1haWwz
LnVzYmFuay5jb22CEG1haWw0LnVzYmFuay5jb22CEG1haWw1LnVzYmFuay5jb22C
EG1haWw5LnVzYmFuay5jb22CEW1haWwxMC51c2JhbmsuY29tghFtYWlsMTMudXNi
YW5rLmNvbYIRbWFpbDE0LnVzYmFuay5jb20wHwYDVR0jBBgwFoAUgqJwdN28Uz/P
e9T3zX+nYMYKTL8wHQYDVR0OBBYEFEkrQFAAvMduT7b0GkyDcX0G/Qw3MAkGA1Ud
EwQCMAAwDQYJKoZIhvcNAQELBQADggEBADJ51+/MV2yN7zTEmDmtSEuJzoRMRoVT
+WTuDcOxG8lhD04XpLqpUm1fR3+WwhuMKEXVUZySAUUw7tUhPLPcnvoCDDXJ7K+A
8xpJxf0X6sU0rbZs427a7aLn7LZq6ZExQechtc6njmfIn/ZjUFMe7ANCtAFKxxXh
Pw/Vv57kBjGUQz2Zb0YiovGpQGjkwcTD5uQp+i20FptpafVC7KMJEEY1leYFuaPA
sdBVB5J5PHcOpw8Cywx75ldi6izoUp505szQUCUsJpDOQxwOs6QvVa4ErsNWOlOu
3Gw0XWgU6Rd2jXwY3xhAbbwCaoble/+neBp6DAPgk/JwRwFQJgE3oro=
-----END CERTIFICATE-----
subject=/C=US/ST=Minnesota/L=Richfield/O=US Bank/OU=ISS/CN=MAIL1.USBANK.COM
issuer=/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2012 Entrust, Inc. - for authorized use only/CN=Entrust Certification Authority - L1K
---
No client certificate CA names sent
---
SSL handshake has read 4799 bytes and written 538 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID: 80AC74E4D8A7B7692DB1303D281CD7163C21C15E3C7CBCA5E03BA2F4A925E86B
    Session-ID-ctx:
    Master-Key: F5D056E07989986086C1E581DCD0CDE8BA79F80F36D32DB9C80AD88CA6D098D8FC25AFA63EE979FDDCE22EC0980B5576
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket:
    0000 - 0e d1 4f e2 21 d9 6f 8b-96 b6 e8 71 9b df 81 d6   ..O.!.o....q....
    0010 - 27 0d af 22 9d 81 35 db-cd 6c 01 e1 5e ed 67 3d   '.."..5..l..^.g=
    0020 - 6a 5c e4 e9 5a a2 9d fc-3a 42 0d 1d db 17 72 51   j\..Z...:B....rQ
    0030 - fd 4d 45 4c 05 28 ef 92-20 41 0c 31 96 94 5a e1   .MEL.(.. A.1..Z.
    0040 - 46 8e 25 c2 df 5e f4 b3-74 66 b9 81 3a b4 7b 9b   F.%..^..tf..:.{.
    0050 - da 4f 68 c6 8f c9 28 ac-75 59 41 00 9c b9 21 14   .Oh...(.uYA...!.
    0060 - c4 3d 10 d2 51 24 d0 b8-4f 9d c8 bc 1a bf be d3   .=..Q$..O.......
    0070 - 0c bc 08 2f 64 b4 4a f9-7c 68 4a 5d ab d8 45 db   .../d.J.|hJ]..E.
    0080 - 93 15 39 c9 a3 31 2f f3-c9 b0 f5 a7 85 e2 fd c4   ..9..1/.........
    0090 - c1 2e c9 fc 8a f2 88 76-a2 06 8e a2 3b d9 1a 7b   .......v....;..{
    00a0 - 91 56 91 4a a5 08 55 d3-c9 44 e9 38 9b 34 f4 3a   .V.J..U..D.8.4.:
    00b0 - d4 b3 69 5c 18 0c 30 6d-32 87 5c 1a 1c 25 7f b7   ..i\..0m2.\..%..

    Start Time: 1424906320
    Timeout   : 300 (sec)
    Verify return code: 19 (self signed certificate in certificate chain)
---
250 STARTTLS

Select allOpen in new window

That one just hung at the '250 STARTTLS' and I CRTL-C'd to get out. Didn't try your one to mail.hprs.local, that gave:

> openssl s_client -connect mail.hprs.local:sslport -SMTP
getservbyname failure for sslport
usage: s_client args

Select allOpen in new window

Then a 'help' listing of possible arguments.

Some of the earlier errors pointed to the certificate not including the signer chain I.e. .crt file needs to include the certificate chain from godaddy.

I contacted goDaddy about instructions for generating certs for Sendmail and their response was that they don't have instructions for that server.

Right now, I'm using a cert from CAcert.org -- that is what I generated those `open s_client` outputs from. I'll switch back to GoDaddy and try again.

I received 2 files from GoDaddy: c5fe0cc8242d6030.crt and gd_bundle.crt. Not sure if either one of those has the "certificate chain" you mentioned. I know you're not too familiar with sendmail. but I have the following settings:

define(`confCACERT_PATH',`/etc/ssl/certs/OHPRS/CACERT2')dnl
define(`confCACERT',`/etc/ssl/certs/OHPRS/CACERT2/CAroot.cer')dnl
define(`confSERVER_CERT',`/etc/ssl/certs/OHPRS/CACERT2/cert.pem')dnl
define(`confSERVER_KEY',`/etc/ssl/certs/OHPRS/CACERT2/privkey.pem')dnl
define(`confCLIENT_CERT',`/etc/ssl/certs/OHPRS/CACERT2/cert.pem')dnl
define(`confCLIENT_KEY',`/etc/ssl/certs/OHPRS/CACERT2/privkey.pem')dnl
define(`confAUTH_OPTIONS', `A')dnl

Supposedly, confCACERT is the root certificate. I've tried using the gd_bundle.crt file here.

confSERVER_CERT is supposedly the certificate received from the issuer (in this example from CAcert.org, but with GoDaddy I was using the c5fe0cc8242d6030.crt  file.

confSERVER_KEY is supposedly my keyfile and is what I used to generate the CSR.

The confCLIENT... entries are for me as a client, which is what I am being when sending to these servers.

Another error I saw in your earlier post with external domain and a tls error that suggest your configuration for your sendmail when it is sending email out to use TLS a with the remote which not sure you want to use that.

Not sure what you mean, are you referring to the confSERVER... settings?

My latest theory is that there is something wrong with my version of openssl. I found numerous postings like http://serverfault.com/questions/588125/openssl-update-1-0-1f-to-1-0-1g-broke-sendmail-ssl23-get-server-hellotlsv1-ale indicating some kind of problem with TLS padding starting with version 1.0.1g. My previous setup used 0.9.8r and I had no problem. What I'm doing at the moment is updating my system to the latest release (slackpgk update-all) to see if this has been changed since I installed this host back August, 2014. If not, and if my configuration looks correct, I might try falling back to openssi 1.0.1f

I'll reset to GoDaddy's certificate and try those `openssl s_client` commands again.

the issues with openssl can be patched through an update.  

your sslport in the reference I posted needs to be replaced with the port you wish to use.  Note the attempt without the ssl3, the response from the destination includes the certificate chain i.e. the public certificates of the entities that signed the certificate.  This is one of the errors I saw that you certificate does not include the chains

The certificate would look like
CA certificate
ca subordinate certificate

your certificate.

use a browser (not chrome as it has a setting that blocks it from accessing a port ...
http://mail.hprs.local:465 and look at the certificate properties.
Compare them to the certificate of the others.

you can use openssl s_client -connect
get the server certificate and look at the listed purposes
one of them will be 2.16.840.1.114414.1.7.23.2 or 1.3.6.1.4.1.11129.2.5.1 or an additional entry indicating that it is beyond the standard server authentication that a web server certificate is.
I think these are the additional purposes that indicate it is a mail exchanger.

Deal with X509 additional attribute/function/purpose.

These days the mail exchanger (mail server) certificate have to include the additional purpose to indicate or the clients connecting to them with a web type certificate are seen as illegitimate.

The same issue is likely the starttls error.

the suggestion Jan made to create your own signed certificate will prove the point since usually self signed certificate have the widest possible function/purpose and will pass the certificate "validation" use test.

did you run the script or use the openssl -newreq to generate the CSR?
Do not remember exactly on the command line how to provide the additional attribute whether it is to have the certificate authorized for multiple names, or to include the additional attribute/policy identifier.

Sheesh! I don't know why I keep trying `slackpkg upgrade-all`. I guess I'm expecting it to work as reliably as Debian's apt-get [dist-]upgrade. The last 2 time I tried slackpkg it trashed my system to an unusable state and I had to restore. I just got finished doing that. I did note, however, that there is an openssl version 0.1.0k, newer than my 0.1.0i. I'll try just grabbing the latest .tar.gz file for that tomorrow.

Meanwhile, I put the GoDaddy cert back after my last post. My sendmail .mc config is now:

define(`confCACERT_PATH',`/etc/ssl/certs/')dnl
define(`confCACERT',`/etc/ssl/certs/OHPRS/GoDaddy/Apache/gd_bundle.crt')dnl
define(`confSERVER_CERT',`/etc/ssl/certs/OHPRS/GoDaddy/Apache/c5fe0cc8242d6030.crt')dnl
define(`confSERVER_KEY',`/etc/ssl/certs/OHPRS/GoDaddy/mail.ohprs.org.key')dnl
define(`confCLIENT_CERT',`/etc/ssl/certs/OHPRS/GoDaddy/Apache/c5fe0cc8242d6030.crt')dnl
define(`confCLIENT_KEY',`/etc/ssl/certs/OHPRS/GoDaddy/mail.ohprs.org.key')dnl
define(`confAUTH_OPTIONS', `A')dnl

Wherein I'm assuming the GoDaddy root cert is in the gd_bundle file. I've attached the `openssl x509 -text -in gd_bundle.crt` output if that sheds any light.

Anyway, all of the TLS requiring client servers who had stopped accepting my messages after I tried the CAcert.org experiment starting accepted my messages again -- except for the original two who prompted this question in the first place: usbank.com and paycor.com.

I'll post the `openssl s_client` results from one that worked and one that didn't and I'll digest the rest of your message tomorrow (well, it's already tomorrow!)

This one started working once the GoDaddy cert was re-installed. It waited for a while at the final "250 CHUNKING" statement, probably waiting on the actual message; I CTRL-C'd out. Even though it has some worrisome messages "(unable to get local issue certificate)", I can send messages successfully to this server.

$ openssl s_client -starttls smtp -connect das-ohio-gov.mail.protection.outlook.com:25
CONNECTED(00000003)
depth=2 CN = Microsoft Internet Authority
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
 0 s:/C=US/ST=WA/L=Redmond/O=Microsoft/OU=Forefront Online Protection for Exchange/CN=mail.protection.outlook.com
   i:/DC=com/DC=microsoft/DC=corp/DC=redmond/CN=MSIT Machine Auth CA 2
 1 s:/DC=com/DC=microsoft/DC=corp/DC=redmond/CN=MSIT Machine Auth CA 2
   i:/CN=Microsoft Internet Authority
 2 s:/CN=Microsoft Internet Authority
   i:/C=IE/O=Baltimore/OU=CyberTrust/CN=Baltimore CyberTrust Root
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIHITCCBgmgAwIBAgIKaJVJ8AABAADfvjANBgkqhkiG9w0BAQUFADCBgDETMBEG
CgmSJomT8ixkARkWA2NvbTEZMBcGCgmSJomT8ixkARkWCW1pY3Jvc29mdDEUMBIG
CgmSJomT8ixkARkWBGNvcnAxFzAVBgoJkiaJk/IsZAEZFgdyZWRtb25kMR8wHQYD
VQQDExZNU0lUIE1hY2hpbmUgQXV0aCBDQSAyMB4XDTE0MDUyOTIyMTk0NloXDTE2
MDUxNTIwNTA1NVowgZkxCzAJBgNVBAYTAlVTMQswCQYDVQQIEwJXQTEQMA4GA1UE
BxMHUmVkbW9uZDESMBAGA1UEChMJTWljcm9zb2Z0MTEwLwYDVQQLEyhGb3JlZnJv
bnQgT25saW5lIFByb3RlY3Rpb24gZm9yIEV4Y2hhbmdlMSQwIgYDVQQDExttYWls
LnByb3RlY3Rpb24ub3V0bG9vay5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
ggEKAoIBAQC//+TcN6C92y7BZE4E9+3VJfxW/QHCbOdk8/W2rZ9NXK+JfgM8t6lD
+Xi9IQflxEnOpuANelypefk5rfpJuiSnGRGMg44xAWQkhhBVynduvDRoddd9ieaC
LIC0rcuyeqpvXnw8MPZdp1nRn12XoOrDhUYBke3JRk9JKys5yOec+g5a65nUxp++
jDtQOHCN60n5MmGZH5a+/EX++ZpyC13SISHEcVLNRDMMHzpmYT3h5JjCe3AhMgTy
qbjavIddv5lAyuGw9UsSpmjdyQ0gLPepfKscZ/5bp6QRT8rOj3d4jTlAbqsjJM6y
PBHxAHXrLiCPC3mn38Eggs7PIAPce47/AgMBAAGjggOAMIIDfDALBgNVHQ8EBAMC
BLAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMHgGCSqGSIb3DQEJDwRr
MGkwDgYIKoZIhvcNAwICAgCAMA4GCCqGSIb3DQMEAgIAgDALBglghkgBZQMEASow
CwYJYIZIAWUDBAEtMAsGCWCGSAFlAwQBAjALBglghkgBZQMEAQUwBwYFKw4DAgcw
CgYIKoZIhvcNAwcwHQYDVR0OBBYEFOdAD77qj+T7cfw6+hwbEjOKBl9DMB8GA1Ud
IwQYMBaAFOvbEV74CZ7Y1mKc/WKd44RKKOEnMIHuBgNVHR8EgeYwgeMwgeCggd2g
gdqGT2h0dHA6Ly9tc2NybC5taWNyb3NvZnQuY29tL3BraS9tc2NvcnAvY3JsL01T
SVQlMjBNYWNoaW5lJTIwQXV0aCUyMENBJTIwMigxKS5jcmyGTWh0dHA6Ly9jcmwu
bWljcm9zb2Z0LmNvbS9wa2kvbXNjb3JwL2NybC9NU0lUJTIwTWFjaGluZSUyMEF1
dGglMjBDQSUyMDIoMSkuY3JshjhodHRwOi8vY29ycHBraS9jcmwvTVNJVCUyME1h
Y2hpbmUlMjBBdXRoJTIwQ0ElMjAyKDEpLmNybDCBrQYIKwYBBQUHAQEEgaAwgZ0w
VQYIKwYBBQUHMAKGSWh0dHA6Ly93d3cubWljcm9zb2Z0LmNvbS9wa2kvbXNjb3Jw
L01TSVQlMjBNYWNoaW5lJTIwQXV0aCUyMENBJTIwMigxKS5jcnQwRAYIKwYBBQUH
MAKGOGh0dHA6Ly9jb3JwcGtpL2FpYS9NU0lUJTIwTWFjaGluZSUyMEF1dGglMjBD
QSUyMDIoMSkuY3J0MD8GCSsGAQQBgjcVBwQyMDAGKCsGAQQBgjcVCIPPiU2t8gKF
oZ8MgvrKfYHh+3SBT4PC7YUIjqnShWMCAWQCAQ0wJwYJKwYBBAGCNxUKBBowGDAK
BggrBgEFBQcDAjAKBggrBgEFBQcDATCBiAYDVR0RBIGAMH6CFSoubWFpbC5lby5v
dXRsb29rLmNvbYIdKi5tYWlsLnByb3RlY3Rpb24ub3V0bG9vay5jb22CG21haWwu
cHJvdGVjdGlvbi5vdXRsb29rLmNvbYILb3V0bG9vay5jb22CHG1haWwubWVzc2Fn
aW5nLm1pY3Jvc29mdC5jb20wDQYJKoZIhvcNAQEFBQADggEBAG0IKQDUPEOjAOv2
RMUAzyveNL590cdIVRNb3qq9kOOAK2HsUJJy8AE6HXEhgAl2kOyeIUKLlO0iYVRe
Viapc0nAcmuGT0AJtNEOaklBBzEAxfMBVsDuo1N9ngGDH4sx0izkM1R6fkN6fjHe
lVWeyne4GnJG//RoiQDIoRcETgLhpr+fd972PupvF13ao+tC3L4MEx6K5KfDY4z9
Fvjz+uPd1Y/6h2PwmxyBR2C5G2hkAsKs7ZD2ZhI5JhI+Sle4JLFDcjhdYVHS/dGo
s5+lCADuoG4gaPkdHplaqHyF5p8kREhlCOlwhEp3c6LXoTjgG75Lu02V1YKy+DZK
v5STRJE=
-----END CERTIFICATE-----
subject=/C=US/ST=WA/L=Redmond/O=Microsoft/OU=Forefront Online Protection for Exchange/CN=mail.protection.outlook.com
issuer=/DC=com/DC=microsoft/DC=corp/DC=redmond/CN=MSIT Machine Auth CA 2
---
Acceptable client certificate CA names
/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
/C=US/O=GeoTrust Inc./OU=(c) 2008 GeoTrust Inc. - For authorized use only/CN=GeoTrust Primary Certification Authority - G3
/C=SI/O=ACNLB
/C=HU/L=Budapest/O=NetLock Kft./OU=Tan\xC3\xBAs\xC3\xADtv\xC3\xA1nykiad\xC3\xB3k (Certification Services)/CN=NetLock Arany (Class Gold) F\xC5\x91tan\xC3\xBAs\xC3\xADtv\xC3\xA1ny
/C=PL/O=Unizeto Technologies S.A./OU=Certum Certification Authority/CN=Certum Trusted Network CA
/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 1999 VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3 Public Primary Certification Authority - G3
/C=SE/O=Carelink/CN=SITHS CA v3
/C=HU/L=Budapest/O=Microsec Ltd./OU=e-Szigno CA/CN=Microsec e-Szigno Root CA
/O=RSA Security Inc/OU=RSA Security 2048 V3
/C=US/O=The Go Daddy Group, Inc./OU=Go Daddy Class 2 Certification Authority
/C=US/O=AffirmTrust/CN=AffirmTrust Networking
/C=US/ST=New Jersey/L=Jersey City/O=The USERTRUST Network/CN=USERTrust RSA Certification Authority
/C=IL/O=StartCom Ltd./CN=StartCom Certification Authority G2
/C=US/O=GeoTrust Inc./CN=GeoTrust Primary Certification Authority
/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2008 VeriSign, Inc. - For authorized use only/CN=VeriSign Universal Root Certification Authority
/C=JP/O=SECOM Trust.net/OU=Security Communication RootCA1
/C=FI/O=Sonera/CN=Sonera Class2 CA
/C=ES/ST=Madrid/L=Madrid/O=IPS Certification Authority s.l. ipsCA/OU=ipsCA/CN=ipsCA Global CA Root/emailAddress=global01@ipsca.com
/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority
/C=si/O=state-institutions/OU=sigen-ca
/O=TeliaSonera/CN=TeliaSonera Root CA v1
/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2
/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2006 VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3 Public Primary Certification Authority - G5
/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
/O=Entrust.net/OU=www.entrust.net/CPS_2048 incorp. by ref. (limits liab.)/OU=(c) 1999 Entrust.net Limited/CN=Entrust.net Certification Authority (2048)
/C=CH/O=WISeKey/OU=Copyright (c) 2005/OU=OISTE Foundation Endorsed/CN=OISTE WISeKey Global Root GA CA
/C=NL/O=Staat der Nederlanden/CN=Staat der Nederlanden Root CA - G2
/O=Cybertrust, Inc/CN=Cybertrust Global Root
/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
/C=FR/ST=France/L=Paris/O=PM/SGDN/OU=DCSSI/CN=IGC/A/emailAddress=igca@sgdn.pm.gouv.fr
/C=NO/O=Buypass AS-983163327/CN=Buypass Class 3 CA 1
/C=PL/O=Unizeto Sp. z o.o./CN=Certum CA
/C=DE/O=TC TrustCenter GmbH/OU=TC TrustCenter Universal CA/CN=TC TrustCenter Universal CA I
/C=EU/O=AC Camerfirma SA CIF A82743287/OU=http://www.chambersign.org/CN=Chambers of Commerce Root
/C=US/O=Network Solutions L.L.C./CN=Network Solutions Certificate Authority
/C=FR/O=Certplus/CN=Class 2 Primary CA
/OU=GlobalSign Root CA - R2/O=GlobalSign/CN=GlobalSign
/C=KR/O=Government of Korea/OU=GPKI/CN=GPKIRootCA1
/C=ES/O=Consejo General de la Abogacia NIF:Q-2863006I/CN=Autoridad de Certificacion de la Abogacia
/C=si/O=state-institutions/OU=sigov-ca
/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority - G2/OU=(c) 1998 VeriSign, Inc. - For authorized use only/OU=VeriSign Trust Network
/C=DE/O=Deutsche Telekom AG/OU=T-TeleSec Trust Center/CN=Deutsche Telekom Root CA 2
/C=US/O=SecureTrust Corporation/CN=SecureTrust CA
/C=CN/O=CNNIC/CN=CNNIC ROOT
/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2009 Entrust, Inc. - for authorized use only/CN=Entrust Root Certification Authority - G2
/C=US/O=U.S. Government/OU=FPKI/CN=Federal Common Policy CA
/C=CZ/CN=I.CA - Standard Certification Authority, 09/2009/O=Prvn\xC3\xAD certifika\xC4\x8Dn\xC3\xAD autorita, a.s./OU=I.CA - Provider of Certification Services
/C=US/O=GTE Corporation/CN=GTE CyberTrust Root
/C=US/O=thawte, Inc./OU=Certification Services Division/OU=(c) 2006 thawte, Inc. - For authorized use only/CN=thawte Primary Root CA
/C=US/O=GTE Corporation/OU=GTE CyberTrust Solutions, Inc./CN=GTE CyberTrust Global Root
/C=US/O=Entrust.net/OU=www.entrust.net/CPS incorp. by ref. (limits liab.)/OU=(c) 1999 Entrust.net Limited/CN=Entrust.net Secure Server Certification Authority
/C=CH/O=SwissSign AG/CN=SwissSign Silver CA - G2
/C=NO/O=Buypass AS-983163327/CN=Buypass Class 2 CA 1
/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA
/C=US/O=Starfield Technologies, Inc./OU=Starfield Class 2 Certification Authority
/C=DE/O=TC TrustCenter GmbH/OU=TC TrustCenter Class 2 CA/CN=TC TrustCenter Class 2 CA II
/C=ES/CN=Autoridad de Certificacion Firmaprofesional CIF A62634068
/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
/C=BE/O=GlobalSign nv-sa/OU=Root CA/CN=GlobalSign Root CA
/C=US/O=Entrust, Inc./OU=www.entrust.net/CPS is incorporated by reference/OU=(c) 2006 Entrust, Inc./CN=Entrust Root Certification Authority
/CN=T\xC3\x9CRKTRUST Elektronik Sertifika Hizmet Sa\xC4\x9Flay\xC4\xB1c\xC4\xB1s\xC4\xB1/C=TR/L=Ankara/O=T\xC3\x9CRKTRUST Bilgi \xC4\xB0leti\xC5\x9Fim ve Bili\xC5\x9Fim G\xC3\xBCvenli\xC4\x9Fi Hizmetleri A.\xC5\x9E. (c) Kas\xC4\xB1m 2005
/C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./CN=Starfield Root Certificate Authority - G2
/C=CN/O=WoSign CA Limited/CN=Certification Authority of WoSign
/C=CH/O=The Federal Authorities of the Swiss Confederation/OU=Services/OU=Certification Authorities/CN=Swiss Government Root CA II
/C=BM/O=QuoVadis Limited/CN=QuoVadis Root CA 2
/C=us/O=U.S. Government/OU=FBCA/CN=Common Policy
/C=GB/ST=Greater Manchester/L=Salford/O=Comodo CA Limited/CN=AAA Certificate Services
/C=AT/O=A-Trust Ges. f. Sicherheitssysteme im elektr. Datenverkehr GmbH/OU=A-Trust-nQual-03/CN=A-Trust-nQual-03
/C=IE/O=Baltimore/OU=CyberTrust/CN=Baltimore CyberTrust Root
/OU=GlobalSign Root CA - R3/O=GlobalSign/CN=GlobalSign
/C=HK/O=Hongkong Post/CN=Hongkong Post Root CA 1
/C=CH/O=SwissSign AG/CN=SwissSign Gold CA - G2
/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
/CN=T\xC3\x9CRKTRUST Elektronik Sertifika Hizmet Sa\xC4\x9Flay\xC4\xB1c\xC4\xB1s\xC4\xB1/C=TR/L=Ankara/O=T\xC3\x9CRKTRUST Bilgi \xC4\xB0leti\xC5\x9Fim ve Bili\xC5\x9Fim G\xC3\xBCvenli\xC4\x9Fi Hizmetleri A.\xC5\x9E. (c) Aral\xC4\xB1k 2007
/C=US/O=thawte, Inc./OU=Certification Services Division/OU=(c) 2008 thawte, Inc. - For authorized use only/CN=thawte Primary Root CA - G3
/C=IT/L=Milan/O=Actalis S.p.A./03358520967/CN=Actalis Authentication Root CA
/C=US/O=AffirmTrust/CN=AffirmTrust Commercial
/C=UY/O=ADMINISTRACION NACIONAL DE CORREOS/OU=SERVICIOS ELECTRONICOS/CN=Correo Uruguayo - Root CA
/C=US/ST=Washington/L=Redmond/O=Microsoft Corporation/CN=Microsoft Root Certificate Authority 2010
/OU=Copyright (c) 1997 Microsoft Corp./OU=Microsoft Corporation/CN=Microsoft Root Authority
/DC=com/DC=microsoft/CN=Microsoft Root Certificate Authority
/CN=NT AUTHORITY
---
SSL handshake has read 14990 bytes and written 566 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-SHA384
    Session-ID: E00D0000B4C253AA956206067340A83B44023A7E383C01BC4B1D2EDB3B7FF21B
    Session-ID-ctx:
    Master-Key: 2BD08E63495DB08571C70B9CAE9FF8F0C427117BB612F9F7AC4DE6F0C3B84BE3B0B923F664F2C0C99C60C0E51CEE7A09
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1424937769
    Timeout   : 300 (sec)
    Verify return code: 20 (unable to get local issuer certificate)
---
250 CHUNKING

Select allOpen in new window

Note that only the following lines are sent to stderr:

depth=2 CN = Microsoft Internet Authority
verify error:num=20:unable to get local issuer certificate
verify return:0
250 CHUNKING

Select allOpen in new window

This next one is one of the problem connections.

$ openssl s_client -starttls smtp -connect mail1.usbank.com:25
CONNECTED(00000003)
depth=2 C = US, O = "Entrust, Inc.", OU = See www.entrust.net/legal-terms, OU = "(c) 2009 Entrust, Inc. - for authorized use only", CN = Entrust Root Certification Authority - G2
verify error:num=19:self signed certificate in certificate chain
verify return:0
---
Certificate chain
 0 s:/C=US/ST=Minnesota/L=Richfield/O=US Bank/OU=ISS/CN=MAIL1.USBANK.COM
   i:/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2012 Entrust, Inc. - for authorized use only/CN=Entrust Certification Authority - L1K
 1 s:/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2012 Entrust, Inc. - for authorized use only/CN=Entrust Certification Authority - L1K
   i:/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2009 Entrust, Inc. - for authorized use only/CN=Entrust Root Certification Authority - G2
 2 s:/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2009 Entrust, Inc. - for authorized use only/CN=Entrust Root Certification Authority - G2
   i:/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2009 Entrust, Inc. - for authorized use only/CN=Entrust Root Certification Authority - G2
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFxTCCBK2gAwIBAgIEUNKF7zANBgkqhkiG9w0BAQsFADCBujELMAkGA1UEBhMC
VVMxFjAUBgNVBAoTDUVudHJ1c3QsIEluYy4xKDAmBgNVBAsTH1NlZSB3d3cuZW50
cnVzdC5uZXQvbGVnYWwtdGVybXMxOTA3BgNVBAsTMChjKSAyMDEyIEVudHJ1c3Qs
IEluYy4gLSBmb3IgYXV0aG9yaXplZCB1c2Ugb25seTEuMCwGA1UEAxMlRW50cnVz
dCBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eSAtIEwxSzAeFw0xNDExMjExNDM2NDda
Fw0xNzExMjAxNzU3NTJaMHAxCzAJBgNVBAYTAlVTMRIwEAYDVQQIEwlNaW5uZXNv
dGExEjAQBgNVBAcTCVJpY2hmaWVsZDEQMA4GA1UEChMHVVMgQmFuazEMMAoGA1UE
CxMDSVNTMRkwFwYDVQQDExBNQUlMMS5VU0JBTksuQ09NMIIBIjANBgkqhkiG9w0B
AQEFAAOCAQ8AMIIBCgKCAQEAgBj+V9PqtKV6c2tq0g4XPA2ZYN7SU2aEhCqYw8I7
E8tKx0GS4baXfUiS/0z8b2ZX4/n53pWQ6ymr4i/pRDBEKKt7Ai34UZoWiBHj+2x2
Ex/+dTer1lSh5h7j4yC7badwbL/KQUFzJtfJLG2Jpoeei2857gUL9qNtwsfI5vC6
ZNDf0XfEZxUoDyY3/wZe298D00xl+WeSfnzee3flMVpPkqlo7RZa4in7PbE9oWXq
yW+45+Wu1wh5W3XBXfndrT0U2a+76MQM/eMLPCZ0hABJOr24ueVTxHCSrSBQyWw5
+OmUqrsr5Y8lKbHWVY5ZB9eQRV/dQc90Lexer5ufQlpdrwIDAQABo4ICGjCCAhYw
CwYDVR0PBAQDAgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAzBgNV
HR8ELDAqMCigJqAkhiJodHRwOi8vY3JsLmVudHJ1c3QubmV0L2xldmVsMWsuY3Js
MEsGA1UdIAREMEIwNgYKYIZIAYb6bAoBBTAoMCYGCCsGAQUFBwIBFhpodHRwOi8v
d3d3LmVudHJ1c3QubmV0L3JwYTAIBgZngQwBAgIwaAYIKwYBBQUHAQEEXDBaMCMG
CCsGAQUFBzABhhdodHRwOi8vb2NzcC5lbnRydXN0Lm5ldDAzBggrBgEFBQcwAoYn
aHR0cDovL2FpYS5lbnRydXN0Lm5ldC9sMWstY2hhaW4yNTYuY2VyMIGwBgNVHREE
gagwgaWCEE1BSUwxLlVTQkFOSy5DT02CEG1haWwyLnVzYmFuay5jb22CEG1haWwz
LnVzYmFuay5jb22CEG1haWw0LnVzYmFuay5jb22CEG1haWw1LnVzYmFuay5jb22C
EG1haWw5LnVzYmFuay5jb22CEW1haWwxMC51c2JhbmsuY29tghFtYWlsMTMudXNi
YW5rLmNvbYIRbWFpbDE0LnVzYmFuay5jb20wHwYDVR0jBBgwFoAUgqJwdN28Uz/P
e9T3zX+nYMYKTL8wHQYDVR0OBBYEFEkrQFAAvMduT7b0GkyDcX0G/Qw3MAkGA1Ud
EwQCMAAwDQYJKoZIhvcNAQELBQADggEBADJ51+/MV2yN7zTEmDmtSEuJzoRMRoVT
+WTuDcOxG8lhD04XpLqpUm1fR3+WwhuMKEXVUZySAUUw7tUhPLPcnvoCDDXJ7K+A
8xpJxf0X6sU0rbZs427a7aLn7LZq6ZExQechtc6njmfIn/ZjUFMe7ANCtAFKxxXh
Pw/Vv57kBjGUQz2Zb0YiovGpQGjkwcTD5uQp+i20FptpafVC7KMJEEY1leYFuaPA
sdBVB5J5PHcOpw8Cywx75ldi6izoUp505szQUCUsJpDOQxwOs6QvVa4ErsNWOlOu
3Gw0XWgU6Rd2jXwY3xhAbbwCaoble/+neBp6DAPgk/JwRwFQJgE3oro=
-----END CERTIFICATE-----
subject=/C=US/ST=Minnesota/L=Richfield/O=US Bank/OU=ISS/CN=MAIL1.USBANK.COM
issuer=/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2012 Entrust, Inc. - for authorized use only/CN=Entrust Certification Authority - L1K
---
No client certificate CA names sent
---
SSL handshake has read 4799 bytes and written 538 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID: 0881BA1F9F13B375CFC249391924306C66F84D171EDB265755A817BEB876BF79
    Session-ID-ctx:
    Master-Key: 4AA78B94D2FB638C52772BE2062DBD8725D54BAA59071082E3048FD10FB690C3BDCE7537BAF56A027DD49E237F4410F5
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket:
    0000 - 0e d1 4f e2 21 d9 6f 8b-96 b6 e8 71 9b df 81 d6   ..O.!.o....q....
    0010 - 01 2c 89 5f eb 9e f6 2d-b7 00 0f 43 88 c5 92 58   .,._...-...C...X
    0020 - eb 42 77 a5 64 91 c9 07-63 64 26 52 a5 4d 51 57   .Bw.d...cd&R.MQW
    0030 - 98 65 86 c8 c8 6b 7f 8a-d6 58 80 b8 fc 4f 1e 51   .e...k...X...O.Q
    0040 - bd 71 8f ec fd 89 98 07-98 36 06 d7 a8 81 b5 61   .q.......6.....a
    0050 - 61 08 a1 fb b6 19 6c 31-d5 03 5a ed cd 04 66 fd   a.....l1..Z...f.
    0060 - 2b fb 8f 52 fc f7 99 81-06 bd a4 f0 55 b1 61 aa   +..R........U.a.
    0070 - 35 6d 92 60 13 78 30 aa-3d 3b d8 c5 0a 3f f8 6c   5m.`.x0.=;...?.l
    0080 - 35 57 12 6a 95 d3 0d 14-c8 4f fc a3 d9 54 0e 46   5W.j.....O...T.F
    0090 - 4c 5d ed ef 43 b5 ce bf-9a 65 ef 03 a0 0c 87 67   L]..C....e.....g
    00a0 - c0 68 9b ec c5 24 df 97-9a cc 67 ae 8b 17 14 85   .h...$....g.....
    00b0 - c5 e9 42 fc af 78 91 a2-94 9c 00 d0 50 9b e8 bb   ..B..x......P...

    Start Time: 1424936414
    Timeout   : 300 (sec)
    Verify return code: 19 (self signed certificate in certificate chain)
---
250 STARTTLS
421 Exceeded allowable connection time, disconnecting.
closed

Select allOpen in new window

This one send only the following to stderr, (Self signed certificate in certificate chain?):

depth=2 C = US, O = "Entrust, Inc.", OU = See www.entrust.net/legal-terms, OU = "(c) 2009 Entrust, Inc. - for authorized use only", CN = Entrust Root Certification Authority - G2
verify error:num=19:self signed certificate in certificate chain
verify return:0

Select allOpen in new window

Your suggested command `openssl s_client -connect mail.hprs.local:25 -SMTP` doesn't work. It says "unknown option -SMTP". I played around with various permutations, but no luck.

To answer some of your questions/comments:

one of the errors I saw that you certificate does not include the chains

Does that still appear to be the case now that I'm using the official GoDaddy certificate?

http://mail.hprs.local:465

Can't access, but if I leave off the port 465 and click on the 'Certificate Error' in the address bar I get the dialog shown in image certView.jpg

did you run the script or use the openssl -newreq to generate the CSR?

I ran the following per the specific instructions from the GoDaddy tech to create the CSR. Beyond that, they were no help and didn't mention anything about "purpose":

openssl req -new -newkey rsa:2048 -nodes -keyout mail.ohprs.org.key -out mail.ohprs.org.csr

I'll digest the rest of your comments tomorrow.

Thanks
gd-bundle.crt.txt
certView.jpg

Arnold: Revisiting your comments ...

get the server certificate and look at the listed purposes
 one of them will be 2.16.840.1.114414.1.7.23.2 or 1.3.6.1.4.1.11129.2.5.1 or an additional entry indicating that it is beyond the standard server authentication that a web server certificate is.

In my GoDaddy cert (c5fe0cc8242d6030.crt) I have "Policy: 2.16.840.1.114413.1.7.23.1". Is that a good thing?

the suggestion Jan made to create your own signed certificate will prove the point since usually self signed certificate have the widest possible function/purpose and will pass the certificate "validation" use test.

I can generate a self signed certificate, but I'm kind of at a loss as to what to do with it? The troublesome recipients are not likely to accept it. What should I do to "test" with it?

btw - have downloaded and built openssl 1.0.2. Will install after business hours (about an hour or so from now).

Recipients aside, not sure why you are configuring your outgoing sendmail function to use TLS when available.

If you only are using this certificate for TLS when your system is the sender, I think you have the wrong certificate type.  I think the certificate you have deals with identifying your server for purposes of others sending through it or to it with TLS.

The client certificate when your sendmail smtp connects and presents the same certificate, it is being rejected.

I understand why you would configure your outgoing sendmail to have the option to exchange email messages in a more secure way (TLS) not sure whether you would actually have to have two different certificates. or have one with multiple purposes that includes the client identification.

you could try to determine the issue with the recipient using the openssl s_client -cert client.crt -key key.crt -starttls smtp -connect remotemx:25
and see what the error if any.
they return.

Mark,

You need to be clear on which part you are looking to resolve.
It looks that you've resolved the issue of incoming TLS requests.

The remaining issue deals with outgoing emails from your server trying to negotiate a TLS session after the connection is established.

You should not validate remote cert, just log validation result.

I installed openssh 1.0.2. No change with failure to send it to "bad" servers. I did get a 100% OK with TLScheck.com (albeit as a server, not client), see attached pdf

Arnold:

Recipients aside, not sure why you are configuring your outgoing sendmail function to use TLS when available.

If I understand your comment correctly, we are a pension system and certain recipients: banks, government office, health insurance companies, etc. require TLS. This is the case for my 2 problem recipients (note that all other institutions that require TLS are able to receive our messages).

If you only are using this certificate for TLS when your system is the sender, I think you have the wrong certificate type.  I think the certificate you have deals with identifying your server for purposes of others sending through it or to it with TLS.

Hmm, well, I got this sendmail config from an example years ago and it seems to work for other sendmail servers. For the office server (before current sendmail installation), we downloaded certs for Exchange from GoDaddy and they worked. I did download an Exchange version for this exercise and instead of the gd_bundle.crt it has gd_iis_intermediates.p7b. Not sure what's in that since openssl x509 -text doesn't recognize it.

Are you suggesting I need a separate/different certificate as the client? How? Certainly GoDaddy doesn't ask whether the cert is for client or server use.

you could try to determine the issue with the recipient using the openssl s_client -cert client.crt -key key.crt -starttls smtp -connect remotemx:25

Here are the results of that (unable to get local issuer certificate):

> openssl s_client -cert GoDaddy/Apache/c5fe0cc8242d6030.crt -key GoDaddy/mail.ohprs.org.key -starttls smtp -connect mail02.paycor.com:25 2>&1 | tee out
depth=1 C = US, O = "Entrust, Inc.", OU = www.entrust.net/rpa is incorporated by reference, OU = "(c) 2009 Entrust, Inc.", CN = Entrust Certification Authority - L1C
verify error:num=20:unable to get local issuer certificate
verify return:0
CONNECTED(00000003)
---
Certificate chain
 0 s:/C=US/ST=Ohio/L=Cincinnati/O=Paycor, Inc/CN=mail02.paycor.com
   i:/C=US/O=Entrust, Inc./OU=www.entrust.net/rpa is incorporated by reference/OU=(c) 2009 Entrust, Inc./CN=Entrust Certification Authority - L1C
 1 s:/C=US/O=Entrust, Inc./OU=www.entrust.net/rpa is incorporated by reference/OU=(c) 2009 Entrust, Inc./CN=Entrust Certification Authority - L1C
   i:/O=Entrust.net/OU=www.entrust.net/CPS_2048 incorp. by ref. (limits liab.)/OU=(c) 1999 Entrust.net Limited/CN=Entrust.net Certification Authority (2048)
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFFTCCA/2gAwIBAgIETCD45jANBgkqhkiG9w0BAQUFADCBsTELMAkGA1UEBhMC
VVMxFjAUBgNVBAoTDUVudHJ1c3QsIEluYy4xOTA3BgNVBAsTMHd3dy5lbnRydXN0
Lm5ldC9ycGEgaXMgaW5jb3Jwb3JhdGVkIGJ5IHJlZmVyZW5jZTEfMB0GA1UECxMW
KGMpIDIwMDkgRW50cnVzdCwgSW5jLjEuMCwGA1UEAxMlRW50cnVzdCBDZXJ0aWZp
Y2F0aW9uIEF1dGhvcml0eSAtIEwxQzAeFw0xMzEwMDYyMjU3NTRaFw0xNTEwMDkw
MjE4MjFaMGMxCzAJBgNVBAYTAlVTMQ0wCwYDVQQIEwRPaGlvMRMwEQYDVQQHEwpD
aW5jaW5uYXRpMRQwEgYDVQQKEwtQYXljb3IsIEluYzEaMBgGA1UEAxMRbWFpbDAy
LnBheWNvci5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDWBV+w
cpnLOjpQ+TNqR5V25+3U4YVhXqlTa29xCcf3qFEgZ33IKhZ0VhpUw+5l2cojLuMn
POduG6n+6K/+blhIPLRhNyDAOqoAVv+LNnJ7tHud1J5d+6gudrCBNB8gj5K/6XAU
qyEtNnKYhVjmIgyyPm14BzhthuaFbMQXqbHZQScm2Lltb3I1In5mjzRMAldhKUXB
H+Oo681MjKyevJdk+EXYzAaXB0uxgYxJX7epFnfwhQGcnqFuRZFTRPeRq/VVY0c1
mA3V2Uk3dGptfTGbvABtHdou8VIX+ouv0EZAwOUhWL4aoXGnLsrCbmEFkbGQ9CIQ
Ldn89E0Zvg0PlhP5AgMBAAGjggGAMIIBfDALBgNVHQ8EBAMCBaAwHQYDVR0lBBYw
FAYIKwYBBQUHAwEGCCsGAQUFBwMCMDMGA1UdHwQsMCowKKAmoCSGImh0dHA6Ly9j
cmwuZW50cnVzdC5uZXQvbGV2ZWwxYy5jcmwwZAYIKwYBBQUHAQEEWDBWMCMGCCsG
AQUFBzABhhdodHRwOi8vb2NzcC5lbnRydXN0Lm5ldDAvBggrBgEFBQcwAoYjaHR0
cDovL2FpYS5lbnRydXN0Lm5ldC8yMDQ4LWwxYy5jZXIwSgYDVR0gBEMwQTA1Bgkq
hkiG9n0HSwIwKDAmBggrBgEFBQcCARYaaHR0cDovL3d3dy5lbnRydXN0Lm5ldC9y
cGEwCAYGZ4EMAQICMBwGA1UdEQQVMBOCEW1haWwwMi5wYXljb3IuY29tMB8GA1Ud
IwQYMBaAFB7xq4kG+EkPATN37hR67hl8kyhNMB0GA1UdDgQWBBRQWvywMsH7+b9J
BDcrwpZ5tt6SpjAJBgNVHRMEAjAAMA0GCSqGSIb3DQEBBQUAA4IBAQBP9l7iKWF2
DU2Hv9jKsbPD9STHBP+a6ZsNiy3a4U4sjWSRAB5LO8NJ+6f6zn/fA2ipGC7xjOzj
jk34a7/Jcclls2rCUVKjln+6KoMTL5HvLNKd3mdL1WdIA/F74UsolNSTKNdFLhjK
tab1QJvOux4yQzByHX8FJvGXqk4eX1UisXehUXsH6njcU1jvPTWRfKBYMftVPRwv
rAtQOifv/dIIAKccMO1XB0g/7ByqrdmEZVVnp9Ze93qlZh36sjhhyirHPa3mOlOK
uENDaO2U2oDZXP4//6SsEbnDMHXnEn+ddua52IGGUp28J+arEfVEh0Fv7xMpqxjS
Q/IZVY7FmzUp
-----END CERTIFICATE-----
subject=/C=US/ST=Ohio/L=Cincinnati/O=Paycor, Inc/CN=mail02.paycor.com
issuer=/C=US/O=Entrust, Inc./OU=www.entrust.net/rpa is incorporated by reference/OU=(c) 2009 Entrust, Inc./CN=Entrust Certification Authority - L1C
---
No client certificate CA names sent
---
SSL handshake has read 3562 bytes and written 538 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID: 8B1C3ADD1C8975B13574777960B6819E109C9C84B8704E1C626B22329F0A37FF
    Session-ID-ctx:
    Master-Key: 3E239E0704E612D125C641A5551D02534CB35D3CF8389CA4C9F4570F6F9B05F1954DFE61360E9F987C2E8FA2883A7A8F
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket:
    0000 - 43 e3 b5 1c 15 3a fe a4-6a fa c0 d3 be 95 20 79   C....:..j..... y
    0010 - 88 2d 21 59 e7 e5 67 b7-95 fc 4c ab 14 a2 a1 01   .-!Y..g...L.....
    0020 - ae ea 48 50 ec 58 c1 bf-60 e6 f1 67 32 ce d1 f0   ..HP.X..`..g2...
    0030 - ba 2e 70 a9 20 37 5a 98-0c 9c 30 8e 30 d8 b4 7e   ..p. 7Z...0.0..~
    0040 - f8 3f 46 ff f6 56 05 36-70 76 c8 77 e8 21 3b 06   .?F..V.6pv.w.!;.
    0050 - 2a f2 0e da d9 e9 39 cd-d4 f0 26 9b 32 40 68 1a   *.....9...&.2@h.
    0060 - c6 42 a8 34 23 8c 56 af-6a fc de 0d cf af 98 7d   .B.4#.V.j......}
    0070 - 6b 7b 90 48 1c c6 cc cd-26 95 19 51 d2 f8 f5 5d   k{.H....&..Q...]
    0080 - 79 26 cc 15 c9 49 45 ad-15 94 e5 08 bb 17 71 a7   y&...IE.......q.
    0090 - bb e4 18 57 c5 68 9d ff-99 2b ca 06 e2 f6 e1 f6   ...W.h...+......
    00a0 - d9 54 c4 0b 42 82 f5 86-dc 1e 5c b6 fc d1 b0 f5   .T..B.....\.....

    Start Time: 1425004482
    Timeout   : 300 (sec)
    Verify return code: 20 (unable to get local issuer certificate)
---
250 STARTTLS

Select allOpen in new window

Even the successful sends give an "unable to get local issuer certificate". Why does e.g. ohioattorneygeneral.gov accept the connection, but paycor.com does not? Why does ohioattorneygeneral.gov end with "250 CHUNKING" and paycor.com ends with "250 STARTTLS"? What are they looking for??!?!?

TLS 1.0 padding hack may not be compatible with old proprietary solutions.

That is a theory of mine, but since I'm now using the new version of openssh and still have the problem it's thrown a damper on my theory. I would have thought the latest version of openssh would have fixed backward compatibility problems.

How can I tell that sendmail is using the new version (maybe I need to reboot)?

How can I tell what version of ssl the recipient machine is using?

You need to be clear on which part you are looking to resolve.
 It looks that you've resolved the issue of incoming TLS requests.

I don't think I ever said I had an incoming TLS issue, but if so -- sorry to have confused an already confusing issue.

The remaining issue deals with outgoing emails from your server trying to negotiate a TLS session after the connection is established.

Yes, so the fundamental question is, why are usbank.com and paycor.com rejecting my TLS connection? I don't think I'm any closer to an answer :(
TLScheck-mail.ohprs.org.pdf

more information:

I change my s_client command to add -CAfile and got rid of the "unable to get local issuer certificate. Yeah! (see below), but still not able to send to recipient:

/var/log/maillog

Feb 26 22:08:09 mail sm-mta[27117]: ruleset=tls_server, arg1=SOFTWARE, relay=mail02.paycor.com, reject=403 4.7.0 TLS handshake failed.
Feb 26 22:08:09 mail sm-mta[27117]: t1OEnKLO016503: to=<PKeer@paycor.com>, delay=2+12:18:49, xdelay=00:00:01, mailer=esmtp, pri=13536724, relay=mail02.paycor.com. [69.61.226.160], dsn=4.0.0, stat=Deferred: 403 4.7.0 TLS handshake failed.
Feb 26 22:08:09 mail sm-mta[27117]: t1NLugQa013458: to=<PKeer@paycor.com>, delay=3+05:11:26, xdelay=00:00:00, mailer=esmtp, pri=17226649, relay=mail02.paycor.com., dsn=4.0.0, stat=Deferred

Select allOpen in new window

I changed my sendmail .mc to:

define(`confCACERT_PATH',`/etc/ssl/certs/')dnl
define(`confCACERT',`/etc/ssl/certs/Go_Daddy_Root_Certificate_Authority_-_G2.pem')dnl
define(`confSERVER_CERT',`/etc/ssl/certs/OHPRS/GoDaddy/Apache/c5fe0cc8242d6030.crt')dnl
define(`confSERVER_KEY',`/etc/ssl/certs/OHPRS/GoDaddy/mail.ohprs.org.key')dnl
define(`confCLIENT_CERT',`/etc/ssl/certs/OHPRS/GoDaddy/Apache/c5fe0cc8242d6030.crt')dnl
define(`confCLIENT_KEY',`/etc/ssl/certs/OHPRS/GoDaddy/mail.ohprs.org.key')dnl
define(`confAUTH_OPTIONS', `A')dnl

Select allOpen in new window

Sendmail experts, is that right?

So, what's *still* wrong?

s_client results:

$ openssl s_client -cert GoDaddy/Apache/c5fe0cc8242d6030.crt -key GoDaddy/mail.ohprs.org.key -CAfile ../Go_Daddy_Root_Certificate_Authority_-_G2.pem -starttls smtp -connect mail02.paycor.com:25 2>&1 | tee paycor_withCA.out
depth=2 O = Entrust.net, OU = www.entrust.net/CPS_2048 incorp. by ref. (limits liab.), OU = (c) 1999 Entrust.net Limited, CN = Entrust.net Certification Authority (2048)
verify return:1
depth=1 C = US, O = "Entrust, Inc.", OU = www.entrust.net/rpa is incorporated by reference, OU = "(c) 2009 Entrust, Inc.", CN = Entrust Certification Authority - L1C
verify return:1
depth=0 C = US, ST = Ohio, L = Cincinnati, O = "Paycor, Inc", CN = mail02.paycor.com
verify return:1
CONNECTED(00000003)
---
Certificate chain
 0 s:/C=US/ST=Ohio/L=Cincinnati/O=Paycor, Inc/CN=mail02.paycor.com
   i:/C=US/O=Entrust, Inc./OU=www.entrust.net/rpa is incorporated by reference/OU=(c) 2009 Entrust, Inc./CN=Entrust Certification Authority - L1C
 1 s:/C=US/O=Entrust, Inc./OU=www.entrust.net/rpa is incorporated by reference/OU=(c) 2009 Entrust, Inc./CN=Entrust Certification Authority - L1C
   i:/O=Entrust.net/OU=www.entrust.net/CPS_2048 incorp. by ref. (limits liab.)/OU=(c) 1999 Entrust.net Limited/CN=Entrust.net Certification Authority (2048)
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFFTCCA/2gAwIBAgIETCD45jANBgkqhkiG9w0BAQUFADCBsTELMAkGA1UEBhMC
VVMxFjAUBgNVBAoTDUVudHJ1c3QsIEluYy4xOTA3BgNVBAsTMHd3dy5lbnRydXN0
Lm5ldC9ycGEgaXMgaW5jb3Jwb3JhdGVkIGJ5IHJlZmVyZW5jZTEfMB0GA1UECxMW
KGMpIDIwMDkgRW50cnVzdCwgSW5jLjEuMCwGA1UEAxMlRW50cnVzdCBDZXJ0aWZp
Y2F0aW9uIEF1dGhvcml0eSAtIEwxQzAeFw0xMzEwMDYyMjU3NTRaFw0xNTEwMDkw
MjE4MjFaMGMxCzAJBgNVBAYTAlVTMQ0wCwYDVQQIEwRPaGlvMRMwEQYDVQQHEwpD
aW5jaW5uYXRpMRQwEgYDVQQKEwtQYXljb3IsIEluYzEaMBgGA1UEAxMRbWFpbDAy
LnBheWNvci5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDWBV+w
cpnLOjpQ+TNqR5V25+3U4YVhXqlTa29xCcf3qFEgZ33IKhZ0VhpUw+5l2cojLuMn
POduG6n+6K/+blhIPLRhNyDAOqoAVv+LNnJ7tHud1J5d+6gudrCBNB8gj5K/6XAU
qyEtNnKYhVjmIgyyPm14BzhthuaFbMQXqbHZQScm2Lltb3I1In5mjzRMAldhKUXB
H+Oo681MjKyevJdk+EXYzAaXB0uxgYxJX7epFnfwhQGcnqFuRZFTRPeRq/VVY0c1
mA3V2Uk3dGptfTGbvABtHdou8VIX+ouv0EZAwOUhWL4aoXGnLsrCbmEFkbGQ9CIQ
Ldn89E0Zvg0PlhP5AgMBAAGjggGAMIIBfDALBgNVHQ8EBAMCBaAwHQYDVR0lBBYw
FAYIKwYBBQUHAwEGCCsGAQUFBwMCMDMGA1UdHwQsMCowKKAmoCSGImh0dHA6Ly9j
cmwuZW50cnVzdC5uZXQvbGV2ZWwxYy5jcmwwZAYIKwYBBQUHAQEEWDBWMCMGCCsG
AQUFBzABhhdodHRwOi8vb2NzcC5lbnRydXN0Lm5ldDAvBggrBgEFBQcwAoYjaHR0
cDovL2FpYS5lbnRydXN0Lm5ldC8yMDQ4LWwxYy5jZXIwSgYDVR0gBEMwQTA1Bgkq
hkiG9n0HSwIwKDAmBggrBgEFBQcCARYaaHR0cDovL3d3dy5lbnRydXN0Lm5ldC9y
cGEwCAYGZ4EMAQICMBwGA1UdEQQVMBOCEW1haWwwMi5wYXljb3IuY29tMB8GA1Ud
IwQYMBaAFB7xq4kG+EkPATN37hR67hl8kyhNMB0GA1UdDgQWBBRQWvywMsH7+b9J
BDcrwpZ5tt6SpjAJBgNVHRMEAjAAMA0GCSqGSIb3DQEBBQUAA4IBAQBP9l7iKWF2
DU2Hv9jKsbPD9STHBP+a6ZsNiy3a4U4sjWSRAB5LO8NJ+6f6zn/fA2ipGC7xjOzj
jk34a7/Jcclls2rCUVKjln+6KoMTL5HvLNKd3mdL1WdIA/F74UsolNSTKNdFLhjK
tab1QJvOux4yQzByHX8FJvGXqk4eX1UisXehUXsH6njcU1jvPTWRfKBYMftVPRwv
rAtQOifv/dIIAKccMO1XB0g/7ByqrdmEZVVnp9Ze93qlZh36sjhhyirHPa3mOlOK
uENDaO2U2oDZXP4//6SsEbnDMHXnEn+ddua52IGGUp28J+arEfVEh0Fv7xMpqxjS
Q/IZVY7FmzUp
-----END CERTIFICATE-----
subject=/C=US/ST=Ohio/L=Cincinnati/O=Paycor, Inc/CN=mail02.paycor.com
issuer=/C=US/O=Entrust, Inc./OU=www.entrust.net/rpa is incorporated by reference/OU=(c) 2009 Entrust, Inc./CN=Entrust Certification Authority - L1C
---
No client certificate CA names sent
---
SSL handshake has read 3562 bytes and written 538 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID: 1B5EECD5C16DBF331753D2EE517BF927B564D9B3A594E2B5E42E20DE471C7114
    Session-ID-ctx:
    Master-Key: 87EE60E15A177836CC0E016BE02931B910A43AE2D4CF9972ABAB5ED5308758EF47C8CBC2694971878F2713EE280F21D9
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket:
    0000 - 43 e3 b5 1c 15 3a fe a4-6a fa c0 d3 be 95 20 79   C....:..j..... y
    0010 - d3 3b 9d 01 03 23 b5 47-18 1e c1 d2 6b 74 7e c2   .;...#.G....kt~.
    0020 - 14 93 80 c9 9f 8f 26 50-1d dd ae a9 81 6b 2a 25   ......&P.....k*%
    0030 - 18 88 f5 ef ac b7 5a 79-35 65 f2 20 02 9e 0a 82   ......Zy5e. ....
    0040 - 63 2e 2e e2 ef 5b 57 74-08 b0 a0 09 c1 5b 51 67   c....[Wt.....[Qg
    0050 - 69 f8 ae e6 fb 9f 6a 09-2c 2c c2 42 2a 30 51 6b   i.....j.,,.B*0Qk
    0060 - 7e 97 27 46 42 c9 eb 45-3e 92 d8 95 49 89 f1 1d   ~.'FB..E>...I...
    0070 - bb a0 1e f6 cc 05 71 26-5c ba 83 76 f8 d5 0f 4a   ......q&\..v...J
    0080 - 44 5a 80 1d 9b 16 1b 7d-86 38 d7 ec bf 18 0e 2b   DZ.....}.8.....+
    0090 - dc e5 84 a2 4e 8f 3e 11-fa 97 0c b8 d1 46 ae 53   ....N.>......F.S
    00a0 - 0c d5 d1 a2 4c 6a d0 14-b8 c1 14 ed e0 b6 1b b6   ....Lj..........

    Start Time: 1425005969
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
250 STARTTLS

Select allOpen in new window

i think you may have to add the -cacert and the signer certs.
Your side is missing the certificate chain.

One option to have two files is to include/add the godaddy Cas certificates atop your client cert I.r.

CA cert
Ca inter Cert
Your server cert.

Try smtp session with the

Ehlo yourservername (greating)
Mail from: <youremailaddress>
Rcpt to: <recipient address>
Data
From: youremailaddress
To: recipient
Subject: test

The message
.
The single period by itself will terminate the message, between mail and rcpt you should receive a 2xx message. After data you should get a 3xx to go ahead after the single period you should get a 2xx indicating the message was successfully accepted.

Arnold:

The Other option is if you have contact with technical folks on the other side ...

Yeah, I think I might contact them tomorrow.

Try smtp session with ...

I've done those before, but again, I'll need to contact someone on the other end before I can do that.

That leaves us with your option #1. At the surface, it doesn't seem too complex. We have 5 files:

1. GoDaddy Root Certificate (apparently pre-installed in /etc/files/certs/Go_Daddy_Root_Certificate_Authority_-_G2.pem?)

2. My key file: mail.ohprs.org.key

3. My CSR: mail.ohprs.org.csr

4. GoDaddy's generated certificate: c5fe0cc8242d6030.crt

5. GoDaddy's generated bundle file: gd_bundle.crt (whatever that is)

I've got 4 places to put/reference these in sendmail:

confCACERT_PATH
confCACERT
confCLIENT_CERT
confCLIENT_KEY

That's all I've got to work with.

Based on this link http://www.sendmail.org/~ca/email/starttls.html, confCACERT has "one or more CA certs ... You should probably put only the CA cert into that file that signed or own cert(s)." Confusing, but the only thing I got back from GoDaddy other than the certificate file was the gd_bundle.crt file. Possibly this goes there?

confCACERT_PATH has "zero or more CA certs ... with symbolic links of their hashes pointing to them. ... hese CA certificates are required to successfully authenticate another entity. The signature of the certificate presented by the other side is checked against these CAs."

The /etc/ssl/certs/ directory has all kinds of symbolically linked files and came pre-installed on my system. That's probably what this confCACERT_PATH is referring to. As I understand it, this statement is telling me that these are used when I am acting as the server to authenticate the sender, so I don't think I care about this folder right now.

The next bit is interesting: "3.Get a cert (make sure the CN is the fully qualified name of your host) and install it as confSERVER_CERT and the private key as confSERVER_KEY (make sure the file is only readable by root or the trusted user). For simplicity, use the same filenames for confCLIENT_CERT and confCLIENT_KEY, respectively."

So, instead of using my mail.ohprs.org.key as the key, perhaps I should use the cert file? Giving:

define(`confCACERT_PATH',`/etc/ssl/certs/')dnl
define(`confCACERT',`/etc/ssl/certs/OHPRS/GoDaddy/Apache/gd_bundle.crt')dnl
define(`confCLIENT_CERT',`/etc/ssl/certs/OHPRS/GoDaddy/Apache/c5fe0cc8242d6030.crt')dnl
define(`confCLIENT_KEY',`/etc/ssl/certs/OHPRS/GoDaddy/Apache/c5fe0cc8242d6030.crt')dnl

What the heck, I'll try that; can't possibly work ... tried it, it didn't.

Meanwhile:

One option to have two files is to include/add the godaddy Cas certificates atop your client cert I.r.

Can you elaborate a bit? How would I combine files? Simply catenate them together?

Installing new SSL toolkit will not make it instantly used.. If sedmail is linked with openssl 0.9.8 it will never go up (or down)

run;
ldd /usr/sbin/sendmail the response will tell you whether they are linked libcrypto dynamically or statically.

usually, if the version is dynamically linked to 0.9.8 unless you get a newer version of sendmail and openssl, the newer 1.x version might break the functionality.

Your configuration files/settings are fine and you can leave them as they are for sendmail.
Not sure whether the client settings used for outgoing are missing another entry that includes the CAcerts, but an option could be to duplicate the CA certs as part of the signed cert.
cat /etc/ssl/certs/OHPRS/GoDaddy/Apache/gd_bundle.crt /etc/ssl/certs/OHPRS/GoDaddy/Apache/c5fe0cc8242d6030.crt > /tmp/combined.crt
mv /etc/ssl/certs/OHPRS/GoDaddy/Apache/c5fe0cc8242d6030.crt /etc/ssl/certs/OHPRS/GoDaddy/Apache/c5fe0cc8242d6030.crt.bak
mv /tmp/combined.crt /etc/ssl/certs/OHPRS/GoDaddy/Apache/c5fe0cc8242d6030.crt and see if that changes the response/behavior from the one site with which you are having issues while you reach out to them.

I'm certainly not opposed to rebuilding sendmail, here's my `ldd /usr/sbin/sendmail` output:

> ldd /usr/sbin/sendmail
        linux-vdso.so.1 (0x00007fff8d0b0000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f657cc2e000)
        libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f657ca14000)
        libssl.so.1 => /lib64/libssl.so.1 (0x00007f657c7a7000)
        libcrypto.so.1 => /lib64/libcrypto.so.1 (0x00007f657c3c8000)
        libsasl2.so.2 => /usr/lib64/libsasl2.so.2 (0x00007f657c1ae000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f657beab000)
        libdb-4.8.so => /lib64/libdb-4.8.so (0x00007f657bb35000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f657b91b000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f657b551000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f657ce67000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f657b334000)

Select allOpen in new window

libcrytpo.so.1 is:

lrwxrwxrwx 1 root root 18 2015-02-24 19:34:02.677877558 -0500 /usr/lib64/libcrypto.so.1 -> libcrypto.so.1.0.0
lrwxrwxrwx 1 root root 18 2015-02-24 19:39:56.170840931 -0500 /lib64/libcrypto.so.1 -> libcrypto.so.1.0.0

Select allOpen in new window

These are from the new build of openssl (on Feb 24th).

Does the fact that crypto.so.1 is listed in the `ldd` output mean it is loaded dynamically (I think the .so bit means dynamic/shared)? If so, is my sendmail therefore using the new openssl?

I have a message in to get a contact at Paycor. I'll wait to get some feedback from that end before trying to Cuisinart the certificate files.

The listing in LDD means the ssl is dynamically linked, so the update to openssl/libraries make the change effective on sendmail without a need to recompile.
Check sendmail as it too may have been updated.

Sorry folks, having trouble getting anyone at the problem sites to respond. Still trying

Do you still have access to your exchange to see how it was configured to communicate with them or whether it has a different client certificate when connecting to them.?

I figured it out with help from recipients tech people.