Question on TLS, MTLS and email server to email server communication

Been trying to read up on this but its confusing. Let me give you an example. I am just talking server to server communication here. No clients unless you want to consider one server a client.

A user sends an email. Their server does not support TLS or encryption of any kind. The receiving gateway supports opportunistic TLS. Does the email from the server to the gateway get encrypted? I can't see how if the sending server has no certificate to encrypt the email.

Reverse this now. User behind the gateway which does support TLS encryption sends an email and the same receiving server receives it. Again, is the email encrypted? The gateway has a certificate but the receiving server does not. Receiving server does not understand TLS so again no encryption, correct?

Now both have certificates and support TLS. Email gets encrypted in both directions, but is this considered MTLS because they both have certificates?

You not believe how many different ways this gets explained and they are probably all right but explained differently.

I have read most of the articles so please don't just post a link. I need someone that understand TLS and email to explain it to me.

Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Adam FarageEnterprise ArchCommented:
I know you said not to point to a link, but check this out. This is possibly the most clear understanding of TLS that Microsoft provides:

Now back to your question...

Opportunistic TLS is enabled on Exchange 2007 forward (I am not sure about Exchange 2003 back as I kinda forgot) but here is how it works..

1) The source server (sending the email) sends an EHLO to the target server (receiving an email)
2) Exchange will run a STARTTLS to the server, and if this is accepted by the target server the target will send the public key to the source server
3) The Exchange server stores the key, and then the connection is encrypted

If the STARTTLS does not get a response, the connection is not encrypted and connects over plain text.

Now as for mutual auth TLS, this is the one that requires configuration and you are essentially "forcing" TLS between the source server and the target.

Internally speaking, since the Exchange server trusts other Exchange servers, TLS is utilized. Between the client machine and the Exchange Mailbox role AND the HUB role, communication is encrypted via RPC.

Does that make a bit more sense?

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Simon Butler (Sembee)ConsultantCommented:
No opportunist TLS in Exchange 2003. It was either on or off. If it is was on, then you lost about 90% of email.

shadowtuckAuthor Commented:
Thanks evrydayzawrkday. The whole point of this post is to find out if all our inbound email is encryptped. Forget Exchange. We have a gateway that encrypts inbound and outbound email but it sounds like there is no guarantee inbound will be encrypted unless the sending server intiates it first. Not all servers do. Unless the sending server initiates a StartTLS command, its going to come over in clear text regardless of whether the receiving gateway supports TLS.

In order to ensure encryption always, sounds like MTLS is needed but that would probably cause a number of emails to drop I am guessing.
Adam FarageEnterprise ArchCommented:

You are absolutely correct. You might get Opp TLS between your smarthost and Exchange, but I cannot guarantee (without using MTLS) that from the smarthost outbound would be encrypted.

To ensure encryption of sensitive data, I would recommend using MutAuth TLS (MTLS) between the smarthost and other companies in which you share confidential data with.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.