Solved

Require TLS Encryption Option Greyed Out of IIS 6 SMTP Relay Outbound Security Interface

Posted on 2013-01-28
12
3,177 Views
Last Modified: 2013-04-23
Hello,

I’m trying to enable TLS authentication on an IIS 6 SMTP relay virtual host, without success.  Although I’ve installed a self-signed cert, the option to “Require TLS Encryption” is greyed out.


To review, here are the steps I took.


1.)      Using IIS 7, I created a self-signed cert
2.)      I exported this cert to a folder on the server
3.)      I imported the cert into the Personal Cert store
4.)      I then tried to fully enable TLS but, the option to “Require TLS Encryption” is greyed out, suggesting the cert is either incorrectly formed, or, the virtual relay host does not see it.



Visuals of the steps I took...


Step One. Create Self-signed Cert via IIS 7 Manager

create self signed cert windows 2008
 

Step Two. Export the cert

export the certificate
 

Step Three. Import the Cert into the Personal Certificate Store

import the cert
 

Step Four. Secure IIS SMTP Virtual Host Using TLS

Require TLS option greyed out
0
Comment
Question by:drmonoe
  • 6
  • 6
12 Comments
 
LVL 16

Accepted Solution

by:
choward16980 earned 500 total points
ID: 38828938
Quote from here:

http://forums.iis.net/t/1155280.aspx/1

'When the IIS6 SMTP Server module looks for a certificate to use for TLS encryption, it seems checks the 'Local_Machine\my' store. I'm not sure where the OpenSSL certificate was placed by the system, but if it's not in the Personal certificates section, then the SMTP server won't be able to find it, and will therefore provide the error listed here.'
1
 

Author Comment

by:drmonoe
ID: 38829453
Yes, I found that during my Google searches but it wasn't immediately helpful.


So I suppose the next question is: how does Local Machine/My Store correspond to a drive location? Or does this refer to the HKLM reg hive?

I'll take a look in the reg to see if it reveals a location.
0
 
LVL 16

Expert Comment

by:choward16980
ID: 38831961
It refers to the certificate store.  You need to click start, type mmc , push enter.

file, add/remove snap-in.

add certificates

local computer

then look at picture as to what it's referencing.  (local machine should be local computer)

My store is personal
Untitled.png
0
PeopleSoft Has Never Been Easier

PeopleSoft Adoption Made Smooth & Simple!

On-The-Job Training Is made Intuitive & Easy With WalkMe's On-Screen Guidance Tool.  Claim Your Free WalkMe Account Now

 

Author Comment

by:drmonoe
ID: 38832240
Yes, overnight I reached that conclusion and added the cert to the correct store.

Cert Store
Unfortunately, this hasn't made the TLS encryption option available.  I've also tried resetting the virtual SMTP host individually and even an iisreset with no success.
0
 
LVL 16

Expert Comment

by:choward16980
ID: 38836724
Investigate the registry keys this article says to disable TLS in IIS.  I'm wondering if an admin before you disabled.

http://support.microsoft.com/kb/187498
0
 
LVL 16

Expert Comment

by:choward16980
ID: 38836734
Also, maybe unrelated/non-correlated (but I've seen MS do worse).  Check your ie options listed here:

http://superuser.com/questions/342148/why-are-some-use-tls-and-use-ssl-options-turned-off
0
 

Author Comment

by:drmonoe
ID: 38836884
Hmmm, interesting.


I'll check those out and get back to you.  Thanks for the leads.
0
 

Author Comment

by:drmonoe
ID: 38839660
Unfortunately, none of those steps applied to this situation.  Truly a mystery as to why IIS is failing to see the cert (which seems to be properly installed and verified).
0
 
LVL 16

Expert Comment

by:choward16980
ID: 38841712
Lol, it's not expired is it?  (had to ask)
0
 

Author Comment

by:drmonoe
ID: 38844330
Ha! That's a fair question (crossing all the T's, etc).

I'm sure the cert is still valid.  It was only issued a few days ago and expires a year from now.  Also, I confirmed it using a tool named SSLDiag.

Oh and I successfully used it to secure the default website with SSL.

So aside from the SMTP relay, all else seems good.
0
 
LVL 16

Expert Comment

by:choward16980
ID: 38845153
0
 

Author Comment

by:drmonoe
ID: 38845211
Yup, I verified that too.  

The cert's subject name definitely matches the virtual server's FQDN.
0

Featured Post

On Demand Webinar - Networking for the Cloud Era

This webinar discusses:
-Common barriers companies experience when moving to the cloud
-How SD-WAN changes the way we look at networks
-Best practices customers should employ moving forward with cloud migration
-What happens behind the scenes of SteelConnect’s one-click button

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

The canonical version of this article is on my web site here: http://iconoun.com/articles/collisions/ A companion presentation is available here: http://iconoun.com/articles/collisions/Unicode_Presentation.pdf
As with any other System Center product, the installation for the Authoring Tool can be quite a pain sometimes. This article serves to help you avoid making these mistakes and hopefully save you a ton of time on troubleshooting :)  Step 1: Make sur…
Viewers will learn how to maximize accessibility options in an Excel workbook for users with accessibility issues.
The viewer will learn how to create two correlated normally distributed random variables in Excel, use a normal distribution to simulate the return on different levels of investment in each of the two funds over a period of ten years, and, create a …

679 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question