• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 8199
  • Last Modified:

IIS 8.5 certificate chain error results in 403.16 error

Hi,

In IIS I am requiring certificates for a directory that authorized users will access with their smartcard that has both an ID and Email/Signing certs. Many-to-one certificate mapping has been set up and one rule enabled to match the cert subject OU field which is consistent across all certificates.

In a browser, I go to the site and click into the directory. I get asked which certificate to use as ActivClient makes them available to Windows. I select one of them (both should work) and then enter and submit my PIN so the smartcard's private key can be accessed to complete the request. The server response is 403.16. Looking at the failed request trace:

ModuleName IIS Web Core
Notification BEGIN_REQUEST
HttpStatus 403
HttpReason Forbidden
HttpSubStatus 16
ErrorCode A certificate chain processed, but terminated in a root certificate which is not trusted by the trust provider. (0x800b0109)

Opening up the mmc certificate snap-in shows that the local computer certificate store has the certificates installed where expected. The CA Root certificate is in the (local machine)/Trust Root Certification Authorities and the intermediate CA certificates (corresponding to the ID and Signing certs) are installed to (local machine)/Intermediate Trusted Root Certification Authorities.

I also ran SSLDiag.exe which reported "Certificate verified." for the SSL certificate. This comes from Network Solutions and has its own root and intermediate certs installed to the same (local machine) trusted and intermediate cert locations. Other SSL enabled pages display fine so no problem there.  

How can I trace this further to find what the underlying problem is? Or believing in the error message, how do I tell the trust provider that the Root CA should be trusted?

Thanks!
0
James Clark
Asked:
James Clark
  • 6
  • 5
1 Solution
 
James ClarkAuthor Commented:
Some additional info on this is that none of the certificates have expired. Also for test purposes, I've set up a directory with a static html file that I'm trying to access. Within IIS manager, the .NET Trust Level feature remains set to the default 'Full (internal)'.
0
 
btanExec ConsultantCommented:
The problem's symptoms may vary depending on the configuration and use of a Certificate Trust List (CTL) on the IIS server.

For your error case, if IIS is not configured to use a CTL, SSL client certificate authentication will fail with the 403.16 error condition. This error occurs because SChannel.dll wrongly considers the client certificate to be untrusted. (NOTE: Having no CTL in use is the default configuration of IIS 8.0. This is configured by having no SendTrustedIssuerList present or by setting SendTrustedIssuerList=0).

In this scenario, the IIS log typically shows a value of 2148204809 in the sc-win32-status field. This translates to error code 0x800b0109, which is defined as CERT_E_UNTRUSTEDROOT.

To workaround these issues, uninstall the non-self-signed certificate from the Local Computer --> Trusted root Certification authorities certificate store on the IIS server. You may consider using the powershell script provided in the article to detect the non-self-signed certificates installed in the Trusted Root Certification Authorities certificate store.

You can use a Windows PowerShell command to find certificates that are put in the Trusted Root Certification Authorities store incorrectly on the local computer. The following command compares the "Issuer" property and the "Subject" property of each certificate in the store, and then outputs details of certificates that do not meet the criteria of a self-signed certificate:
Get-Childitem cert:\LocalMachine\root -Recurse | Where-Object {$_.Issuer -ne $_.Subject} | Format-List * | Out-File "c:\computer_filtered.txt"
0
 
James ClarkAuthor Commented:
Thanks for pointing me to Powershell. I ran the command and the CA Root certificate wasn't listed so as a sanity check, I reversed the condition so that {$_.Issuer -eq $_.Subject}  and it was in the list with lots of interesting info so it does appear to be installed correctly. I also verified the intermediate CA certs as well. Lot's to love about Powershell and the certificate provider! For anyone else interested in this, check out these links:

http://technet.microsoft.com/en-us/library/hh847807.aspx
http://technet.microsoft.com/en-us/library/hh847761.aspx

Back to the problem though, I'll add that the client certificate is accepted by IIS at least to the extent that I can see it in the log files. I added a custom log field for the server variable {CERT_SUBJECT} and my client cert is listed on all the failed 403.16 requests.

To debug, I also created a new site in IIS from scratch. Same setup with a basic SSL encrypted static .html home page with a link to the 'cert required' sub directory that has only a one static html file. I setup many to one mapping again and reset and re-entered the user account password that the mapping uses. Even with the most simple setup, I get the same result with SSL working on the home page and 403.16 on the directory page.

I guess I'll delete all the certs, reboot, and try again.
0
Shaping tomorrow’s technology leaders, today

The leading technology companies all recognize the growing need for gender diversity. Through its Women in IT scholarship program, WGU is working to reverse this trend by empowering more women to earn IT degrees and become tomorrow’s tech-industry leaders.  

 
btanExec ConsultantCommented:
I do highly suspect is the client cert and most of the time 403.16 is a problem with the client cert. The highest hint is that client certificate was created by a certification authority that the IIS computer does not trust. Possible error revolve such as below http://bogdanbrinzarea.wordpress.com/2009/05/12/http-403-16-forbidden-client-certificate-is-ill-formed-or-is-not-trusted-by-the-web-server/
HTTP 403.16 Forbidden: Client certificate untrusted or invalid.
HTTP 403.16 Forbidden: Client certificate is ill-formed or is not trusted by the web server.

The new server certificate was located only in the Personal store for the Computer account and not on the Trusted Root Certificates Authorities store of the same account.

More on setting up SSL here

Using SSL in kernel mode requires storing SSL binding information in two places. First, the binding is stored in %windir%\System32\inetsrv\config\applicationHost.config for your site. When the site starts, IIS sends the binding to HTTP.sys, and HTTP.sys starts listening for requests on the specified IP:Port (this works for all bindings). Second, the SSL configuration associated with the binding is stored in the HTTP.sys configuration. Use the netsh command at a command prompt to view SSL binding configuration stored in HTTP.sys as in the following example:

netsh http show sslcert

When a client connects and initiates an SSL negotiation, HTTP.sys looks in its SSL configuration for the IP:Port pair to which the client connected. The HTTP.sys SSL configuration must include a certificate hash and the name of the certificate store before the SSL negotiation will succeed.

Troubleshooting Tip: If you're having trouble with an SSL binding, verify that the binding is configured in ApplicationHost.config, and that the HTTP.sys store contains a valid certificate hash and store name for the binding.

 Another minor pt is Crypto API (used by IIS for cert verification) rejects certificates if the root certification authority certificates are not installed in the local computer Trusted Root Certification Authorities certificate store.

 There is a possibility some failure is due to a group policy that forces the IIS computer to "Trust Only Enterprise Root Stores." If this policy is in enabled, the authentication will still fail, even if the CA is a Trusted Root Store. To work around this issue, remove the Group Policy Trust only Enterprise Root stores option for the domain. To do this, perform the following steps:
Start the Default Domain Policy Group Policy Editor.
Select Computer Settings, choose Computer Configuration, and then select Windows Settings.
Choose Security Settings, select Public Key Policies and then choose Trusted Root Certification Authorities.
Right-click Trusted Root CA node, and then select Properties.
Disable the Trust only Enterprise Root stores option.
0
 
James ClarkAuthor Commented:
A quick update in that I can 100% guarantee that there is no problem with the client certs or CA certs either. The Root CA literally supports millions of smart cards so there is no doubt about the technical integrity of the client certs. Plus I use the smart card daily for other purposes without error.

I think you are on to something regarding the CTL. SendTrustedIssuerList is not set and the netsh command shows the CTL Identifier and Store Name are both Null. The problem is I don't have a running Windows 2003 or 2008 server anymore to create a CTL using the old IIS GUI. I've hit this issue a couple of times recently and found little help on the web to show how to create a CTL on either 2008R2 or 2012R2 servers which I do have access to.

If I can't figure this out tonight, I'll probably just get an Amazon EC2 spot instance tomorrow of a 2003 server to work on and create a CTL. I can't believe Microsoft has made it this hard recently to do what appears to be a fairly simple task!
0
 
btanExec ConsultantCommented:
The CTL issue is more the issue if you see IIS log shows a value of 2148204809 in the sc-win32-status field. This translates to error code 0x800b0109, which is defined as CERT_E_UNTRUSTEDROOT. You can catch the workaround which is the uninstalled which you did as well and the mention of "SendTrustedIssureList") http://support.microsoft.com/kb/2802568

But there is other sharing that root cause is during the handshake protocol for client certificate authorization, the server sends a list of Trusted Root Certification Authorities to the client.  The client will in this case only provide Client Certificates, issued by one of these Trusted Root Certification Authorities.  The problem was that the Trusted Root Certification Authorities list was too long on that particular server, so it was truncated before sent to the client.  Unluckily, Root Certificate Authority was truncated from the list, so the handshake failed. The suggested two solutions to solve are:
The first solution is to clean up the Trusted Root Certification Authorities store (Local Machine) and remove all unnecessary certificates. Be aware that you don’t remove certificates that are required by Windows.
 
A second solution is to configure Schannel to no longer send the list of trusted root certification authorities during the TLS/SSL handshake process.  This can be done by adding this registry entry on the web server:
        HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL
        Value name: SendTrustedIssuerList
        Value type: REG_DWORD
        Value data: 0 (False)
Similar context, this http://support.microsoft.com/kb/2464556 also suggested 3 method for considerations pertaining to CTL related issue and Method 3 touch on the registry key again.
Method 1: Remove some trusted root certificates
Method 2: Configure Group Policy to ignore the list of trusted certification authorities on the computer that hosts the UC client
Method 3: Configure Schannel to no longer send the list of trusted root certification authorities during the TLS/SSL handshake process
Can be very taxing to troubleshoot the certificate if fresh installation doesn't work, not any easier though to get through
0
 
James ClarkAuthor Commented:
The best solution appears to be to use a CTL (instead of trying not to) so that's the path I'm on.

Thanks to Amazon EC2, I launched a free tier Windows 2003 server and created a CTL to file using my existing root cert. The CTL file was then moved to and imported into my 2012 server. So far so good and in MMC I see the CTL under the local machine\Intermediate Certification Authorities\Certificate Trust List. All the CTL properties match the expected values and looks like it is ready for use except for the message "This certificate trust list is not digitally signed and cannot be verified."

So to create a signing certificate I follow the advice at http://stackoverflow.com/questions/84847/how-do-i-create-a-self-signed-certificate-for-code-signing-on-windows and use makecert to create a new self signed CA root cert and a code-signing (SPC) certificate and then use pvk2pfx to create a pfx for each cert that combines public/private keys for import. The new CA root cert goes into the local machine trusted root store and I let the wizard pick where the SPC cert goes but I placed it in both personal and Trusted People stores during testing. The SPC cert shows an OK certification path back to the new root CA.

The issue now is that when I edit the CTL which runs the Certificate Trust List wizard to sign my CTL, the only option is "Select from Store..." and clicking on the button brings up a dialog "No certificate available - No certificates meet the application criteria".  Is there a particular -eku value I should be using with makecert to ensure my CA / SPC certs are created as Windows needs them to be? I looked at the wincrypt.h file from the SDK which didn't clarify anything!

This needs to be resolved since netsh won't create a binding with the CTL as it is currently configured.

Thanks again!
0
 
btanExec ConsultantCommented:
CTL  is a list of self-signed certificates for the CAs whose certificates are to be trusted by your organization.. Whenever you create a certificate trust list, you need to authorize it by signing the certificate trust list with a certificate issued by an already trusted certification authority. Hence, due to the certificate Usages as specified in Enhanced Key Usage as shared, I do suspect it validates the specified certificate in the following ways:

Verifies the presence of the Basic Constraints extension and its value, which must be either Subject Type=End Entity or unspecified.
Verifies the value of the Enhanced Key Usage property, which must contain Code Signing and may also contain Lifetime Signing. Any other EKUs are prohibited.
Verifies the value of the KeyUsage (KU) property, which must be either Unset or DigitalSignature.
Verifies the existence of a private key exists.
Verifies whether the certificate is active, hasn’t expired, and hasn't been revoked.

In short, I believe acquiring a certificate that did only have Code Signing as an Enhanced Key Usage may help. This faq has info on the various EKU http://social.technet.microsoft.com/wiki/contents/articles/1760.windows-root-certificate-program-technical-requirements-version-2-0.aspx

Each root certificate will be associated with a minimum set of EKU Object Identifiers (OIDs) to enable the supported product or business scenario.

The following is a list of EKUs allowed to CAs.
Server Authentication =1.3.6.1.5.5.7.3.1
Client Authentication =1.3.6.1.5.5.7.3.2
Secure E-mail =1.3.6.1.5.5.7.3.4

Other EKUs may be granted if the CA is able to provide additional or specific justification:

Code Signing =1.3.6.1.5.5.7.3.3 (see Explanatory Note for qualifying your root for the code signing EKU)[8]
Time stamping =1.3.6.1.5.5.7.3.8
OCSP =1.3.6.1.5.5.7.3.9  (see Explanatory Note for qualifying your root for the OCSP EKU)[9]
Encrypting File System =1.3.6.1.4.1.311.10.3.4
IPsec (Tunnel, User) =1.3.6.1.5.5.7.3.6, 1.3.6.1.5.5.7.3.7
Document Signing = 1.3.6.1.4.1.311.10.3.12
0
 
James ClarkAuthor Commented:
As it turns out, the CTL does not need to be signed. Of great help was a recent blog post at http://tokyoimage.wordpress.com/2014/07/06/smart-card-cac-authentication-with-iis-8-5/ which basically walked me through exactly what I needed to do.

I would add a couple of things though to help anyone trying to do the same...

- for the netsh http add sslcert command, don't forget if using Powershell to quote the appid='{...}' curly braces otherwise it results in "The parameter is incorrect." In retrospect it seems more obvious but this threw me and I mistakenly thought netsh was complaining about the input.

- if and when netsh does complain about the input, it will probably be a 1312 error. This basically means it can't find the certhash it was given to anything in the user's MY store (a.k.a. Personal \ Certificates in mmc) where it looks by default. Since https.sys will also need to access this same cert and does not want to access it through a user's store, create a duplicate cert in the WebHosting store.

Thanks for your help!
0
 
James ClarkAuthor Commented:
Helpful in understanding 403.16 CERT_E_UNTRUSTEDROOT root errors and some of the SCHANNEL options.
0
 
btanExec ConsultantCommented:
thanks for sharing :)
0

Featured Post

New feature and membership benefit!

New feature! Upgrade and increase expert visibility of your issues with Priority Questions.

  • 6
  • 5
Tackle projects and never again get stuck behind a technical roadblock.
Join Now