Smart Card Logon failure KDC certificate CERT_TRUST_IS_NOT_VALID_FOR_USAGE

We are trying to enable Smart Card Logon.

When we attempt to logon with a Smart Card we get "The Kerberos Protocol encounterd an error while validating the KDC certificate during Smart Card Logon."

In the system log we see the following event:

Event ID 9
The certificate is not valid for the requested usage.
The client has failed to validate the Domain Controller certificate for DC.domain.com. The following error was returned from the certificate validation process: The certificate is not valid for the requested usage.

Looking in the CAPI log we see that the domain controller cert is passing the CRL checks, but is returning:
CERT_TRUST_IS_NOT_VALID_FOR_USAGE

We are using all 3rd party certificates.
The CA certificates have been added to the correct CA stores via Group Policy.
The root is in the Trusted Root Certificate store.
The 2 intermediate CA's are in the Intermediate CA store.

The CA certificates have all be added to the NTAuth store.

All the domain controllers have certificates, issued by the above CA's.

The smart card certificates are issued by the above CA's.

certutil -urlfetch -dcinfo verify says the KDC certs on all of the domain controllers are valid.

I can't figure out what I'm missing.   Why are the clients not trusting the domain controller certificates for the required usage?
BarryBasAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

btanExec ConsultantCommented:
Referencing the smartcard login, I am suspecting this step to check on the username in AD. Minimally the Subject Name/Subject Alternative Name of the user certificate must contain the user's User Principal Name (UPN).
6.The KDC compares the UPN in the certificate with the UPN on the user object in the directory. The KDC also verifies the signature on the certificate to ensure that it was issued by a CA that's trusted in the Active Directory forest, such as an Enterprise CA.
http://blogs.msdn.com/b/windowsvistanow/archive/2008/03/26/smart-card-logon-and-authentication.aspx

Also the smartcard certificate has specific format requirements, extracted part of it for check
Enhanced Key Usage =•Client Authentication (1.3.6.1.5.5.7.3.2)
 (The client authentication OID) is only required if a certificate is used for SSL authentication.)
•Smart Card Logon (1.3.6.1.4.1.311.20.2.2)

•Subject Alternative Name = Other Name: Principal Name= (UPN). For example:
UPN = user1@name.com
 The UPN OtherName OID is : "1.3.6.1.4.1.311.20.2.3"
 The UPN OtherName value: Must be ASN1-encoded UTF8 string

•Subject = Distinguished name of user. This field is a mandatory extension, but the population of this field is optional.

6.There are two predefined types of private keys. These keys are Signature Only(AT_SIGNATURE) and Key Exchange(AT_KEYEXCHANGE). Smartcard logon certificates must have a Key Exchange(AT_KEYEXCHANGE) private key type in order for smartcard logon to function correctly.
https://support.microsoft.com/en-us/kb/281245
BarryBasAuthor Commented:
The UPN for the users is set to match the UPN on the smartcards.

Both the user certificates and the Domain Controller Certificates have the appropriate Enhanced Key Usage settings.

Still getting the above errors.
btanExec ConsultantCommented:
I am thinking if the steps for user mapping
Active DIrectory User Configuration

As these certificates are issued by the government, they don’t contain any specific information that allows Active Directory to find out to which user should be authenticated. In order to resolve that we can add a name mapping to a user. And this is the second drawback. If you want to put EID authentication in place you’ll have to have some sort of process or tool that allows users to link their EID to their Active Directory User Account
using a copy of the smartcard cert template
Testing the authentication

An easy way to see if a user logged on using smart card or username/password is the query for the user his group memberships on the client. When users  log on with a smart card they get the This organization certificate group SID added to their logon token. This is a well-known group (S-1-5-65-1) that was introduced with Windows 7/ Windows 2008 R2
and since using 3rd party CA, to also ensure not only it is in the NTAuth store but knowing your clients and servers can only download the content of the AD NTauth store if they have auto-enrollment enabled.
Keep in mind that the NTauth store exists both locally on the client/servers and in Active Directory. An easy way to view/manipulate the NTauth store in Active Directory is the pkview.msc management console which you typically find on a CA. Right-click the root and choose manage AD containers to view the store.

A second important fact regarding the NTauth store. Whilst you might see the require CA certificate in the store in AD, your clients and servers will only download the content of the AD NTauth store IF they have auto-enrollment configured!
http://setspn.blogspot.sg/2014/10/configure-windows-logon-with-electronic.html

May want to unjoin and rejoin the client to domain again esp if only happened to this client and esp if there are other clients logging into the DC using this smartcard. Also to check client own event log for any errors during the login period.
Determine the Perfect Price for Your IT Services

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden with our free interactive tool and use it to determine the right price for your IT services. Download your free eBook now!

BarryBasAuthor Commented:
The certificates have a UPN that uniquely identifies the user and we've updated the UPN in active directly to match that value.  So, I'm pretty sure the user certificates are correct.  

Looking in the CAPI log on the domain controller, we can see that the Domain Controller is validating the user certificate and it is passing the CRL checks.

We have also verify that the NTAUTH store is getting propagated on the servers and workstations.  The task scheduler is enabled and if you do a certutil -enterprise -viewstore ntauth on the workstations it lists all of the appropriate CA certificates.

We have already tried removing and rejoining the workstation from the domain.  It had no impact.

The above event log information is from the workstation.  It is the workstation that is saying the Domain Controller certificate is not trusted for the requested usage.

On the domain controller, everything seems to be working normally.  The KDC service starts with no problems.  We can see the client authentication request, it passes all of the certificate checks.

The client just isn't trusting the Domain Controllers response.
btanExec ConsultantCommented:
The client root store should have both the AD and 3rd part CA cert. Include also respective intermediate cert bundle accordingly. Same goes for server end which you has verified. Am thinking to have only those cert rather have other multiple cert at client end with such cert usage. I assume your client smartcard has only one smartcard login cert.
btanExec ConsultantCommented:
May want to also review this on for using 3rd party CA http://blogs.technet.com/b/instan/archive/2011/05/17/smartcard-logon-using-certificates-from-a-3rd-party-on-a-domain-controller-and-kdc-event-id-29.aspx

Specifically the "Certificate and configuration problem" section in below. Know that you have verified those I supposed https://support.microsoft.com/en-gb/kb/281245
BarryBasAuthor Commented:
btan,  thanks for all the suggestions.

I have already reviewed those links as well.  All of those settings and requirements have been verified and met.

"certutil -urlfetch -dcinfo verify" on both the domain controllers and the workstations indicate that the kdc certificates are good.  certutil indicates that each domain controller has a valid KDC certificate.

So, what is the logon process looking for that certutil isn't?

This is very frustrating.  This is not my first time enabling smartcard logon, I've done it at other places.  Have never run into a problem like this.
btanExec ConsultantCommented:
thanks for sharing, I do find it strange as well. I am suspecting if there are earlier updates or fixed (within OS or smartcard provider) that would have caused such situation. I am suspecting if it can also be due to mandating FIPS crypto provider in policy but seems unlikely based on the error so far.. the whole flow for card login is as per below but it is not hinting any oversights. There is autoenrollment required to download but since the cert is verify to be in the right cert store, it should not matters. The link is more of internal CA which is not in your case unless there are some running of internal root CA or ent CA in the AD server concurrently (though not certain if that can conflict due to enumeration for the first trusted CA cert or even http CRP url etc)..https://technet.microsoft.com/en-us/library/92aea8c5-c617-4a04-9356-e0ad8dd4cea9(v=ws.10)#BKMK_SmartCardLogonFlowVista
BarryBasAuthor Commented:
btan, that was a very interesting article.  Had not seen that one yet.

From what I can figure out we seem to be failing at step 15 of the smartcard logon flow for windows vista and windows 7.

I believe this is the case, because I can see events on the domain controller that it is properly verifying the users certificate, in the capi2 log.

why isn't the workstation trusting the KDC certificate?  It doesn't make sense.
btanExec ConsultantCommented:
Agree as well, hopefully this discussion forum shed some light as it seems to be in the same predicament you are in now.
For future reference the CAPI event log is in:
Event Viewer (local) > Windows Logs > Application and Service Logs > Microsoft > Windows > CAPI2
I found the issue.  Apparently the CA root was not fully trusted by the NTAuth store, which is extremely weird since I used the Enterprise PKI snap in to add the public key of the Root CA to the NTAuth store.  Below is the CAPI error:
<Result value="800B0112">A certification chain processed correctly, but one of the CA certificates is not trusted by the policy provider.</Result>
The odd thing is that the "certutil -dspublish -f filename NTAuthCA" command did not work claiming that the cert already existed, but when I instead used  "certutil -enterprise -addstore NTAuth CA_CertFilename.cer" a certificate got added and certutil replied with public key added (I confirmed by checking the registry HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\EnterpriseCertificates\NTAuth\Certificates).  I find this very odd.

I get it now:
        certutil -dspublish adds the certificate to the domains Enterprise NTAuth Store
certutil -enterprise -addstore adds the certificate to the local cache on the server.  If Auto-enroll is not enabled at a domain group policy level the cert in the Enterprise NTAuth Store will not replicate to the other machines and will need to be added manually via certutil.
https://social.technet.microsoft.com/forums/windowsserver/en-US/a3891055-53d6-4af2-b9b4-e0bff7f81a94/how-to-create-3rd-party-ca-ntauth-trust-without-autoenroll
BarryBasAuthor Commented:
Had already tried adding certificates manually to the workstations.  It didn't work, but I tried again just to make sure, for each cert I got an reply that the cert was already in the ntauth store.

Smartcard logon still not working.

I'll be out for the weekend.   Maybe a fresh look at this Monday morning, after the weekend, will help.

Thanks.
btanExec ConsultantCommented:
getting stranger but interesting. I was re-reading on the liner on "....the direct issuer of the potential KDC certificate needs to be in the NTAuth store of the DC and all certificates in the chain (except the Root CA cert) need to pass revocation checking ..." and in your context, not sure if KDC cert is issueed by EntCA or 3rdparty CA. Regardless, issuer for either must be in NTAuth store as specified.
http://social.technet.microsoft.com/wiki/contents/articles/3824.updated-requirements-for-a-windows-server-2008-r2-domain-controller-certificate-from-a-3rd-party-ca.aspx

another is ref the link which stated WINS issue to resolve DNS
...We have found that some computers, don't reflect the DNS suffix in the "Full computer name" and other computers, were using the manual list entry for the DNS suffix search order on the NIC.  We also found out that our VPN software changes the DNS Suffix search order on the NIC's from option 1 (what I call automatic, as it takes the DNS suffix in the "Full Computer Name" and uses that for finding other computers), to option 2 (where you have to manually enter a list of DNS suffix's to search through), when you connect the VPN, and then it goes back to option 1 after you disconnect the VPN....

WINS made the difference before, so we didn't notice the problem.  Now without WINS short name resolution, the computers have to have the DNS suffix to be able to find network resources.  And if the slightest misconfiguration is going on, they can't see other basic network servers performing vital functions.  Such as CA CRL download lists, mapped drives, and connection to other servers.  That FQDN qualifier on the workstation/server names is the only thing that allows the systems to be properly found in a purely DNS environment.
https://social.technet.microsoft.com/Forums/en-US/d56366a6-65f6-4f20-a6e1-a5c7c42bef86/securitykerberos-event-id-9-smart-card-not-working-for-login-due-to-crl-download-failure?forum=winserverDS

Also thinking to take step back, and re-review the potential failure pts ... to stay focus and try to isolate issue causing this

a) The KDC validates the user's certificate (time, path, and revocation status) to ensure that the certificate is from a trusted source.....The domain controller verifies the signature and uses the public key from the user's certificate to prove that the request originated from the owner of the private key that corresponds to the public key. The KDC also verifies that the issuer is trusted and appears in the NTAUTH certificate store.

b) The client validates the reply from the KDC (time, path, and revocation status). It first verifies the KDC's signature by the construction of a certification path from the KDC's certificate to a trusted root CA, and then it uses the KDC's public key to verify the reply signature.

c) Kerberos is case sensitive. Problems can occur in an environment using host names with mixed case. In the world of Kerberos, appserver1.EXAMPLE.COM and appserver1.example.com are not the same. Check that DNS resolves host names with consistent case.

d) Expand Certificates (Local Computer) and click Personal. You should see a certificate with the FQDN of your domain controller. If there is no certificate, your first troubleshooting step is to force a Group Policy update by executing the following command on one of your domain controllers:
C:\>gpupdate /force

e) Certificate mapping is based on the UPN that is contained in the subjectAltName (SAN) field of the certificate. Client certificates that do not contain the subjectAltName extension in the certificate are also supported.

f) The KDC root certificate and the smart card logon certificate on the card must have an HTTP CRL distribution point listed in its certificate. The CRL distribution point must have a valid CRL published
BarryBasAuthor Commented:
btan,  thanks for the interesting articles.  I'm learning more than I ever thought I'd need to for smartcard logon.  I can tell from the network traffic that the workstations are downloading the CRL, so I don't think that applies.

Regarding your other points.

a) we are seeing events on the domain controller, were it is validating the user certificate.

b) this seems to be were we are failing, when the client attempts to validate the KDC certificate from the domain controller, we are getting the not trusted for usage error.

c) I'll look more into case sensitivity being an issue.   Since the workstation and server are both selecting the proper certificate and the workstation is attempting to validate the certificate, it would seem the names match.

d) On the domain controllers we have the appropriate certificate in the personal store and it's trust chain validates back to the root CA.

e) The certificates do have the UPN extention and the Active Directory accounts have been updated to match the UPN on the server.

f)  The certificates do have an HTTP crl distribution point.  and a CRL has been published.
btanExec ConsultantCommented:
if so then all are in the right track and I can only see that unless certain updates on the Windows OS is culprit otherwise the debug for smartcard msg have to be review again - not the best means as searching needle in haystack but we no worst off. https://technet.microsoft.com/en-us/library/ff404296(v=ws.10).aspx

But before that maybe worthwhile to verify the section stated "Can the certificate on the smart card be validated on the domain controller?" in below
To verify that the certificate chain can be built on the DC, perform the following:

Export a copy of the smart card certificate; either from the CA, or by running:
Certutil -scinfo

On a workstation with the smart card inserted in the reader.

Open the certificate, go to details, and click the "Copy to file" button. Export the certificate to file, and copy this exported certificate to the authenticating domain controller. At the command prompt, run the following:

Certutil -verify -urlfetch cerexport.cer

If the certificate is not trusted because the root certificate is not in the trusted root store of the DC, the following will be displayed at the bottom of the output: ...
http://blogs.msdn.com/b/windowsvistanow/archive/2008/03/26/smart-card-logon-and-authentication.aspx. If already done so, we back to the debug means which I see it also going no very specific means to the concluding certain outcome. The worst case is rebuild but I understand it is great pain and no guarantee too...
BarryBasAuthor Commented:
btan,  I have exported my public cert from my smartcard and it does validate fine on the domain controller.

This is very frustrating.

I've opened a support case with Microsoft on this.  Having a bit of difficulty since it's been assigned to a support group in India and their working hours and mine don't match up.  Trying to get them to re-assign it to someone how works the same hours I do.

If they come up with anything, I'll let  you know.
btanExec ConsultantCommented:
Good - I understand the "heat" to go deep and see nothing - keep us updated, only those MS folks can now reveal if this is really a bug issue or even they are in dark and gray understanding
BarryBasAuthor Commented:
Finally figured out what was causing our problem.

There was nothing wrong with our Domain Controller or Smart Card Certificates.

A number of years ago, the team the builds our workstation images added the 3rd party root CA certificate to the workstation image.  It was added with usage restrictions.

So, even though we had all of our group policies set correcting and had properly published the CA certificates in active directory, the workstations were using the locally stored copy of the root CA, not the one published in Active Directory.

We kept checking to ensure the CA certificates we had published in Active Directory were correct.  But, overlooked that it was possible the certificates were added locally.

Removing the locally stored version of the certificate and allowing the version published in Active Directory to take effect corrected the problem.

SmartCard logon began working as soon as the certificate was removed.  No reboot was needed.

We've developed a script that will remove the root CA certificate from the registry on the workstations.

You need to look for the thumbprint of the Certificate in either of 2 locations in the registry.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\EnterpriseCertificates\AuthRoot\Certificates
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates\AuthRoot\Certificates

Delete the key that contains the thumbprint of the certificate you want to remove.  Need to be careful, there could be certificates in these location that you need, only delete the ones you don't want.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
btanExec ConsultantCommented:
Thanks we probably have to look further and deeper to locally as shared which in prev posting stated local computer for personal store but we should also check the RootCA in trusted store
d) Expand Certificates (Local Computer) and click Personal. You should see a certificate with the FQDN of your domain controller. If there is no certificate, your first troubleshooting step is to force a Group Policy update by executing the following command on one of your domain controllers:
 C:\>gpupdate /force
BarryBasAuthor Commented:
We opened a case with Microsoft and worked with them to resolve the issue.
btanExec ConsultantCommented:
thanks for sharing.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Server 2008

From novice to tech pro — start learning today.