L2TP VPN - error 810 with non-domain clients
Hi ,
I have successfully setup a L2TP VPN server with MS TMG 2010. The clients that are part of the domain can successfully establish the connection.
I have my own CA that I use to generate the certificates.
When I generate the exact same certificate with the same CA for a client that is not part of any domain then I get the 810 error.
Specifically, the client I am testing is on Windows 7. It belongs to a WORKGROUP (Windows default) and it is freshly installed with nothing else on it. It is called "burgvpc-PC".
I have test with another non-domain client - same problem.
I made sure that the CA certificate is in the clients stores (in Trusted Root and Intermediate Certification Authorities - for both use and computer stores).
I also have the client certificate in the computer store.
In the VPN trace logs on the client, the only relevant information I get is "RASDiag: Mapping to new errorcode: 810 (ERROR_VPN_BAD_CERT) instead of 786 (ERROR_OKLEY_NO_CERT)"
There is not information why Windows considers it a bad certificate.
I have also tried the subject name for the certificate to be both "burgvpc-PC" and "burgvpc-PC.WORKGROUP" - both give the same error.
Does anyone have any idea how the certificate should be different when the client is not part of the domain?
I have successfully setup a L2TP VPN server with MS TMG 2010. The clients that are part of the domain can successfully establish the connection.
I have my own CA that I use to generate the certificates.
When I generate the exact same certificate with the same CA for a client that is not part of any domain then I get the 810 error.
Specifically, the client I am testing is on Windows 7. It belongs to a WORKGROUP (Windows default) and it is freshly installed with nothing else on it. It is called "burgvpc-PC".
I have test with another non-domain client - same problem.
I made sure that the CA certificate is in the clients stores (in Trusted Root and Intermediate Certification Authorities - for both use and computer stores).
I also have the client certificate in the computer store.
In the VPN trace logs on the client, the only relevant information I get is "RASDiag: Mapping to new errorcode: 810 (ERROR_VPN_BAD_CERT) instead of 786 (ERROR_OKLEY_NO_CERT)"
There is not information why Windows considers it a bad certificate.
I have also tried the subject name for the certificate to be both "burgvpc-PC" and "burgvpc-PC.WORKGROUP" - both give the same error.
Does anyone have any idea how the certificate should be different when the client is not part of the domain?
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Yes sure there are,
From the TMG site, you have some possiblities to check, what is going on. First some additional steps:
1.) First make sure, the TMG is up to date SP2, RU2 is the last one, which is build 7.0.9193.540
2.) Check your certificates on the TMG, the TMG does not take necessarily the cert you assign, it sometimes take the first one who fits. Just had the case with a client, used PSK and cert based L2TP and regognized, the TMG sends not the certificate I configured in NAP
I had two certs, one with serverauthentication one with IPSec, assigned the server authentication and TMG send the IPSec, which was nowhere assigned. But you should be able to identifiy the cert thumbprint on the client.
According to this, also make sure, what the client has as certificates. To much can disturb the conection in the same way.
3.) Check if there are differences between domain members and not domain members, i.e policies, which influences the local windows firewall. For testing, disable all fireus and antivirus products.
4.) The client:
If I use L2TP / IPSec, I get for PSK as well as cert based authentication two security events for
IPSec fast modus and IPSec main modus.
The last one shows the authentification of the machines, so either PSK or cert, and if with cert, it shows me the thrumprint of the TMG cert, the cleint gets.
Be patient, the log has some delay....,
On the TMG site, there are several options....
1.) Event log
2.) NAP Accountig log (you have to enable the log in NAP (i.e. file based - IAS Format)
3.) The protocol log in TMG (Log & Reports)
4.) The Diagnostic Logging in TMG (Troubleshooting)
5.) The TMG Data Packager (TMG BPA unter Tools)
The last one installs also MS Network Monitor, the is hardcore but logs everything
But I guess the problem seems to be on the client as P2TP seems to fail.
I just test, which logs you can see in different combinations.
So come back in a moment
From the TMG site, you have some possiblities to check, what is going on. First some additional steps:
1.) First make sure, the TMG is up to date SP2, RU2 is the last one, which is build 7.0.9193.540
2.) Check your certificates on the TMG, the TMG does not take necessarily the cert you assign, it sometimes take the first one who fits. Just had the case with a client, used PSK and cert based L2TP and regognized, the TMG sends not the certificate I configured in NAP
I had two certs, one with serverauthentication one with IPSec, assigned the server authentication and TMG send the IPSec, which was nowhere assigned. But you should be able to identifiy the cert thumbprint on the client.
According to this, also make sure, what the client has as certificates. To much can disturb the conection in the same way.
3.) Check if there are differences between domain members and not domain members, i.e policies, which influences the local windows firewall. For testing, disable all fireus and antivirus products.
4.) The client:
If I use L2TP / IPSec, I get for PSK as well as cert based authentication two security events for
IPSec fast modus and IPSec main modus.
The last one shows the authentification of the machines, so either PSK or cert, and if with cert, it shows me the thrumprint of the TMG cert, the cleint gets.
Be patient, the log has some delay....,
On the TMG site, there are several options....
1.) Event log
2.) NAP Accountig log (you have to enable the log in NAP (i.e. file based - IAS Format)
3.) The protocol log in TMG (Log & Reports)
4.) The Diagnostic Logging in TMG (Troubleshooting)
5.) The TMG Data Packager (TMG BPA unter Tools)
The last one installs also MS Network Monitor, the is hardcore but logs everything
But I guess the problem seems to be on the client as P2TP seems to fail.
I just test, which logs you can see in different combinations.
So come back in a moment
And another thing to check...
The user you use has dial in rihgt via AD, right?
The user you use has dial in rihgt via AD, right?
OK, to bring some light in the darkness....
TMG has the settings to enable / disable PPTP, L2TP/IPSec and SSTP
Also you can create on the Remote Access Policy the allowed Authentication Policies for
MSChap-v2 and EAP as well as a PresharedKey
If EAP is enabled, then you have to make additional settings to the NAP configuration.
There you find Policies - Network Polices and Connection Request Policies.
In the Network Policy, you have a Forefront TMG Default Policy.
On the Contraints tab, you find the setting MS-CAHP-v2 as well as EAP settings where you can define up to three different EAP modes
- Smartcard
- PEAP
- EAP-MSCHAP-V2
The first two have a certificate, ist has to be valid, but cannot see it on the client.
Now the methods are defined, which are allowed !!! Thats all
But If you think, that this forces the client to use cert based authentication, then you make the deal without MS logic....
To get certbased authentication to run, you need a certificate, which is assigned nowhere, it just has to reside in the computers cert store.
I created my own template for this on my CA to make sure it will renew automatically.
The cert is a copy of the Webserver template, I added beside the existing Server Authentication the "IP security, IKE intermediate" setting for the key usage and also changed the template to ask for the names during the request. Then I requested that cert on the TMG and put the external name as subject, and the internal + external name as Alternative Subject Name.
On the client, I created several VPN setting in all combinations. For the computer PSK and cert based, for the user MS-Chap-v2, EAP-MS-Chap-v2, PEAP and cert based, just to see, what happens on the client.
With the certificate, all connections went fine, PSK used PSK and cert base used the cert. On the client I could see, that the computer cert is my special IPSEC cert, and the user cert is my user cert.
When I deleted my IPSec cert on TMG, all cert based connection where denied, but after rebooting the TMG, I could logon again, but in this case, all connection felt back to PSK, no certs are used at all. So readded the Cert with no change, restarted the firewall services and the certs are working again on the client.
Means in fact, to get cert based authentication work, you need the described certificate on the TMG and better to make a reboot.
Last try was to delete the computer cert from the client. Nevertheless it is not seen anywhere, it brakes the computer authentication.
The information about the certificate are logges on both system in the securtity log with the task category IPSec Main Mode.
TMG has the settings to enable / disable PPTP, L2TP/IPSec and SSTP
Also you can create on the Remote Access Policy the allowed Authentication Policies for
MSChap-v2 and EAP as well as a PresharedKey
If EAP is enabled, then you have to make additional settings to the NAP configuration.
There you find Policies - Network Polices and Connection Request Policies.
In the Network Policy, you have a Forefront TMG Default Policy.
On the Contraints tab, you find the setting MS-CAHP-v2 as well as EAP settings where you can define up to three different EAP modes
- Smartcard
- PEAP
- EAP-MSCHAP-V2
The first two have a certificate, ist has to be valid, but cannot see it on the client.
Now the methods are defined, which are allowed !!! Thats all
But If you think, that this forces the client to use cert based authentication, then you make the deal without MS logic....
To get certbased authentication to run, you need a certificate, which is assigned nowhere, it just has to reside in the computers cert store.
I created my own template for this on my CA to make sure it will renew automatically.
The cert is a copy of the Webserver template, I added beside the existing Server Authentication the "IP security, IKE intermediate" setting for the key usage and also changed the template to ask for the names during the request. Then I requested that cert on the TMG and put the external name as subject, and the internal + external name as Alternative Subject Name.
On the client, I created several VPN setting in all combinations. For the computer PSK and cert based, for the user MS-Chap-v2, EAP-MS-Chap-v2, PEAP and cert based, just to see, what happens on the client.
With the certificate, all connections went fine, PSK used PSK and cert base used the cert. On the client I could see, that the computer cert is my special IPSEC cert, and the user cert is my user cert.
When I deleted my IPSec cert on TMG, all cert based connection where denied, but after rebooting the TMG, I could logon again, but in this case, all connection felt back to PSK, no certs are used at all. So readded the Cert with no change, restarted the firewall services and the certs are working again on the client.
Means in fact, to get cert based authentication work, you need the described certificate on the TMG and better to make a reboot.
Last try was to delete the computer cert from the client. Nevertheless it is not seen anywhere, it brakes the computer authentication.
The information about the certificate are logges on both system in the securtity log with the task category IPSec Main Mode.
ASKER
Hi Bembi,
Thanks for all you comments.
Note that I am not using EAP certificate authentication. It is username/password auth with MS-CHAPv2. But IPSec still uses a machine certificate on both the server and the client because I am not using PSK. As I have already mentioned, the server setup works because my computers that are members of the domain connect just fine.
The problem is on the client computer when the client is NOT a domain member.
Normally in the certificate common name of the client you would specify the FQDN of the client such as pc1.domain.local
But the computers with the problem are non domain members. They belong to a workgroup so they have no DNS name.
So subject I use is simply the computer name, although I have tried different options for the subject name. Please check my original post.
The below security event on the TMG server confirms that the problem is the client and that its certificate is not valid.
So do you know what should be different on the certificate when a computer does not belong to the domain?
Log Name: Security
Source: Microsoft-Windows-Security
Date: 8/5/2012 4:08:47 PM
Event ID: 4652
Task Category: IPsec Main Mode
Level: Information
Keywords: Audit Failure
User: N/A
Computer: NFSIFWL000002.natfsi.local
Description:
An IPsec Main Mode negotiation failed.
Local Endpoint:
Principal Name: -
Network Address:
Keying Module Port: 500
Local Certificate:
SHA Thumbprint: -
Issuing CA: -
Root CA: -
Remote Endpoint:
Principal Name: -
Network Address:
Keying Module Port: 60434
Remote Certificate:
SHA thumbprint: -
Issuing CA: -
Root CA: -
Additional Information:
Keying Module Name: IKE
Authentication Method: Certificate
Role: Responder
Impersonation State: Not enabled
Main Mode Filter ID: 77447
Failure Information:
Failure Point: Remote computer
Failure Reason: IKE failed to find valid machine certificate. Contact your Network Security Administrator about installing a valid certificate in the appropriate Certificate Store.
State: Sent second (KE) payload
Initiator Cookie: ba4ca9b34ea468b2
Responder Cookie: aad530cac5e26acb
Thanks for all you comments.
Note that I am not using EAP certificate authentication. It is username/password auth with MS-CHAPv2. But IPSec still uses a machine certificate on both the server and the client because I am not using PSK. As I have already mentioned, the server setup works because my computers that are members of the domain connect just fine.
The problem is on the client computer when the client is NOT a domain member.
Normally in the certificate common name of the client you would specify the FQDN of the client such as pc1.domain.local
But the computers with the problem are non domain members. They belong to a workgroup so they have no DNS name.
So subject I use is simply the computer name, although I have tried different options for the subject name. Please check my original post.
The below security event on the TMG server confirms that the problem is the client and that its certificate is not valid.
So do you know what should be different on the certificate when a computer does not belong to the domain?
Log Name: Security
Source: Microsoft-Windows-Security
Date: 8/5/2012 4:08:47 PM
Event ID: 4652
Task Category: IPsec Main Mode
Level: Information
Keywords: Audit Failure
User: N/A
Computer: NFSIFWL000002.natfsi.local
Description:
An IPsec Main Mode negotiation failed.
Local Endpoint:
Principal Name: -
Network Address:
Keying Module Port: 500
Local Certificate:
SHA Thumbprint: -
Issuing CA: -
Root CA: -
Remote Endpoint:
Principal Name: -
Network Address:
Keying Module Port: 60434
Remote Certificate:
SHA thumbprint: -
Issuing CA: -
Root CA: -
Additional Information:
Keying Module Name: IKE
Authentication Method: Certificate
Role: Responder
Impersonation State: Not enabled
Main Mode Filter ID: 77447
Failure Information:
Failure Point: Remote computer
Failure Reason: IKE failed to find valid machine certificate. Contact your Network Security Administrator about installing a valid certificate in the appropriate Certificate Store.
State: Sent second (KE) payload
Initiator Cookie: ba4ca9b34ea468b2
Responder Cookie: aad530cac5e26acb
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Resolved it myself, Bembi's comments helped in the troubleshooting to verify all is in place but ultimately it was not related to the issue.
ASKER
I do not have anything in the system log, just the application and security log (check below).
The computers that are members of the domain can connect successfully, so that should rule out any issues with the TMG Server L2TP certificate.
The "Include windows logon domain" is turned off and also the "automatically use the logon information".
The NAP is not limiting anything at this stage or giving any errors.
Is there any option other than VPN tracing options that can give me more detail on the L2TP negotiations?
Log Name: Security
Source: Microsoft-Windows-Security
Date: 8/2/2012 1:39:20 PM
Event ID: 5061
Task Category: System Integrity
Level: Information
Keywords: Audit Failure
User: N/A
Computer: burgvpc-PC
Description:
Cryptographic operation.
Subject:
Security ID: SYSTEM
Account Name: BURGVPC-PC$
Account Domain: WORKGROUP
Logon ID: 0x3e7
Cryptographic Parameters:
Provider Name: Microsoft Software Key Storage Provider
Algorithm Name: Not Available.
Key Name: le-IPSec!0028Offlinereques
Key Type: User key.
Cryptographic Operation:
Operation: Open Key.
Return Code: 0x80090016
Log Name: Application
Source: RasClient
Date: 8/2/2012 1:36:39 PM
Event ID: 20227
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: burgvpc-PC
Description:
CoId={02AF5DD7-008A-46B4-A
Log Name: Application
Source: RasClient
Date: 8/2/2012 1:36:38 PM
Event ID: 20221
Task Category: None
Level: Information
Keywords: Classic
User: N/A
Computer: burgvpc-PC
Description:
CoId={02AF5DD7-008A-46B4-A
Dial-in User = vpntest
VpnStrategy = L2TP
DataEncryption = Require maximum
PrerequisiteEntry =
AutoLogon = No
UseRasCredentials = Yes
Authentication Type = CHAP/MS-CHAPv2
Ipv4DefaultGateway = Yes
Ipv4AddressAssignment = By Server
Ipv4DNSServerAssignment = By Server
Ipv6DefaultGateway = Yes
Ipv6AddressAssignment = By Server
Ipv6DNSServerAssignment = By Server
IpDnsFlags =
IpNBTEnabled = Yes
UseFlags = Private Connection
ConnectOnWinlogon = No
IPsec authentication for L2TP = Machine certificate.