L2TP IPSec VPN - Requesting Client Authentication Certificate

Posted on 2006-07-13
Last Modified: 2008-01-09

I have setup Windows Server 2003 RRAS for L2TP IPSec VPN and it works when I use a pre-shared key, however I want to make it work with certificates and have been having some trouble.

I have an Enterprise Root CA install on our domain controller and I have had it issue a Computer Certificate for Server Authentication to the RRAS Server, which is also running ISA 2004.

The problem is that I don't seem to be able to get an appropriate certificate to a VPN client (not a member of the domain) - I receive error 786 "The L2TP Connection attempt failed because there is no valid machine certificate on your computer for security authentication"

I can access the certificate server from the client I am trying to setup by connecting the VPN via PPTP or using a pre-shared key and going to http://servername/certsrv but need some guidance on what to from here please.

Really, a step-by-step do this do that guide for the certificate part of this setup (RRAS server and clients) would be useful.

Question by:ITHELP-BOCS
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 6

Expert Comment

ID: 17106821

1. from the http://servername/certsrv website you need to request a new certificate.
2. If the machine you are using is not logged onto the domain already, a prompt to supply domain credentials appears.
3.  In the initial Welcome screen of the Certificate server, click Request a Certificate, and then click Next.
4.  In the Choose Request Type screen, click Advanced Request, and then click Next.
5.  In the Advanced Certificate Requests screen, click Submit a certificate request to this CA using a form, and then click Next.
6.  In the Advanced Certificate Request screen for the Certificate Template option, select Administrator.
7.  Under Key Options, click Microsoft Base Cryptographic Provider v1.0, Signature for Key Usage and 1024 for Key Size.
8.  Leave the Create new key set option enabled (you can clear the Container Name check box unless you want to specify a specific name), and then click Use local machine store.
9.  Leave all the other options set to the default value unless you need to make a specific change.
10.  Click Submit.
11.  The Certificate Issued screen should appear. Click Install this Certificate. The Certificate Installed screen should appear with the message:
Your new certificate has been successfully Installed

After the certificate is installed, verify the location of the certificate by using the Certificate (Local Computer) snap-in in Microsoft Management Console (MMC). Your certificate should appear under Personal.

Author Comment

ID: 17118557
SkIllblock - Thanks for your quick response and appologies for my slow one!

Still not working, I tried to follow your instructions but the screens on CERTSRV I saw were slightly different although similar enough to follow.

Screen shots here;

The certificate is installed on my PC in the correct location but I still get error 786 "The L2TP Connection attempt failed because there is no valid machine certificate on your computer for security authentication".

Any other ideas or items to check?

Thanks again!


Expert Comment

ID: 17120356
alright then maybe the problem lies not with the actual certificate server or the client system, maybe the problem lies with the RRAS server.

what authentication provider and accounting provider have you supplied?
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!


Author Comment

ID: 17120391
authentication provider = windows authentication
accounting provider = none

allow custom IPSec policy for L2TP connection = enabled with a pre-shared key specified
(would this stop certificate based connections)?

Author Comment

ID: 17120415
The only option selected on authentication methods is:


Expert Comment

ID: 17120567
try remove the custom IPSec policy in RRAS, this maybe the problem.
also make sure that the client vpn connection > properties  > security doesnt have a pre-shared key entered in.

Expert Comment

ID: 17120580
Just a note
Check the ms website

and make sure that those points under error 786 are valid.

Author Comment

ID: 17121404
Found this Microsoft article very useful and I have now managed to successful a connect a domain member to the L2TP IPSec VPN using certificates rather than a pre-shared key:

My problem remains - how do I get a non-domain member to connect? i.e. a users home PC.

Expert Comment

ID: 17121872
try these steps for the client system that is not apart of the domain.
(if at first this doesnt work, delete all the certificates that have been assigned to the computer/user, and then try again)

Author Comment

ID: 17126413
Thanks for your help and suggestions, I have now succeeded in obtaining a certificate on a non-domain member and establishing a connection using it, the following web pages provided the necessary information;
(Section:Computer authentication by IPSec)

Basically, because we are using our own Enterprise Root CA which isn't trusted by non-domain members by default, it was necessary to also click the following links on /CERTSRV;

1) Download a CA certificate, certificate chain, or CRL
2) To trust certificates issued from this certification authority, install this CA certificate chain.

The CA Certificate Chain should by placed into the Trusted Root Certification Authorities store (the system placed it into the CURRENT USER store by default but I moved it to the LOCAL COMPUTER store) and the certificate issued to the computer from the /CERTSRV request needs to be in PERSONAL/CERTIFICATES (if the USE LOCAL MACHINE STORE box is selected it will be placed into LOCAL COMPUTER as well).

One more question though, I don't want to have to go through the request process using /CERTSRV on every non-domain member computer I want to connect! If I export these certificates to file (on floppy etc) or just request them to file in the first place, should I be able to re-use them on other computers? and have multiple computers connecting simultaneously using the same certificates?

I will be doing more investigation and will post the information if I can get it.

Will shortly be assigning you the points, just want to conclude all the questions while the question is still open.

Thanks again,

Accepted Solution

SkUllbloCk earned 500 total points
ID: 17128438
Well thats fantastic Neil
I was half way through trying to replicate your situation here in my office, but thankfully now i dont have to do all that tampering.
I had a feeling it might have been with the trusting and location that the certificates were stored.
I came to this conclusing after i had setup a stand-alone CA and issued the certificate to the client PC, on the vpn connection settings for the client i told it to use a smart card or other certificate option, then when trying to connect, i got the error message that the certificate could not be found. After some reading up on MS websites, i found that the certificate issued was being placed in the personal folder as a user certificate, i was playing around with moving and importing the certifiacte to the computer certifiacte store, but still hadnt gotten to the trusted section yet by the time i went home last night.

I suggest you try selecting the advanced certifiacte option save to file, and then also save the Certificate chain aswell.
then try installing those on teh client system.


Author Comment

ID: 17172392
Haven't had time test using the same certificates on multiple PC's but I ASSUME it would work!

Thanks for the help you provided SkUllbloCk.

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Organizations create, modify, and maintain huge amounts of data to help their businesses earn money and generally function.  Typically every network user within an organization has a bit of disk space to store in process items and personal files.   …
by Batuhan Cetin In this article I will be guiding through the process of removing a failed DC metadata from Active Directory (hereafter, AD) using the ntdsutil tool in a Windows Server 2003 environment. These steps are not necessary in a Win…
In this video, viewers will be given step by step instructions on adjusting mouse, pointer and cursor visibility in Microsoft Windows 10. The video seeks to educate those who are struggling with the new Windows 10 Graphical User Interface. Change Cu…
Add bar graphs to Access queries using Unicode block characters. Graphs appear on every record in the color you want. Give life to numbers. Hopes this gives you ideas on visualizing your data in new ways ~ Create a calculated field in a query: …

624 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question