[Last Call] Learn how to a build a cloud-first strategyRegister Now


VPN with IPSec (L2TP) and Certificates or Preshared Keys

Posted on 2006-05-25
Medium Priority
Last Modified: 2007-11-27
Microsoft and Google simply aren't helping me out anymore.  Hopefully someone here can.

I setup SBS 2003 with the Remote Access Wizard from the "To Do" list.
I was able to immediatly setup a client vpn to access the server using PPTP.
So far so good.
Then I tried switching the protocol to L2TP IPSec VPN and it quickly went downhill from there.

I'm trying to go as secure as possible so I went the harder route first and tried using certificates.
Recently I switched from that and tried just using a preshared key.
I've received the can't connect errors.
Error 789
Error 798
Error 792
And probably a few more...

I have a SonicWall Pro Firewall that has these ports on DMZ: 500(UDP), IPSec 4500(UDP), IPSec 1701(UDP) and IPSec (ESP)
I don't believe the Firewall is an issue at this point as I'm testing behind the firewall and just pointing the client to the local server name.

I guess for now if I can get the "easier" way to work (preshared keys). then I'll be happy and figure out certificates afterwards.
My most current error if you'd like to help from there is:
Error 792  (Microsoft document doesn't help on this - http://support.microsoft.com/default.aspx?scid=kb;en-us;Q299307)

I'll beat you to it and post some of the "suggested reading" I have a feeling you'll send me to. =-)
(Just my list of recent bookmarks... some may not apply or be for the wrong server type)
Note: I have SBS 2003 Standard and NO ISA.

Question by:g127404
  • 6
  • 5
LVL 78

Expert Comment

by:Rob Williams
ID: 16765813
I appreciate it's not an answer to your question but are not all of the Pro series Sonicwall routers also VPN routers? Have you considered setting up an IPSec connection to the router itself using either another VPN router, or the Sonicwall VPN client ? This is easier to set up, more secure, more stable, and offers better performance.
Having said that, a 792 error, as you are likely aware, is usually due to mis-matched keys, missing certificates or blocked fragmented packets. L2TP tunnels are difficult to diagnose here, when you are not hands on, as there are so many variables with router ports, certificates, policies and filters.

Author Comment

ID: 16769698
Yes, the Pro series Sonicwall does act as a VPN router.  We have a hardware Sonicwall -> Sonicwall VPN setup and working already.  I was hoping to use the Windows VPN method instead of hardware (mucho cheaper).  As far as Sonicwall software for the client... I had thought of that but wanted to try Windows.  The hardware to hardware setup we have now is veeery slow when it comes to VPN traffic and I thought it might be different with another software approach.

I tried giving as much information as possible on my senario to help narrow down the possibilites.
Router ports are open
Not using certificates yet (just preshared keys)
... I don't think I've done much with policies and filters.  Are they needed to get it working or just make it more secure.  Maybe that's where I've gone wrong.
LVL 78

Expert Comment

by:Rob Williams
ID: 16769823
>>"We have a hardware Sonicwall -> Sonicwall VPN setup and working already."
Is this on the same router?
If so, my understanding is if the router is using IPSec for it's own VPN connection you cannot do IPSec pass-trough as well. L2TP/IPSec requires that, so I question if it is possible. The basic Windows PPTP VPN uses PPTP/GRE protocol so most routers will allow it to pass even with an IPSec VPN established, where it is a different protocol.
The SonicWall client would be IPSec as well,l but connects to the router, not a device behind it, so you can have the site to site tunnel as well as VPN software clients connecting simultaneously.
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!


Author Comment

ID: 16770010
I'll definetly look into that and probably make a call to Sonicwall...

..however I'm trying this behind the firewall in our internal network.  I have my gateway/dns pointing to the VPN server and am just using the server name not FQDN to connect such as: "Server1"
Shouldn't this still work regardless of the firewall?
LVL 78

Accepted Solution

Rob Williams earned 2000 total points
ID: 16770549
If on the LAN side of the firewall, yes, it will definitely work. The issue I was referring to, only applies to accessing from the public side of the Firewall.
However, L2TP with IPSec is not an easy task like PPTP. You need to set up the server with certificates, access policies, filters for the policies, and user permissions, as well as the client and user certificates. If interested the following are good references for the necessary steps:

Author Comment

ID: 16771530
Is IAS a requirement to getting L2TP working?
LVL 78

Expert Comment

by:Rob Williams
ID: 16771576
No, not all. Actually having it adds another "dimension", i.e. more work.

Author Comment

ID: 16772572
Well, I finally got it working.  Thank you RobWill for your help.
I actually bought a Microsoft Press book called: Windows Small Business Server 2003
It walks you through the whole process.  To tell you the truth I'm not sure what I missed because it looked like I had done everything.
In the end I restarted all the services involved on the server and then it worked.
net stop policyagent
net start policyagent
net stop remoteaccess
net start remoteaccess

Much thanks again for pointing out those references and shedding some light on this difficult technology.

One last question and I'll let you go (and give you the points).

I put my logon information in the vpn connection the first time.
I disconnected and tried it again (to make sure it wasn't a fluke!) and this time it didn't ask for my credentials.  I made sure I have the options selected to prompt for name, password, certificate, etc.
In order to be the most secure I'd like to require a username/password entered each time.  Is this possible or is it just the first time in.
LVL 78

Expert Comment

by:Rob Williams
ID: 16773146
Odd that it didn't request at least a password. After logging in the first time, some information is cached, and the user's name from the last time the connection was used (from that user profile) should be automatically entered in the box, but unless you selected "save password" you should always be asked for a password. I don't have one here to look at but there is an "options" section on the client, that should allow you to ask to be prompted, or not, for user name and password.

Pat yourself on the back. Not an easy task to set up. I don't think I have done any since Win2000, and I remember it's not fun. One little over-site and it doesn't connect. The good news is, it works well once configured correctly.

Author Comment

ID: 16773173
Yes, I've made sure the "options" section on the client is checked to prompt for password.. but to no avail.

I'm just hoping it will ask me when I'm not behind the firewall and already logged into the network.  Anyway... that's a different problem for a different day.  I'm "patting myself on the back" today. =-)

Thanks again Rob.  Good work.
LVL 78

Expert Comment

by:Rob Williams
ID: 16773249
Forgot you are still behind the firewall, if using domain names/accounts it might use cached credentials. Will be interesting to see if remotely performs differently.
Actually I wasn't a lot of help, but thanks.

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The articles for turning off the Client firewall policy on the internet are for SBS 2008 and don't really help for SBS 2011. They actually moved the Client firewall policy. In 2011, the client firewall policy has moved to the SBS computers conta…
I’m often asked about newer and larger USB drives connected to SBS2008 and 2011 failing Windows Server Backup vs the older USB drives not failing. As disk space continues to grow and drive technology change SBS2008 and some SBS2011 end up with the f…
Is your OST file inaccessible, Need to transfer OST file from one computer to another? Want to convert OST file to PST? If the answer to any of the above question is yes, then look no further. With the help of Stellar OST to PST Converter, you can e…
With just a little bit of  SQL and VBA, many doors open to cool things like synchronize a list box to display data relevant to other information on a form.  If you have never written code or looked at an SQL statement before, no problem! ...  give i…
Suggested Courses

830 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question