Troubleshoot VPN to Windows 2003 SBS Premium with ISA 2004 and Trend Micro WFBS

I've been painstakingly creating a virtual duplicate of our physical Windows 2003 Small Business Server Premium, and I've finally got everything working except the VPN connections. This is despite having restored an exact backup of the ISA 2004 configuration from the physical server (known good) to the virtual server.

When attempting to connect via VPN to the virtual server using a laptop that can connect to the physical server without a problem, the connection times out and a message is displayed saying that the L2TP security could not be negotiated.

Research led me to suspect that Trend Micro Worry-Free Business Security might be the factor, so I tried disabling the Trend Micro Personal Firewall Service, which may have been adding layers of confusion and over-restriction on top of the ISA 2004 firewall. I still could not VPN in at that point (from a machine that VPN's into the physical server without a problem). So, I learned how to uninstall the Personal Firewall component of Trend Micro WFBS.

I will try standing up the virtual server tonight when our users are off the network. Until then, does anyone have any possible suggestions that could improve my chances of success with this VPN?

Thank you.

- Jason
Who is Participating?
upcjdidnerConnect With a Mentor Author Commented:
With the help of a consultant we came to the conclusion that ISA wasn't designed to be a VPN device in a virtualized environment; we went with a dedicated VPN appliance sitting outside the virtual server, and that worked.

Thanks all for your input.

- Jason
Keith AlabasterEnterprise ArchitectCommented:
Define 'Cannot Connect' - do you get no result or an error message?
What do you see in the ISA real time log monitor? Is the client vpn traffic even getting to the virtual ISA/SBS box?
upcjdidnerAuthor Commented:

When I filter for Failed VPN connections or Initiated VPN Connections for the last 7 days, I get no results.

Do you think this supports the notion that the Trend Micro firewall was blocking VPN connection attempts before they even got to ISA?


Simple Misconfiguration =Network Vulnerability

In this technical webinar, AlgoSec will present several examples of common misconfigurations; including a basic device change, business application connectivity changes, and data center migrations. Learn best practices to protect your business from attack.

Keith AlabasterEnterprise ArchitectCommented:
Yes - it is likely but your planned tests will prove that one way or another.
upcjdidnerAuthor Commented:
I just ran the same logging query in ISA Server on the physical SBS server and sure enough, that one correctly lists the VPN connections. This further proves that my VPN connection attempts weren't even reaching ISA last night.

Do you suspect any other variables aside from the Trend Micro firewall?
upcjdidnerAuthor Commented:
OK - here's what I have from the client: Error 792: The L2TP connection attempt failed because security negotiation timed out.

And from the ISA Server logging, each connection attempt yielded the following:

Action: Denied Connection
Client IP:
Protocol: IKE Client

The client is using a Verizon Wireless mobile broadband hotspot (with which I connect just fine to the physical server). When I do an IPConfig on my computer, I get the IP address I suppose Verizon is NATing for me?

Your thoughts on a cause or remedy?
upcjdidnerAuthor Commented:
The resolution for this question may be instructive to other people with a similar issue. It would have saved me many hours if it was out there in a search result.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.