Troubleshoot VPN to Windows 2003 SBS Premium with ISA 2004 and Trend Micro WFBS

I've been painstakingly creating a virtual duplicate of our physical Windows 2003 Small Business Server Premium, and I've finally got everything working except the VPN connections. This is despite having restored an exact backup of the ISA 2004 configuration from the physical server (known good) to the virtual server.

When attempting to connect via VPN to the virtual server using a laptop that can connect to the physical server without a problem, the connection times out and a message is displayed saying that the L2TP security could not be negotiated.

Research led me to suspect that Trend Micro Worry-Free Business Security might be the factor, so I tried disabling the Trend Micro Personal Firewall Service, which may have been adding layers of confusion and over-restriction on top of the ISA 2004 firewall. I still could not VPN in at that point (from a machine that VPN's into the physical server without a problem). So, I learned how to uninstall the Personal Firewall component of Trend Micro WFBS.

I will try standing up the virtual server tonight when our users are off the network. Until then, does anyone have any possible suggestions that could improve my chances of success with this VPN?

Thank you.

- Jason
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Keith AlabasterEnterprise ArchitectCommented:
Define 'Cannot Connect' - do you get no result or an error message?
What do you see in the ISA real time log monitor? Is the client vpn traffic even getting to the virtual ISA/SBS box?
upcjdidnerAuthor Commented:

When I filter for Failed VPN connections or Initiated VPN Connections for the last 7 days, I get no results.

Do you think this supports the notion that the Trend Micro firewall was blocking VPN connection attempts before they even got to ISA?


Keith AlabasterEnterprise ArchitectCommented:
Yes - it is likely but your planned tests will prove that one way or another.
Check Out How Miercom Evaluates Wi-Fi Security!

It's not just about Wi-Fi connectivity anymore. A wireless security breach can cost your business large amounts of time, trouble, and expense. Plus, hear first-hand from Miercom on how WatchGuard's Wi-Fi security stacks up against the competition plus a LIVE demo!

upcjdidnerAuthor Commented:
I just ran the same logging query in ISA Server on the physical SBS server and sure enough, that one correctly lists the VPN connections. This further proves that my VPN connection attempts weren't even reaching ISA last night.

Do you suspect any other variables aside from the Trend Micro firewall?
upcjdidnerAuthor Commented:
OK - here's what I have from the client: Error 792: The L2TP connection attempt failed because security negotiation timed out.

And from the ISA Server logging, each connection attempt yielded the following:

Action: Denied Connection
Client IP:
Protocol: IKE Client

The client is using a Verizon Wireless mobile broadband hotspot (with which I connect just fine to the physical server). When I do an IPConfig on my computer, I get the IP address I suppose Verizon is NATing for me?

Your thoughts on a cause or remedy?
upcjdidnerAuthor Commented:
With the help of a consultant we came to the conclusion that ISA wasn't designed to be a VPN device in a virtualized environment; we went with a dedicated VPN appliance sitting outside the virtual server, and that worked.

Thanks all for your input.

- Jason

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
upcjdidnerAuthor Commented:
The resolution for this question may be instructive to other people with a similar issue. It would have saved me many hours if it was out there in a search result.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.