SBS 2011 - RDP & VPN Disconnect

I've got a real stumper here and stuck for an answer. I'm trying to roll-out a new server with SBS 2011 and things are going good, except the VPN disconnects frequently, and using RDP via RWA disconnects, usually it's brief and can re-dial immediately. But it happens often and randomly. Multiple clients, machine and user accounts all have the same issue.

I have verified it's not the router/firewall. I've used 3 different ones and get the same results. I've also replaced all the cabling and switches. There are no physical level errors.

I've updated the server with the latest patches ; the logs don't really produce any useful information. They just show the normal connect / disconnect information. Everything looks normal in the logs. The only error the client recieves is the generic Error 800 ; L2TP failure. Usually redial will reconnect immediately, sometimes not. If it doesn't connect we receive the Error 800.

I have replaced the NIC in the server, 3 different interfaces. Same error resutls.

Everything looks good in the SBS Console ; I've run the Network connectivity wizards multiple times and get green lights on everything.  Which, BTW, this is my first experience with SBS and I am not a fan of how controlling the Console is. I realize it's useful for those who want simpler administration. It's just seems the Wizard is not so Wizardly sometimes.!

I'm kind of out of ideas at this point.