Link to home
Start Free TrialLog in
Avatar of Gerhardpet
GerhardpetFlag for Canada

asked on

RDC not working using OpenVPN

I recently got myself a new notebook with Win7 with all windows updates installed. Right from the start I was getting the following error when connecting to servers at my customer's using remote desktop connection (RDC) and OpenVPN

because of an error in data encryption, this session will end. please try connecting to the remote computer again.

I thought that something was wrong on the notebook but over the weekend I installed all the lasted updates on my desktop (where I did not have the problem) and now I have the same problem on the desktop.

I've been scratching my head on this one. The problem is that I only get the error on any server I try to connect to using OpenVPN. All of my customers have Untangle with OpenVPN except for 2. I use OpenVPN to connect to them and one of the other customer is use a SonicWall client. No problems with the SonicWall client and RDC.

The only time I have this problem when it is OpenVPN client to an Untangle box with all Microsoft updates installed. I'm running the latest OpenVPN client from openvpn.net

Has anyone else had this problem? Any ideas?
Avatar of Member_2_6582184
Member_2_6582184
Flag of Germany image

Since you seem to be able to initialize the connection, I suspect this is NLA. Try to switch it off for testing, here the technet paper on NLA:
http://technet.microsoft.com/en-us/library/cc732713.aspx

And a howto to switch it off:
http://www.2x.com/blog/2013/03/news/disabling-network-level-authentication-for-remote-desktop-services-connections-2/
Avatar of Gerhardpet

ASKER

You mean switch it off on the server or my desktop?
Both, depending if your server only accepts NLA. Go to RDP settings and allow 'any version' - then setup your RDP client not to use NLA.

However, I most commonly use OpenVPN and had never issues with this and run all RDP sessions with NLA on. If it works with NLA off, then the trust chain in the certificates is somehow broken. OpenVPN might not be directly the cause then but some domain names or IPs.
In my case I think it's got to be OpenVPN. I'm now connected to a Server using SonicWall and and 2 servers without a VPN. All 3 RDC sessions are active and working as I write this.

Then I have 5 customers with Untangle/OpenVPN and none of them work.

I get this on all of them

because of an error in data encryption, this session will end. please try connecting to the remote computer again.
Even with NLA disabled?

The error message is from NLA, broken trust chain. Use TLS and you are fine for now. Then check your trust settings and if there are subnets not allowed.

Again, OpenVPN is not the cause, but something else in the settings of either the server or the client making the connection.
Very strange. I made the connection to 3 different servers (at different customers) with no VPN. No issues. Then right after I tried 2 customer servers with OpenVPN and wola it worked.

Now all of them work. I have not made any changes to my desktop nor the servers
Really hard to tell. IMHO OpenVPN has nothing to do with it - as a tunnel connection this should be (and is) transparent.
But since you need a tunnel network, that IP/network segment you get from OpenVPN might be the cause
Well for now it fixed itself and I'll if it happen again. I'll leave this question open for now
I don't get this. Back to the same problem after rebooting my computer. And it is only a problem if connecting via OpenVPN. In all case it is OpenVPN/Untangle at both ends

Using SonicWall for VPN there is not problem. Also a have 2 servers where I can connect to directly without a VPN and it works fine too.

I wish I could once and for all find a solution to this
Have a look at this thead:
Untange / OpenVPN / RDP
http://forums.untangle.com/openvpn/30898-rdp-over-openvpn.html
Untagle says its not them, but a bug in Windows TCP stack:
http://social.technet.microsoft.com/Forums/windowsserver/en-US/3e4e9d8a-cf6a-4e7a-9072-f9ecd3f17a72/because-of-an-error-in-data-encryption-i-get-disconnect-rdc

It seems the user did solve this by disabling large send offload in the NIC of the windows machine:
To workaround this issue, set the host NIC property "IPv4 Large Send Offload" to disabled.
Alternatively, bind RDP to a non-TOE interface, or one with working TOE.

Worth testing, but not derivable in the long run for servers. The second method, bind RDP to another interface where TOE is disabled, is far better but may not posible.
But since you do not have this with sonic wall, untangle seems to be is involved here, no matter what they say.
But since you do not have this with sonic wall, untangle seems to be is involved here, no matter what they say.

You are right and also I can make an RDP connection without using a VPN. I will try the links
Maybe another option would be to switch to Ipsec at the Untangle sites?
SOLUTION
Avatar of Gerhardpet
Gerhardpet
Flag of Canada image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Found the problem myself