PCOIP protocol time through vmware view security server

When I try to connect to my VDI outside of our local network my connection times out when logging into the virtual machine using PCOIP. We are using View 5.2 It connects to the connection server, shows us our entitled pools, connects to a desktop then hangs up and times out when actually signing into the desktop. A desktop is even assigned  to the user. It appears to only be happening with PCOIP. The only weird thing I seem to be seeing in the log on the desktop is:

2013-06-03T12:59:40.178-04:00 INFO  (0414-0E58) <MessageFrameWorkDispatch>
[wsnm_desktop] Session CONNECTED (PCOIP) with CHANGED CREDENTIALS:
expected DOMAIN\user, got DOMAIN\DomainAdministrator

Open in new window

Everythign works like a charm on our local network tho. Except we will intermittently get an error that the PCOIP protocol is not available at this time. That is solved by just simply reattempting the login.

We are also not using IPSec for these machines.

Vmware view 5.2
Windows 7 x64
Windows Server 2k8
Who is Participating?

[Webinar] Streamline your web hosting managementRegister Today

bbayachekConnect With a Mentor Author Commented:
The problem ended up being a routing issue.

The way we have it setup is our Public IP is being NATed to a private IP inside our DMZ. The IP address in the DMZ is associated with a NIC on our security server. That security server has a second NIC that is setup with a private IP that is in our Private LAN.

Because both NICs had the same metric some of the traffic was being routed incorrectly. To fix it we created a perpetual static route on the server:
route add mask -p

Open in new window

All looks good now.

Sounds like an MTU black hole problem.  Basically, the smaller packets negotiating the session go through OK, but the large packets with the screen shots get dropped because they exceed the allowed MTU size on one of the network hops.

Are you using a VPN?  Most VPN clients will allow you to set the MTU size.

If you're not able to do that, you can set the MTU size on your network card.

One test is to ping the View server with packets of various sizes.  On Windows, you use the -l switch to set the size (that's a lower-case "L"):

ping -l 1500

ping -l 1350

ping -l 100

just FYI PCoIP uses udp so if you  are running a ssl vpn it will not work. try ipsec.
Making Bulk Changes to Active Directory

Watch this video to see how easy it is to make mass changes to Active Directory from an external text file without using complicated scripts.

bbayachekAuthor Commented:
If I VPN in and try to connect I have no problems. The problem I am having is through a Security Server.

I tried pinging the security server, the connection server and the vcenter server from the VM with 65500 bytes of data and had no problem.
You need to ping the VM from the client.
bbayachekAuthor Commented:
I can't ping the VM from the Client. The VM is in my enterprise network and the client is in the WAN.

PCOIP works perfect inside our network. It just fails when accessing Via the WAN
Is it a WAN or a VPN?

Why and where are pings blocked?  It seems likely that whatever is blocking the pings is also blocking PCoIP.
bbayachekAuthor Commented:
I am trying to connect over a WAN, not VPN. I can ping the public IP and everything but there is no way for me to directly ping the Virtual Desktop that I am connecting to. The Virtual desktop resides in a Private LAN not the WAN.

I am using PCOIP with the View Client over a WAN through a View Security server, not a VPN. There is no need to use VPN when connecting through a Security server.

I have a public IP address that is setup to NAT to my security server IP address. The security server IP address sits in my DMZ. My security server has 2 NICS (one in the DMZ, the other in my private LAN). The security server then hands off the connection attempt to my connection server which resides only in my private network. The connection server gets the request, offers up the entitled desktops, talks to the entitled desktop and begins the sign on process on the actual virtual desktop. Therefore network transmission up to and including the virtual desktop has no problem. Now the question is why is the log in timing out? Is there possibly data flow not going back out through the firewall to the WAN.

Again pcoip DOES work internally on my local LAN. I have no problems with PCOIP locally. My problem is ONLY via the WAN and I am NOT using VPN. Strictly going through the security server.

I hope that clears things up?
OK, I think we're getting our terminology crossed up.

Typically when I hear "WAN," I think that everything is part of the same private network.  

Are you using "WAN" interchangeably with "Internet?"

What firewall product are you using?
See this article for required ports (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1027217).  It looks to me like maybe you're missing TCP/UDP 4172.  

TCP connections are stateful, so usually you only have to set up permissions in one directlion.  UDP connections are not stateful, so you often have to set up rules to allow the traffic in both directions.  Make sure you allow UDP 4172 outbound from the security server to the client.  (I think you have to allow source IP <security server> source port UDP 4172 destination IP <client/any> UDP port <any>.)
bbayachekAuthor Commented:
Yes, that is correct, I was using it interchangeably for internet. Sorry about that.

These are the ways I am able to communicate over port 4172

Virtual Desktop ---> Vcenter server = No communication over 4172
Virtual Desktop ---> Connection Server = Communication over 4172
Virtual Desktop ---> Security Server = Communication over 4172

Vcenter Server ---> Virtual Desktop = No communication over 4172
Vcenter Server ---> Connection server = Communication over 4172
Vcenter Server ---> Security Server = Communication over 4172

Connection Server ---> Vcenter server = No Communication over 4172
Connection Server ---> Virtual Desktop = No Communication over 4172
Connection Server ---> Security Server = Communication over 4172

Security Server ---> Vcenter server = No Communication over 4172
Security Server ---> Connection Server = No Communication over 4172
Security Server ---> Virtual Desktop = No communication over 4172

Open in new window

I'm not sure why the security server cannot get anywhere with 4172 tho. I have the windows firewalls disabled to rule them out and my Cisco firewall has an inside rule that says port 4172 Source/Destination = ANY/ANY.
I think the traffic needed is client->security server and security server->client.

(Is this a Cisco ASA?  Can you provide the static and dynamic NAT rules pertaining to the security server?)
bbayachekAuthor Commented:
There was no firewall issue with regards to the PCOIP protocol, and RDP may or may not have worked entirely the whole time. The issue turned out to be a routing issue that was fixed with a static route.
All Courses

From novice to tech pro — start learning today.