VRA 7.5 can connect to remote console

Eduardo Alvarado
Eduardo Alvarado used Ask the Experts™
on
We noticed that we can't open a remote console for a deployed VM from VRA web interface, we get the below error:

"Cannot connect to remote console. Verify that the machine is powered on and connected to the network."

The VMs are powered On, on Vcenter side the VMs look healthy.

This is what i have tried so far:

Ping from Host to vcenter where the VRA VM resides:
	[root@host:~] ping vcenter
	PING vcenter (10.93.104.30): 56 data bytes
	64 bytes from 10.93.104.30: icmp_seq=0 ttl=64 time=0.186 ms
	64 bytes from 10.93.104.30: icmp_seq=1 ttl=64 time=0.282 ms
64 bytes from 10.93.104.30: icmp_seq=2 ttl=64 time=0.343 ms

	--- vcenter ping statistics ---
	3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.186/0.270/0.343 ms

Open in new window



-Curl to port 902 from vRA to host works fine:
            
[replica] vra1:~ # curl -vvv telnet://host.domain.local:902
		* Rebuilt URL to: telnet://host.domain.local:902/
		*   Trying 10.93.104.27...
		* TCP_NODELAY set
		* [b]Connected to[/b] hosts.domain.local (10.93.104.27) port 902 (#0)
		220 VMware Authentication Daemon Version 1.10: SSL Required, ServerDaemonProtocol:SOAP, MKSDisplayProtocol:VNC , VMXARGS supported, NFCSSL supported/t

Open in new window



-Connection to port 902 from vRA to host:
	[root@host:~] esxcli network ip connection list | grep 902
	tcp         0       0  10.93.104.27:902                10.245.253.3:40456  [b]ESTABLISHED[/b]     67166  newreno  busybox
	tcp         0       0  :::902                          :::0                LISTEN          67166  newreno  busybox
	tcp         0       0  0.0.0.0:902                     0.0.0.0:0           LISTEN          67166  newreno  busybox

Open in new window


In Security properties on VRA this is already set:
 vra1:/etc/vcac # grep -i timeout security.properties
consoleproxy.timeout.connectionInitMs=20000

Open in new window


Infrastructure -> DEM status -> all are online
Infrastructure -> log:
Error:
The underlying connection was closed: An unexpected error occurred on a receive.
Inner Exception: Certificate is not trusted (RemoteCertificateChainErrors). Subject: C=US, CN=vm-vcenter.domain.local Thumbprint: DBDF5C8DDAF5C4AE34A55AB995DFF56C14B13181

Open in new window


Stack trace:
at System.Web.Services.Protocols.WebClientProtocol.GetWebResponse(WebRequest request)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
at VMware.vSphere.VimService.RetrieveServiceContent(ManagedObjectReference _this)
at DynamicOps.VMWareModel.Interface.VSphereSession.Connect(String userName, String password)
at DynamicOps.VMWareModel.Interface.VSphereInterface.ConnectToVCenter(Uri connectionUri, String userName, String password)
at DynamicOps.Web.VMRC.Vmrc.OnInit(EventArgs e)
Inner Exception: at DynamicOps.Common.GlobalCertificateValidationManager.ThrowUntrustedCertificateException(SslPolicyErrors sslPolicyErrors, X509Certificate certificate)
at DynamicOps.Common.GlobalCertificateValidationManager.ServerCertificateValidation(Object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
at System.Net.ServerCertValidationCallback.Callback(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Net.ServerCertValidationCallback.Invoke(Object request, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
at System.Net.Security.SecureChannel.VerifyRemoteCertificate(RemoteCertValidationCallback remoteCertValidationCallback, ProtocolToken& alertToken)
at System.Net.Security.SslState.CompleteHandshake(ProtocolToken& alertToken)
at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result)
at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.ConnectStream.WriteHeaders(Boolean async)

Open in new window


I'm new to VRA and this is my first issue so not sure what all that means but seems a certificate issue, what else can I do/check?

This is a VRA cluster of 3 appliances, another thing that I noticed is that vra1 appliance is not set as Master, could this be a possible cause?

Cluster Connection Status: Connected.

vra1: Replica UP/Async
vra2: Master / UP
vra3: Replica UP/Async
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®

Author

Commented:
Time configuration is:
ESXI where VRA vm resides: EDT
Vcenter where VRA vm resides: EDT
VRA1,2,3: UTC(4 hours of difference)
Unless it is a "well known" acronym (such as GMT or UTC when mentioning time), it is always a good idea to include the expanded acronym on its first occurance in a description.

I presume that you mean vRealize Automation.

How many VMs is this occuring on ?
How many VMs are deployed ?

Please confirm each of the prerequisites from here are met.

Author

Commented:
Prerequisites met as shown in the first post, as I thought I was a cert issue. Issue is resolved.
There was a problem with certificate chain for the all the VCs after VRA upgrade.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial