Solved

VMWARE ESX server appears as disconnected in vSphere client.

Posted on 2014-03-31
15
1,972 Views
Last Modified: 2014-04-09
During the last few weeks, I have a problem in our VMWARE ESX cluster (Rel.3.5, build 207095): 1 of the 9 servers appears as disconnected in the vSphere client of our vCenter Server (both: Rel.4.0.0, build 258672), while it seems to have a normal operation (the ESX Server console is displayed on the system's screen, it accepts login connections and any Linux command via SSH and it seems to have access to all the VMFS volumes on the shared storage engines). Whenever I reboot that server, it seems to be connected to the cluster ("Connected" state and "Normal" status) for a few seconds (less than 1 minute) and then goes into "Not responding" state. Same thing happens whenever I re-connect that server to the cluster (by right-clicking on its entity in the vSphere client and selecting "Reconnect"). Restarting the VMWARE management services via command-line on the server ("service mgmt-vmware restart" and "service vmware-vpxa restart") did not resolve the problem.

Advice is always welcome - thanks in advance.

Panos Tsapralis,
Athens, GREECE.
0
Comment
Question by:Panos Tsapralis
  • 8
  • 5
  • 2
15 Comments
 
LVL 25

Expert Comment

by:Zephyr ICT
Comment Utility
This could be a few things, best to go hunt the eventlogs on the host that shows the issue, the locations of the logs can be found in this article.

There's also an article of steps to take to troubleshoot disconnects, did you check this? It can be found here

Might be an issue with hostd or vpxa agent though ... The hostd.log is of interest in that case.
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
What version of VMware vSphere are you using?

have you checked the host logs?
0
 

Author Comment

by:Panos Tsapralis
Comment Utility
The hosts are running ESX Server, Rel.3.5, build 207095, and the Virtual-Center Server and client are at version 4.0.0, build 258672. I have read through the various logs on the disconnected host and couldn't find anything that could seem to be related to my problem. However, when examining the "vpxd" logs on the Virtual Center server, I noticed that there are several messages stating "Marked (hostname) as dirty". I believe that these messages are related to my problem (since they are not generated for any other host of my cluster). I am hereby attaching a small sample of the "vpxd.log" file to this message, so that other people. reading this discussion, have a chance to look at it (in this file, "kronos.phonemarketing.gr" is the hostname of the host in question).
vmware-esx-cluster-vcenter-vpxd-.txt
0
 
LVL 25

Expert Comment

by:Zephyr ICT
Comment Utility
Hmmm, any changes in firewall settings? Also check out this KB might be interesting: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1029919
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
Are the hosts local to vCenter server?
0
 

Author Comment

by:Panos Tsapralis
Comment Utility
Andrew, the ESX hosts are, indeed, local to the vCenter server (I suppose that you mean that the cluster and the vCenter server are communicating through the same LAN) - actually, the vCenter server is itself a VM on one of the ESX hosts (not the disconnected one!...).

spravtek, the Windows firewall is turned off on the vCenter system and I have verified that the vCenter system and the disconnected ESX host are communicating with each other (using "ping" commands in both directions). I am going to carry out the testing procedure, described in the VMWARE KB article that you mentioned and see what comes out (although I do not really believe that the problem is in the UDP/TCP ports of the vCenter system, because - in that case - I would experience the same problem on my other ESX hosts as well, right?...).
0
 

Author Comment

by:Panos Tsapralis
Comment Utility
I have carried out the connectivity testing procedure of VMWARE KB article #1029919 (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1029919) and I have verified that the disconnected ESX host "speaks" to the Virtual Center server through UDP port "902".

What else is there to look at?...
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
TCP 443 and TCP 902
0
 

Author Comment

by:Panos Tsapralis
Comment Utility
Andrew, I have verified that TCP port "443" of the Virtual Center system is accepting connections from the disconnected host (that was expected, anyway, since "https://<vcenterserverhostname>/" is accessible from anywhere within my LAN).

Also, entering "telnet disconnectedesxhostname 902" from a command-line window or opening the page "http://disconnectedesxhostname:902/" in a browser within the Virtual Center server (towards the disconnected ESX host) produces the following message:

"220 VMware Authentication Daemon Version 1.10: SSL Required, ServerDaemonProtocol:SOAP, MKSDisplayProtocol:VNC , "

This the same message that I get when using the same commands towards any other (connected) host in my cluster. Therefore, I assume that TCP port "902" is also open for connections on the troubled host.
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
do you ever get disconnected from the server?

does vCenter Server ever connect?
0
 

Author Comment

by:Panos Tsapralis
Comment Utility
When I connect to the troubled host via SSH or the browser, I do not get disconnected (and I feel very confident that the test, that I described earlier - putting a VM on that server - will verify that the host works fine...). When I try to connect from within vCenter Server (right-click on thw disconnected host and select "Connect"), the host stays connected for up to 60 seconds and then returns itself to "Not responding/Alert".

In this Vmware KB article:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1012382#vcenter_4.x

I see that TCP ports "623" and "5989" are also used in the connection between the vCenter system and the ESX hosts. Should I test these ports as well (I'm going to anyway...)?
0
 

Author Comment

by:Panos Tsapralis
Comment Utility
Nope - neither TCP port "623" nor "5989" are used between the Virtual Center server and any ESX host in my environment...
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
Management traffic is carried via 443 tcp. (between vCenter and Host)
0
 

Accepted Solution

by:
Panos Tsapralis earned 0 total points
Comment Utility
I solved the problem based on advice in this VMWARE KB article:

http://kb.vmware.com/selfservice/documentLinkInt.do;jsessionid=3EF75AC47C91118D42A44F26F15216F4?micrositeID=null&externalID=1011647

(The IP address of the Virtual Center system must be entered into the "vCenter Server Managed IP" field of the Runtime Settings of the vCenter Server Settings dialog in the vSphere client).

Thanks to all for being helpful...
0
 

Author Closing Comment

by:Panos Tsapralis
Comment Utility
The advice of the article, mentioned in the solution, fits exactly to this issue.
0

Featured Post

VMware Disaster Recovery and Data Protection

In this expert guide, you’ll learn about the components of a Modern Data Center. You will use cases for the value-added capabilities of Veeam®, including combining backup and replication for VMware disaster recovery and using replication for data center migration.

Join & Write a Comment

Will try to explain how to use the VMware feature TAGs in the VMs and create Veeam Backup Jobs using TAGs. Since this article is too long, I will create second article for the Veeam tasks.
HOW TO: Install and Configure VMware vSphere Hypervisor 6.5 (ESXi 6.5), Step by Step Tutorial with screenshots. From Download, Checking Media, to Completed Installation.
Teach the user how to rename, unmount, delete and upgrade VMFS datastores. Open vSphere Web Client: Rename VMFS and NFS datastores: Upgrade VMFS-3 volume to VMFS-5: Unmount VMFS datastore: Delete a VMFS datastore:
Advanced tutorial on how to run the esxtop command to capture a batch file in csv format in order to export the file and use it for performance analysis. He demonstrates how to download the file using a vSphere web client (or vSphere client) and exp…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now