vSphere displays 2 ESXi servers as not responding
Hi all, during my study for vcp I have managed to break my ESXi install many times and have got some real good advise from Experts Exchange. It is frustrating but I'm learning lots in the mean time. Well i log on this morning and boom another problem.
My set-up
Hardware - Laptop
vmware workstation 9
2 x ESXi 5.1 installed as virtuals under workstation with about 5 hosts each.
1 2008 R2 virtual under workstation that runs vSphere.
Storage is using Starwind to a 2nd installed SSD drive deployed as iSCSI target etc.
This morning when i log onto the vSphere 2008 server and launch vSphere client both my ESXi hosts show as not responding (please see screen shot). I have rebooted everything and removed both Hosts from vsphere and then added them back. When i add them back everything is normal, I am able to access the guests and then after about 5 minutes they grey-out and show not responding again. The 2 ESXi hosts are still up and i can log onto them and ping them etc. I've searched Experts for an answer but can't find one. Anyone has any ideas would be much appreciated.
Thanks
Jason
vSphere-not-responding.png
My set-up
Hardware - Laptop
vmware workstation 9
2 x ESXi 5.1 installed as virtuals under workstation with about 5 hosts each.
1 2008 R2 virtual under workstation that runs vSphere.
Storage is using Starwind to a 2nd installed SSD drive deployed as iSCSI target etc.
This morning when i log onto the vSphere 2008 server and launch vSphere client both my ESXi hosts show as not responding (please see screen shot). I have rebooted everything and removed both Hosts from vsphere and then added them back. When i add them back everything is normal, I am able to access the guests and then after about 5 minutes they grey-out and show not responding again. The 2 ESXi hosts are still up and i can log onto them and ping them etc. I've searched Experts for an answer but can't find one. Anyone has any ideas would be much appreciated.
Thanks
Jason
vSphere-not-responding.png
Which version of ESXi are you using?
It sounds like it could be storage related?
otherwise, I think we are going to start looking at the logs:-
/var/log - vmkernel.log
also do you set you logs to log to a LUN or Syslog server, because the logs are not persistent, and will disappear when you reboot the servers.
It sounds like it could be storage related?
otherwise, I think we are going to start looking at the logs:-
/var/log - vmkernel.log
also do you set you logs to log to a LUN or Syslog server, because the logs are not persistent, and will disappear when you reboot the servers.
ASKER
Hi chaps.
1. If i look at the "alarms" tab for both Hosts i see an alert that corresponds to the time of not responding error. The alerts states for both:
192.168.17.171
Alert
Host connection and power state
2008vSphere.vsphere.local
5/7/2014 9:45:38 AM
My laptop has 16GB. Each host is assigned 6GB and at present none of the guests are running. My laptop reports 11GB of memory in use so i thing memory is OK. My data store still has 55GB availible.
I'm running ESXi 5.1. I just did a test or re-connecting one host and it stays up for about 90 seconds but during that time I was able to access a guest which is on the storage so I think this rules storage out possibly?
Andrew, I have accessed the log file using a SSH client, is that how you would do it? I have uploaded the file (see attachment) to be honest it doesn't mean a lot to me but would appreciate someone taking a look and pointing anything out that I can use etc.
Thanks both.
vmkernel.log
1. If i look at the "alarms" tab for both Hosts i see an alert that corresponds to the time of not responding error. The alerts states for both:
192.168.17.171
Alert
Host connection and power state
2008vSphere.vsphere.local
5/7/2014 9:45:38 AM
My laptop has 16GB. Each host is assigned 6GB and at present none of the guests are running. My laptop reports 11GB of memory in use so i thing memory is OK. My data store still has 55GB availible.
I'm running ESXi 5.1. I just did a test or re-connecting one host and it stays up for about 90 seconds but during that time I was able to access a guest which is on the storage so I think this rules storage out possibly?
Andrew, I have accessed the log file using a SSH client, is that how you would do it? I have uploaded the file (see attachment) to be honest it doesn't mean a lot to me but would appreciate someone taking a look and pointing anything out that I can use etc.
Thanks both.
vmkernel.log
ASKER
Oh! I have just switched to datastore and it is showing as inactive
Yes, this can be an issue with ESXi, if the iSCSI datastore disappears, the ESXi servers, keep polling the iSCSI datastore, and ESXi hosts can go responsive.
datastore disappearing is a common issue.
datastore disappearing is a common issue.
ASKER
Yep i just rebooted everything and reseated the SSD but after about 90 seconds the datastore shows as inactive. You were spot-on with it being storage. Is there anything i can do about it?
I would look closely at the build of ESXi, are you using the latest version and build ?
ASKER
Im using ESXi 5.0.0 vmkernal release build 469512
Is it possible to patch update the builds if so can you tell me how that is achieved please?
Is it possible to patch update the builds if so can you tell me how that is achieved please?
Okay, that release is very old, it's the first ESXi release e.g. GA
since that release there have been 15 updates, the latest is:-
ESXi 5.0 Patch 7, Build 1489271 released 23 Jan 2014.
You can get the updates from VMware Patch Portal here
https://www.vmware.com/patchmgr/findPatch.portal
and follow my EE Article
HOW TO: Upgrade from VMware vSphere Hypervisor ESXi 5.1 to VMware vSphere Hypervisor ESXi 5.5 for FREE
It's the same procedure and commands, just change the version numbers and files.
since that release there have been 15 updates, the latest is:-
ESXi 5.0 Patch 7, Build 1489271 released 23 Jan 2014.
You can get the updates from VMware Patch Portal here
https://www.vmware.com/patchmgr/findPatch.portal
and follow my EE Article
HOW TO: Upgrade from VMware vSphere Hypervisor ESXi 5.1 to VMware vSphere Hypervisor ESXi 5.5 for FREE
It's the same procedure and commands, just change the version numbers and files.
ASKER
Thanks dude i will take a look.
About my problem, I have managed a workaround for the problem.
I removed vsphere server and SQL and the vsphere client from the 2008 server, rebooted and then redeployed the vsphere server and client. I then created a new datastore and added the 2 ESXi hosts back and now everything seems stable.
Here's what i don't understand. I have to create a new data centre for the new vsphere server. Now when i browse the data store it shows me all my virtual servers and uploads etc but how does it show them when this is a new data centre - How does the new data centre auto plug into the data store?
Ammended for mistakes.
About my problem, I have managed a workaround for the problem.
I removed vsphere server and SQL and the vsphere client from the 2008 server, rebooted and then redeployed the vsphere server and client. I then created a new datastore and added the 2 ESXi hosts back and now everything seems stable.
Here's what i don't understand. I have to create a new data centre for the new vsphere server. Now when i browse the data store it shows me all my virtual servers and uploads etc but how does it show them when this is a new data centre - How does the new data centre auto plug into the data store?
Ammended for mistakes.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Doh! :)
ASKER
Thank you again Andrew.
I believe these hosts are utilizing entire resources (CPU /Memory /Disk)
Try to ackoledge the alerts clear the warnings. and check. If possible increase the capacity.