ESXi Hosts unknown shutdown

I have two DL380 servers.  One is a G6 and the other a G7 running ESXi 4.1, in vcenter.  This morning at 0708 both host shutdown.  Upon arrival at the site, both hosts had power and just needed to be powered on.  No other servers were shut down or rebooted in the rack.  The servers are also configured to turn on when power is applied, but did not do so.  They are also attached to a UPS.  The UPS log showed two low voltage events shortly before the servers went down, each was about 6 seconds.

We believe the UPS is faulty, but it does not indicate so in our monitoring software.

What logs can I look at on the ESXi servers to get any information on what may have happened?
asrvwizAsked:
Who is Participating?
 
Andrew Hancock (VMware vExpert / EE MVE^2)Connect With a Mentor VMware and Virtualization ConsultantCommented:
Correct ESXi does not keep logs, through a reboot.

You need to create a Syslog server, and get ESXi to send the alerts to a remote Syslog server.

if the server has just died, stopped, powered off, its unlikely anything will be recorded.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)Connect With a Mentor VMware and Virtualization ConsultantCommented:
look at

/var/log/vmkernel.log

BUT, you may find the events have gone, because there is little or no storage for ESXinto store events, if the servers have been restarted, the logs will be cleared.

you need to setup a Syslog server for ESXi servers to log events remotely toma syslog server.

see here to enable Syslog

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1016621
0
 
asrvwizAuthor Commented:
No vmkernel.log in /var/log  We do have syslogging enabled to our SIM appliance, I will have to pull reports to see if I can find anything.
0
Improve Your Query Performance Tuning

In this FREE six-day email course, you'll learn from Janis Griffin, Database Performance Evangelist. She'll teach 12 steps that you can use to optimize your queries as much as possible and see measurable results in your work. Get started today!

 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
sorry, its now called messages
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
you wil see older logs .0.gz etc
0
 
andyalderCommented:
There may be some info on the server healt log, but you'd have to reboot from SmartStart CD to read that.

What make/model of UPS is it?
0
 
asrvwizAuthor Commented:
Andyalder,

The UPS is an APC Smart 3000 RM XL.  We moved some of the load off of it and ran a calibration  and a self test both passed.  We think the UPS is bad but we are currently remodeling the space and will be replacing all UPS's with one.  We already have a generator behind the UPS.


Hanccocka,

So it is var/log/messages?  It must get overwrote when it boots.  We do capute that log, but I could not find anything in it.
0
 
andyalderConnect With a Mentor Commented:
SmartUPS are generally OK, the PSUs in these servers need pure sine wave which they provide (unless faulty).
0
 
asrvwizAuthor Commented:
Thanks for the help Everyone!
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.