VMware ESXi host 5.1 stays connected, but powers off vm's

Hey,

We've learned the hard way that when you set your VMWare server into maintenance, the servers (VM's) will be shut down, might this be the cause??

Could you expand on the details, of make, model of host, VMware Tools installed, Virutal Machine OS, disk type, Host Memory, CPUs....

is the host on HCL....

check VM vmware.logs in the vmware folder....

check /var/log/vmkernel.log

do all the VMs power off or shutdown?

event logs on VMs - do you get unexpected shutdown, or power off ?

Version of ESXi 5.1 ?

Sure. It's an HP Proliant ML350 G6 running VMware ESXi 5.1.0 1900470 with 8 CPU's x 2.4GHz with 24 GB RAM. It's on the HCL. All the VM's power off and on the Windows Server, it shows an unexpected shutdown. In the vmkernel.log file, the only error I'm seeing is this (which looks like the DVD Drive):

ScsiDeviceIO: 2331: Cmd(0x41240040c5c0) 0x4d, CmdSN 0x16 from world 14536 to dev "naa.600508b1001cc9ac4a3b94a37e825430" failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
2014-08-07T14:00:37.431Z

So, it looks like the VMs are not powering off, they are failing and crashing.

Any info in the vmware.log ?

How many VMs, and do all fail ?

The only 2 VM's are a Windows 2008r2 file server and a linux based WAN optimization appliance. I put some of the failure output from both vmlog's below:

Think it's the CD drive? Thanks so much again.

ConfigDB: Failed to load /usr/lib/vmware/config
2014-08-07T10:59:52.092Z| vmx| I120: ConfigDB: Failed to load ~/.vmware/config
2014-08-07

PREF Failed to load user preferences.
2014-08-07T10:59:52.097Z| vmx| I120

| vmx| I120: Vix: [6315 mainDispatch.c:475]: VMAutomation: Initializing VMAutomation.
2014-08-07T10:59:52.132Z| vmx| I120: Vix: [6315 mainDispatch.c:772]: VMAutomationOpenListenerSocket() listening
2014-08-07T10:59:52.145Z| vmx| I120: Vix: [6315 mainDispatch.c:3870]: VMAutomation_ReportPowerOpFinished: statevar=0, newAppState=1870, success=1 additionalError=0
2014-08-07T10:59:52.145Z| vmx| I120: Transitioned vmx/execState/val to poweredOff
2014-08-07T10:59:52.145Z| vmx| I120: Vix: [6315 mainDispatch.c:3870]: VMAutomation_ReportPowerOpFinished: statevar=1, newAppState=1873, success=1 additionalError=0
2014-08-07T10:59:52.145Z| vmx| I120: Vix: [6315 mainDispatch.c:3870]: VMAutomation_ReportPowerOpFinished: statevar=2, newAppState=1877, success=1 additionalError=0
2014-08-07T10:59:52.145Z| vmx| I120: Vix: [6315 mainDispatch.c:3870]: VMAutomation_ReportPowerOpFinished: statevar=3, newAppState=1881, success=1 additionalError=0
2014-08-07T10:59:52.145Z| vmx| I120: UTIL: Change file descriptor limit from soft 2320,hard 2320 to soft 4096,hard 4096.
2014-08-07T10:59:52.145Z| vmx| I120: PTSC: using TSC + offset (-350122277952)
2014-08-07T10:59:52.145Z| vmx| I120: PTSC: hardware TSCs are synchronized.
ObtainHardwareID unexpected failure: 22.
2014-08-07T10:59:52.149Z| vmx| W110: Hostinfo_MachineID ObtainHardwareID failure (Invalid argument); providing default.
2014-08-07T10:59:52.149Z| vmx| I120: [0x23506000-0x23EFDDEC): /bin/vmx
2014-08-07T10:59:52.643Z| vmx| I120: changing directory to /vmfs/volumes/5373d9e5-8b5f29f6-04a2-d8d385c0a89e/CDC-VX-1000/.
2014-08-07T10:59:52.643Z| vmx| I120: Config file: /vmfs/volumes/5373d9e5-8b5f29f6-04a2-d8d385c0a89e/CDC-VX-1000/CDC-VX-1000.vmx
2014-08-07T10:59:52.644Z| vmx| I120: Vix: [6315 mainDispatch.c:3870]: VMAutomation_ReportPowerOpFinished: statevar=1, newAppState=1873, success=1 additionalError=0
2014-08-07T10:59:52.644Z| vmx| I120: Vix: [6315 mainDispatch.c:3870]: VMAutomation_ReportPowerOpFinished: statevar=2, newAppState=1878, success=1 additionalError=0
2014-08-07T10:59:53.726Z| vmx| I120: VMXVmdbCbVmVmxExecState: Exec state change requested to state poweredOn without reset, soft, softOptionTimeout: 0.
2014-08-07T10:59:53.726Z| vmx| I120: Tools: sending 'OS_PowerOn' (state = 3) state change request
2014-08-07T10:59:53.726Z| vmx| I120: Tools: Delaying state change request to state 3.
2014-08-07T10:59:53.726Z| vmx| W110: PowerOn
2014-08-07T10:59:53.737Z| vmx| I120: Vix: [6315 mainDispatch.c:3870]:

serial0: Overrun
2014-08-07T11:00:16.358Z| vcpu-1

Error VIX_E_UNRECOGNIZED_COMMAND_IN_GUEST in VMAutomationTranslateGuestRpcError(): Tools failed to recognize guest command op code=62, result string="Unknown Command".
2014-08-07T11:00:41.266Z| vcpu-2| I120: TOOLS Received tools.set.version rpc call, version = 2147483647.
2014-08-07T11:00:41.266Z| vcpu-2| I120: ToolsSetVersionWork did nothing; new tools version

2014-08-07T10:59:47.792Z| vmx| W110: ObtainHardwareID unexpected failure: 22.
2014-08-07T10:59:47.792Z| vmx| W110: Hostinfo_MachineID ObtainHardwareID failure (Invalid argument); providing default.

2014-08-07T11:00:36.512Z| vcpu-2| I120: CDROM ide0:0: CMD 0x5a (MODE SENSE(10)) FAILED (key 0x5 asc 0x24 ascq 0)

2014-08-07T11:00:48.216Z| vmx| I120: USBIO: Detected usb-storage class
2014-08-07T11:00:50.406Z| vmx| I120: GuestRpcSendTimedOut: message to toolbox-dnd timed out.
2014-08-07T11:00:51.885Z| vcpu-1| I120: Unknown int 10h func 0x0000
2014-08-07T11:00:52.396Z| vcpu-0| I120: SVGA enabling SVGA

Do you have cdroms in the VM?

The host has one. I just disabled the connected one on the vm. Thanks!

Did you use the HP OEM version of ESXi before we continue?

Yes, I did. Thanks.

Thanks! I'll give that a try. I really appreciate it.

Appreciate the help!