akhalighi
asked on
Two ESX servers get rebooted randomly at the same time
Hello Experts
We have 3 ESXi 4.0 servers , from last week ... two of them get rebooted at the same time. one time it happened @ 1:30 AM on Friday and next time was today @ 8:00 AM.
we had some cabling changes in serevr room last week , so I thought maybe it's a power issue , but ESX servers are connected to different outlets and other servers are connected to the same outlets...
How I can investigate this issue ? log files are very complicated and they are getting rolled over after ESX reboot . I extracted all zip files inside report bundles but didn't get anything for the time of reboot.
We have 3 ESXi 4.0 servers , from last week ... two of them get rebooted at the same time. one time it happened @ 1:30 AM on Friday and next time was today @ 8:00 AM.
we had some cabling changes in serevr room last week , so I thought maybe it's a power issue , but ESX servers are connected to different outlets and other servers are connected to the same outlets...
How I can investigate this issue ? log files are very complicated and they are getting rolled over after ESX reboot . I extracted all zip files inside report bundles but didn't get anything for the time of reboot.
Do you have the power management for the cluster turned on?
ASKER
as far as I know , there is no cluster . there are 3 ESX servers under a data center.
I checked power management option under configuration , it's empty : attached the screenshot.
vmware.jpg
I checked power management option under configuration , it's empty : attached the screenshot.
vmware.jpg
Reason for previous posting is that DPM (power management) can be enabled on a cluster of ESX/ESXi hosts. If you do not have the cluster, as I have highlighted above, then other option to look for are: 1) cron job running on the servers themselves, 2) roles and who has ability to reboot the hosts. I will check my test environment further for anything else I can find.
ASKER
Thanks frevere
there is no clusters fo sure .
As for roles ; there is only one and that's domain administrator that can connect to Vcenter. when I select the ESX servers in question I can see all events ( like powering up VMs , etc ) but there is nothing about ESX server reboot.
One important thing ; I am not sure if they get really rebooted ... we came in the morning and noticed that VMs are off . I am checking now to confirm if they are really rebooted.
there is no clusters fo sure .
As for roles ; there is only one and that's domain administrator that can connect to Vcenter. when I select the ESX servers in question I can see all events ( like powering up VMs , etc ) but there is nothing about ESX server reboot.
One important thing ; I am not sure if they get really rebooted ... we came in the morning and noticed that VMs are off . I am checking now to confirm if they are really rebooted.
ASKER
This was in message log ; I think that's an indication of a reboot.
~~~~~~~~~~~~~
Nov 30 13:05:59 syslogd started: BusyBox v1.9.1-VMware-visor-654
Nov 30 13:05:59 vmklogger: Successfully daemonized
~~~~~~~~~~~~~
I received alerts from monitoring system @ around 8:04 AM EST ; which is 13:04 GTZ ; this shows that reboot happened afterwards.
~~~~~~~~~~~~~
Nov 30 13:05:59 syslogd started: BusyBox v1.9.1-VMware-visor-654
Nov 30 13:05:59 vmklogger: Successfully daemonized
~~~~~~~~~~~~~
I received alerts from monitoring system @ around 8:04 AM EST ; which is 13:04 GTZ ; this shows that reboot happened afterwards.
it looks as though you have SSH enabled on your ESXi host. I would check the server for a cron job that is running.
How is the power in generall in your datacenter. Could it be a surge or drop that the ESXi servers might be more sensitive to?
ASKER
thanks . would you tell me how to check a running cron job ?
Not positive (you may want to google this) but you might want to look in the /var/spool/cron folder. Also check the rc.local in the /etc/ directory for any commands that would reschedule the job. To check for a running cron job, run the following: cat /var/run/crond.pid
ASKER
hmmm . okay ... I'll give this a try . but this wasn't happening before and we didn't change anything . I cannot imagine how come a cron job started occuring all of the sudden.
http://www.jules.fm/Logbook/files/add_cron_job_vmware.html for adding a cron job to ESXi. Also has link for enabling SSH. If not refer to http://www.vm-help.com/esx40i/ESXi_enable_SSH.php for enabling SSH instructions. You know glad you asked this question. I didn't know about having to run the auto backup script.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Issue didn't happened for some time . I close this question for now. Thanks for your helps