Server lights green, but no access to anything
Posted on 2011-09-19
One of our clients has an HP Proliant ML350 that has been having a recurring problem. Of course, it doesn't happen all the time, but it happens often enough that it seems like there is a pattern. Here is the most recent event log entry related to this issue:
The previous system shutdown at 10:07:19 PM on 9/16/2011 was unexpected (Event ID: 6008)
When this happens (usually on Friday evening, it seems), the server lights are all showing a green status (either solid, or flashing, as appropriate), but no server resources are accessible, nor can we access the server via its own monitor, nor can we address it via RDP. We are forced to shutdown the server manually, then start it back up. After restarting, we get the above-mentioned log entry that the shutdown was unexpected.
We initially suspected that the old UPS attached to it was causing the trouble, since it was not an adaptive sinewave-capable UPS, which is required by all the recent Proliant ML 350's. We replaced the UPS with an adaptive sinewave-capable UPS thinking that we would fix the problem, but it did not. There are two power supplies in this server. One goes to the UPS, the other to a dedicated circuit. The dedicated circuit is not otherwise protected, but we are going to fix that right now (surge protector, another UPS, etc.).
There are 17 entries with Event ID 6008 dating back almost exactly a year (back to 9/21/10). Seven of these entries occurred within a couple minutes of 10:10pm local time. The other 10 seem more random. It's weird to me that there are almost half that occur at almost exactly the same time. The backup is scheduled to run at 11:00pm local time, so it's not triggering the problem. We don't have any other scheduled tasks near 10:00pm.
Any ideas where to start? Thanks for your suggestions!