Urgent: VMware ESX Server Roll Backup on Reboot

Hi All,

One of our customers has physically reset (using the power button) their ESX server. Upon powering it back on, it looks to have reverted to an old snapshot (number of years ago). What is the likelihood of being able to get this back? What steps should I take to mitigate risk of losing the data since the snapshot was taken?

We do have a valid backup, but obviously booting into the valid state would be much better than having to restore the systems (2 VMs) from backup.

Also, can anyone offer any explanation to how this happened given there was no human intervention other than rebooting the server? It seems like a critical flaw in the design if the system boots into an old snapshot if the power is lot.

Thanks in advance.
Who is Participating?
JimmyDailyConnect With a Mentor Author Commented:
Thanks for the feedback, it seems as though we've worked out what's happening here. Someone on-site actually powered on the old SBS server that was P2V'ed way back, but never formatted. So when we were remotely accessing the box, we were actually accessing the old server, and not the VM. Seems as if the VM host is still offline.

We'll know definitively tomorrow morning. Cheers.
After the ESX host reboot, are you saying one of the *VMs* reverted to an old snapshot? Hmm...not sure why that would have happened exactly. I haven't heard of this happening before.

I personally just stay away from snapshots as they can tend to be a nuiscance. If a VM fails...beit from update or whatever, I'll just restore it from a backup. I use Veeam to backup all my VMs. I do so a couple times during the week, so I would lose a day, but all my 'data' is on a SAN...not a VM (or on any of my physical servers either), so there really isn't a problem restoring a VM from a couple days off. Anyway...my recommendation is, if you have a backup copy of the VM (if this is what you meant), to restore that backup to your host.

Well that makes sense. Glad you got it figured out :)

On that point I've seen a few odd things.
Recently (believe it or not) a path failed on the SAN, I won't go into which SAN but it's a big boys, anyway both filers panicked and restarted..
The next stages were quite amazing.
ESX host which had plenty of memory and CPU available seemed to continue to run the guests in memory..
BUT a few guests were nasty. I had one which came back after a little work which had a different static IP than it had prior to the SAN failure..
Another which appeared to revert to a snapshot as I think it was being VCB'ed at the time.
About 20 which just hung and needed a restart.
1 which corrupted the Notes address book.
But all up that's pretty remarkable out of 400 odd guests any a few issues, and considering we had the entire shop running again in about 2 hours.
Me I was on a trip at the time and worked from the passenger seat via NextG.. recovering systems etc.
Just a short tail of Murphy's Law..
All Courses

From novice to tech pro — start learning today.