Got a bit of a problem with a Windows 2012 R2 box and would appreciate any suggestions. it isn't a massively vital system (internal instant messaging server) and I sill have a few other potential restore options before rebuilding but I wonder whether I have missed something blindingly obvious and would like to see if can solve issue, apart from anything else before rebooting the other 8 or so VM!
This is running on VMware ESXi with two hosts running to a iSCSI SAN. There is also a DR site with VM replication setup
This started at the weekend with a routine reboot. The server hadn't been rebooted for a few months, possibly 4-5 months and it had a small number of updates + a load of optional ones.
I didn't create a VM snapshot before it rebooted, apart from anything else it had just backed up and replicated to DR site and was routine updates.
But... it disappeared, checking the VM console it was in a cycle of failed boot and automatic repair. It shows "inaccessible boot device" but just that, no code like I would normally expect to give a hint.
At that point I took a snapshot which I reverted to at various points before trying different repairs and restore.
Safe mode, command prompt, last known good etc. all failed the same of course
The command prompt repair had an issue in that the local admin password for this box did not match documentation and would not work.
So after a boot off Windows 2012 R2 DVD. Login to command prompt that way.
Carried out checks with command line tools, Bcdedit, bootrec etc. All looks OK, partitions look OK etc. right drive active, BCD looks OK etc.
Looked with DISM etc. to see what updates were put on. = approx. 4 .NET framework updates. Multiple other updates pending but not yet installed
kb2894856 = net framework 4.5
KB3010788 = OLE on Windows 7?
KB3032663 = net framework 4.5.2
KB3037579 = net framework 4.5.2
kb3045717 = Narrator
KB3094486 = KDS
Pending updates - tried clearing using online suggestions removing pending XML files, and registry entries
Reset local admin password using Linux NTPASSWD boot disc. It advised it thought password might already be blank (I later tried blank password which did not work either)
So then command prompt repairs same as above using built in recovery command prompt
Boot logging did not help, did not seem to drop the log file.
No obvious memory dmp files.
Then tried a restored using Symantec SSR boot CD restoring C: and System State from backup few days before any updates (19/9).
Still the same.
I cloned VM to new one and moved to other host
Not surprisingly the same...
Restored latest backup from SSR using boot CD - system state and C: drive. Same
Restored oldest backup from SSR using boot CD. Same
Today: Built fresh new empty VM
Restored in full newest backup before reboot - same
So... any suggestions I might be missing?
I have two other options open to me I think:
1. Use recovery at the DR site and see if that boots and bring back to main site.
2. Build new OS, installs apps and data....
I've not had to recover from a VMware replication before as previously used Storage based replication and for small companies without a SAN Hyper-V replication methods. It looks like once I do a recovery the option of going back to recover again will be gone. I am not convinced the recovered machine will be any different though will have a few snapshots to go back through trying.