Windows OS
--
Questions
--
Followers
Top Experts
Windows 2012 R2 - inaccessible boot device
Got a bit of a problem with a Windows 2012 R2 box and would appreciate any suggestions. it isn't a massively vital system (internal instant messaging server) and I sill have a few other potential restore options before rebuilding but I wonder whether I have missed something blindingly obvious and would like to see if can solve issue, apart from anything else before rebooting the other 8 or so VM!
This is running on VMware ESXi with two hosts running to a iSCSI SAN. There is also a DR site with VM replication setup
This started at the weekend with a routine reboot. The server hadn't been rebooted for a few months, possibly 4-5 months and it had a small number of updates + a load of optional ones.
I didn't create a VM snapshot before it rebooted, apart from anything else it had just backed up and replicated to DR site and was routine updates.
But... it disappeared, checking the VM console it was in a cycle of failed boot and automatic repair. It shows "inaccessible boot device" but just that, no code like I would normally expect to give a hint.
At that point I took a snapshot which I reverted to at various points before trying different repairs and restore.
Safe mode, command prompt, last known good etc. all failed the same of course
The command prompt repair had an issue in that the local admin password for this box did not match documentation and would not work.
So after a boot off Windows 2012 R2 DVD. Login to command prompt that way.
Carried out checks with command line tools, Bcdedit, bootrec etc. All looks OK, partitions look OK etc. right drive active, BCD looks OK etc.
Looked with DISM etc. to see what updates were put on. = approx. 4 .NET framework updates. Multiple other updates pending but not yet installed
kb2894856 = net framework 4.5
KB3010788 = OLE on Windows 7?
KB3032663 = net framework 4.5.2
KB3037579 = net framework 4.5.2
kb3045717 = Narrator
KB3094486 = KDS
Pending updates - tried clearing using online suggestions removing pending XML files, and registry entries
Reset local admin password using Linux NTPASSWD boot disc. It advised it thought password might already be blank (I later tried blank password which did not work either)
So then command prompt repairs same as above using built in recovery command prompt
Boot logging did not help, did not seem to drop the log file.
No obvious memory dmp files.
Then tried a restored using Symantec SSR boot CD restoring C: and System State from backup few days before any updates (19/9).
Still the same.
I cloned VM to new one and moved to other host
Not surprisingly the same...
Restored latest backup from SSR using boot CD - system state and C: drive. Same
Restored oldest backup from SSR using boot CD. Same
Today: Built fresh new empty VM
Restored in full newest backup before reboot - same
So... any suggestions I might be missing?
I have two other options open to me I think:
1. Use recovery at the DR site and see if that boots and bring back to main site.
2. Build new OS, installs apps and data....
I've not had to recover from a VMware replication before as previously used Storage based replication and for small companies without a SAN Hyper-V replication methods. It looks like once I do a recovery the option of going back to recover again will be gone. I am not convinced the recovered machine will be any different though will have a few snapshots to go back through trying.
thanks
Steve
This is running on VMware ESXi with two hosts running to a iSCSI SAN. There is also a DR site with VM replication setup
This started at the weekend with a routine reboot. The server hadn't been rebooted for a few months, possibly 4-5 months and it had a small number of updates + a load of optional ones.
I didn't create a VM snapshot before it rebooted, apart from anything else it had just backed up and replicated to DR site and was routine updates.
But... it disappeared, checking the VM console it was in a cycle of failed boot and automatic repair. It shows "inaccessible boot device" but just that, no code like I would normally expect to give a hint.
At that point I took a snapshot which I reverted to at various points before trying different repairs and restore.
Safe mode, command prompt, last known good etc. all failed the same of course
The command prompt repair had an issue in that the local admin password for this box did not match documentation and would not work.
So after a boot off Windows 2012 R2 DVD. Login to command prompt that way.
Carried out checks with command line tools, Bcdedit, bootrec etc. All looks OK, partitions look OK etc. right drive active, BCD looks OK etc.
Looked with DISM etc. to see what updates were put on. = approx. 4 .NET framework updates. Multiple other updates pending but not yet installed
kb2894856 = net framework 4.5
KB3010788 = OLE on Windows 7?
KB3032663 = net framework 4.5.2
KB3037579 = net framework 4.5.2
kb3045717 = Narrator
KB3094486 = KDS
Pending updates - tried clearing using online suggestions removing pending XML files, and registry entries
Reset local admin password using Linux NTPASSWD boot disc. It advised it thought password might already be blank (I later tried blank password which did not work either)
So then command prompt repairs same as above using built in recovery command prompt
Boot logging did not help, did not seem to drop the log file.
No obvious memory dmp files.
Then tried a restored using Symantec SSR boot CD restoring C: and System State from backup few days before any updates (19/9).
Still the same.
I cloned VM to new one and moved to other host
Not surprisingly the same...
Restored latest backup from SSR using boot CD - system state and C: drive. Same
Restored oldest backup from SSR using boot CD. Same
Today: Built fresh new empty VM
Restored in full newest backup before reboot - same
So... any suggestions I might be missing?
I have two other options open to me I think:
1. Use recovery at the DR site and see if that boots and bring back to main site.
2. Build new OS, installs apps and data....
I've not had to recover from a VMware replication before as previously used Storage based replication and for small companies without a SAN Hyper-V replication methods. It looks like once I do a recovery the option of going back to recover again will be gone. I am not convinced the recovered machine will be any different though will have a few snapshots to go back through trying.
thanks
Steve
Zero AI Policy
We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.
If the partitions and data are all there when the VM gets booted via .ISO file, then one place to look is in the settings for the VM in VMware. Has anything changed there? Is the boot order still correct and Drive 0 still the OS VMDK file?
Also, you could try to rebuild the boot configuration database in-guest. So, boot a Windows installer .ISO file and at CMD:
bootrec /rebuildbcdASKER CERTIFIED SOLUTION
membership
Log in or create a free account to see answer.
Signing up is free and takes 30 seconds. No credit card required.
Thank you both. I have tried the bootrec command already at some point though I can soon try it again.
I think I am write in thinking MBR must be right because we've got past that, Boot record should be OK as we have hit the Windows logo and spinner and must be at the Kernel stage which made me think logically would be storage drivers etc.
I've had ones before with storage controller issues but would have seen a code, we just get this:
Yeah backups... they are carried out using Symantec system recovery 2013 from within the VM to images. I've got daily images going back 2 weeks and restoring those over the VM or into a fresh VM with same virtual hardware and get the same boot issue.
I wouldn't normally be too worried about an isolated machine but have 8 other 2012 R2 servers built by the same people that I inherited on the same VM infrastructure, same AV etc. which I am not keen on rebooting just at the mo!
My next option then might be the replications to DR site using VM replication appliance but not a procedure I have done before myself, looks easy enough but I've not seen a document yet identifying best practice - i.e. whether I can preserve the state of the replication, copy it and recover that or whether the only option is to do a "recover" and then hope that boots and I presume clone while on the snapshot that want to keep into a fresh VM.
steve
I think I am write in thinking MBR must be right because we've got past that, Boot record should be OK as we have hit the Windows logo and spinner and must be at the Kernel stage which made me think logically would be storage drivers etc.
I've had ones before with storage controller issues but would have seen a code, we just get this:
Yeah backups... they are carried out using Symantec system recovery 2013 from within the VM to images. I've got daily images going back 2 weeks and restoring those over the VM or into a fresh VM with same virtual hardware and get the same boot issue.
I wouldn't normally be too worried about an isolated machine but have 8 other 2012 R2 servers built by the same people that I inherited on the same VM infrastructure, same AV etc. which I am not keen on rebooting just at the mo!
My next option then might be the replications to DR site using VM replication appliance but not a procedure I have done before myself, looks easy enough but I've not seen a document yet identifying best practice - i.e. whether I can preserve the state of the replication, copy it and recover that or whether the only option is to do a "recover" and then hope that boots and I presume clone while on the snapshot that want to keep into a fresh VM.
steve
EARN REWARDS FOR ASKING, ANSWERING, AND MORE.
Earn free swag for participating on the platform.
Sorry crossed over there I think. As to the backups.... the backups have not reported any errors and I have tested in terms of restoring files from every VM and proved that works fine (and still do) and the disks and partitions all show as restored OK to a new VM too which behaves the same as the original so I suspect backup is working but that the OS had an issue that was going to crop up after next reboot and nothing to do with (those) updates and is probably on all the backups too.
Would appreciate any advice you could give please on bringing back using the DR replica, though gut feeling is going to end up with the same issue it may well be a case of build a new OS, apps and restore the data back (which also appears OK).
thanks
Would appreciate any advice you could give please on bringing back using the DR replica, though gut feeling is going to end up with the same issue it may well be a case of build a new OS, apps and restore the data back (which also appears OK).
thanks
SOLUTION
membership
Log in or create a free account to see answer.
Signing up is free and takes 30 seconds. No credit card required.
I know. That is what I have inherited, on the (lengthy) list to change within the time I have available to work on these systems. Yes it is VMware vSphere replication, with a replication appliance machine running.
I paused the replication as soon as I knew there was an issue so we have 7 earlier replications:
Not an area I have used like I say for recovery but from what i read if I set off recovery I'll end up with a VM with multiple snapshots. I assume then best would be to clone the machine to try a boot and with NIC offline.
Steve
I paused the replication as soon as I knew there was an issue so we have 7 earlier replications:
Not an area I have used like I say for recovery but from what i read if I set off recovery I'll end up with a VM with multiple snapshots. I assume then best would be to clone the machine to try a boot and with NIC offline.
Steve
Okay so I assume you don’t know how to restore the earliest replica
It’s good replication has been paused because based on config the earliest replicas can just get erased
It’s good replication has been paused because based on config the earliest replicas can just get erased
Get a FREE t-shirt when you ask your first question.
We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.
Yes with only few days worth I thought would be best. Will look at Recovery tomorrow I think.
Steve
Steve
I've got time there on Tuesday and from what I can see do it as:
1. recover the most recent entry from DR site. That should leave me with a VM with previous replications as snapshots
2. copy the VM
3. Boot that up on DR site. If it does boot, do test reboots, add the missing updates and boot again etc. and try and break it!
4. if not revert to earlier snapshot and try the same again.
If/when it works copy the VM back to main site, clean up and set replication back up.
Does that sound sensible, I suppose step 2 isn't actually necessary but feels better working on a copy.
Steve
1. recover the most recent entry from DR site. That should leave me with a VM with previous replications as snapshots
2. copy the VM
3. Boot that up on DR site. If it does boot, do test reboots, add the missing updates and boot again etc. and try and break it!
4. if not revert to earlier snapshot and try the same again.
If/when it works copy the VM back to main site, clean up and set replication back up.
Does that sound sensible, I suppose step 2 isn't actually necessary but feels better working on a copy.
Steve
Yes, that's correct, but I would restore the oldest replica.
EARN REWARDS FOR ASKING, ANSWERING, AND MORE.
Earn free swag for participating on the platform.
Thank you. Given up now, the recovery process worked perfectly but all 6 replicas held the same issues as did 2 weeks worth of backups. I restored other VM from scratch using same procedure and it was fine so clearly that OS broke itself weeks or months ago.
Time for new OS, app installs and recover data from the backups...
Time for new OS, app installs and recover data from the backups...
Windows OS
--
Questions
--
Followers
Top Experts
This topic area includes legacy versions of Windows prior to Windows 2000: Windows 3/3.1, Windows 95 and Windows 98, plus any other Windows-related versions including Windows Mobile.