ms 2016 server stuck in merge

nigelbeatson
nigelbeatson used Ask the Experts™
on
we have an ongoing issue with cloudcare where we cannot log on to our vm servers.
we have 2 vms, 1 our 2016 dc and the other our 2016 exchange server.

once we get into this “lock out” situation we try closing down the servers but they just hang, so the only way we have been able to resolve this is to hard reset the server.

we have just done this again and now both vms are stuck starting up, where the status shows "merge in progress", but the progress is just sat at 26% for our dc and 7% for our exchange server.

we really need these servers back online, so does anyone know how we can tidy this up and get them operational?

any help much appreciated - desperate!!

thanks
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®

Author

Commented:
i forgot to mention we do have a checkpoint for both vm's and wondered if i applied them, we could get out of this startup / merge situation?

they are both 3 weeks old, but having never carried this out before, wanted to check whether this would help or not and make sure it only changed the os and not the data?

how about exchange? would we still keep the current data in that too?

any help appreciated.

thanks
Patrick BogersDatacenter platform engineer Lindows

Commented:
Hi,

It sounds like the checkpoint is being merged and since you say it is three weeks old this can take serious time if it is a high load server. IMHO you have to wait.
Advise, never keep checkpoints this long, they are no means as a backup.

Cheers.

Author

Commented:
we took this when we started experiencing the problem.

our problem is that the servers are NOT starting. they are in startup mode, ie we get the windows logo and the progress bar, but the servers just stay that way.

we need these back now!

help!
Become a CompTIA Certified Healthcare IT Tech

This course will help prep you to earn the CompTIA Healthcare IT Technician certification showing that you have the knowledge and skills needed to succeed in installing, managing, and troubleshooting IT systems in medical and clinical settings.

Author

Commented:
why would the servers just decide to merge an ancient checkpoint? i never asked them to??
Patrick BogersDatacenter platform engineer Lindows

Commented:
Hi,

Best practive is to let hyper-v deal with it but if you are sure it cannot you can shutndown the vms and try powershell
Get-VMSnapshot -VMName | Remove-VMSnapshot

Cheers

Edit: keeping snapshots that long is a very good recipy for data corruption as you are finding out now. This automerge, as it seems, sound like Hyper-V has decided it has no use for the snapshot so it merges it. Merge does not mean it is rolling back the vm, it just means the current instance is becoming the soul file for the VM.

Author

Commented:
unfortunately, im not sure of anything anymore! all i can say is that both servers are just sat starting up and the status in the hyperv manager shows the merge taking place. not sure why they are stuck at the same progress point.

i have tried running powershell on the hyperv host but although the cmd window opens, we just get a blinking cursor!

should i call microsoft?
Patrick BogersDatacenter platform engineer Lindows

Commented:
If you have support that would be a good option yes.

Author

Commented:
we have to pay! so if you think they would be the best people to resolve this, i will have to do it.

i cant (for some reason) get powershell to start all the way, we just get a blinking cursor!
Datacenter platform engineer Lindows
Commented:
Thats very strange, sounds like your host is in distress for resources. I cannot imagine but let me ask anyway, there are no AV or antimalware apps on the hyperv host right? Those could lock up the vm files.

Btw, Have you tried rebooting the host completely?

I would call cloudcare and ask if they know why this server is behaving poorly.
Philip ElderTechnical Architect - HA/Compute/Storage

Commented:
Click start on the host and type: ResMon.exe [CTRL+SHFT+ENTER] --> UAC --> Disk tab --> Disk Activity dropdown --> Sort largest Total (B/Sec)

Snip that and post please. The disk subsystem is probably maxed while the merges are running.

Please be patient.

And, it's better to use backup as a fallback than snapshots/checkpoints as the latter soak up free disk space as they grow!

Author

Commented:
Thanks Patrick, yes we do have anti virus and malware on our host, but we did exclude key folders and files. Are you suggesting that we should not run this on the base host?

I did reboot the host, but it ended up being the big hard reset. The problem was that when we get locked out, when we try close down, it hangs when closing down, when we select reset, it hangs on resetting. The only option I was aware of was to power off the server. This is when we end up with the merging problems.

I did eventually reset (power off the server) as you suggested and it did then restart and the merge completed in a timely fashion. I have now deleted the checkpoints, and was frustrated to see it set about yet another merge!! Aaaggh! This is now completed and we have restarted all fully, and all seems to be well.

We do run a full system backup for each vm, but due to the problems with the server going down constantly, we only have one that is 5 days old. We have reported this to cloudcare nearly a month ago, and I am STILL waiting for any feedback from their development team. In the meantime, I have the choice of running the server without protection, or run protection and have to reset the server constantly! So frustrating!!

Many thanks to all.

Author

Commented:
Many thanks to all who tried to assist.
Philip ElderTechnical Architect - HA/Compute/Storage

Commented:
What's "CloudCare" please? There are _lots_ of results that come up.
Patrick BogersDatacenter platform engineer Lindows

Commented:
Hi again,

Glad to hear you are back up and running and yes, if the hyperV host is not serving frontend requests so basically is not reachable from the outside i would leave AV off.
Protection on the vms themselfes is another story.

Cheers!

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial