VMWare - Remove snapshot hung at 95%

I have a VMWare guest on a host running ESXi 4.1 that has been hung at Remove Snapshot (95%) for over 60 hours.  The snapshot creation and removal process was started by our backup software - Trilead VM Explorer.

Inspection of files in datastore show no file activity for more than 50 hours.

Guest OS is completely inaccessible.

Is there a way to stop the deletion process?
gtkdwDirector, ITAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Just been writing a blog about this....

this is my EE Article

HOW TO: VMware Snapshots :- Be Patient

If you fiddle, mess with the process, the virtual machine disk, could be corrupted.

Can you send me the following:-

1. disk type, datastore type, e.g. number or disks, SATA or SAS, RAID etc

2. screenshot of the datastore

3. size of snapshot, how many, how long has been running?

4. is the VM running? (ON)?

5. file time stamp changing?

other than kill all processes, possibly restart Host, there is no way to cancel or stop.
0
gtkdwDirector, ITAuthor Commented:
Thanks for your quick response.  

Should I be troubled by the fact that there has been no apparent file activity in over 48 hours - kb.vmware.com/kb/1007566.

If the deletion is progressing, shouldn't I see some activity?
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Can you send me the following:-

1. disk type, datastore type, e.g. number or disks, SATA or SAS, RAID etc

2. screenshot of the datastore

3. size of snapshot, how many, how long has been running?

4. is the VM running? (ON)?

5. file time stamp changing?

other than kill all processes, possibly restart Host, there is no way to cancel or stop.

it''s possible it's hung....but if I can have the above info, I can give you a more informed response.
0
The Ultimate Tool Kit for Technolgy Solution Provi

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy for valuable how-to assets including sample agreements, checklists, flowcharts, and more!

gtkdwDirector, ITAuthor Commented:
Disk type = 2 x 1tb SAS drives in Raid 1, internal to the host

Datastore screenshot
there appear to be 5 snapshot files, the oldest created about 10 days ago

VM is running, but not accessible in any way (ping, console, RDP)

File timestamps have note changed since Friday evening
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
The snapshot merge process, has hung.

Your 5 snapshots are small, and should have merged by now.  

A 250GB snapshot can merge on a two disk RAID 1 SAS in 33 hours.

there is a risk, that cancelling, killing the process, could corrupt the snapshots.

but you will have to restart the process.

first shutdown and stop the Backup Application.
0
gtkdwDirector, ITAuthor Commented:
Backup application is stopped.  How do I kill/restart the process?
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
1. Restart vCenter Server service. - if using vCenter Server - Step 1 applies.

2. Restart Network Management Agents.

3. If the process is truly hung, you will need to restart the Host Server.

you can also use the following at the console or remotely via ssh...

esxcli vm process list (obtain process lists)

soft, hard and force kills

esxcli vm process kill -t soft -w world-id
esxcli vm process kill -t hard -w world-id
esxcli vm process kill -t force -w world-id

doing the above, can cause other issues, and a restart is often better.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
gtkdwDirector, ITAuthor Commented:
Had to restart the Host, but everything came up roses.  Whew!!!!!

Do you have any recommendations about removing the snapshots manually?  I have attached an image of the current situation in Snapshot Manager.

Snapshot Manager
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
I would

1. first create a new snapshot.
2. wait 60 seconds.
3. check snapshots.
4. Delete ALL - and wait...
0
gtkdwDirector, ITAuthor Commented:
Thanks.  I'll wait until after business hours for that - just in case.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Network Architecture

From novice to tech pro — start learning today.