Solved

VDP Snapshots left after powering down VDP

Posted on 2014-07-19
3
1,549 Views
Last Modified: 2014-07-19
We have a VMware 5.1 cluster with VDP backup.
I was storage vmotioning several vm's the other day and the VDP got stupid and was running for more than 12 hours.  Users were complaining about the slow down so I forcibly shut down the VDP servers. (They were taking wayyyyy to long to shut down)  Now I have vdp snapshots left with my VMs.  I tried to vstorage motion one vm and it hung at 81%.  After a rather long process of removing the vm and recreating and pointing to the vmdk it is back up but do not want to go thru the process for 6 more VMs.
If I power up VDP will it clean up?
Is there any other way to clean them up? (is consolidate safe?)

Any help is appreciated...  very stressed out right now.
Thanks
0
Comment
Question by:brian_appliedcpu
  • 2
3 Comments
 
LVL 117

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE) earned 500 total points
Comment Utility
You are not really, going to like what I about to tell you, but ALL backup applications which backup VMs, even VMware, and Veeam etc, will leave VMs running on snapshots after backup which is *BAD*.

It's a part of VMware Administration you need to get used to, and check every day after backup, they are not running on a snapshot, and it's been like its forever, since VMware invented and released ESX 1.0 (I've been there with them since then!)

So either check manually, or set a vCenter Alarm, to warn you of Snapshots, or run scripts etc

When a VM runs on a snapshot performance will be poor, and at the next backup, the situation gets worse.....

VDP WILL NOT solve this on restart you will need to complete some admin....

Have a read of my EE Article

HOW TO: VMware Snapshots :- Be Patient

Sometimes, you will not see any snapshots listed in Snapshot Manager.

How to solve the issue....

1. Take a new snapshot.
2. Wait 60 seconds
3. Select Delete ALL and BE PATIENT!

Snapshot merging/deletion can take, seconds, minutes, hours, days and weeks, DO NOT be tempted to stop/cancel the operation when started, fiddle with vCenter, stop vCenter Server, stop services, fiddle with Hosts etc - you will corrupt the virtual disk!

Walk away, get a coffee, do not stare at the screen, it's likely to sit at 90, 95 or 99% for hours, and look hung, but it will complete.

this depends on how large the snapshot and fast your storage is...

So before you continue, read my EE Article Above, I feel your pain, screenshot datastore folders, and lets see what the Snapshot sizes are....

and Welcome to Snapshot Hell - snapshots are evil...

Andy

and when Merging a Snapshot performance of the VM gets worse, so if you can turn OFF the VM, the better!
0
 
LVL 2

Author Closing Comment

by:brian_appliedcpu
Comment Utility
Thank you!
Shut them down one at a time and followed your advice.
They consolidated at a rate of around 1GB / Min so the 10GB snapshots went fast.
Had one VM that shut down but VCenter thought it was still up.... hung.... I hate my job.
So even trying to do an esxcli shutdown with hard and force did nothing
esxcli vms vm kill --type=[soft,hard,force] --world-id=WorldNumber
Had to restart the host, then it came up as invalid
Had to remove from the vcenter and re-add.  During that process the vmx file icon was not valid so it did not want to add... refresh and it changed.... The Lord is good, it was a huge exchange server.
Added it back, then booted fine, then down again, do the snapshot dance per your instructions and take it back up.  Store mounted fine and on to the next.
Did I say I hate my job....
Thank you for your assistance.
I am still curious why it hung on shutdown and why when I vstorage motioned a vm that had a snapshot that it hung on 81% for 12+ hours, if you think you can answer them I can start a new question.
Thanks again you were a life saver.
Brian
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
Thanks for your kinds words...

1. VDP does take an age to shutdown, and sometimes it remains unresponsive and hangs! (still in latest version!).

2. I would *never* be confident in Storage vMotioning a VM with a snapshot! Deal with the snapshot first!
0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

Suggested Solutions

Create your own, high-performance VM backup appliance by installing NAKIVO Backup & Replication directly onto a Synology NAS!
Exchange server is not supported in any cloud-hosted platform (other than Azure with Azure Premium Storage).
Teach the user how to use configure the vCenter Server storage filters Open vSphere Web Client:  Navigate to vCenter Server Advanced Settings: Add the four vCenter Server storage filters: Review the advanced settings: Modify the values of the four v…
This video shows you how easy it is to boot from ISO images for virtual machines with the ISO images stored on a local datastore on the ESXi host.

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now