Solved

VDP Snapshots left after powering down VDP

Posted on 2014-07-19
3
1,612 Views
Last Modified: 2014-07-19
We have a VMware 5.1 cluster with VDP backup.
I was storage vmotioning several vm's the other day and the VDP got stupid and was running for more than 12 hours.  Users were complaining about the slow down so I forcibly shut down the VDP servers. (They were taking wayyyyy to long to shut down)  Now I have vdp snapshots left with my VMs.  I tried to vstorage motion one vm and it hung at 81%.  After a rather long process of removing the vm and recreating and pointing to the vmdk it is back up but do not want to go thru the process for 6 more VMs.
If I power up VDP will it clean up?
Is there any other way to clean them up? (is consolidate safe?)

Any help is appreciated...  very stressed out right now.
Thanks
0
Comment
Question by:brian_appliedcpu
  • 2
3 Comments
 
LVL 118

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE) earned 500 total points
ID: 40206329
You are not really, going to like what I about to tell you, but ALL backup applications which backup VMs, even VMware, and Veeam etc, will leave VMs running on snapshots after backup which is *BAD*.

It's a part of VMware Administration you need to get used to, and check every day after backup, they are not running on a snapshot, and it's been like its forever, since VMware invented and released ESX 1.0 (I've been there with them since then!)

So either check manually, or set a vCenter Alarm, to warn you of Snapshots, or run scripts etc

When a VM runs on a snapshot performance will be poor, and at the next backup, the situation gets worse.....

VDP WILL NOT solve this on restart you will need to complete some admin....

Have a read of my EE Article

HOW TO: VMware Snapshots :- Be Patient

Sometimes, you will not see any snapshots listed in Snapshot Manager.

How to solve the issue....

1. Take a new snapshot.
2. Wait 60 seconds
3. Select Delete ALL and BE PATIENT!

Snapshot merging/deletion can take, seconds, minutes, hours, days and weeks, DO NOT be tempted to stop/cancel the operation when started, fiddle with vCenter, stop vCenter Server, stop services, fiddle with Hosts etc - you will corrupt the virtual disk!

Walk away, get a coffee, do not stare at the screen, it's likely to sit at 90, 95 or 99% for hours, and look hung, but it will complete.

this depends on how large the snapshot and fast your storage is...

So before you continue, read my EE Article Above, I feel your pain, screenshot datastore folders, and lets see what the Snapshot sizes are....

and Welcome to Snapshot Hell - snapshots are evil...

Andy

and when Merging a Snapshot performance of the VM gets worse, so if you can turn OFF the VM, the better!
0
 
LVL 2

Author Closing Comment

by:brian_appliedcpu
ID: 40206658
Thank you!
Shut them down one at a time and followed your advice.
They consolidated at a rate of around 1GB / Min so the 10GB snapshots went fast.
Had one VM that shut down but VCenter thought it was still up.... hung.... I hate my job.
So even trying to do an esxcli shutdown with hard and force did nothing
esxcli vms vm kill --type=[soft,hard,force] --world-id=WorldNumber
Had to restart the host, then it came up as invalid
Had to remove from the vcenter and re-add.  During that process the vmx file icon was not valid so it did not want to add... refresh and it changed.... The Lord is good, it was a huge exchange server.
Added it back, then booted fine, then down again, do the snapshot dance per your instructions and take it back up.  Store mounted fine and on to the next.
Did I say I hate my job....
Thank you for your assistance.
I am still curious why it hung on shutdown and why when I vstorage motioned a vm that had a snapshot that it hung on 81% for 12+ hours, if you think you can answer them I can start a new question.
Thanks again you were a life saver.
Brian
0
 
LVL 118
ID: 40206738
Thanks for your kinds words...

1. VDP does take an age to shutdown, and sometimes it remains unresponsive and hangs! (still in latest version!).

2. I would *never* be confident in Storage vMotioning a VM with a snapshot! Deal with the snapshot first!
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

This is an issue that we can get adding / removing permissions in the vCSA 6.0. We can also have issues searching for users / groups in the AD (using your identify sources). This is how one of the ways to handle this issues and fix it.
In this article, I will show you HOW TO: Install VMware Tools for Windows on a VMware Windows virtual machine on a VMware vSphere Hypervisor 6.5 (ESXi 6.5) Host Server, using the VMware Host Client. The virtual machine has Windows Server 2016 instal…
Teach the user how to join ESXi hosts to Active Directory domains Open vSphere Client: Join ESXi host to AD domain: Verify ESXi computer account in AD: Configure permissions for domain user in ESXi: Test domain user login to ESXi host:
This tutorial will walk an individual through the steps necessary to enable the VMware\Hyper-V licensed feature of Backup Exec 2012. In addition, how to add a VMware server and configure a backup job. The first step is to acquire the necessary licen…

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

27 Experts available now in Live!

Get 1:1 Help Now