Solved

VDP Snapshots left after powering down VDP

Posted on 2014-07-19
3
1,755 Views
Last Modified: 2014-07-19
We have a VMware 5.1 cluster with VDP backup.
I was storage vmotioning several vm's the other day and the VDP got stupid and was running for more than 12 hours.  Users were complaining about the slow down so I forcibly shut down the VDP servers. (They were taking wayyyyy to long to shut down)  Now I have vdp snapshots left with my VMs.  I tried to vstorage motion one vm and it hung at 81%.  After a rather long process of removing the vm and recreating and pointing to the vmdk it is back up but do not want to go thru the process for 6 more VMs.
If I power up VDP will it clean up?
Is there any other way to clean them up? (is consolidate safe?)

Any help is appreciated...  very stressed out right now.
Thanks
0
Comment
Question by:brian_appliedcpu
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
3 Comments
 
LVL 120

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 40206329
You are not really, going to like what I about to tell you, but ALL backup applications which backup VMs, even VMware, and Veeam etc, will leave VMs running on snapshots after backup which is *BAD*.

It's a part of VMware Administration you need to get used to, and check every day after backup, they are not running on a snapshot, and it's been like its forever, since VMware invented and released ESX 1.0 (I've been there with them since then!)

So either check manually, or set a vCenter Alarm, to warn you of Snapshots, or run scripts etc

When a VM runs on a snapshot performance will be poor, and at the next backup, the situation gets worse.....

VDP WILL NOT solve this on restart you will need to complete some admin....

Have a read of my EE Article

HOW TO: VMware Snapshots :- Be Patient

Sometimes, you will not see any snapshots listed in Snapshot Manager.

How to solve the issue....

1. Take a new snapshot.
2. Wait 60 seconds
3. Select Delete ALL and BE PATIENT!

Snapshot merging/deletion can take, seconds, minutes, hours, days and weeks, DO NOT be tempted to stop/cancel the operation when started, fiddle with vCenter, stop vCenter Server, stop services, fiddle with Hosts etc - you will corrupt the virtual disk!

Walk away, get a coffee, do not stare at the screen, it's likely to sit at 90, 95 or 99% for hours, and look hung, but it will complete.

this depends on how large the snapshot and fast your storage is...

So before you continue, read my EE Article Above, I feel your pain, screenshot datastore folders, and lets see what the Snapshot sizes are....

and Welcome to Snapshot Hell - snapshots are evil...

Andy

and when Merging a Snapshot performance of the VM gets worse, so if you can turn OFF the VM, the better!
0
 
LVL 2

Author Closing Comment

by:brian_appliedcpu
ID: 40206658
Thank you!
Shut them down one at a time and followed your advice.
They consolidated at a rate of around 1GB / Min so the 10GB snapshots went fast.
Had one VM that shut down but VCenter thought it was still up.... hung.... I hate my job.
So even trying to do an esxcli shutdown with hard and force did nothing
esxcli vms vm kill --type=[soft,hard,force] --world-id=WorldNumber
Had to restart the host, then it came up as invalid
Had to remove from the vcenter and re-add.  During that process the vmx file icon was not valid so it did not want to add... refresh and it changed.... The Lord is good, it was a huge exchange server.
Added it back, then booted fine, then down again, do the snapshot dance per your instructions and take it back up.  Store mounted fine and on to the next.
Did I say I hate my job....
Thank you for your assistance.
I am still curious why it hung on shutdown and why when I vstorage motioned a vm that had a snapshot that it hung on 81% for 12+ hours, if you think you can answer them I can start a new question.
Thanks again you were a life saver.
Brian
0
 
LVL 120
ID: 40206738
Thanks for your kinds words...

1. VDP does take an age to shutdown, and sometimes it remains unresponsive and hangs! (still in latest version!).

2. I would *never* be confident in Storage vMotioning a VM with a snapshot! Deal with the snapshot first!
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
vm size discrepency 2 48
Internal CA server 6 91
VMotion Direct Attached Storage 9 92
vSphere VM Sysprepped + deleted a snapshot 3 62
When converting a physical machine to a virtual machine using VMware vCenter Converter Standalone or vCenter Converter Enterprise, if an adapter type is not selected during the initial customization the resulting virtual machine may contain an IDE d…
In this article, I will show you HOW TO: Suppress Configuration Issues and Warnings Alert displayed in Summary status for ESXi 6.5 after enabling SSH or ESXi Shell.
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …
Teach the user how to use create log bundles for vCenter Server or ESXi hosts Open vSphere Web Client: Generate vCenter Server and ESXi host log bundle:  Open vCenter Server Appliance Web Management interface and generate log bundle: Open vCenter Se…

756 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question