asked on
VMWare Snapshot Consolidation after failed snapshot removal?
I had a VM server stop responding during work hours. I was unable to reset the VM due to an incomplete snapshot removal initiated by Veeam. Ultimately, we stopped that removal, rebooted the server (it wouldn't power on still even after the stopping, so I had to reboot), and got the VM up and running.
Now, I have two Veeam temporary snapshots that are showing up. It seems that one of the drives has 2 delta disks, but the other three drives have just the one. The 2 disks is likely due to the interrupted snapshot.
Veeam suggested Consolidation, which I agree with, especially since it's a database server and changes have been made, so I wouldn't want data loss. My vSphere client though is showing a "No" in the "Needs consolidation" column. I have never done a consolidation, so I'm not sure what to expect. Is this my best (only) option? Will it even run if "needs consolidation" says no? Any concerns I should have (I'll obviously do a backup beforehand.)
Thanks.
Now, I have two Veeam temporary snapshots that are showing up. It seems that one of the drives has 2 delta disks, but the other three drives have just the one. The 2 disks is likely due to the interrupted snapshot.
Veeam suggested Consolidation, which I agree with, especially since it's a database server and changes have been made, so I wouldn't want data loss. My vSphere client though is showing a "No" in the "Needs consolidation" column. I have never done a consolidation, so I'm not sure what to expect. Is this my best (only) option? Will it even run if "needs consolidation" says no? Any concerns I should have (I'll obviously do a backup beforehand.)
Thanks.
VirtualizationVeeamVMware
Last Comment
Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)
ASKER CERTIFIED SOLUTION
THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
Yea, all you need to do is to right-click and then perform the consolidate on the VM snapshot.
Do not use the consolidate option, the consolidation option is dumb! Please follow my post "as is".
If you are still unsure of the details of my post please post and ask questions.
If you are still unsure of the details of my post please post and ask questions.
ASKER
Andrew, the Veeam tech was worried that the original snapshot "hanging" would cause issue when I try to delete. However I have been able to backup this VM since then, so everything should be intact in Veeam.
I am currently planning to try your suggestion tomorrow since we're closed on the 25th giving it more time to run. I'm assuming the new snapshot helps ensure it's in a more consistent state if the original hanging one does have some issues?
Thanks.
I am currently planning to try your suggestion tomorrow since we're closed on the 25th giving it more time to run. I'm assuming the new snapshot helps ensure it's in a more consistent state if the original hanging one does have some issues?
Thanks.
That is *always* a problem with the removal of snapshots, because they can provide high I/O usage on the datastore.
Snapshots can seem to hang, but they will delete, you just need to be very patient, and do not mess, restart VM, shutdown VM, stop and restart the host, restart vcenter server etc.
Please check that Veeam backup server does not currently have the parent vmdk disk of the VM attached. if so make sure you remove that disk from it's inventory.
Which could indicate your datastore is too slow, and under performing for the virtual machines hosted on the datastore.
The new snapshots ensures the chain is corrected and intact.
You need to ensure after every backup you check VMs are not running on a snapshot, this has always been a problem since snapshots were invented!
Use scripts, alarms to check daily, or as part of your daly VMware Admin manual check.
Snapshots can seem to hang, but they will delete, you just need to be very patient, and do not mess, restart VM, shutdown VM, stop and restart the host, restart vcenter server etc.
Please check that Veeam backup server does not currently have the parent vmdk disk of the VM attached. if so make sure you remove that disk from it's inventory.
Which could indicate your datastore is too slow, and under performing for the virtual machines hosted on the datastore.
The new snapshots ensures the chain is corrected and intact.
You need to ensure after every backup you check VMs are not running on a snapshot, this has always been a problem since snapshots were invented!
Use scripts, alarms to check daily, or as part of your daly VMware Admin manual check.
ASKER
Thanks. Just to confirm, "Please check that Veeam backup server does not currently have the parent vmdk disk of the VM attached. " means to just check if it's still an attached drive in the Windows OS and not something to check in Veeam itself?
Check the Veeam VM (if you have a Veeam VM) and check the Edit Settings no additional disk of the parent VM is currently attached via hot-add function!
ASKER
So after all that, I created the new snapshot and the Delete All surprisingly only took 15 minutes to complete. Everything seems to be working ok so far. However, this bring up the related question - I had set up an hourly Veeam backup during production hours in order to reduce the recovery point time for this database. However, now I'm not sure whether to re-enable this schedule, or make it every few hours, or just not run it during work hours, given what happened. Any thoughts on how best to proceed?
Thanks.
Thanks.
All VMware VM aware backup products use the snapshot method to backup VMs. (So I'm afraid this snapshot left after a backup, always needs to be checked)
Usually VMs are left running on snapshots because the datastore is too slow, or the VM has high I/O (activity), also the issue is if a VM runs on a snapshot performance is also affected whilst the VM is running on the snapshot for the period of the backup.
and at the end of the backup....the snapshot is then deleted/merged - this can result in the snapshot not being removed, and you have to di it manually.
You need to look at your environment, and make a decision, maybe Veeam is not the best product to backup the VM and database, and a backup of the VM and it's database once a day out of hours, at quiet times, e.g. 3am
and use SQL Backups every hour throughout the day to a drive on the SQL server. (or offline, NAS, Web, Cloud etc)
Usually VMs are left running on snapshots because the datastore is too slow, or the VM has high I/O (activity), also the issue is if a VM runs on a snapshot performance is also affected whilst the VM is running on the snapshot for the period of the backup.
and at the end of the backup....the snapshot is then deleted/merged - this can result in the snapshot not being removed, and you have to di it manually.
You need to look at your environment, and make a decision, maybe Veeam is not the best product to backup the VM and database, and a backup of the VM and it's database once a day out of hours, at quiet times, e.g. 3am
and use SQL Backups every hour throughout the day to a drive on the SQL server. (or offline, NAS, Web, Cloud etc)
What's the purpose of the consolidate function on the vSphere ?
I believe the last steps after closing the VM snapshot is to consolidate the VMDK using the right click menu option.
I believe the last steps after closing the VM snapshot is to consolidate the VMDK using the right click menu option.
ASKER
Thank you Andrew for your help.
no problems.