VMWare Snapshot Consolidation after failed snapshot removal?
I had a VM server stop responding during work hours. I was unable to reset the VM due to an incomplete snapshot removal initiated by Veeam. Ultimately, we stopped that removal, rebooted the server (it wouldn't power on still even after the stopping, so I had to reboot), and got the VM up and running.
Now, I have two Veeam temporary snapshots that are showing up. It seems that one of the drives has 2 delta disks, but the other three drives have just the one. The 2 disks is likely due to the interrupted snapshot.
Veeam suggested Consolidation, which I agree with, especially since it's a database server and changes have been made, so I wouldn't want data loss. My vSphere client though is showing a "No" in the "Needs consolidation" column. I have never done a consolidation, so I'm not sure what to expect. Is this my best (only) option? Will it even run if "needs consolidation" says no? Any concerns I should have (I'll obviously do a backup beforehand.)
Thanks.
Now, I have two Veeam temporary snapshots that are showing up. It seems that one of the drives has 2 delta disks, but the other three drives have just the one. The 2 disks is likely due to the interrupted snapshot.
Veeam suggested Consolidation, which I agree with, especially since it's a database server and changes have been made, so I wouldn't want data loss. My vSphere client though is showing a "No" in the "Needs consolidation" column. I have never done a consolidation, so I'm not sure what to expect. Is this my best (only) option? Will it even run if "needs consolidation" says no? Any concerns I should have (I'll obviously do a backup beforehand.)
Thanks.
Do more with
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
What you must do if the VM is running on a snapshot is as follows
1. Take a new snapshot
2. Wait 120 seconds
3. Select Delete All
Wait and be patient
What you also want to check, is if Veeam Backup and Replication still has the parent disk attached, otherwise you will not be able to delete/merge the snapshot.
Also have a read of my supporting article about snapshots here on EE.
HOW TO: VMware Snapshots :- Be Patient
1. Take a new snapshot
2. Wait 120 seconds
3. Select Delete All
Wait and be patient
What you also want to check, is if Veeam Backup and Replication still has the parent disk attached, otherwise you will not be able to delete/merge the snapshot.
Also have a read of my supporting article about snapshots here on EE.
HOW TO: VMware Snapshots :- Be Patient
Do not use the consolidate option, the consolidation option is dumb! Please follow my post "as is".
If you are still unsure of the details of my post please post and ask questions.
If you are still unsure of the details of my post please post and ask questions.
Andrew, the Veeam tech was worried that the original snapshot "hanging" would cause issue when I try to delete. However I have been able to backup this VM since then, so everything should be intact in Veeam.
I am currently planning to try your suggestion tomorrow since we're closed on the 25th giving it more time to run. I'm assuming the new snapshot helps ensure it's in a more consistent state if the original hanging one does have some issues?
Thanks.
I am currently planning to try your suggestion tomorrow since we're closed on the 25th giving it more time to run. I'm assuming the new snapshot helps ensure it's in a more consistent state if the original hanging one does have some issues?
Thanks.
That is *always* a problem with the removal of snapshots, because they can provide high I/O usage on the datastore.
Snapshots can seem to hang, but they will delete, you just need to be very patient, and do not mess, restart VM, shutdown VM, stop and restart the host, restart vcenter server etc.
Please check that Veeam backup server does not currently have the parent vmdk disk of the VM attached. if so make sure you remove that disk from it's inventory.
Which could indicate your datastore is too slow, and under performing for the virtual machines hosted on the datastore.
The new snapshots ensures the chain is corrected and intact.
You need to ensure after every backup you check VMs are not running on a snapshot, this has always been a problem since snapshots were invented!
Use scripts, alarms to check daily, or as part of your daly VMware Admin manual check.
Snapshots can seem to hang, but they will delete, you just need to be very patient, and do not mess, restart VM, shutdown VM, stop and restart the host, restart vcenter server etc.
Please check that Veeam backup server does not currently have the parent vmdk disk of the VM attached. if so make sure you remove that disk from it's inventory.
Which could indicate your datastore is too slow, and under performing for the virtual machines hosted on the datastore.
The new snapshots ensures the chain is corrected and intact.
You need to ensure after every backup you check VMs are not running on a snapshot, this has always been a problem since snapshots were invented!
Use scripts, alarms to check daily, or as part of your daly VMware Admin manual check.
Thanks. Just to confirm, "Please check that Veeam backup server does not currently have the parent vmdk disk of the VM attached. " means to just check if it's still an attached drive in the Windows OS and not something to check in Veeam itself?
Check the Veeam VM (if you have a Veeam VM) and check the Edit Settings no additional disk of the parent VM is currently attached via hot-add function!
So after all that, I created the new snapshot and the Delete All surprisingly only took 15 minutes to complete. Everything seems to be working ok so far. However, this bring up the related question - I had set up an hourly Veeam backup during production hours in order to reduce the recovery point time for this database. However, now I'm not sure whether to re-enable this schedule, or make it every few hours, or just not run it during work hours, given what happened. Any thoughts on how best to proceed?
Thanks.
Thanks.
All VMware VM aware backup products use the snapshot method to backup VMs. (So I'm afraid this snapshot left after a backup, always needs to be checked)
Usually VMs are left running on snapshots because the datastore is too slow, or the VM has high I/O (activity), also the issue is if a VM runs on a snapshot performance is also affected whilst the VM is running on the snapshot for the period of the backup.
and at the end of the backup....the snapshot is then deleted/merged - this can result in the snapshot not being removed, and you have to di it manually.
You need to look at your environment, and make a decision, maybe Veeam is not the best product to backup the VM and database, and a backup of the VM and it's database once a day out of hours, at quiet times, e.g. 3am
and use SQL Backups every hour throughout the day to a drive on the SQL server. (or offline, NAS, Web, Cloud etc)
Usually VMs are left running on snapshots because the datastore is too slow, or the VM has high I/O (activity), also the issue is if a VM runs on a snapshot performance is also affected whilst the VM is running on the snapshot for the period of the backup.
and at the end of the backup....the snapshot is then deleted/merged - this can result in the snapshot not being removed, and you have to di it manually.
You need to look at your environment, and make a decision, maybe Veeam is not the best product to backup the VM and database, and a backup of the VM and it's database once a day out of hours, at quiet times, e.g. 3am
and use SQL Backups every hour throughout the day to a drive on the SQL server. (or offline, NAS, Web, Cloud etc)
Premium Content
You need an Expert Office subscription to comment.Start Free Trial