[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 3611
  • Last Modified:

Snapshot

This week suddenly two virtual machines on our shop stopped because of to little space on the hosts storage. This is because we didn't think of the extra amount of storage that is required when using snapshots on a Exchange server on this host. The host storage is 270GB. Only 200GB is predefined by the virtual machines, but snapshot files are living there own life's besides and are now nearly reaching 70GB,
Can we safely delete all snapshot's from the snapshot manager in ESXi 4.0, and is it correct that data on the snapshot files will be merged with the "original" storage upon deletions?

snapshotmanager.jpg
storage.jpg
0
elit2007
Asked:
elit2007
  • 8
  • 5
  • 2
  • +3
1 Solution
 
coolsport00Commented:
Yes, if you choose to "Delete All", your snapshots will transfer its data to its parent disk in the snap tree. Once the data is transferred, then that orig snap is deleted. This process continues up the "snap tree" until all data is 'commmited' and all snaps are deleted in the tree.

~coolsport00
0
 
coolsport00Commented:
I highly recommend reading these 2 articles on snaps:
Understanding Snaps - http://kb.vmware.com/kb/1015180
Snap Best Practices - http://kb.vmware.com/kb/1025279

BTW...with the amt of snaps you have, and the size of them all, don't expect this process to end anytime soon. It'll take a day...maybe longer.

~coolsport00
0
 
Paul SolovyovskyCommented:
The snapshots are part of the original vmdk and you will still need the room for them.  When a snapshot is taken the original vmdk is put in read mode and all new data is being written to the snapshot vmdk.  When you delete the snapshot it will write the snapshot to the original vmdk and will make it just as large as it is with snapshots and you will need more space during the process for the snapshots to write the data back to the original.  Snapshots are unlike SAN snapshots, they are a part of the VM.
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
elit2007Author Commented:
Do we need to shut the vm down first?
0
 
coolsport00Commented:
No
0
 
coolsport00Commented:
And do keep in mind what "pausolov" has added, and what I suggested in my orig post...you will need the space on your datastore equal to the size of all your snaps as the data in all the snaps will be 'commited' to the VMDK.

~coolsport00
0
 
elit2007Author Commented:
Are you sure paulolo?  If you take a look at the screenshot of the data browser you can see the two virtual disks on the top of the list (104GB and 52GB). They are set to be predefined/static.  They cannot grow, and they are just limits and not what actually is stored at the moment. Therefore I don’t understand why I can’t release some storage space when the snapshot files has been merged to the original virtual disk?
0
 
Danny McDanielClinical Systems AnalystCommented:
when you try to commit a snapshot while a VM is running, it will take another snapshot for the VM to run on while it consolidates the other snapshots.  So, if you don't have any space available or not very much, you will want to power down the VM to prevent another crash.
0
 
elit2007Author Commented:
Good idea :-) Only got 2GB left. So I'm I right about my last post?
0
 
coolsport00Commented:
You will need more than 2GB left to successfully commit your snaps. See this KB that discusses "consolidating" a bit more:
http://kb.vmware.com/kb/1007849

~coolsport00
0
 
coolsport00Commented:
You could clone your VM & that removes snapshots during that process, if you have enough space on another datastore to clone it to.

~coolsport00
0
 
Paul SolovyovskyCommented:
Either clone or take vmware converter and take another image of it inside the OS, this will just backup the used blockes, I've had issues cloning VMs that have snapshots.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Before you do any of the above are you using vDR? (VMware Data Recovery)
0
 
elit2007Author Commented:
Right now I'm moving the second VM to free up 50GB.
I'm using Veeam backup. But the backup job stopped working when the host ran out of storage. I think this is because Veeam backup don't get enough space for bacakup snapshot. See error message:
Production datastore 'Storage (tiger)' is getting low on free space (2,13 GB left), and may run out of free disk space completely due to open snapshots.

Removing snapshot
RemoveSnapshot failed, snapshotRef "64-snapshot-140", timeout "3600000"
The object has already been deleted or has not been completely created
Veeam Backup will attempt to remove snapshot during the next job cycle, but you may consider removing snapshot manually.
Possible causes for snapshot removal failure:
 - Network connectivity issue, or vCenter Server is too busy to serve the request
 - ESX host was unable to process snapshot removal request in a timely manner
 - Snapshot was already removed by another application
BackupDiskFull failed
Client error: Failed to open VDDK disk [[Storage (tiger)] Exchange/Exchange-000016.vmdk] ( is read-only mode - [true] )
Failed to open VMDK.
Logon attempt with parameters [VC/ESX: [10.1.254.181];Port: 443;Login: [root];VMX Spec: [moref=64];Snapshot mor: [64-snapshot-140];Transports: [san]] failed because of the following errors:
Failed to open VMDK.
Logon attempt with parameters [VC/ESX: [10.1.254.181];Port: 902;Login: [root];VMX Spec: [moref=64];Snapshot mor: [64-snapshot-140];Transports: [nbd]] failed because of the following errors:

Server error: End of file

Open in new window

0
 
coolsport00Commented:
Veeam does use snapshotting and yes...if there isn't enough space to create its own snap to b/u, it will fail.

~coolsport00
0
 
elit2007Author Commented:
Hello again, this is what I ended up with. First I ran vConverter inside the virtual machine and convertet it to another VM host. Then I shut down the virtual server and pushed "Delete all" in ESXi Snapshot Manager.
Then I went to sleep and when I cheked the server next morning all snapshots where gone from the Snapshot manager, but no snapshotfiles where deleted from the storage. The virtual machine was still running on the last snapshot disk. So I gave up this server and booted up the server that I used vConverter to build. And now it's running on that server. No more use of snapshots for me :0)
0
 
coolsport00Commented:
Yeah...snapshots are to be used solely as a temporary thing -- create a snap, push out an update to your VM, then after you confirm all is well (after a day or 2 at most) DELETE the snap. :) Both KBs I posted earlier explain it in a bit more detail. Glad you're good to go now...

Regards,
~coolsport00
0
 
noblehoboCommented:
Also note that if a VM is powered off, the snapshot consolidation process requires very little free space for functionality (about 16 MB).  

Note that if a VM is powered-off for snapshot consolidation (delete or delete-all), the VM must remain powered off until the consolidation is complete.  Changing the VM power state during snapshot consolidation can potentially break the process.  Conversely, if the VM is left powered-on, it should stay powered-on.  This is why ensuring enough space exists is paramount.

Powering off a VM can also free up space by deleting the VM swap file.  By default, if a VM does not have a memory reservation set, a swap file equal in size to the VM's RAM allocation is created at power on.  i.e. a VM with 8 GB RAM has an 8 GB swap file.  Powering off the VM deletes the 8 GB swap file and will free up that amount of space on the datastore.  This is often used when troubleshooting snapshot issues where the datastore has run out of space.

To confirm that a snapshot consolidation process is complete, command-line monitoring via an ESX host can be performed - see KB article 1007566 - http://kb.vmware.com/kb/1007566.

Kind regards,

-noblehobo
0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

  • 8
  • 5
  • 2
  • +3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now