Solved

Snapshot

Posted on 2011-03-19
18
3,454 Views
Last Modified: 2012-08-14
This week suddenly two virtual machines on our shop stopped because of to little space on the hosts storage. This is because we didn't think of the extra amount of storage that is required when using snapshots on a Exchange server on this host. The host storage is 270GB. Only 200GB is predefined by the virtual machines, but snapshot files are living there own life's besides and are now nearly reaching 70GB,
Can we safely delete all snapshot's from the snapshot manager in ESXi 4.0, and is it correct that data on the snapshot files will be merged with the "original" storage upon deletions?

snapshotmanager.jpg
storage.jpg
0
Comment
Question by:elit2007
  • 8
  • 5
  • 2
  • +3
18 Comments
 
LVL 40

Expert Comment

by:coolsport00
Comment Utility
Yes, if you choose to "Delete All", your snapshots will transfer its data to its parent disk in the snap tree. Once the data is transferred, then that orig snap is deleted. This process continues up the "snap tree" until all data is 'commmited' and all snaps are deleted in the tree.

~coolsport00
0
 
LVL 40

Expert Comment

by:coolsport00
Comment Utility
I highly recommend reading these 2 articles on snaps:
Understanding Snaps - http://kb.vmware.com/kb/1015180
Snap Best Practices - http://kb.vmware.com/kb/1025279

BTW...with the amt of snaps you have, and the size of them all, don't expect this process to end anytime soon. It'll take a day...maybe longer.

~coolsport00
0
 
LVL 42

Expert Comment

by:paulsolov
Comment Utility
The snapshots are part of the original vmdk and you will still need the room for them.  When a snapshot is taken the original vmdk is put in read mode and all new data is being written to the snapshot vmdk.  When you delete the snapshot it will write the snapshot to the original vmdk and will make it just as large as it is with snapshots and you will need more space during the process for the snapshots to write the data back to the original.  Snapshots are unlike SAN snapshots, they are a part of the VM.
0
 
LVL 1

Author Comment

by:elit2007
Comment Utility
Do we need to shut the vm down first?
0
 
LVL 40

Expert Comment

by:coolsport00
Comment Utility
No
0
 
LVL 40

Expert Comment

by:coolsport00
Comment Utility
And do keep in mind what "pausolov" has added, and what I suggested in my orig post...you will need the space on your datastore equal to the size of all your snaps as the data in all the snaps will be 'commited' to the VMDK.

~coolsport00
0
 
LVL 1

Author Comment

by:elit2007
Comment Utility
Are you sure paulolo?  If you take a look at the screenshot of the data browser you can see the two virtual disks on the top of the list (104GB and 52GB). They are set to be predefined/static.  They cannot grow, and they are just limits and not what actually is stored at the moment. Therefore I don’t understand why I can’t release some storage space when the snapshot files has been merged to the original virtual disk?
0
 
LVL 16

Expert Comment

by:danm66
Comment Utility
when you try to commit a snapshot while a VM is running, it will take another snapshot for the VM to run on while it consolidates the other snapshots.  So, if you don't have any space available or not very much, you will want to power down the VM to prevent another crash.
0
 
LVL 1

Author Comment

by:elit2007
Comment Utility
Good idea :-) Only got 2GB left. So I'm I right about my last post?
0
Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

 
LVL 40

Expert Comment

by:coolsport00
Comment Utility
You will need more than 2GB left to successfully commit your snaps. See this KB that discusses "consolidating" a bit more:
http://kb.vmware.com/kb/1007849

~coolsport00
0
 
LVL 40

Expert Comment

by:coolsport00
Comment Utility
You could clone your VM & that removes snapshots during that process, if you have enough space on another datastore to clone it to.

~coolsport00
0
 
LVL 42

Expert Comment

by:paulsolov
Comment Utility
Either clone or take vmware converter and take another image of it inside the OS, this will just backup the used blockes, I've had issues cloning VMs that have snapshots.
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
Before you do any of the above are you using vDR? (VMware Data Recovery)
0
 
LVL 1

Author Comment

by:elit2007
Comment Utility
Right now I'm moving the second VM to free up 50GB.
I'm using Veeam backup. But the backup job stopped working when the host ran out of storage. I think this is because Veeam backup don't get enough space for bacakup snapshot. See error message:
Production datastore 'Storage (tiger)' is getting low on free space (2,13 GB left), and may run out of free disk space completely due to open snapshots.

Removing snapshot
RemoveSnapshot failed, snapshotRef "64-snapshot-140", timeout "3600000"
The object has already been deleted or has not been completely created
Veeam Backup will attempt to remove snapshot during the next job cycle, but you may consider removing snapshot manually.
Possible causes for snapshot removal failure:
 - Network connectivity issue, or vCenter Server is too busy to serve the request
 - ESX host was unable to process snapshot removal request in a timely manner
 - Snapshot was already removed by another application
BackupDiskFull failed
Client error: Failed to open VDDK disk [[Storage (tiger)] Exchange/Exchange-000016.vmdk] ( is read-only mode - [true] )
Failed to open VMDK.
Logon attempt with parameters [VC/ESX: [10.1.254.181];Port: 443;Login: [root];VMX Spec: [moref=64];Snapshot mor: [64-snapshot-140];Transports: [san]] failed because of the following errors:
Failed to open VMDK.
Logon attempt with parameters [VC/ESX: [10.1.254.181];Port: 902;Login: [root];VMX Spec: [moref=64];Snapshot mor: [64-snapshot-140];Transports: [nbd]] failed because of the following errors:

Server error: End of file

Open in new window

0
 
LVL 40

Expert Comment

by:coolsport00
Comment Utility
Veeam does use snapshotting and yes...if there isn't enough space to create its own snap to b/u, it will fail.

~coolsport00
0
 
LVL 1

Author Comment

by:elit2007
Comment Utility
Hello again, this is what I ended up with. First I ran vConverter inside the virtual machine and convertet it to another VM host. Then I shut down the virtual server and pushed "Delete all" in ESXi Snapshot Manager.
Then I went to sleep and when I cheked the server next morning all snapshots where gone from the Snapshot manager, but no snapshotfiles where deleted from the storage. The virtual machine was still running on the last snapshot disk. So I gave up this server and booted up the server that I used vConverter to build. And now it's running on that server. No more use of snapshots for me :0)
0
 
LVL 40

Accepted Solution

by:
coolsport00 earned 500 total points
Comment Utility
Yeah...snapshots are to be used solely as a temporary thing -- create a snap, push out an update to your VM, then after you confirm all is well (after a day or 2 at most) DELETE the snap. :) Both KBs I posted earlier explain it in a bit more detail. Glad you're good to go now...

Regards,
~coolsport00
0
 

Expert Comment

by:noblehobo
Comment Utility
Also note that if a VM is powered off, the snapshot consolidation process requires very little free space for functionality (about 16 MB).  

Note that if a VM is powered-off for snapshot consolidation (delete or delete-all), the VM must remain powered off until the consolidation is complete.  Changing the VM power state during snapshot consolidation can potentially break the process.  Conversely, if the VM is left powered-on, it should stay powered-on.  This is why ensuring enough space exists is paramount.

Powering off a VM can also free up space by deleting the VM swap file.  By default, if a VM does not have a memory reservation set, a swap file equal in size to the VM's RAM allocation is created at power on.  i.e. a VM with 8 GB RAM has an 8 GB swap file.  Powering off the VM deletes the 8 GB swap file and will free up that amount of space on the datastore.  This is often used when troubleshooting snapshot issues where the datastore has run out of space.

To confirm that a snapshot consolidation process is complete, command-line monitoring via an ESX host can be performed - see KB article 1007566 - http://kb.vmware.com/kb/1007566.

Kind regards,

-noblehobo
0

Featured Post

Get up to 2TB FREE CLOUD per backup license!

An exclusive Black Friday offer just for Expert Exchange audience! Buy any of our top-rated backup solutions & get up to 2TB free cloud per system! Perform local & cloud backup in the same step, and restore instantly—anytime, anywhere. Grab this deal now before it disappears!

Join & Write a Comment

Suggested Solutions

This article is an update and follow-up of my previous article:   Storage 101: common concepts in the IT enterprise storage This time, I expand on more frequently used storage concepts.
This article will show you how to create an ISO CD-ROM/DVD-ROM image (*.iso), and MD5 checksum signature, for use with VMware vSphere Hypervisor 6.5 (ESXi 6.5). It's a good idea to compare checksums, because many installations fail because of a corr…
Teach the user how to install vSphere Update Manager  Console to Windows system:  Install vSphere Update Manager: Configure vSphere Update Manager plug-in in vSphere Client: Verify vSphere Update Manager settings in vSphere Client:
This Micro Tutorial steps you through the configuration steps to configure your ESXi host Management Network settings and test the management network, ensure the host is recognized by the DNS Server, configure a new password, and the troubleshooting…

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now