Solved

Snapshot

Posted on 2011-03-19
18
3,486 Views
Last Modified: 2012-08-14
This week suddenly two virtual machines on our shop stopped because of to little space on the hosts storage. This is because we didn't think of the extra amount of storage that is required when using snapshots on a Exchange server on this host. The host storage is 270GB. Only 200GB is predefined by the virtual machines, but snapshot files are living there own life's besides and are now nearly reaching 70GB,
Can we safely delete all snapshot's from the snapshot manager in ESXi 4.0, and is it correct that data on the snapshot files will be merged with the "original" storage upon deletions?

snapshotmanager.jpg
storage.jpg
0
Comment
Question by:elit2007
  • 8
  • 5
  • 2
  • +3
18 Comments
 
LVL 40

Expert Comment

by:coolsport00
ID: 35173110
Yes, if you choose to "Delete All", your snapshots will transfer its data to its parent disk in the snap tree. Once the data is transferred, then that orig snap is deleted. This process continues up the "snap tree" until all data is 'commmited' and all snaps are deleted in the tree.

~coolsport00
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 35173116
I highly recommend reading these 2 articles on snaps:
Understanding Snaps - http://kb.vmware.com/kb/1015180
Snap Best Practices - http://kb.vmware.com/kb/1025279

BTW...with the amt of snaps you have, and the size of them all, don't expect this process to end anytime soon. It'll take a day...maybe longer.

~coolsport00
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 35173120
The snapshots are part of the original vmdk and you will still need the room for them.  When a snapshot is taken the original vmdk is put in read mode and all new data is being written to the snapshot vmdk.  When you delete the snapshot it will write the snapshot to the original vmdk and will make it just as large as it is with snapshots and you will need more space during the process for the snapshots to write the data back to the original.  Snapshots are unlike SAN snapshots, they are a part of the VM.
0
Manage your data center from practically anywhere

The KN8164V features HD resolution of 1920 x 1200, FIPS 140-2 with level 1 security standards and virtual media transmissions at twice the speed. Built for reliability, the KN series provides local console and remote over IP access, ensuring 24/7 availability to all servers.

 
LVL 1

Author Comment

by:elit2007
ID: 35173122
Do we need to shut the vm down first?
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 35173131
No
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 35173139
And do keep in mind what "pausolov" has added, and what I suggested in my orig post...you will need the space on your datastore equal to the size of all your snaps as the data in all the snaps will be 'commited' to the VMDK.

~coolsport00
0
 
LVL 1

Author Comment

by:elit2007
ID: 35173230
Are you sure paulolo?  If you take a look at the screenshot of the data browser you can see the two virtual disks on the top of the list (104GB and 52GB). They are set to be predefined/static.  They cannot grow, and they are just limits and not what actually is stored at the moment. Therefore I don’t understand why I can’t release some storage space when the snapshot files has been merged to the original virtual disk?
0
 
LVL 16

Expert Comment

by:danm66
ID: 35173319
when you try to commit a snapshot while a VM is running, it will take another snapshot for the VM to run on while it consolidates the other snapshots.  So, if you don't have any space available or not very much, you will want to power down the VM to prevent another crash.
0
 
LVL 1

Author Comment

by:elit2007
ID: 35173347
Good idea :-) Only got 2GB left. So I'm I right about my last post?
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 35173385
You will need more than 2GB left to successfully commit your snaps. See this KB that discusses "consolidating" a bit more:
http://kb.vmware.com/kb/1007849

~coolsport00
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 35173415
You could clone your VM & that removes snapshots during that process, if you have enough space on another datastore to clone it to.

~coolsport00
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 35173461
Either clone or take vmware converter and take another image of it inside the OS, this will just backup the used blockes, I've had issues cloning VMs that have snapshots.
0
 
LVL 119
ID: 35173607
Before you do any of the above are you using vDR? (VMware Data Recovery)
0
 
LVL 1

Author Comment

by:elit2007
ID: 35174407
Right now I'm moving the second VM to free up 50GB.
I'm using Veeam backup. But the backup job stopped working when the host ran out of storage. I think this is because Veeam backup don't get enough space for bacakup snapshot. See error message:
Production datastore 'Storage (tiger)' is getting low on free space (2,13 GB left), and may run out of free disk space completely due to open snapshots.

Removing snapshot
RemoveSnapshot failed, snapshotRef "64-snapshot-140", timeout "3600000"
The object has already been deleted or has not been completely created
Veeam Backup will attempt to remove snapshot during the next job cycle, but you may consider removing snapshot manually.
Possible causes for snapshot removal failure:
 - Network connectivity issue, or vCenter Server is too busy to serve the request
 - ESX host was unable to process snapshot removal request in a timely manner
 - Snapshot was already removed by another application
BackupDiskFull failed
Client error: Failed to open VDDK disk [[Storage (tiger)] Exchange/Exchange-000016.vmdk] ( is read-only mode - [true] )
Failed to open VMDK.
Logon attempt with parameters [VC/ESX: [10.1.254.181];Port: 443;Login: [root];VMX Spec: [moref=64];Snapshot mor: [64-snapshot-140];Transports: [san]] failed because of the following errors:
Failed to open VMDK.
Logon attempt with parameters [VC/ESX: [10.1.254.181];Port: 902;Login: [root];VMX Spec: [moref=64];Snapshot mor: [64-snapshot-140];Transports: [nbd]] failed because of the following errors:

Server error: End of file

Open in new window

0
 
LVL 40

Expert Comment

by:coolsport00
ID: 35176383
Veeam does use snapshotting and yes...if there isn't enough space to create its own snap to b/u, it will fail.

~coolsport00
0
 
LVL 1

Author Comment

by:elit2007
ID: 35196852
Hello again, this is what I ended up with. First I ran vConverter inside the virtual machine and convertet it to another VM host. Then I shut down the virtual server and pushed "Delete all" in ESXi Snapshot Manager.
Then I went to sleep and when I cheked the server next morning all snapshots where gone from the Snapshot manager, but no snapshotfiles where deleted from the storage. The virtual machine was still running on the last snapshot disk. So I gave up this server and booted up the server that I used vConverter to build. And now it's running on that server. No more use of snapshots for me :0)
0
 
LVL 40

Accepted Solution

by:
coolsport00 earned 500 total points
ID: 35197459
Yeah...snapshots are to be used solely as a temporary thing -- create a snap, push out an update to your VM, then after you confirm all is well (after a day or 2 at most) DELETE the snap. :) Both KBs I posted earlier explain it in a bit more detail. Glad you're good to go now...

Regards,
~coolsport00
0
 

Expert Comment

by:noblehobo
ID: 35794366
Also note that if a VM is powered off, the snapshot consolidation process requires very little free space for functionality (about 16 MB).  

Note that if a VM is powered-off for snapshot consolidation (delete or delete-all), the VM must remain powered off until the consolidation is complete.  Changing the VM power state during snapshot consolidation can potentially break the process.  Conversely, if the VM is left powered-on, it should stay powered-on.  This is why ensuring enough space exists is paramount.

Powering off a VM can also free up space by deleting the VM swap file.  By default, if a VM does not have a memory reservation set, a swap file equal in size to the VM's RAM allocation is created at power on.  i.e. a VM with 8 GB RAM has an 8 GB swap file.  Powering off the VM deletes the 8 GB swap file and will free up that amount of space on the datastore.  This is often used when troubleshooting snapshot issues where the datastore has run out of space.

To confirm that a snapshot consolidation process is complete, command-line monitoring via an ESX host can be performed - see KB article 1007566 - http://kb.vmware.com/kb/1007566.

Kind regards,

-noblehobo
0

Featured Post

Connect further...control easier

With the ATEN CE624, you can now enjoy a high-quality visual experience powered by HDBaseT technology and the convenience of a single Cat6 cable to transmit uncompressed video with zero latency and multi-streaming for dual-view applications where remote access is required.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Will try to explain how to use the VMware feature TAGs in the VMs and create Veeam Backup Jobs using TAGs. Since this article is too long, I will create second article for the Veeam tasks.
In this article, I will show you HOW TO: Suppress Configuration Issues and Warnings Alert displayed in Summary status for ESXi 6.5 after enabling SSH or ESXi Shell.
Teach the user how to install log collectors and how to configure ESXi 5.5 for remote logging Open console session and mount vCenter Server installer: Install vSphere Core Dump Collector: Install vSphere Syslog Collector: Open vSphere Client: Config…
This Micro Tutorial steps you through the configuration steps to configure your ESXi host Management Network settings and test the management network, ensure the host is recognized by the DNS Server, configure a new password, and the troubleshooting…

828 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question