Solved

Snapshot

Posted on 2011-03-19
18
3,470 Views
Last Modified: 2012-08-14
This week suddenly two virtual machines on our shop stopped because of to little space on the hosts storage. This is because we didn't think of the extra amount of storage that is required when using snapshots on a Exchange server on this host. The host storage is 270GB. Only 200GB is predefined by the virtual machines, but snapshot files are living there own life's besides and are now nearly reaching 70GB,
Can we safely delete all snapshot's from the snapshot manager in ESXi 4.0, and is it correct that data on the snapshot files will be merged with the "original" storage upon deletions?

snapshotmanager.jpg
storage.jpg
0
Comment
Question by:elit2007
  • 8
  • 5
  • 2
  • +3
18 Comments
 
LVL 40

Expert Comment

by:coolsport00
ID: 35173110
Yes, if you choose to "Delete All", your snapshots will transfer its data to its parent disk in the snap tree. Once the data is transferred, then that orig snap is deleted. This process continues up the "snap tree" until all data is 'commmited' and all snaps are deleted in the tree.

~coolsport00
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 35173116
I highly recommend reading these 2 articles on snaps:
Understanding Snaps - http://kb.vmware.com/kb/1015180
Snap Best Practices - http://kb.vmware.com/kb/1025279

BTW...with the amt of snaps you have, and the size of them all, don't expect this process to end anytime soon. It'll take a day...maybe longer.

~coolsport00
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 35173120
The snapshots are part of the original vmdk and you will still need the room for them.  When a snapshot is taken the original vmdk is put in read mode and all new data is being written to the snapshot vmdk.  When you delete the snapshot it will write the snapshot to the original vmdk and will make it just as large as it is with snapshots and you will need more space during the process for the snapshots to write the data back to the original.  Snapshots are unlike SAN snapshots, they are a part of the VM.
0
 
LVL 1

Author Comment

by:elit2007
ID: 35173122
Do we need to shut the vm down first?
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 35173131
No
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 35173139
And do keep in mind what "pausolov" has added, and what I suggested in my orig post...you will need the space on your datastore equal to the size of all your snaps as the data in all the snaps will be 'commited' to the VMDK.

~coolsport00
0
 
LVL 1

Author Comment

by:elit2007
ID: 35173230
Are you sure paulolo?  If you take a look at the screenshot of the data browser you can see the two virtual disks on the top of the list (104GB and 52GB). They are set to be predefined/static.  They cannot grow, and they are just limits and not what actually is stored at the moment. Therefore I don’t understand why I can’t release some storage space when the snapshot files has been merged to the original virtual disk?
0
 
LVL 16

Expert Comment

by:danm66
ID: 35173319
when you try to commit a snapshot while a VM is running, it will take another snapshot for the VM to run on while it consolidates the other snapshots.  So, if you don't have any space available or not very much, you will want to power down the VM to prevent another crash.
0
 
LVL 1

Author Comment

by:elit2007
ID: 35173347
Good idea :-) Only got 2GB left. So I'm I right about my last post?
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 
LVL 40

Expert Comment

by:coolsport00
ID: 35173385
You will need more than 2GB left to successfully commit your snaps. See this KB that discusses "consolidating" a bit more:
http://kb.vmware.com/kb/1007849

~coolsport00
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 35173415
You could clone your VM & that removes snapshots during that process, if you have enough space on another datastore to clone it to.

~coolsport00
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 35173461
Either clone or take vmware converter and take another image of it inside the OS, this will just backup the used blockes, I've had issues cloning VMs that have snapshots.
0
 
LVL 118
ID: 35173607
Before you do any of the above are you using vDR? (VMware Data Recovery)
0
 
LVL 1

Author Comment

by:elit2007
ID: 35174407
Right now I'm moving the second VM to free up 50GB.
I'm using Veeam backup. But the backup job stopped working when the host ran out of storage. I think this is because Veeam backup don't get enough space for bacakup snapshot. See error message:
Production datastore 'Storage (tiger)' is getting low on free space (2,13 GB left), and may run out of free disk space completely due to open snapshots.

Removing snapshot
RemoveSnapshot failed, snapshotRef "64-snapshot-140", timeout "3600000"
The object has already been deleted or has not been completely created
Veeam Backup will attempt to remove snapshot during the next job cycle, but you may consider removing snapshot manually.
Possible causes for snapshot removal failure:
 - Network connectivity issue, or vCenter Server is too busy to serve the request
 - ESX host was unable to process snapshot removal request in a timely manner
 - Snapshot was already removed by another application
BackupDiskFull failed
Client error: Failed to open VDDK disk [[Storage (tiger)] Exchange/Exchange-000016.vmdk] ( is read-only mode - [true] )
Failed to open VMDK.
Logon attempt with parameters [VC/ESX: [10.1.254.181];Port: 443;Login: [root];VMX Spec: [moref=64];Snapshot mor: [64-snapshot-140];Transports: [san]] failed because of the following errors:
Failed to open VMDK.
Logon attempt with parameters [VC/ESX: [10.1.254.181];Port: 902;Login: [root];VMX Spec: [moref=64];Snapshot mor: [64-snapshot-140];Transports: [nbd]] failed because of the following errors:

Server error: End of file

Open in new window

0
 
LVL 40

Expert Comment

by:coolsport00
ID: 35176383
Veeam does use snapshotting and yes...if there isn't enough space to create its own snap to b/u, it will fail.

~coolsport00
0
 
LVL 1

Author Comment

by:elit2007
ID: 35196852
Hello again, this is what I ended up with. First I ran vConverter inside the virtual machine and convertet it to another VM host. Then I shut down the virtual server and pushed "Delete all" in ESXi Snapshot Manager.
Then I went to sleep and when I cheked the server next morning all snapshots where gone from the Snapshot manager, but no snapshotfiles where deleted from the storage. The virtual machine was still running on the last snapshot disk. So I gave up this server and booted up the server that I used vConverter to build. And now it's running on that server. No more use of snapshots for me :0)
0
 
LVL 40

Accepted Solution

by:
coolsport00 earned 500 total points
ID: 35197459
Yeah...snapshots are to be used solely as a temporary thing -- create a snap, push out an update to your VM, then after you confirm all is well (after a day or 2 at most) DELETE the snap. :) Both KBs I posted earlier explain it in a bit more detail. Glad you're good to go now...

Regards,
~coolsport00
0
 

Expert Comment

by:noblehobo
ID: 35794366
Also note that if a VM is powered off, the snapshot consolidation process requires very little free space for functionality (about 16 MB).  

Note that if a VM is powered-off for snapshot consolidation (delete or delete-all), the VM must remain powered off until the consolidation is complete.  Changing the VM power state during snapshot consolidation can potentially break the process.  Conversely, if the VM is left powered-on, it should stay powered-on.  This is why ensuring enough space exists is paramount.

Powering off a VM can also free up space by deleting the VM swap file.  By default, if a VM does not have a memory reservation set, a swap file equal in size to the VM's RAM allocation is created at power on.  i.e. a VM with 8 GB RAM has an 8 GB swap file.  Powering off the VM deletes the 8 GB swap file and will free up that amount of space on the datastore.  This is often used when troubleshooting snapshot issues where the datastore has run out of space.

To confirm that a snapshot consolidation process is complete, command-line monitoring via an ESX host can be performed - see KB article 1007566 - http://kb.vmware.com/kb/1007566.

Kind regards,

-noblehobo
0

Featured Post

Complete VMware vSphere® ESX(i) & Hyper-V Backup

Capture your entire system, including the host, with patented disk imaging integrated with VMware VADP / Microsoft VSS and RCT. RTOs is as low as 15 seconds with Acronis Active Restore™. You can enjoy unlimited P2V/V2V migrations from any source (even from a different hypervisor)

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
"Error while attempting to connect to View Composer" 4 213
Windows VM unexpected shutdown 7 54
VM migration error 7 40
Veeam Manuall Backup 2 45
Will try to explain how to use the VMware feature TAGs in the VMs and create Veeam Backup Jobs using TAGs. Since this article is too long, I will create second article for the Veeam tasks.
HOW TO: Install and Configure VMware vSphere Hypervisor 6.5 (ESXi 6.5), Step by Step Tutorial with screenshots. From Download, Checking Media, to Completed Installation.
Teach the user how to delpoy the vCenter Server Appliance and how to configure its network settings Deploy OVF: Open VM console and configure networking:
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …

932 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now