Solved

How to reclaim free space on a ESXi datastore?

Posted on 2010-09-15
12
6,552 Views
Last Modified: 2012-05-10
I have two vm setup on ESXi (Windows 2003 and Ubuntu).  Both have their own datastores and we haven't used Ubuntu yet, so it almost empty.  Windows 2003 vm, however is constantly running out of space. I have this error:

 msg.hbacommon.outofspace:there is no more space for vurtual disk xxx Windows 2003-000002.vmdk.  You may be able to continue this session by freeing disk space on the relevant partition.

I used to have a few snapshots on this machine simultaneously, and after deleting them in vSphere client I seem to have some leftovers. As shown on attached screenshots, in addition to the Windows 2003.vmdk file, I have Windows 2003-000001, 000002 and 000003 files.  Windows 2003.vmx file (see attached) points to Windows 2003-000002 file, but I have no idea as to which files to delete or how else to reclaim free space.
.encoding = "UTF-8"
config.version = "8"
virtualHW.version = "4"
nvram = "XXX Windows 2003.nvram"
deploymentPlatform = "windows"
virtualHW.productCompatibility = "hosted"
unity.customColor = "|23C0C0C0"
tools.upgrade.policy = "useGlobal"
powerType.powerOff = "soft"
powerType.powerOn = "default"
powerType.suspend = "hard"
powerType.reset = "soft"

displayName = "XXX Windows 2003"
extendedConfigFile = "XXX Windows 2003.vmxf"
floppy0.present = "TRUE"

numvcpus = "2"
scsi0.present = "TRUE"
scsi0.sharedBus = "none"
scsi0.virtualDev = "lsilogic"
memsize = "2048"
scsi0:0.present = "TRUE"
scsi0:0.fileName = "XXX Windows 2003-000002.vmdk"
scsi0:0.deviceType = "scsi-hardDisk"
ide0:0.present = "TRUE"
ide0:0.fileName = "CD/DVD Drive 1"
ide0:0.deviceType = "atapi-cdrom"
ide0:0.startConnected = "FALSE"
floppy0.startConnected = "FALSE"
floppy0.fileName = "Floppy Drive 1"
ethernet0.present = "TRUE"
ethernet0.wakeOnPcktRcv = "FALSE"
ethernet0.networkName = "Research Machine Network"
ethernet0.addressType = "generated"
ethernet1.present = "TRUE"
ethernet1.wakeOnPcktRcv = "FALSE"
ethernet1.networkName = "VM Network"
ethernet1.addressType = "generated"
guestOS = "winnetstandard"
uuid.location = "56 4d 6b 9c 28 39 0c 39-0a 6e 1e ce 73 00 3d 4c"
uuid.bios = "56 4d 6b 9c 28 39 0c 39-0a 6e 1e ce 73 00 3d 4c"
vc.uuid = "52 36 58 a8 25 0c b7 88-ac 16 a8 88 81 94 ec 49"
sched.cpu.min = "0"
sched.cpu.units = "mhz"
sched.cpu.shares = "normal"
sched.mem.minsize = "0"
sched.mem.shares = "normal"

ethernet0.generatedAddress = "00:0c:29:00:3d:4c"
ethernet1.generatedAddress = "00:0c:29:00:3d:56"
tools.syncTime = "FALSE"
cleanShutdown = "FALSE"
replay.supported = "FALSE"
sched.swap.derivedName = "/vmfs/volumes/4b97d46e-0e4e6f50-05ff-001b242daf36/XXX Windows 2003/XXX Windows 2003-79b83bab.vswp"
scsi0:0.redo = ""
vmotion.checkpointFBSize = "4194304"
ethernet0.generatedAddressOffset = "0"
ethernet1.generatedAddressOffset = "10"
hostCPUID.0 = "0000000168747541444d416369746e65"
hostCPUID.1 = "00040f320002080000002001178bfbff"
hostCPUID.80000001 = "00040f320000034f0000001febd3fbff"
guestCPUID.0 = "0000000168747541444d416369746e65"
guestCPUID.1 = "00040f320000080000002001078bbbff"
guestCPUID.80000001 = "00040f320000034f00000009ebd3bbff"
userCPUID.0 = "0000000168747541444d416369746e65"
userCPUID.1 = "00040f320002080000002001078bbbff"
userCPUID.80000001 = "00040f320000034f00000009ebd3bbff"
evcCompatibilityMode = "FALSE"

ethernet1.startConnected = "FALSE"

guestOSAltName = "Microsoft Windows Server 2003, Standard Edition (32-bit)"

checkpoint.vmState.readOnly = "FALSE"
checkpoint.vmState = ""

Open in new window

VM-screenshot.png
datastore-screenshot.png
0
Comment
Question by:lolekbolek
  • 6
  • 3
  • 2
  • +1
12 Comments
 
LVL 10

Expert Comment

by:BloodRed
ID: 33685905
The VM is still running off of a snapshot.  Can you post a screenshot of the Snapshot Manager window for this VM?
0
 
LVL 22

Expert Comment

by:Luciano Patrão
ID: 33685912
Hi

First, have create some virtual disks, then removed? Before, or after a snapshot?

Check if this can fix you problem

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1012383

Jail
0
 
LVL 22

Expert Comment

by:Luciano Patrão
ID: 33685935
Hi

Sorry forget to add the next link

http://virtrix.blogspot.com/2007/06/vmware-dreadful-sticky-snapshot.html

Its old, but can fix many problems with snapshots

Jail
0
 
LVL 28

Expert Comment

by:bgoering
ID: 33686342
You need to clean up your snapshots - but you will need some free space in order to clean up the snapshots. This is because when deleting the snapshot vmware will merge changes from the snapshot back into the base disk before deleting the snapshot disk, then repeat that operation for each snapshot level that is defined. The extra space is required for the merge operation.

Often that is kind of difficult if you have no free space to give. If you have additional space laying around consider adding an extent to the datastore that is hosting your Windows vm. If not you may be able to get by with using vmware converter (free download from web site) to convert your ESXi based Windows vm to another platform like VMware Workstation. The convert process will also merge the changes, but do it on the destination. Don't worry if you don't have VMware Workstation, you won't need it because it is just creating files on your disk that define the vm as if you were to run it on VMware server.

At this point I would download the free VMware player application and power up your converted VM for a sanity check that it is indeed all there and looks as you would expect.

After that conversion is completed successfully and you have verified it with VMware player, you can completely delete your source ESXi vm and free up all of its space on the datastore.

Finally use converter again, and this time go the other way with the source as your offline image, and the destination as your ESXi server.

Good Luck
0
 

Author Comment

by:lolekbolek
ID: 33687247
Thank you for the suggestions.  I followed them - adding a snapshot and then going to Snapshot Manager -> Delete All.  It took two hours to delete it, but it showed task status as "complete" in vSphere Client and there is nothing shown in Snapshot Manager (see attached).

However, my disk is still almost full.  The only thing that changed, I now have a very tiny file named "XXX Windows 2003-000004.vmdk" and VM -> Edit Settings now points the virtual drive to this file and not the "XXX Windows 2003-000002.vmdk" as before.  Does it mean that it is safe to delete:

"XXX Windows 2003-000001.vmdk"
"XXX Windows 2003-000002.vmdk"
"XXX Windows 2003-000003.vmdk"

or/and should I try the VMware vCenter Converter as suggested above? Because having the virtual drive pointing to a 000004.vmdk instead of the original one is not right?
after-removing-ALL-snapshots-1.png
after-removing-ALL-snapshots-2.png
0
 
LVL 28

Expert Comment

by:bgoering
ID: 33687347
Nothing so far as snapshots are safe to delete manually... You can do it, and point your .vmx file back to your base disk, but you will lose all changes since the first snapshot was created. Most folks don't want that.
0
Control application downtime with dependency maps

Visualize the interdependencies between application components better with Applications Manager's automated application discovery and dependency mapping feature. Resolve performance issues faster by quickly isolating problematic components.

 
LVL 28

Expert Comment

by:bgoering
ID: 33687357
If you have vCenter server (not the free ESXi) you could also try cloneing the vm to to another datastore if you have one with more room - rather than the converter method I posted earlier.
0
 

Author Comment

by:lolekbolek
ID: 33687390
Hi bgoering, thank you.  In my situation, could I delete the -0000x files and then export the VM so I can import it (as a substitute for a snapshot)?

And one more question - shall I just rename the

scsi0:0.fileName = "XXX Windows 2003-000004.vmdk"

line in .vmx to point to "XXX Windows 2003.vmdk"?  Is it ok that it had no word "-flat" in the name?
0
 
LVL 28

Expert Comment

by:bgoering
ID: 33687420
If you don't mind losing the data you can do that. If you have both a

XXX Windows 2003.vmdk (would be a small descripter file)
and
XXX Windows 2003-flat.vmdk (would be size of virtual disk)

keep them both and point .vmx to XXX Windows 2003.vmdk

If you only have a large XXX Windows 2003.vmdk that is all you need to keep.

Note that you can't really trust the datastore browser as sometimes it doesn't show you the flat files, from the console go to the datastore and do a "ls" to see what is there.
0
 
LVL 28

Expert Comment

by:bgoering
ID: 33687444
A process I use to back up running vms on a free ESXi is as follows:

Using Veeam FastSCP

1. Delete (commit) all snapshots - will leave just the base disk
2. Create a snapshot - freezes I/O to base disk and points .vmx to new snapshot
3. Back up VM folder except the new -00001 file and .vmx using Veeam FastSCP
4. Delete the snapshot you just created for the backup - points .vmx back to base disk
5. Back up .vmx file using Veeam FastSCP

Doing this leaves everything in the correct state where if you need to restore you just copy the entire backup folder back to ESXi and add vm to inventory and go

Good Luck
0
 

Author Comment

by:lolekbolek
ID: 33688780
Thank you again, bgoering.  I am much closer now, but have a couple steps in your process to clarify.

Should the vm be running when I delete/create snapshots?
Do I have to do your steps 1 and 2 using FastSCP or the ESXi Snapshot Manager?

I've already created a screenshot and used "Delete All" (both using Snapshot Manager).  After this, the scsi0:0.fileName in .vmx is pointing to a new file "XXX Windows 2003-000004.vmdk" (a snapshot file?)  I've backed up the whole directory using FastSCP (great app - thanks a lot).  Where I'm stack is what files on the attached FastSCP screenshot can I delete and in what order.

Sorry to bug you, I am almost done!
Veeam-screenshot.png
0
 
LVL 28

Accepted Solution

by:
bgoering earned 500 total points
ID: 33691915
Taking the questions in order - It doesn't make any difference if the vm is running or not to create and delete snapshots. Steps 1 & 2 are done using the snapshot manager. If you are willing to have the vm down while backing it up most of the process isn't necessary. If the vm is down you can just copy off the entire folder then bring it back up.

You are having trouble with the delete process because you are out of space on your datastore - so changes can't be merged back in. You largest snapshot is the -000001 at 24.50 GB, so if you are willing to lose the data you can manually delete all of the -00000* files and point the vmx to the base disk.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

This is an issue that we can get adding / removing permissions in the vCSA 6.0. We can also have issues searching for users / groups in the AD (using your identify sources). This is how one of the ways to handle this issues and fix it.
Will try to explain how to use the VMware feature TAGs in the VMs and create Veeam Backup Jobs using TAGs. Since this article is too long, I will create second article for the Veeam tasks.
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …
This Micro Tutorial walks you through using a remote console to access a server and install ESXi 5.1. This example is showing remote access and installation using a Dell server. The hypervisor is the very first component of your virtual infrastructu…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now