[Last Call] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1534
  • Last Modified:

vSphere 5.1: Space reclamation with NFS datastores

We are running vSphere 5.1 using NFS datastores on NetApp. On Saturday, we removed/deleted a 150GB disk (thin provisioned) from one of our VMs. We backed up all the data on the disk -- about 100 GB. We took notes before and after the move.

The datastore volume has 540 GB capacity.

Provisioned space before deletion: 594 GB. After deletion, 436 GB.
Free space before deletion: 232 GB. After deletion, 235 GB.

I always thought that with NFS datastrores, space reclamation was done automatically. Why are we not seeing the free space go up?
0
pzozulka
Asked:
pzozulka
  • 4
  • 2
  • 2
  • +2
1 Solution
 
Ganesh Kumar ACommented:
If you delete a file on a VMFS Datastore, the file system marks the affected blocks as unoccupied and reusable.
Unfortunately this information is not passed on to the storage system which believes that the free blocks are actually still in use.

With vSphere 5 VMWare introduced a new VAAI primitive called Thin provisioning block space reclamation (UNMAP).
This VAAI primitive should ensure that the storage system is informed if something is deleted and that the capacity is automatically released.

However, performance problems occured as these operations caused high storage IO and high latency.
VMWare has therefore recommended to disable the feature (actually it is disabled by default).

With ESXi Update 1 the command line tool “vmkfstools” was enhanced with an unmap parameter.
So it is now possible to release unused space manually as needed (but make sure that this is also supported from your storage vendor!)
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
How are you provisioning your datastores on your NetApp?

Are you using Virtual Storage Console ?

Are you using the VAAI plugins for NFS ?

or see here

http://www.running-system.com/how-to-reclaim-dead-space-from-thin-provisioned-vmfs-datastores/
0
 
pzozulkaAuthor Commented:
The datastores are thin provisioned on the NetApp. We are using Virtual Storage Console, but not VAAI plugins.

Also, for each Host in vCenter's Configuration tab, under Storage, all datastores are listed as Type = NFS, and only the local ESXi disk is listed as VMFS.
0
Visualize your virtual and backup environments

Create well-organized and polished visualizations of your virtual and backup environments when planning VMware vSphere, Microsoft Hyper-V or Veeam deployments. It helps you to gain better visibility and valuable business insights.

 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
It's recommended to deploy the VAAI plugins, and configure your hosts as recommended for NFS
0
 
pzozulkaAuthor Commented:
Her'es all I see when I run esxcli storage vmfs extent list
~ # esxcli storage vmfs extent list
Volume Name            VMFS UUID                            Extent Number  Device Name                           Partition
---------------------  -----------------------------------  -------------  ------------------------------------  ---------
i1whlesx02:datastore1  5270c817-4a522f52-083c-000af72f52a8              0  naa.6b8ca3a0f6d1f60019cd00c6124da7ca          4

Open in new window


As you can see, the only thing showing up is the local datastore (local disks on the esx host). None of the NetApp datastores are showing because they are all NFS. So I don't think the VMFS suggestions above apply.
0
 
gheistCommented:
Deploy VAAI and deduplication. It will save lots of headaches.
So small deletion most likely comes from deduplication - dupe data stays on disk to feed other copies.
0
 
pzozulkaAuthor Commented:
Thanks, I'll deploy VAAI. We already use deduplication. Not sure how it applies to reclaiming space from a deleted VMDK on an NFS datastore.
0
 
gheistCommented:
If there is deduplication enabled on volume deleting file will free less than that because of heap of zero blocks and heap of OS files shared with other VMs. Hope that explains where space disappeared.
With VAAI eager zeroed volumes are allocated in an instant, and VM cloning too. And it is VMWARE recommendation to use it. v1.0.21 is the latest one, and you can deploy it via VUM at regulat update cycle.
0
 
pzozulkaAuthor Commented:
Hmm...not quite. I can't say the space really disappeared. It just feels like it is not being reclamed after deleting the VMDK. See below.

BeforeBeforeAfter After
0
 
robocatCommented:
It sounds like the deleted disk was caught in a NetApp snapshot. The free space will only be reclaimed if all Netapp snapshots that were created before deleting the disk have been deleted/expired.

Use NetApp system manager and check for snapshots on the volume. Delete snapshots if necessary.
0

Featured Post

Important Lessons on Recovering from Petya

In their most recent webinar, Skyport Systems explores ways to isolate and protect critical databases to keep the core of your company safe from harm.

  • 4
  • 2
  • 2
  • +2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now