Understanding Thin Disk

If I understand in Vmware when creating VM with thin disk , the space is not allocated from the beginning, however it is still set up theoretically, for instance you set it up to 100 GB /Thin disk

Correct, if you create a Thin Disk of 100GB, initially at Creation by ESXi/vCenter Server, a small stub is allocated on the datastore, which takes up very little space.

What if ,for instance with windows OS, you can go and use disk Management you extend the volume to its maximum. Then at this stage it is still the same as creating VM with thick disk, because the space was not completely allocated through vmware but completely allocated from within windows disk management.

You CANNOT go into to Disk Management and EXTEND the physical disk (which it's virtual because it's MAX is always going to be 100GB)

The OS has no concept of what a thin disk is, and will see the entire NTFS partition/volume as 100GB.

So inside the OS, it's always the MAX of 100GB disk, how your format it, and partition is up to you. But if you allocate ALL it will be 100GB.

BUT, as the OS formats the disk, and allocates 1's to the sectors of the disk, the thin disk will expand and inflate, to potentially all 100GB.

<<You CANNOT go into to Disk Management and EXTEND the physical disk (which it's virtual because it's MAX is always going to be 100GB) >>

So If you set up a thin disk in Vmware with size 100 GB, then in Windows it will be set up to 100 GB
if that 's the case what is the benefit of thin disk, as no other VM can use  the unused space since Windows own the space already ?
 

Although the OS may see the disk as 100GB.

It may NOT all be used!

It's all about used space or provisioned space....
 
on the DATASTORE!

1. Create a virtual machine with a thin disk of 100GB.

2. Install e.g. Windows 10 fully patched.

3. The size is likely to be approx 7-15GB.

That is the size on the datastore!

So other VMs, can use that space.... e.g. 100GB-7-15GB

What you need to be careful OF, if all VMs, inflate their disks, and you don't end up over provisioning, and not having the space on the datastore.

Consider this.
Your available datastore space is 1TB ...
You are setting up an application SQL server, 2 DC, web server, fileserver
In a five year projection
SQL server needs 300 GB and 300GB 
Each DC is 100GB
Webserver 250GB
Fileserver 300GB
The total demand on datastore is 1.35TB but you only have 1TB

On the SQL server the first disk thin provision with max size of 300, second disk full provision 300
Each DC thin provision 100GB
Webserver thin provision 250GB
Filleserver thin provision 300GB

If the OS takes up 50GB for OS SQL is about 20GB
So you have 250Gb for the OS for all, 20GB for SQL, and 300GB second SQL DATA drive
So you have the platform deployed with 570GB consumed.

<<Although the OS may see the disk as 100GB. >>
Do you think it can see 100GB today, and in a week it can see less, if other VMs have used some of it ?

No, you are not allocating shared resources to VMs, you are sharing data storage space each individual VM will have allocated space from the shared datastore space.
If the space of the datastore runs out Of space and your VM that currently is using 10GB and need an additional 1byte, it will error out possibly crash depending on what process was running..

it really depends how quickly the OS writes to all the space!

its about the viewpoint

Try thinking about it in 3D, if you look at the processor and the disk from the side, a thick disk will look like a box on the left and a tube of storage to the right which is 100TB long
If you now turn the viewpoint 90 degrees and look from the processor you see 100TB

with a Thin disk the view from the processor is still 100TB (cuz thats what the storage system reports is the size), but from the side it initially looks like a very short tube that slowly gets longer as data is written to it, it can only reach a max of 100TB

using a thin disk now means that you can provision your application/system for all the expansion its likely to get over the next 5 years, but only buy enough physical storage to handle your current requirement

I do not have my LAB set up yet to verify it, but I might get a grasp of it by patience.
Well, let 's say I create Windows VM with Thin Disk and set up the size to 100 GB

if I login to the VM and check for instance Disk Management. What should I see ? C: Drive with 100GB ?

what do you think you will see and why ?

I guess 100 GB.

But if I use 50 GB , and other VMs need the other  50 GB (assuming the datastore is very low in space)

So I may log back in  to the computer and see almost 100 GB used up ? even though almost half of it by other VMs ?

The VM does not see over subscription of the datastore.
If you have 10 VMs with 100gb allocated max space, but each only uses 30GB.
In disk manager on each they will show 100GB allocation with how much is used, %free.

If two of the systems ramp up a demand on storage
The other 8 will only crash when they need additional storage that is unavailable in. Datastore.

100GB is correct.

And Why does Disk Management see/detect 10GB?

At present you’ve score 50% out of a possible 100%

Think about it this way.  VMware manages the hardware platform.  When you create a disk and set it as "thin provision", that tells VMware to tell the OS that there is 100GB there( assuming you set the size as 100GB ).  Since VMWare controls the hardware platform and its services, the OS has no idea that VMWare is lying about whats its been given.  This is similar to the fact that the disk you are placing that virtual disk on is probably much larger than 100GB but the OS doesnt know it, it only sees what VMWare allows it to know.  The OS trusts this information because VMWare is emulating the information that the OS is used to looking at.  

Thin provisioning is great in most cases.  If you have a storage platform, you can even thin provision storage LUNs that way as well, thus you are lying to VMWare which is also lying to the OS.  If you have a spinning disk storage system, then you may want to avoid using thin provisioning on database drives.  If you have solid state storage, then thats most likely, no longer a worry.  

The one warning i always give people about thin provisioning:  Know whats your needs are going to be!  If you thin provision a storage platform 10x beyond, you are probably ok.  If you do it 100x beyond, you could put the whole thing in jeopardy ( you would want to keep a close eye on the LUNs if you use those or on the storage platforms overall storage space ).  

The good news about VMWare, using vmfs 6 ( i think ) and above on a LUN, will allow the OS to release the space back to the LUN.  So if you thin provision 100GB and an out of control process fills up the drive, once deleted, the OS will release that space back to VMFS and VMFS will release the space back to the LUN.    VVOLs do this by default ( you dont use VMFS 6 ).  This wont happen quickly, its usually a few hours before you see the space back unless you force a cleanup manually.

As far as the storage device goes, it will handle the data placement on disk.  Yes, you may end up with data all over the place, but generally, storage units like SAN/NAS have data collection systems to clean that up ( move data around, assuming you were worried about a fragmentation issue ).  DO NOT defrag LUNs or VVOLs.  If something is out of wack on a vm on the storage side, migrate to a new LUN.

Sorry Guys for the delay.
I created new VM with Thin Disk. I can see in Windows VM DIsk Management:
so if I understand the computer will use disk 0 to its fullest size 120 GB, even if  other VMs run out of disk space  and there is no more space to allocate to the them from the storage ?

NO!
As one  can not squeeze water from a rock, one VM can not get addiitonal space from a datastore where there is non left.

An airline flight booking is an apt example.
Based on industry there are 15% of people who cancel their booking. So they overbook.
This is what thin provisioning does. IT OVERBOOKS the storage resource.
Like in airplanes where there are times that the cancelation rate is below the 15% meaning a flight is over subscribed and someone has to be rescheduled.
So it happens here. Each VM is told that it has X amount of total storage to use as it sees fit.
This is based on a 10% annual storage consumption.
There are situation,where there is a higher than expected demand on resources, such that storage consumption of three of the VMs went from 10% annually to 30%

While the VM sees that is has 100GB, it is currently using 20GB of real storage space. the datastore available space is 0GB.
there are x number of additional thin provisioned vms with different distribution of Allocated space (provisioned) and actual used space.
Any one of the VMs that needs additional storage resource in this situation will be met with an error that there is no MORE space in the datastore and likely will crush.
unlike planes and people,
once the datastore runs out of space, stuff stops working. until space is increased.

I guess I was not clear. let 's say:

I have Shared Storage : Storage1 (360GB)
I have VM1 with thin disk : 120 GB
I have VM2 with thin disk : 240 GB

120GB+240GB=360GB

VM1 has used only 20 GB out of 120GB and VM2 is running out of space(it has used all 360 GB ).
Now Will VM2 gets any storage to use from VM1 that has 100GB not used ?
if the answer is No, then I do not see the purpose of thin disk.

Now Will VM2 gets any storage to use from VM1 that has 100GB not used ?

No.

You have defined VM2 with a maximum hard disk size of 240GB, a VM cannot automatically change it's specification!

But you could re-size VM1, to 20GB, and then that would recover the 80GB from the datastore which hosts VM1, and then manually add it to VM2!

(that's another question if you want to know how to do it!)

and VM2 would then have 320GB.

What is the point, as Andrew noted, thoughsinceypuusethin provisioning,toto used is actually 260GB  meaning the shared storage still has 100GB available
You should look at Anrew's articles onthis topic.
You do not nee to reclaim space to allocate additional pace to vm2

But you could re-size VM1, to 20GB, and then that would recover the 80GB from the datastore which hosts VM1, and then manually add it to VM2!

Do you resize VM1  from within Vmware or from within Windows ?

If these are thin provisioned, you do not have to downsize, release space from VM1 to oversubscribed by increasing the allocation to vm2

Do you resize VM1  from within Vmware or from within Windows ?

both, I did suggest if you really want to know the method post a new question!

The whole point of thin provisioning to allow you to oversubscribe your resources ( space in this case ) while still being able to allocate a reasonable amount of resources to each VM.  Will all the VMs need that space, probably not, so the hope is that this doesnt happen.  If you intend on using the space to its full capacity, then you need to go back to capacity planning 101 ( which says, if you know you need more space, then buy more space ).  Just like many things in life, an elastic/flexible pool of space is needed to allow for the expansion and contraction of space during normal operations.  Thin provisioning gives you this option for storage.  Without thin provisioning in use, you would end up tying up a bunch of storage space that does not get used and costs you more money to just sit there since it was allocated to one VM instead of shared.

Per Andrew:

 Do you resize VM1  from within Vmware or from within Windows ?                                    

       both, I did suggest if you really want to know the method post a new question!                  

I believe I got it now:

So if VM1 was assigned Thin disk of 100 GB, it will still own whole 100 GB , it does not give  one Byte away automatically even if it is using only 20 GB, and other VMs are starving for disk space.
However, Administrator can reduce the size of VM1 disk from Vsphere console then from Windows Disk Management

However , if my  above statement is true, then I do not see the difference between Thin and Thick disk   , because with thick disk VM1 will own 100 GB and does not give away any space of its disk to other VMs, and the Administrator can reduce its size using vsphere and windows disk management.

though reading here: https://blogs.vmware.com/vsphere/2014/05/thick-vs-thin-disks-flash-arrays.html

it says :

how is the unused space available for use by other VMs ?

So if VM1 was assigned Thin disk of 100 GB, it will still own whole 100 GB , it does not give  one Byte away automatically even if it is using only 20 GB, and other VMs are starving for disk space.

No. it does not. But a thin is only thick when it's fully inflated to 100GB (maybe!).

However, Administrator can reduce the size of VM1 disk from Vsphere console then from Windows Disk Management

that's wrong as well. that does not work.

 So Thin  disk for VM1 even if it is assigned 100 GB and also windows shows it as 100 GB disk size, in reality it is not ?
if the Datastore runs out of space, other VMs will be able to use unused space  on VM1 ?

The Bottom line, actually I do not see the benefit of Thin Disk, if a VM that needs more space can not get it from a VM that has plenty of space.

You keep circling back
Datastore is the repository
The VMs use data from it.
A VM2 does not and can not access the space of another VM for its own storage.

Consider a bench with no arm rests. While it can fit comfortably four people. One can say ten people can use the bench to rest.
Point being, not all ten can use the bench at the same time. While three people can use the bench, they have not taken the time to use it. It does not one of the others can use "their bench time" as you jeep referring on vm2 needing storage space will use vm1's space.

You keep circling back 

that's true

The one thing that has not been mentioned to me is that Datastore can run out of space while VM1 still thinks that it has space. probably true

let me explain to you:

Datastore has 200 GB
VM1 and VM2 both assigned thin disks of 100 GB each

VM2 has used all 100GB
VM1 has used only 20 GB out of its 100GB provisioned

Vmware Administrator can go and extend VM2 thin disk with extra 60 GB , so its size will be 160 GB.
let's say VM2 filled out all 160GB

VM1 now thinks it has 80GB of disk space, let's say it will use another 20GB of space that will make it use  40GB out of 100GB, it thinks it is still has 60GB of space, but it is not.

At this stage VM1 will run out of disk space even though it has used only 40 GB out of initial provisioning of 100GB

Am I correct ?

I guess that's what arnold has also commented

So Thin  disk for VM1 even if it is assigned 100 GB and also windows shows it as 100 GB disk size, in reality it is not ?
if the Datastore runs out of space, other VMs will be able to use unused space  on VM1 ?

It is from a Windows Point of View.

BUT what is used, is what is used on the datastore.

The one thing that has not been mentioned to me is that Datastore can run out of space while VM1 still thinks that it has space. probably true

Well isn't that obvious, if you have a 500GB datastore, and have

6 x VMs all with 100GB Thin Disks.

THAT IS CALLED OVER SUBSCRIPTION!

Same is a true with CPU and MEMORY.

It's called Management and Capacity Planning.

Consider the following.
Two people share a a pair of oxygen tanks. That can allocate oxygen quantity for use by each person.
For matching the example, total volume is 160sq/ft
Person A Is allocated 60sq ft
Person B is allocated 100sq ft

Let's work as through the gauge is reflecting how much oxygen has been used versus reminds.
On the dive, person B has had a respiratory deficiency that flared up in the dive.
Person A consumed  20sq ft
Person B consumed 100 sq ft.

They now need to decide to go back up.
Once the remaining 40sq ft are consumed, person A neighter can survive no matter how little Person A used from their allocated portion.

Back to the VM, the VM1 being allocated 100GB of which it is currently consuming 20Gb
While VM2 located 100 GB consuming all of it. And now has been extended or an additional 60gb for a total of 160gb

As long s the datastore has free space the VMs can use the space thinly provisioned to the,.
Once the datastore runs out because vm2 required more space that speced for such that after several expansions, VM1 remains using 25Gb vm2 consumed the additional 60Gb and was allocated another 40gb for the new total size of 200GB for vm2
Once vm2 hit 175Gb both VMs crashed on the next attempt to use ace no longer available in the datatore.

You keep circling back
that's true

The one thing that has not been mentioned to me is that Datastore can run out of space while VM1 still thinks that it has space. probably true

let me explain to you:

Datastore has 200 GB
VM1 and VM2 both assigned thin disks of 100 GB each

VM2 has used all 100GB
VM1 has used only 20 GB out of its 100GB provisioned

Vmware Administrator can go and extend VM2 thin disk with extra 60 GB , so its size will be 160 GB.
let's say VM2 filled out all 160GB

VM1 now thinks it has 80GB of disk space, let's say it will use another 20GB of space that will make it use  40GB out of 100GB, it thinks it is still has 60GB of space, but it is not.

At this stage VM1 will run out of disk space even though it has used only 40 GB out of initial provisioning of 100GB

Am I correct ?

I guess that's what arnold has also commented

Yes, you are correct, because you have overprovisioned BEYOND what the datastore is capable of providing.

Thanks arnold 
I guess I got.

It is all from the Datastore Perspective....if it runs out of space, then even if VM1 has used 1 Byte out 100 GB, it will still run out of disk space and crashes.

**Hopefully there is a tool that gives you the trend of each VM, that tells you how fast a VM is using the datastore space.

Correct.

When the VMDK cannot grow anymore to it's defined maximum, e.g. 100GB.

Test it and try it and see it for yourself.

That's why we have Labs. Thoery is all fine, but it's the practice we can actually see things fail. By Applying what we learn.

When doing “thin” provisioning VM just mark the maximum possible size for the VMDK.  In Windows the VM BIOS or UEFI report to OS the maximum size of the disk device.  When Windows initialise, allocate and format the disk, it just write the file table of the volume thus no actual data had been written to the data block.  That’s why you cannot see big changes on VMDK size since only file table was changed not all data blocks content.