DSTechTeam
asked on
VMWare Datastore not accurately reflecting Free Space in regards to Snapshots.
Our backup takes what appear to be custom snapshots that do not get listed within Snapshot Manager. One of our clients went down Sunday and while troubleshooting it I found that there was 1TB of free space listed on the Datastore, but in reality 3 of the VMs had become stunned due to running out of space completely. There were 46 snapshots in the folder for one of the VMs. Though I resolved this issue by creating a 0 space snapshot and then running a consolidation, I'm STILL left with the question:
VMWare Version 5.5.0
Why is the Datastore not accurately reflecting Free Space in regards to snapshots?
VMWare Version 5.5.0
Why is the Datastore not accurately reflecting Free Space in regards to snapshots?
ASKER CERTIFIED SOLUTION
membership
Create a free account to see this answer
Signing up is free and takes 30 seconds. No credit card required.
ASKER
It was running slow, yes. That much is clear but this is off topic.
These snapshots are generated by the backup program acting erroneously as the VMWare server was telling the program that it could not Quiesce but was actually creating snapshots successfully anyway. However that is not the point of this post.
@Mr Tortur
Storage is directly located in the server. Yes I did to a refresh and we also have an external monitor in Labtech that did also not correctly report the size as being completely full.
@Andrew
The base size is about 50% of the capacity of the Datastore's total size: 2.18 TB total and 1.08TB free (according to the Datastore's read.) Ultimately 50% of the space was basically taken up by snapshots of a single VM on the server but never reflected - which is crazy.
Problem is that when it quit on me on Sunday it had reported nothing wrong but had 46 snapshots that were created as a result of another issue but they in fact did live on the Datastore and were not being accounted for. One created each day for the last month and a half, and the Storage never reported all that space being taken.
Is that Free Space number only accounting for space not being Virtual Drive provisioned - not Snapshots? Because that's basically how it is acting. Essentially I return to my main question of
Why is the Datastore not accurately reflecting Free Space in regards to snapshots?
These snapshots are generated by the backup program acting erroneously as the VMWare server was telling the program that it could not Quiesce but was actually creating snapshots successfully anyway. However that is not the point of this post.
@Mr Tortur
Storage is directly located in the server. Yes I did to a refresh and we also have an external monitor in Labtech that did also not correctly report the size as being completely full.
@Andrew
The base size is about 50% of the capacity of the Datastore's total size: 2.18 TB total and 1.08TB free (according to the Datastore's read.) Ultimately 50% of the space was basically taken up by snapshots of a single VM on the server but never reflected - which is crazy.
Problem is that when it quit on me on Sunday it had reported nothing wrong but had 46 snapshots that were created as a result of another issue but they in fact did live on the Datastore and were not being accounted for. One created each day for the last month and a half, and the Storage never reported all that space being taken.
Is that Free Space number only accounting for space not being Virtual Drive provisioned - not Snapshots? Because that's basically how it is acting. Essentially I return to my main question of
Why is the Datastore not accurately reflecting Free Space in regards to snapshots?
Datastore space should be correct, sometimes however the vCenter Server inventory does not update correctly, requiring a refresh, or restart.
All provisioned space, should be reported correctly. (Server virtual disks and snapshots as well)
Also is this local storage of shared storage.
Without screenshots, it is difficult to diagnose.
Are you running latest versions of VMware vSphere 5.5? (vCenter Server and Host OS)
All provisioned space, should be reported correctly. (Server virtual disks and snapshots as well)
Also is this local storage of shared storage.
Without screenshots, it is difficult to diagnose.
Are you running latest versions of VMware vSphere 5.5? (vCenter Server and Host OS)
ASKER
So bar the refresh which did not change anything I'm to believe that VMWare is reliable in reporting (which is serious) for less than 105 days? (Which is the current uptime.)
Nope, just local storage.
I'm sorry I was unclear:
This a server on it's own - dedicated local storage.
VMWare ESXi 5.5.0 2068190.
And I fail to see how it should matter at all but 2008 R2.
I do not have screenshots of when it was full completely, unfortunately as my concern was getting the company back online. My focus is now to figure out WHY the space changes were not reflected and as far as I know Labtech Virtualization Manager does a refresh every time it accesses the VMWare server on the storage to see the reflected size changes.
Nope, just local storage.
I'm sorry I was unclear:
This a server on it's own - dedicated local storage.
VMWare ESXi 5.5.0 2068190.
And I fail to see how it should matter at all but 2008 R2.
I do not have screenshots of when it was full completely, unfortunately as my concern was getting the company back online. My focus is now to figure out WHY the space changes were not reflected and as far as I know Labtech Virtualization Manager does a refresh every time it accesses the VMWare server on the storage to see the reflected size changes.
Is the datastore space being reported correctly now ?
Labtech Virtualization Manager probably only reads the information offered by vCenter Server.
Are vCenter Server and Labtech are consistent.
Your version of ESXi 5.5, is U2, not certain of vCenter Server, as you do not mention, but that was released in September 2014, there have been 10 releases since then, so it's old and not up to date.
I would recommend, a more update version, if this is Production.
Labtech Virtualization Manager probably only reads the information offered by vCenter Server.
Are vCenter Server and Labtech are consistent.
Your version of ESXi 5.5, is U2, not certain of vCenter Server, as you do not mention, but that was released in September 2014, there have been 10 releases since then, so it's old and not up to date.
I would recommend, a more update version, if this is Production.
ASKER
It does look like it's reporting correctly now. I'm wondering if it wasn't refreshing automatically, even though it looked like it had.
I suppose we can call it for an update at this point for the client - do you know if there was a Storage refresh issue fix since any of the releases off-hand?
I suppose we can call it for an update at this point for the client - do you know if there was a Storage refresh issue fix since any of the releases off-hand?
There have been many many issues fixed with hosts and vCenter Server, over the last 15 months, and there have been updates to the Inventory Service.
ASKER
I found the cause.
The Storage refresh never occurs - UNLESS you have a vCenter server, which forces the refresh. Anything without a vCenter does not auto-refresh.
This is pretty critical to know. I will not be forgetting this anytime soon. I appreciate your help Andrew, and will try to get that host updated as a matter of course regardless.
The Storage refresh never occurs - UNLESS you have a vCenter server, which forces the refresh. Anything without a vCenter does not auto-refresh.
This is pretty critical to know. I will not be forgetting this anytime soon. I appreciate your help Andrew, and will try to get that host updated as a matter of course regardless.
What type of storage are you using? SAN/NAS snapshots may use the space allocated for the VMs, especially if you don't manage them and/or delete VMs that are still being held by the snapshot and not showing up on the datastore but are still part of LUN/Volume
So you are not using or have vCenter Server to configure Alerts on datastore usage ?
Hi,
ok, well, good to know.
So if I summarize, you don't have a vcenter, only a standlone host, and when you refresh it manually it is displaying the good space stats, but it does not make auto-refresh on its own.
ok, well, good to know.
So if I summarize, you don't have a vcenter, only a standlone host, and when you refresh it manually it is displaying the good space stats, but it does not make auto-refresh on its own.
But a few pointers the datastore should be at least 20-30% free to allow for snapshots which backup apps use.
Secondly 46 snapshots also means 46 days whilst a VM is running on a snapshot performance of a VM will be poor.
VMware admins need to check daily for left over snapshots.
Otherwise all these snapshots will cause a deterioration in performance of the datastore.