• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 2996
  • Last Modified:

ESXI 4.1 Reported wrong amount of free space on datastore.

Hello Experts.

A Vmware ESXi 4.1 is reporting 264GB (bug?) of free space

One of the vm's run out of disk space and stopped.

How can i get the true free space?

Free space
0
bastouw
Asked:
bastouw
  • 11
  • 10
  • 2
1 Solution
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
login into the console or remotely via ssh

and use

df

or vdf

if you re-fresh of rescan the storage these values may change and update.
0
 
SebastianAbbinantiCommented:
The VM could have also hit a hard max if you are using thin provisioning. The VMDK will not automatically grow beyond its configured size. So if the VM actually used up all the allocated space, this could be the issue.

Thanks,
S.
0
 
bastouwAuthor Commented:
df -h is reporting 78GB free, and after "refresh" is changed to 78GB as well.
is auto-refresh possible?

SebastianAbbinanti:
The virtual disks are thick provisioned, to avoid this problem i just run into.
It run out of disk space due to temp snapshot's while running veeam replication.

Is it possible to set a treshhold of some sort?, i would rather have a crashed backup/replication than a crashed production server.
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
it will auto refresh over time, and catchup!

sometimes datastore changes are not immediate.

you can set alerts on the datastore to provide alerts and snapshot alerts.

Make sure you have enough free space on the datastore for active snashots.

also checkout my EE article on Snapshots and why they are evil...

http://www.experts-exchange.com/Software/VMWare/A_10300-HOW-TO-VMware-Snapshots-Be-Patient.html
0
 
bastouwAuthor Commented:
While browsing the datastoresas, i see a huge amound of snapshotfiles.

Veeam is running at the moment,
and the snapshot manager is reporting none for this vm.
in other vm's i can see the snapshot files veeam is using.

and the amount seems silly, nobody is using the  vm's right now, is correct or is something going wrong here?

snapshot files
0
 
bastouwAuthor Commented:
I run into some major problems in the past with snapshots, especially when they are nested.
but veeam is generating the snapshot files at the moment.

i will read the article hanccocka.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
wait for ALL backups to finish, and then post back.
0
 
bastouwAuthor Commented:
Can take a couple of days as this is the initial replication of a 720GB vm.
The free space is slowly decreasing,  i can free up another 20GB bij moving a vm to the other datastore.

but i cannot afford to run into problems, but lets cross that bridge when i there.
0
 
SebastianAbbinantiCommented:
If this disk itself is a good state, you can clone the VM. This will a clone with all of the snapshots consolidated. If for no other reason, this could give you some piece of mind.

Thanks,
S.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
You need to work out if the VM is currently running on Snapshot, or running on a snapshot because of Veeam Backup and Replication Job.

either way if you have very low datastore space, you run the risk of the VM stopping because it runs out of disk space.

a few solutions

1. increase datastore space
2. move VM to a new datastore with more space
3. alter configuration to relocate snapshots to a snapshot datastore.
0
 
bastouwAuthor Commented:
There is a veeam replication running for 55 hours at the moment, it's for 70% complete.
The free space is slowly decreasing and wend from 78GB to 68GB right now.

do you advice to abort the replication and make more space first?
or let the job continue and abort if free < 20GB?

what minimum space do you recommend anyway?

I give the veeam job info also

veeam job info
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
How many VMs do you have on the datastore?

you really should have a working free space of at least 20% of the datastore.

It depends on how "busy" your server is, because the snapshot changes are changes occuring in the VM, whilst the backup/replication is occuring.

how long did it take to use 10GB ?
0
 
bastouwAuthor Commented:
The 10GB was is 48 hours aprox, but this was in the weekend, and now it's monday and this will go faster.

10:19 = 65.73GB
12:19 = 65.00GB

so 730MB in 2 hours will be approx 2.92GB what will predict 63.11GB at the end of the working day.

There are 5 VM

1x 20GB + 1x 2TB (satastore) XP WS for Veeam, no snapshot
1x 20GB XP test, no snapshot
1x 50GB production (not busy), no snapshot
1x 60GB production (not busy), no snapshot
1x 720GB production (busy), snapshots (we are talking about this one)
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
I think you will have to monitor very carefully, stopping the Veeam Replication job, may cancel the snapshot disk.

but the real issue is the lack of datastore space.
0
 
bastouwAuthor Commented:
my plan is indeed to monitor carefully and abort if space go below 30GB.

(long ago i could only dream of having 30GB storage, where did time go?) :)
0
 
bastouwAuthor Commented:
The job completed successfully, at the lowest point the free space was 50GB
After deleting all the snapshots it is 274GB (=22.8% on 1200GB)

so in theory all is well for now.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
what you need to work out is time to take a backup/replication versus how busy the server is at that time.

it could be beneficial to move the replication time out of hours, otherwise if not possible increase the datastore space, move snapshot to new datastore.
0
 
bastouwAuthor Commented:
Does create delete snapshots in a replication cycle have a lot of impact on the day to day performance?,

I am aware that the topic is slowly changing from the original question.

I can also close this one and ask this in a other question, if you prefer.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Yes, when deleting a snapshot, this can cause a delay to network communications to the VM.

It's still sort of snapshot related.....if it goes too far off, I'll let you know!

Andy
0
 
bastouwAuthor Commented:
I will have to ask the people off the company it it is acceptable or not, i have already put a QoS in place what limits the upload traffic during office hours, that works well, but the short freeze ups, i must ask them in a couple of days.

There is a "sensitive" database connection with a client that must keep running all the time
and there are no disconnects between server and client, and i know for a fact that that one is really really sensitive for even the shortest of hick ups, but that keeps connected, so vmware is doing a good job there.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
We ALWAYS complete and recommend replication out of hours, or busy times, and I know this is difficult in 24/7 environments, but Snapshots, Creation or Deletion can cause Exchange/File Shares to Go Offline!
0
 
bastouwAuthor Commented:
Thanks a lot for the help.

I close this question now.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
No problems, keep a check on your snaps and storage space.
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

  • 11
  • 10
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now