VMWare There is no more space for virtual disk

Sounds like you have run out of physical space on a hard drive.. in the start menu type in disk cleanup and see if it can auto find you some space to clear.  I like to use Sequoia View to visually see what is taking up space on my hard drive to see what I can delete that is the largest.. but if you're outta space you may not be able to download/install it... http://download.cnet.com/SequoiaView/3000-2248_4-10549662.html

I was able to power off the VM but it won't let me power on, it comes back with the same error.
I do have around 4TB of space in the Datastore.

Please help!!!

We answered this before, and you deleted the question!

You have three snapshots associated with this virtual machine, when a virtual machine writes to a snapshot file, it grow and grows, and grows.... you get it.... until the datastore is FULL, and then the VM fails. Snapshots are missed because they are left by virtual machines, after backup applications, or you have forgotten, after using manual snapshots.

Also VMs run poorly, e.g. performance is poor when on a snapshot.

We can work with you, to get this issue solved...in the meantime, have a read of snapshots in my EE Article, and hopefully when I submit this time, the questions is not deleted!

see my EE Article

HOW TO: VMware Snapshots :- Be Patient

What I need from you, is to follow my instructions EXACTLY and do not stray of the path, and meddle, or mess around, otherwise corruption can occur, if not done already.

1. Do you have any other datastores ?

2. Do you have other VMs which share the same datastore ?

we need some screenshots of the datastore, storage space, and a screenshot of te error message, remember WE CANNOT see what you are seeing, we need to ask you!

screenshots would help.

@Steve .... this issue is occurring on the VMware vSphere ESXi host, not in the VM.

Yes, only one datastore
Yes, many VMs sharing the datastore

I have 3 snapshots for this VM.

I KNOW you have 3 snapshots. (you deleted the other question! before I could respond!)

SCREENSHOTS PLEASE of datastore and storage space.

I need to see the storage space, left, with a single datastore, how much is FREE.

So, what do I do, please help!!!

Ok, schreenshots are coming

okay, good, let me have the screenshots, and I can advise. Be patient, I'll hold you hand, but please do not mess, or fiddle, and make the situation worse.

Datastore-summary.JPG

Datastore-contents.JPG

Now I'm waiting for your comments, let me know if you need more information.
Thanks.

this is a little odd, because it does state 4.5TB free.

can you screenshot the error message, and also the settings of the VM Disk, in the VM.

So this is a vSAN datastore ?

what is the capacity of the datastore ?

it states, 40TB, but 75TB provisioned, are you using Thin Virtual Machine disks, and you've overprovisioned your datastore ?

and now this is a combination snapshot and thin provision.

The virtual disk in question is 9TB in size, and the users have used about 5TB.
Take a look at the Datastore contents and you will see the 9TB file there.
error.JPG

Yes this is a vSAN.
40TB in capacity.

can we check the disk settings, to check it's a snapshot.

e.g. Right Click the VM Edit settings, and select the disk, and lets check,

are these thin disks ?

okay, let me catch up and check the screenshots

Settings.JPG

Can I unmount the problematic virtual disk, and start the server?
I have a whole bunch of other information there that people need to access.

Andrew, any ideas?

I have a whole bunch of other information there that people need to access.

I'm afraid at present you are DOWN, and you will need to advise them of that.

okay, we need to deal with the snapshots here....

now, snapshot merging (deleting) can take many seconds, minutes, hours or days, depending upon the size and speed of your datastore. You will need to be very patient.

1. do not cancel the operation.
2. do not reboot the host servers.
3. do not power on the VM.
4. do not restart networking agents
5. do not restart vCenter Server.
6. basically do not do anything other than walk away, grab a coffee, tea, red bull....

and BE PATIENT

it will possibly sit at 90, 95, 99% and look like it's hung BUT it will complete....

You need to free up space on the datastore.

1. there is a folder called isos, can we move it or delete it some where, move to local PC etc

is there any other folders we can delete or move, storing isos on vSAN is expensive usage of stoarge ?

Yes, I can delete the isos folder.

do I do that?

Andrew please help me out man, I'm having a heart attack here.

Click the isos folder, and remove all the files, right click and delete each file.

then remove the folder, again right click delete.

I just tried but it said that it can't delete the iso file.
isoerror.JPG

you will have to check, if it is connected to any VMs, and disconnect and then delete.

I just check and release the access to this iso file.
But I'm still can't delete it.
This is how the other VMs configuration looks like.
DVD.JPG

But why is it saying that it has 4.5TB of Free space, I don't understand.

I have a Virtual Disk in a different VM that we could remove, and that would free some space.
But you will need to guide me through that process.

Does this sound like a good alternative to free up some space?

Anything that can release space, will help.

Lets not worry about what things state, we need free-up space....to start the snapshot process.

Andrew,
I really need some other users to access information in other virtual disks I have on that server.
Is it an option to disable the virtual disk that is full, so that I can restart the server?

I can also see that in the image attached to this message the VM has reached the 17.19TB of space it has available.
Is it an option to grow the virtual disk in question so that it can let me start the VM and then I can delete a lot of files I know I can delete?

17TB.JPG

The snapshots need to be handled correctly, deleting files in he VM is not going to help the snapshot.

The snapshot MUST be dealt with.

I'm afraid all your disks on this server, have snapshots attached.

Even if you could remove a disk, to get it started it's only a matter of time, before it would fail again, because it could not write to disk because of the snapshot, and the danger is you could the  LOSE ALL or the data in the VM could become corrupted.

Did you delete the VM, and free up some space ? and we can proceed.

The quicker you free up space, the quicker we can start the snapshot merge.

What if I go to the snapshot manager and delete the snapshots from all the VMs from there, would that help?

In the Snapshot manager there is also an option that says consolidate.
Wouldn't that delete some of the snapshots?

Okay, please follow my instructions.

did you free up space on the datastore.

In the Snapshot manager there is also an option that says consolidate.
Wouldn't that delete some of the snapshots?

Possibly, I did post originally, do not stray off the path and follow my instructions, otherwise you will end up in a world, you do not want to be!

What if I go to the snapshot manager and delete the snapshots from all the VMs from there, would that help?

FREE UP SOME SPACE.

Did you delete the VM and free space ?

Andrew,
I just removed a virtual disk with a 1TB size.
How can we check if we have enough space now?

Question:
I went to the machine edit settings and click on removed the 1TB virtual disk that I was not using.
But when I go to the browse the data store it seems like the vmdk files are still there, should I remove those vmdk files by hand?
1TB-VDFREE.JPG

Also shouldn't I delete just one or 2 snapshots first?
I would like to have at least 1 snapshot left.

Delete the virtual machine disks manually, you probably did not select delete from disk.

Snapshot are NOT BACKUPS.

REMOVE THEM ALL. DELETE ALL.

What are you going to do with them ALL, rollback the server and lose all your data ?

Ok, I have never deleted vmdk files, so I need your help to pinpoint the ones that need to be deleted.
From the image attached to this message I think I have to delete the following files:

YYCDP-FS-01-2.vmdk
YYCDP-FS-01-2-000001.vmdk
YYCDP-FS-01-2-000002.vmdk

This is because the virtual drive deleted was 1TB in size and there are no other 1TB virtual drives in this server.

Am I correct in the file identification?
1TB-VDFREE.JPG

If that is the assumption you are making then those are the files that need deleting, the 0001 and 0002 are snapshots.

You don't use Snapshots are Backups do you because there seem to snapshots on this VM ?

this is very bad practice, I'm not surprised you are running out of storage space, and have used up 40TB.

Ok, I have deleted the vmdk files in question.
Now I have to go into the snapshots and delete them all, as per your instructions above.

Correct?

That is correct DELETE ALL.

Make sure you select DELETE ALL.

Ok, DELETE ALL snapshots it is.
The process just began,
Wish me luck

It is at 25%

It is still at 25%.
How long does this normally take?

now, snapshot merging (deleting) can take many seconds, minutes, hours or days, depending upon the size and speed of your datastore. You will need to be very patient.

But 25% is quite fast...

and 25% does not mean it's not doing anything, as the indicator is a bit rubbish.

Have you been manually snapshotting ALL your VMs ?

It is now at 26%
So much for the very fast.

But it is still going.

It is now at 50%
While this process completes, I will open a new case to discuss snapshots. Can you take that case?

Yes, see the Progress Bar is really rubbish, if it's jumping fro 25%, 26%, 50%...

If you post another question myself or other experts will be glad to assist.

It is still at 50%.
Should I also delete all the snapshots from the other VMs?

I just created a new question to discuss snapshots, please feel free to join in.

It is at 52%
You are right we don't know when this is going to end.

You should really know how and when to use these, you are not the first VMware Admin, that's been caught by them, or the last.

We would recommend , not use them and backup first, and then apply changes.

You do have BACKUPS of your VMs ?

What do you use ?

If you are going to use them, Snapshot - Apply Patches - Patches Worked - Delete Snapshot.

This LONG delay is caused BY YOU, leaving the VM on a snapshot too long, and it's going to take this length of time, possibly to fix ALL your VMs.

and you've wasted lots of space, and huge overhead on your SAN, with all these snapshots.

and performance will be poor.

You are right, Mea Culpa.

It's at 55%
I just hope the server starts when this snapshot deleting finishes.

The merging operation is removing ALL the DELTA Snapshots from the vSAN SAN, and free-up the storage.

and you should do this WITH ALL YOUR VMs.

But this will allow the server to start normally.

Got stuck at 55%

Question:
If I close the vSphere client, will the process running in the server?
DELETESNAP55.JPG

it will appear to stick.

the process is completing in the back ground on the server, you don't need the client open.

Still at 55%
How do I know if it is still running or if it got stuck?

Be Patient.

Look at the files in the VM Folder.

Can I click "close" in the Snapshot Manager without affecting the process?
SNAPMANAGER.JPG

Yes, you can click close.

the process is completing in the background on the server.

Very bad use of Snapshots!

Have you not noticed your servers getting slower.....

I just look into the datastore and it looks like it is processing the heavy heaters, snapshot files for the 5TB and 9TB virtual disks.

Some of the 00000X.vmdk files from the small virtual drives are gone, so I guess that was the supposed 55% of the process.

My estimation is that this is going to take at least all night.

Should I be able to see the size of any of these files going down during the process?
VMDKFILELIST.JPG

not really.

again, it's best to walk away, and not watch it!

Do you not have a Backup program for VMs ?

What do you use to Backup VMs ?

I'm about to disappear to my bed, I'll be back later.

Andrew,
I'm going home, watching 55% not moving is torture.
I will connect remotely and I'll let you know if the process status changed.

Thanks.

It's at 76%
I check the files and it is now processing the last virtual drive "the big one" 9 TB

Question:
Once the process ends successfully, I can just try to power on the VM, correct?

Question:
Once the process ends successfully, I can just try to power on the VM, correct?

Yes, you can power on, you will find, all the snapshots should have gone, and you've got much storage space back.

and remember you other VMs, could also be at risk of failure, because of the Snapshots.

Good Morning Andrew,
The server is back on, thanks for everything.
:-)

For the other servers, same procedure?

1. power off
2. Snapshot manager
3. Delete all
4. Be patient

Mr. Andrew,
I will take care of the snapshots in the other VMs.
I would probably gone crazy without your guidance by now, so thank you again for your patience.

Best Regards,
Cargex

Hey I'm glad your back in business.

For the other servers, same procedure?

- almost

1. power off
2. Snapshot > Take Snapshot
2a. Wait 1-2 minutes
3. Snapshot Manager and Then - Delete all
4. Be patient