Link to home
Create AccountLog in
VMware

VMware

--

Questions

--

Followers

Top Experts

Avatar of cargex
cargex

VMWare There is no more space for virtual disk
Guys,
I'm getting this error.
And when I try to power off the machine it doesn't let me because of the same message.

I try to access via console and nothing it doesn't let me enter the console same error.

So what do I do?

Zero AI Policy

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


Avatar of Steve AgnewSteve Agnew🇺🇸

Sounds like you have run out of physical space on a hard drive.. in the start menu type in disk cleanup and see if it can auto find you some space to clear.  I like to use Sequoia View to visually see what is taking up space on my hard drive to see what I can delete that is the largest.. but if you're outta space you may not be able to download/install it... http://download.cnet.com/SequoiaView/3000-2248_4-10549662.html

Avatar of cargexcargex

ASKER

I was able to power off the VM but it won't let me power on, it comes back with the same error.
I do have around 4TB of space in the Datastore.

Please help!!!

We answered this before, and you deleted the question!

You have three snapshots associated with this virtual machine, when a virtual machine writes to a snapshot file, it grow and grows, and grows.... you get it.... until the datastore is FULL, and then the VM fails. Snapshots are missed because they are left by virtual machines, after backup applications, or you have forgotten, after using manual snapshots.

Also VMs run poorly, e.g. performance is poor when on a snapshot.

We can work with you, to get this issue solved...in the meantime, have a read of snapshots in my EE Article, and hopefully when I submit this time, the questions is not deleted!

see my EE Article

HOW TO: VMware Snapshots :- Be Patient

What I need from you, is to follow my instructions EXACTLY and do not stray of the path, and meddle, or mess around, otherwise corruption can occur, if not done already.

1. Do you have any other datastores ?

2. Do you have other VMs which share the same datastore ?

we need some screenshots of the datastore, storage space, and a screenshot of te error message, remember WE CANNOT see what you are seeing, we need to ask you!

screenshots would help.

@Steve .... this issue is occurring on the VMware vSphere ESXi host, not in the VM.

Reward 1Reward 2Reward 3Reward 4Reward 5Reward 6

EARN REWARDS FOR ASKING, ANSWERING, AND MORE.

Earn free swag for participating on the platform.


Avatar of cargexcargex

ASKER

Yes, only one datastore
Yes, many VMs sharing the datastore

Avatar of cargexcargex

ASKER

I have 3 snapshots for this VM.

I KNOW you have 3 snapshots. (you deleted the other question! before I could respond!)

SCREENSHOTS PLEASE of datastore and storage space.

I need to see the storage space, left, with a single datastore, how much is FREE.

Free T-shirt

Get a FREE t-shirt when you ask your first question.

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


Avatar of cargexcargex

ASKER

So, what do I do, please help!!!

Avatar of cargexcargex

ASKER

Ok, schreenshots are coming

okay, good, let me have the screenshots, and I can advise. Be patient, I'll hold you hand, but please do not mess, or fiddle, and make the situation worse.

Reward 1Reward 2Reward 3Reward 4Reward 5Reward 6

EARN REWARDS FOR ASKING, ANSWERING, AND MORE.

Earn free swag for participating on the platform.


Avatar of cargexcargex

ASKER

see attached image
Datastore-summary.JPG

Avatar of cargexcargex

ASKER

see attached image
Datastore-contents.JPG

Avatar of cargexcargex

ASKER

Now I'm waiting for your comments, let me know if you need more information.
Thanks.

Free T-shirt

Get a FREE t-shirt when you ask your first question.

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


this is a little odd, because it does state 4.5TB free.

can you screenshot the error message, and also the settings of the VM Disk, in the VM.

So this is a vSAN datastore ?

what is the capacity of the datastore ?

it states, 40TB, but 75TB provisioned, are you using Thin Virtual Machine disks, and you've overprovisioned your datastore ?

and now this is a combination snapshot and thin provision.

Avatar of cargexcargex

ASKER

The virtual disk in question is 9TB in size, and the users have used about 5TB.
Take a look at the Datastore contents and you will see the 9TB file there.
error.JPG

Avatar of cargexcargex

ASKER

Yes this is a vSAN.
40TB in capacity.

Reward 1Reward 2Reward 3Reward 4Reward 5Reward 6

EARN REWARDS FOR ASKING, ANSWERING, AND MORE.

Earn free swag for participating on the platform.


can we check the disk settings, to check it's a snapshot.

e.g. Right Click the VM Edit settings, and select the disk, and lets check,

are these thin disks ?

okay, let me catch up and check the screenshots

Avatar of cargexcargex

ASKER

see attached image of settings
Settings.JPG

Free T-shirt

Get a FREE t-shirt when you ask your first question.

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


Avatar of cargexcargex

ASKER

Can I unmount the problematic virtual disk, and start the server?
I have a whole bunch of other information there that people need to access.

Avatar of cargexcargex

ASKER

Andrew, any ideas?

I have a whole bunch of other information there that people need to access.

I'm afraid at present you are DOWN, and you will need to advise them of that.

okay, we need to deal with the snapshots here....

now, snapshot merging (deleting) can take many seconds, minutes, hours or days, depending upon the size and speed of your datastore. You will need to be very patient.

1. do not cancel the operation.
2. do not reboot the host servers.
3. do not power on the VM.
4. do not restart networking agents
5. do not restart vCenter Server.
6. basically do not do anything other than walk away, grab a coffee, tea, red bull....

and BE PATIENT

it will possibly sit at 90, 95, 99% and look like it's hung BUT it will complete....

You need to free up space on the datastore.

1. there is a folder called isos, can we move it or delete it some where, move to local PC etc

is there any other folders we can delete or move, storing isos on vSAN is expensive usage of stoarge ?

Reward 1Reward 2Reward 3Reward 4Reward 5Reward 6

EARN REWARDS FOR ASKING, ANSWERING, AND MORE.

Earn free swag for participating on the platform.


Avatar of cargexcargex

ASKER

Yes, I can delete the isos folder.

Avatar of cargexcargex

ASKER

do I do that?

Avatar of cargexcargex

ASKER

Andrew please help me out man, I'm having a heart attack here.

Free T-shirt

Get a FREE t-shirt when you ask your first question.

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


Click the isos folder, and remove all the files, right click and delete each file.

then remove the folder, again right click delete.

Avatar of cargexcargex

ASKER

I just tried but it said that it can't delete the iso file.
isoerror.JPG

you will have to check, if it is connected to any VMs, and disconnect and then delete.

Reward 1Reward 2Reward 3Reward 4Reward 5Reward 6

EARN REWARDS FOR ASKING, ANSWERING, AND MORE.

Earn free swag for participating on the platform.


Avatar of cargexcargex

ASKER

I just check and release the access to this iso file.
But I'm still can't delete it.
This is how the other VMs configuration looks like.
DVD.JPG

Avatar of cargexcargex

ASKER

But why is it saying that it has 4.5TB of Free space, I don't understand.

Avatar of cargexcargex

ASKER

I have a Virtual Disk in a different VM that we could remove, and that would free some space.
But you will need to guide me through that process.

Does this sound like a good alternative to free up some space?

Free T-shirt

Get a FREE t-shirt when you ask your first question.

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


Anything that can release space, will help.

Lets not worry about what things state, we need free-up space....to start the snapshot process.

Avatar of cargexcargex

ASKER

Andrew,
I really need some other users to access information in other virtual disks I have on that server.
Is it an option to disable the virtual disk that is full, so that I can restart the server?

Avatar of cargexcargex

ASKER

I can also see that in the image attached to this message the VM has reached the 17.19TB of space it has available.
Is it an option to grow the virtual disk in question so that it can let me start the VM and then I can delete a lot of files I know I can delete?

Reward 1Reward 2Reward 3Reward 4Reward 5Reward 6

EARN REWARDS FOR ASKING, ANSWERING, AND MORE.

Earn free swag for participating on the platform.


Avatar of cargexcargex

ASKER

see attached image
17TB.JPG

The snapshots need to be handled correctly, deleting files in he VM is not going to help the snapshot.

The snapshot MUST be dealt with.

I'm afraid all your disks on this server, have snapshots attached.

Even if you could remove a disk, to get it started it's only a matter of time, before it would fail again, because it could not write to disk because of the snapshot, and the danger is you could the  LOSE ALL or the data in the VM could become corrupted.

Did you delete the VM, and free up some space ? and we can proceed.

The quicker you free up space, the quicker we can start the snapshot merge.

Avatar of cargexcargex

ASKER

What if I go to the snapshot manager and delete the snapshots from all the VMs from there, would that help?

Free T-shirt

Get a FREE t-shirt when you ask your first question.

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


Avatar of cargexcargex

ASKER

In the Snapshot manager there is also an option that says consolidate.
Wouldn't that delete some of the snapshots?

Okay, please follow my instructions.

did you free up space on the datastore.

In the Snapshot manager there is also an option that says consolidate.
Wouldn't that delete some of the snapshots?

Possibly, I did post originally, do not stray off the path and follow my instructions, otherwise you will end up in a world, you do not want to be!

What if I go to the snapshot manager and delete the snapshots from all the VMs from there, would that help?

FREE UP SOME SPACE.

Did you delete the VM and free space ?

Avatar of cargexcargex

ASKER

Andrew,
I just removed a virtual disk with a 1TB size.
How can we check if we have enough space now?

Reward 1Reward 2Reward 3Reward 4Reward 5Reward 6

EARN REWARDS FOR ASKING, ANSWERING, AND MORE.

Earn free swag for participating on the platform.


ASKER CERTIFIED SOLUTION
Avatar of Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)🇬🇧

Link to home
membership
Log in or create a free account to see answer.
Signing up is free and takes 30 seconds. No credit card required.
Create Account

Avatar of cargexcargex

ASKER

Question:
I went to the machine edit settings and click on removed the 1TB virtual disk that I was not using.
But when I go to the browse the data store it seems like the vmdk files are still there, should I remove those vmdk files by hand?
1TB-VDFREE.JPG

Avatar of cargexcargex

ASKER

Also shouldn't I delete just one or 2 snapshots first?
I would like to have at least 1 snapshot left.

Delete the virtual machine disks manually, you probably did not select delete from disk.

Snapshot are NOT BACKUPS.

REMOVE THEM ALL. DELETE ALL.

What are you going to do with them ALL, rollback the server and lose all your data ?

Free T-shirt

Get a FREE t-shirt when you ask your first question.

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


Avatar of cargexcargex

ASKER

Ok, I have never deleted vmdk files, so I need your help to pinpoint the ones that need to be deleted.
From the image attached to this message I think I have to delete the following files:

YYCDP-FS-01-2.vmdk
YYCDP-FS-01-2-000001.vmdk
YYCDP-FS-01-2-000002.vmdk

This is because the virtual drive deleted was 1TB in size and there are no other 1TB virtual drives in this server.

Am I correct in the file identification?
1TB-VDFREE.JPG

If that is the assumption you are making then those are the files that need deleting, the 0001 and 0002 are snapshots.

You don't use Snapshots are Backups do you because there seem to snapshots on this VM ?

this is very bad practice, I'm not surprised you are running out of storage space, and have used up 40TB.

Avatar of cargexcargex

ASKER

Ok, I have deleted the vmdk files in question.
Now I have to go into the snapshots and delete them all, as per your instructions above.

Correct?

Reward 1Reward 2Reward 3Reward 4Reward 5Reward 6

EARN REWARDS FOR ASKING, ANSWERING, AND MORE.

Earn free swag for participating on the platform.


That is correct DELETE ALL.

Make sure you select DELETE ALL.

Avatar of cargexcargex

ASKER

Ok, DELETE ALL snapshots it is.
The process just began,
Wish me luck

Avatar of cargexcargex

ASKER

It is at 25%

Free T-shirt

Get a FREE t-shirt when you ask your first question.

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


Avatar of cargexcargex

ASKER

It is still at 25%.
How long does this normally take?

now, snapshot merging (deleting) can take many seconds, minutes, hours or days, depending upon the size and speed of your datastore. You will need to be very patient.

But 25% is quite fast...

and 25% does not mean it's not doing anything, as the indicator is a bit rubbish.

Have you been manually snapshotting ALL your VMs ?

Avatar of cargexcargex

ASKER

It is now at 26%
So much for the very fast.

But it is still going.

Reward 1Reward 2Reward 3Reward 4Reward 5Reward 6

EARN REWARDS FOR ASKING, ANSWERING, AND MORE.

Earn free swag for participating on the platform.


Avatar of cargexcargex

ASKER

It is now at 50%
While this process completes, I will open a new case to discuss snapshots. Can you take that case?

Yes, see the Progress Bar is really rubbish, if it's jumping fro 25%, 26%, 50%...

If you post another question myself or other experts will be glad to assist.

Avatar of cargexcargex

ASKER

It is still at 50%.
Should I also delete all the snapshots from the other VMs?

I just created a new question to discuss snapshots, please feel free to join in.

Free T-shirt

Get a FREE t-shirt when you ask your first question.

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


Avatar of cargexcargex

ASKER

It is at 52%
You are right we don't know when this is going to end.

You should really know how and when to use these, you are not the first VMware Admin, that's been caught by them, or the last.

We would recommend , not use them and backup first, and then apply changes.

You do have BACKUPS of your VMs ?

What do you use ?

If you are going to use them, Snapshot - Apply Patches - Patches Worked - Delete Snapshot.

This LONG delay is caused BY YOU, leaving the VM on a snapshot too long, and it's going to take this length of time, possibly to fix ALL your VMs.

and you've wasted lots of space, and huge overhead on your SAN, with all these snapshots.

and performance will be poor.

Avatar of cargexcargex

ASKER

You are right, Mea Culpa.

It's at 55%
I just hope the server starts when this snapshot deleting finishes.

Reward 1Reward 2Reward 3Reward 4Reward 5Reward 6

EARN REWARDS FOR ASKING, ANSWERING, AND MORE.

Earn free swag for participating on the platform.


The merging operation is removing ALL the DELTA Snapshots from the vSAN SAN, and free-up the storage.

and you should do this WITH ALL YOUR VMs.

But this will allow the server to start normally.

Avatar of cargexcargex

ASKER

Got stuck at 55%

Question:
If I close the vSphere client, will the process running in the server?
DELETESNAP55.JPG

it will appear to stick.

the process is completing in the back ground on the server, you don't need the client open.

Free T-shirt

Get a FREE t-shirt when you ask your first question.

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


Avatar of cargexcargex

ASKER

Still at 55%
How do I know if it is still running or if it got stuck?

Be Patient.

Look at the files in the VM Folder.

Avatar of cargexcargex

ASKER

Can I click "close" in the Snapshot Manager without affecting the process?
SNAPMANAGER.JPG

Reward 1Reward 2Reward 3Reward 4Reward 5Reward 6

EARN REWARDS FOR ASKING, ANSWERING, AND MORE.

Earn free swag for participating on the platform.


Yes, you can click close.

the process is completing in the background on the server.

Very bad use of Snapshots!

Have you not noticed your servers getting slower.....

Avatar of cargexcargex

ASKER

I just look into the datastore and it looks like it is processing the heavy heaters, snapshot files for the 5TB and 9TB virtual disks.

Some of the 00000X.vmdk files from the small virtual drives are gone, so I guess that was the supposed 55% of the process.

My estimation is that this is going to take at least all night.

Should I be able to see the size of any of these files going down during the process?
VMDKFILELIST.JPG

not really.

again, it's best to walk away, and not watch it!

Do you not have a Backup program for VMs ?

What do you use to Backup VMs ?

Free T-shirt

Get a FREE t-shirt when you ask your first question.

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


I'm about to disappear to my bed, I'll be back later.

Avatar of cargexcargex

ASKER

Andrew,
I'm going home, watching 55% not moving is torture.
I will connect remotely and I'll let you know if the process status changed.

Thanks.

Avatar of cargexcargex

ASKER

It's at 76%
I check the files and it is now processing the last virtual drive "the big one" 9 TB

Question:
Once the process ends successfully, I can just try to power on the VM, correct?

Reward 1Reward 2Reward 3Reward 4Reward 5Reward 6

EARN REWARDS FOR ASKING, ANSWERING, AND MORE.

Earn free swag for participating on the platform.


Question:
Once the process ends successfully, I can just try to power on the VM, correct?

Yes, you can power on, you will find, all the snapshots should have gone, and you've got much storage space back.

and remember you other VMs, could also be at risk of failure, because of the Snapshots.

Avatar of cargexcargex

ASKER

Good Morning Andrew,
The server is back on, thanks for everything.
:-)

For the other servers, same procedure?

1. power off
2. Snapshot manager
3. Delete all
4. Be patient

Avatar of cargexcargex

ASKER

Mr. Andrew,
I will take care of the snapshots in the other VMs.
I would probably gone crazy without your guidance by now, so thank you again for your patience.

Best Regards,
Cargex

Free T-shirt

Get a FREE t-shirt when you ask your first question.

We believe in human intelligence. Our moderation policy strictly prohibits the use of LLM content in our Q&A threads.


Hey I'm glad your back in business.

For the other servers, same procedure?
- almost

1. power off
2. Snapshot > Take Snapshot
2a. Wait 1-2 minutes
3. Snapshot Manager and Then - Delete all
4. Be patient
VMware

VMware

--

Questions

--

Followers

Top Experts

VMware, a software company founded in 1998, was one of the first commercially successful companies to offer x86 virtualization. The storage company EMC purchased VMware in 1994. Dell Technologies acquired EMC in 2016. VMware’s parent company is now Dell Technologies. VMware has many software products that run on desktops, Microsoft Windows, Linux, and macOS, which allows the virtualizing of the x86 architecture. Its enterprise software hypervisor for servers, VMware vSphere Hypervisor (ESXi), is a bare-metal hypervisor that runs directly on the server hardware and does not require an additional underlying operating system.