Hyper-V running out of space while trying to create a snapshot.

HyperV ManagerSnapshotVMs paused-Critical stateHi,Virtual Hard Disk FolderVM stuck during shutdownOptions under Action menu
 I accidently created a snapshot on the VM "FT1" and there are two problems:
(1) Every so many seconds, when I open this VM in Hyper-V manager, it goes to "pause" state showing the screen in shaded color. When I start it, it goes back to shaded/pause  state.
(2) Now I am getting "Out of Disk" message on my Hyper-V server.

I did not intend to create a snapshot at all. I was expecting a snapshot manager screen as I chose "snapshot". But I think it is trying to create a snapshot and the server ran out of space. It seems that the logical thing to do is to cancel "snapshot creation process" because there is no hard drive space for the snapshot to be created in the first place, delete the temporary file "FT1_2481848F-1B1E-4667-A1A2-985A4E532138.avhdx"(18.9GB), and restart the "FT1" VM.

What should I do?

** I forgot to mention that I was trying to shut down the VM, but that is even stuck.
LVL 1
sgleeAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

sgleeAuthor Commented:
To Moderator,

 I have not seen any response so far. Did I select the right category?

Thanks
David Johnson, CD, MVPRetiredCommented:
remove-vmsnapshot -name vmname

Open in new window

or in dire situation
get-vm | remove-vmsnaphot

Open in new window


Critical/Paused VM's can be turned off
sgleeAuthor Commented:
@David
So when I run remove-vmsnapshot -name FT1_2481848F-1B1E-4667-A1A2-985A4E532138.avhdx in CMD, not only it will delete .avhdx file, it will stop the snapshot making process?
Get a highly available system for cyber protection

The Acronis SDI Appliance is a new plug-n-play solution with pre-configured Acronis Software-Defined Infrastructure software that gives service providers and enterprises ready access to a fault-tolerant system, which combines universal storage and high-performance virtualization.

sgleeAuthor Commented:
I ran the command:
PS C:\Users\Administrator> remove-vmsnapshot -name "D:\Hyper-V\Virtual Hard Disks\FT1_2481848F-1B1E-4667-A1A2-985A4E532138.avhdx"

and I get the following prompt:

cmdlet Remove-VMSnapshot at command pipeline position 1
Supply values for the following parameters:
VMName[0]:
sgleeAuthor Commented:
I googled on "hard drive running out of space  read some articles  and so far, what I read, they all deal with the situation where snapshot file is completely created. At that point, you can delete the snapshot from HyperV manager.
However, in my case, snapshot has not been created and the hard drive has run out of space.
David Johnson, CD, MVPRetiredCommented:
get-help remove-vmsnapshot

Open in new window



if powershell is prompting for something then wouldn't it be prudent to supply what it is needed?
It wants the vmname so type in the vmname as it shows in vm manager
sgleeAuthor Commented:
So is removing temporary file the best course of action?
Won’t  hyperv Server trying to resume creating a new snapshot?
sgleeAuthor Commented:
Moderator,
 
 I appreciate the effort of @David.
 However,  I like to work with another expert on this case. The reasons is that I am not getting answers from current expert in a timely manner in order to find the working solution.
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
That .AVHDX file is 18GB in size. That's a lot of change in the few short minutes since the snapshot being taken.

What is the storage layout on the host server?

The .AVHDX file can be mounted to access all "current" data in it.

The only way out of this given the fact that the partition the parent VHDX file is resting on is full is one of two:

1: Manually merge the .AVHDX file back into the parent. Note that this would require the two files to be copied to a partition that allows for a new parent file to be created which based on the Snip is ~420GB + ~420GB + 18GB for the .AVHDX.

2: Mount both parent and differencing disk and copy the data from differencing to parent. Copy the .AVHDX off. Delete the original .AVHDX. Boot/Start the VM.

1 is the safest route while 2 is risky. The backups are known good?
sgleeAuthor Commented:
HyperV Server HD Layout@Philip
(1) What is the storage layout on the host server? -->  C drive has W2012 OS.  D drive contains VMs.  F and G are external USB hard drives.

(2) The .AVHDX file can be mounted to access all "current" data in it.  --> I DIDN'T want to create a snapshot for FT1 VM. It happened by mistake. By clicking "Snapshot", I thought I would see Snapshot Manager like in VMWare, but unfortunately it started to create unwanted/unnecessary snapshot. The original FT1.VHDX was healthy and nothing was wrong.

(3) Can I somehow tell Hyper-V to stop creating this snapshot file (no space in HD currently and I don't need to create one)?
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
Do the properties for the FT1 VM show the .AVDHX or .VHDX file for its virtual disk?

What is the VM doing that it has such a high rate of change in a little over an hour?

Are the backups known good?

I'd set the parent .VHDX file as the virtual disk for FT1 then cut and paste that .AVHDX file onto one of the external drives to free up the space to get things going again. Any changed data would be had in the .AVHDX file anyway as mentioned it can be mounted to access data in it.
sgleeAuthor Commented:
Properties of FT1 VM(1) Does the properties for the FT1 VM show the .AVDHX or .VHDX file for its virtual disk? --> .AVDHX
(2) What is the VM doing that it has such a high rate of change in a little over an hour?
 -->  I was removing some programs from FT1 VM and rebooted a few times in the midst. It also has ACCESS database which is constantly updated by users all day.
(3) Are the backups known good? --> I need to investigate.
(4) I'd set the parent .VHDX file as the virtual disk for FT1 --> I think I can do that, but I think I need to "shutdown" this VM to do that. Right?
(5) then cut and paste that .AVHDX file onto one of the external drives to free up the space to get things going again. ---> I can do that.
(6)  Any changed data would be had in the .AVHDX file anyway as mentioned it can be mounted to access data in it. --> There maybe some records updated here and there, but users can re-enter them easily.
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
Right click the VM and turn it off. Then flip the disk to the parent VHDX. Don't fire the VM until the .AVHDX file is moved off the partition.

Oh, and change the vCPU count to at least two. That way if there's a runaway process in-guest there's one free thread to allow the OS to respond to a call to log on.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
sgleeAuthor Commented:
(1) Right click the VM and turn it off. Then flip the disk to the parent VHDX (2) change the vCPU count to at least two
--> I can do that. About 1 hour ago, I started copying FT1.VHDX file to an external USB hard drive just in case. It is at 45% and I think it will be finished in two hours. Then I will try those two things you mentioned.
I will report back as soon as copying is finished.
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
10-4.
sgleeAuthor Commented:
HD Space on D drive (VMs)VHD Backup FolderFT1 VM Settings ChangeSnapshot@Philip
(1) Both FT1.vhdx and FT1_2481848F-1B1E-4667-A1A2-985A4E532138.avhdx have been backed up to external USB hard drive and D drive now has 25GB free space.
(2) I changed Hard Drive to FT1.vhdx and Processor to 2 virtual processors.
(3)  I am ready to start FT1 VM, Before starting it,  do I need to do anything with what is still in Snapshots section (in red box)? As you know, incomplete snapshot file is no longer on D drive.
sgleeAuthor Commented:
D drive after starting FT1 VMSnapshot folder with BIN fileFT1 VM in Safe Mode ShadedHaving confirmed that there was 25GB of free space on D drive, I started FT1. It came up with Safe mode/Safe mode networking/Normal mode ... etc. I choose Safe mode.
After scrolling some DLL files, it went shaded again. At that time, I checked D drive space and it was down to 82MB.
Then I realized that there is BIN file in D:\Hyper-V\Snapshots\E35C23F9-828B-458F-BA65-8386CD9E1CED folder with 13GB in size.
I am going to turn off the FT1 VM, backup  this BIN folder to the external USB hard drive, delete this file and restart FT1 VM.
I will report back.
sgleeAuthor Commented:
@Philip,
 
 I am going to hold off starting FT1 VM again until I hear from you to discuss where we go from here.
 The reason is that even though I freed up hard drive space (25GB) by moving FT1_2481848F..avhdx to external USB HD, as soon as I started FT1 VM,  that space was down to 82MB. I don't know what just ate up that space that quick.
 Maybe we should remove Snapshot completely before starting FT1 VM again?
 Anyway I like to wait until we get to discuss.
 If I don't hear from you tonight EST, then we will discuss tomorrow AM.
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
Is there another .AVHDX file being created when the VM gets started?
sgleeAuthor Commented:
@philip
I will be in front of the computer in the next 3-4 hours. Would you be available then?
sgleeAuthor Commented:
Hard Drive Space StatusRecycled Bin in D driveContents of Recycled Bin in D driveFiles in Recycled Bin@Philip,

 I discovered that $RECYCLED.BIN has three folders with different dates, however they all had the same file - FT1_xxx.AVHDX  that I deleted.
 Can I go ahead and delete these folders?
sgleeAuthor Commented:
I checked the D drive space usage by major folders - Snapshots, Virtual Hard Disks and Virtual Machines
(1)
D:\Hyper-V\Snapshots has one folder with the size of 13.2GB:
 E35C23F9-828B-458F-BA65-8386CD9E1CED.bin is 13GB
E35C23F9-828B-458F-BA65-8386CD9E1CED.vsv 10MB
(2)
D:\Hyper-V\Virtual Hard Disks has 4 vhdx files for 4 VMs: 436GB
FT1.vhdx (419GB),
XP1 (20GB),  
XP2 (13GB),
XP3 (4GB)
(3)
D:\Hyper-V\Virtual Machines: 27.4GB. The breakdown of this folder is as follows:

D:\Hyper-V\Virtual Machines\3C42CC16-1DF4-4926-B73C-DCDA5C7C572C (2GB) .BIn, .VSV files
D:\Hyper-V\Virtual Machines\80DD39C2-7E9C-43BB-B306-D26245601821(Empty)
D:\Hyper-V\Virtual Machines\902E887F-E764-46CE-B379-0B62E821BB52 (Empty)
D:\Hyper-V\Virtual Machines\CA4375D2-EB65-46C3-85A5-49B83C8EDDB1 (26GB) .BIn, .VSV files
D:\Hyper-V\Virtual Machines\DDB0D718-BCBA-43D5-8668-3696AEB1B394 (20MB)

Is it typical to see 26GB size folder in Virtual Machines folder?
Also somehow I need to remove 13GB file in Snapshots folder.
sgleeAuthor Commented:
Snapshot for FT1 VMIt is a long story, but I managed to start FT1 VM and all looks good in it.
Since FT1 VM hard drive is now using FT1.vhdx (not .AVHDX file ) , can we remove what is in Snapshots section?
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
If the .AVHDX file is backed up then empty the Recycle Bin.

Any update on the backup story?

Leave the VM OFF.

Set up a new VM and use the option to choose existing storage. Choose the parent VHDX file. Set up the vRAM and vCPU settings along with the vNIC the same.

Boot the VM. Verify and set the network configuration then reboot.

If that works, then delete the original VM in Hyper-V Manager.

Any newer needed data would be found in the .AVHDX file on the external drive.
sgleeAuthor Commented:
D drive spaceVM BIN file(1) Having deleted RECYCLED folder, D drive has 20GB.
(2) D:\Hyper-V\Virtual Machines\CA4375D2-EB65-46C3-85A5-49B83C8EDDB1 folder has CA4375D2-EB65-46C3-85A5-49B83C8EDDB1.bin file 26GB in size. Is this normal? Does this have something to do with  snapshot?
(3) Any update on the backup story? --> I have a complete backup of every single file in FT1 VM in an external USB HD.
(4) Set up a new VM --> I see what you are trying to do here. I think I may want to do that next weekend. At this point, FT1 is up and running and all workstation PCs can access files and folders from the server. So I like to keep it as it is, since Monday is tomorrow.

Can you shed some light on (2)?
sgleeAuthor Commented:
Current Status of VMsI rebooted Hyper-v server and all VMs are running like they used to and D drive has 18GB space left. We are good for now.
"Set up a new VM and use the option to choose existing storage. Choose the parent VHDX file. Set up the vRAM and vCPU settings along with the vNIC the same." --> I thought about this now and this makes sense. Since new virtual machine will be using existing good FT1.VHDX file as hard drive, no more drive space/additional RAM or CPU would be required as long as I turn off current FT1 VM, right?
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
Answer to the .BIN question: Yes, this is normal. Hyper-V lays down a file that is the equivalent to the amount of virtual memory assigned to each VM. One needs to keep this in mind when planning storage.
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
And yes, the current VM FT1 would not consume any resources when off. So, the "new" version would be the one to do that.
sgleeAuthor Commented:
I will try to create a new VM for FT1 (and delete FT1 VM) in the next few days as it sounds safe and simple.
I will report back.
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
Hang on, but since the Hyper-V reboot the VMs are now running normally. Does that include FT1? If it does, then there's no point making any further changes?
sgleeAuthor Commented:
Yes FT1 is up and running. However it has a trace of failed snapshot (as seen in the hyper/v manager screenshot) and incomplete avhdx file is not even in the D drive as I moved  that file into external USB HD.
So it is a kind of misinformation and nuisance. If I can somehow clear that, then I do not need to create a new VM.
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
This should show it:
Get-VMSnapshot -VMName FT1

Open in new window

This should delete it:
Get-VMSnapshot -VMName FT1 | Remove-VMSnapshot

Open in new window

sgleeAuthor Commented:
I can delete the Snapshot while FT1 VM is turned on?
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
I don't see why not. It's just a listing at this point with no actual data associated with it.

Was there any data in the .AVHDX file that needed to be copied back?
sgleeAuthor Commented:
"Was there any data in the .AVHDX file that needed to be copied back?" --> No. FT1.vhdx had everything up to the point & time when I, by mistake, clicked Snapshot option.
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
Good to hear.
sgleeAuthor Commented:
Snapshot OptionFyi, I thought, if I click "Snapshot", it would take me to some type of snapshot manager like in VMWare vShpere Client program.
I did not need to create a snapshot for any reason and I learned my lesson a hard way.
sgleeAuthor Commented:
Snapshot Removal CommandAfter pressing [Enter] key after "get-vmsnapshot -vmname FT1 | remove-vmsnapshot", the cursor is just blinking.
I will try this again after hours.
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
Status?
sgleeAuthor Commented:
The cursor is still blinking at the same spot.
Let me wait until everyone is done for the day, re-open Powershell and try to run Get-VMSnapshot -VMName FT1 | Remove-VMSnapshot.
I will report back this evening.
sgleeAuthor Commented:
Philip,
Since all VMs are running well, I am not going to bother with erasing snapshot for FT1. The reason is that since I have moved avhdx file to external usb HD, it is not going to find the file that it will try to delete.
I just performed the same test in my Test W2016 Hyper-v server where I created a snapshot(they call it checkpoint now), moved avhdx file to another folder and tried to delete the checkpoint. It displayed an error complaining about avhdx file not found.
So at this point, I like to wrap up this case and I like to thank you for your help.
sgleeAuthor Commented:
Thanks Philip for your help and I really appreciate it, particularly, over the weekend!
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
No worries, glad to help.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Virtualization

From novice to tech pro — start learning today.