sglee
asked on
Hyper-V running out of space while trying to create a snapshot.
Hi,
I accidently created a snapshot on the VM "FT1" and there are two problems:
(1) Every so many seconds, when I open this VM in Hyper-V manager, it goes to "pause" state showing the screen in shaded color. When I start it, it goes back to shaded/pause state.
(2) Now I am getting "Out of Disk" message on my Hyper-V server.
I did not intend to create a snapshot at all. I was expecting a snapshot manager screen as I chose "snapshot". But I think it is trying to create a snapshot and the server ran out of space. It seems that the logical thing to do is to cancel "snapshot creation process" because there is no hard drive space for the snapshot to be created in the first place, delete the temporary file "FT1_2481848F-1B1E-4667-A1 A2-985A4E5 32138.avhd x"(18.9GB) , and restart the "FT1" VM.
What should I do?
** I forgot to mention that I was trying to shut down the VM, but that is even stuck.
I accidently created a snapshot on the VM "FT1" and there are two problems:
(1) Every so many seconds, when I open this VM in Hyper-V manager, it goes to "pause" state showing the screen in shaded color. When I start it, it goes back to shaded/pause state.
(2) Now I am getting "Out of Disk" message on my Hyper-V server.
I did not intend to create a snapshot at all. I was expecting a snapshot manager screen as I chose "snapshot". But I think it is trying to create a snapshot and the server ran out of space. It seems that the logical thing to do is to cancel "snapshot creation process" because there is no hard drive space for the snapshot to be created in the first place, delete the temporary file "FT1_2481848F-1B1E-4667-A1
What should I do?
** I forgot to mention that I was trying to shut down the VM, but that is even stuck.
remove-vmsnapshot -name vmname
or in dire situationget-vm | remove-vmsnaphot
Critical/Paused VM's can be turned off
ASKER
@David
So when I run remove-vmsnapshot -name FT1_2481848F-1B1E-4667-A1A 2-985A4E53 2138.avhdx in CMD, not only it will delete .avhdx file, it will stop the snapshot making process?
So when I run remove-vmsnapshot -name FT1_2481848F-1B1E-4667-A1A
ASKER
I ran the command:
PS C:\Users\Administrator> remove-vmsnapshot -name "D:\Hyper-V\Virtual Hard Disks\FT1_2481848F-1B1E-46 67-A1A2-98 5A4E532138 .avhdx"
and I get the following prompt:
cmdlet Remove-VMSnapshot at command pipeline position 1
Supply values for the following parameters:
VMName[0]:
PS C:\Users\Administrator> remove-vmsnapshot -name "D:\Hyper-V\Virtual Hard Disks\FT1_2481848F-1B1E-46
and I get the following prompt:
cmdlet Remove-VMSnapshot at command pipeline position 1
Supply values for the following parameters:
VMName[0]:
ASKER
I googled on "hard drive running out of space read some articles and so far, what I read, they all deal with the situation where snapshot file is completely created. At that point, you can delete the snapshot from HyperV manager.
However, in my case, snapshot has not been created and the hard drive has run out of space.
However, in my case, snapshot has not been created and the hard drive has run out of space.
get-help remove-vmsnapshot
if powershell is prompting for something then wouldn't it be prudent to supply what it is needed?
It wants the vmname so type in the vmname as it shows in vm manager
ASKER
So is removing temporary file the best course of action?
Won’t hyperv Server trying to resume creating a new snapshot?
Won’t hyperv Server trying to resume creating a new snapshot?
ASKER
Moderator,
I appreciate the effort of @David.
However, I like to work with another expert on this case. The reasons is that I am not getting answers from current expert in a timely manner in order to find the working solution.
I appreciate the effort of @David.
However, I like to work with another expert on this case. The reasons is that I am not getting answers from current expert in a timely manner in order to find the working solution.
That .AVHDX file is 18GB in size. That's a lot of change in the few short minutes since the snapshot being taken.
What is the storage layout on the host server?
The .AVHDX file can be mounted to access all "current" data in it.
The only way out of this given the fact that the partition the parent VHDX file is resting on is full is one of two:
1: Manually merge the .AVHDX file back into the parent. Note that this would require the two files to be copied to a partition that allows for a new parent file to be created which based on the Snip is ~420GB + ~420GB + 18GB for the .AVHDX.
2: Mount both parent and differencing disk and copy the data from differencing to parent. Copy the .AVHDX off. Delete the original .AVHDX. Boot/Start the VM.
1 is the safest route while 2 is risky. The backups are known good?
What is the storage layout on the host server?
The .AVHDX file can be mounted to access all "current" data in it.
The only way out of this given the fact that the partition the parent VHDX file is resting on is full is one of two:
1: Manually merge the .AVHDX file back into the parent. Note that this would require the two files to be copied to a partition that allows for a new parent file to be created which based on the Snip is ~420GB + ~420GB + 18GB for the .AVHDX.
2: Mount both parent and differencing disk and copy the data from differencing to parent. Copy the .AVHDX off. Delete the original .AVHDX. Boot/Start the VM.
1 is the safest route while 2 is risky. The backups are known good?
ASKER
@Philip
(1) What is the storage layout on the host server? --> C drive has W2012 OS. D drive contains VMs. F and G are external USB hard drives.
(2) The .AVHDX file can be mounted to access all "current" data in it. --> I DIDN'T want to create a snapshot for FT1 VM. It happened by mistake. By clicking "Snapshot", I thought I would see Snapshot Manager like in VMWare, but unfortunately it started to create unwanted/unnecessary snapshot. The original FT1.VHDX was healthy and nothing was wrong.
(3) Can I somehow tell Hyper-V to stop creating this snapshot file (no space in HD currently and I don't need to create one)?
(1) What is the storage layout on the host server? --> C drive has W2012 OS. D drive contains VMs. F and G are external USB hard drives.
(2) The .AVHDX file can be mounted to access all "current" data in it. --> I DIDN'T want to create a snapshot for FT1 VM. It happened by mistake. By clicking "Snapshot", I thought I would see Snapshot Manager like in VMWare, but unfortunately it started to create unwanted/unnecessary snapshot. The original FT1.VHDX was healthy and nothing was wrong.
(3) Can I somehow tell Hyper-V to stop creating this snapshot file (no space in HD currently and I don't need to create one)?
Do the properties for the FT1 VM show the .AVDHX or .VHDX file for its virtual disk?
What is the VM doing that it has such a high rate of change in a little over an hour?
Are the backups known good?
I'd set the parent .VHDX file as the virtual disk for FT1 then cut and paste that .AVHDX file onto one of the external drives to free up the space to get things going again. Any changed data would be had in the .AVHDX file anyway as mentioned it can be mounted to access data in it.
What is the VM doing that it has such a high rate of change in a little over an hour?
Are the backups known good?
I'd set the parent .VHDX file as the virtual disk for FT1 then cut and paste that .AVHDX file onto one of the external drives to free up the space to get things going again. Any changed data would be had in the .AVHDX file anyway as mentioned it can be mounted to access data in it.
ASKER
(1) Does the properties for the FT1 VM show the .AVDHX or .VHDX file for its virtual disk? --> .AVDHX
(2) What is the VM doing that it has such a high rate of change in a little over an hour?
--> I was removing some programs from FT1 VM and rebooted a few times in the midst. It also has ACCESS database which is constantly updated by users all day.
(3) Are the backups known good? --> I need to investigate.
(4) I'd set the parent .VHDX file as the virtual disk for FT1 --> I think I can do that, but I think I need to "shutdown" this VM to do that. Right?
(5) then cut and paste that .AVHDX file onto one of the external drives to free up the space to get things going again. ---> I can do that.
(6) Any changed data would be had in the .AVHDX file anyway as mentioned it can be mounted to access data in it. --> There maybe some records updated here and there, but users can re-enter them easily.
(2) What is the VM doing that it has such a high rate of change in a little over an hour?
--> I was removing some programs from FT1 VM and rebooted a few times in the midst. It also has ACCESS database which is constantly updated by users all day.
(3) Are the backups known good? --> I need to investigate.
(4) I'd set the parent .VHDX file as the virtual disk for FT1 --> I think I can do that, but I think I need to "shutdown" this VM to do that. Right?
(5) then cut and paste that .AVHDX file onto one of the external drives to free up the space to get things going again. ---> I can do that.
(6) Any changed data would be had in the .AVHDX file anyway as mentioned it can be mounted to access data in it. --> There maybe some records updated here and there, but users can re-enter them easily.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
(1) Right click the VM and turn it off. Then flip the disk to the parent VHDX (2) change the vCPU count to at least two
--> I can do that. About 1 hour ago, I started copying FT1.VHDX file to an external USB hard drive just in case. It is at 45% and I think it will be finished in two hours. Then I will try those two things you mentioned.
I will report back as soon as copying is finished.
--> I can do that. About 1 hour ago, I started copying FT1.VHDX file to an external USB hard drive just in case. It is at 45% and I think it will be finished in two hours. Then I will try those two things you mentioned.
I will report back as soon as copying is finished.
10-4.
ASKER
@Philip
(1) Both FT1.vhdx and FT1_2481848F-1B1E-4667-A1A 2-985A4E53 2138.avhdx have been backed up to external USB hard drive and D drive now has 25GB free space.
(2) I changed Hard Drive to FT1.vhdx and Processor to 2 virtual processors.
(3) I am ready to start FT1 VM, Before starting it, do I need to do anything with what is still in Snapshots section (in red box)? As you know, incomplete snapshot file is no longer on D drive.
(1) Both FT1.vhdx and FT1_2481848F-1B1E-4667-A1A
(2) I changed Hard Drive to FT1.vhdx and Processor to 2 virtual processors.
(3) I am ready to start FT1 VM, Before starting it, do I need to do anything with what is still in Snapshots section (in red box)? As you know, incomplete snapshot file is no longer on D drive.
ASKER
Having confirmed that there was 25GB of free space on D drive, I started FT1. It came up with Safe mode/Safe mode networking/Normal mode ... etc. I choose Safe mode.
After scrolling some DLL files, it went shaded again. At that time, I checked D drive space and it was down to 82MB.
Then I realized that there is BIN file in D:\Hyper-V\Snapshots\E35C2 3F9-828B-4 58F-BA65-8 386CD9E1CE D folder with 13GB in size.
I am going to turn off the FT1 VM, backup this BIN folder to the external USB hard drive, delete this file and restart FT1 VM.
I will report back.
After scrolling some DLL files, it went shaded again. At that time, I checked D drive space and it was down to 82MB.
Then I realized that there is BIN file in D:\Hyper-V\Snapshots\E35C2
I am going to turn off the FT1 VM, backup this BIN folder to the external USB hard drive, delete this file and restart FT1 VM.
I will report back.
ASKER
@Philip,
I am going to hold off starting FT1 VM again until I hear from you to discuss where we go from here.
The reason is that even though I freed up hard drive space (25GB) by moving FT1_2481848F..avhdx to external USB HD, as soon as I started FT1 VM, that space was down to 82MB. I don't know what just ate up that space that quick.
Maybe we should remove Snapshot completely before starting FT1 VM again?
Anyway I like to wait until we get to discuss.
If I don't hear from you tonight EST, then we will discuss tomorrow AM.
I am going to hold off starting FT1 VM again until I hear from you to discuss where we go from here.
The reason is that even though I freed up hard drive space (25GB) by moving FT1_2481848F..avhdx to external USB HD, as soon as I started FT1 VM, that space was down to 82MB. I don't know what just ate up that space that quick.
Maybe we should remove Snapshot completely before starting FT1 VM again?
Anyway I like to wait until we get to discuss.
If I don't hear from you tonight EST, then we will discuss tomorrow AM.
Is there another .AVHDX file being created when the VM gets started?
ASKER
@philip
I will be in front of the computer in the next 3-4 hours. Would you be available then?
I will be in front of the computer in the next 3-4 hours. Would you be available then?
ASKER
ASKER
I checked the D drive space usage by major folders - Snapshots, Virtual Hard Disks and Virtual Machines
(1)
D:\Hyper-V\Snapshots has one folder with the size of 13.2GB:
E35C23F9-828B-458F-BA65-83 86CD9E1CED .bin is 13GB
E35C23F9-828B-458F-BA65-83 86CD9E1CED .vsv 10MB
(2)
D:\Hyper-V\Virtual Hard Disks has 4 vhdx files for 4 VMs: 436GB
FT1.vhdx (419GB),
XP1 (20GB),
XP2 (13GB),
XP3 (4GB)
(3)
D:\Hyper-V\Virtual Machines: 27.4GB. The breakdown of this folder is as follows:
D:\Hyper-V\Virtual Machines\3C42CC16-1DF4-492 6-B73C-DCD A5C7C572C (2GB) .BIn, .VSV files
D:\Hyper-V\Virtual Machines\80DD39C2-7E9C-43B B-B306-D26 245601821( Empty)
D:\Hyper-V\Virtual Machines\902E887F-E764-46C E-B379-0B6 2E821BB52 (Empty)
D:\Hyper-V\Virtual Machines\CA4375D2-EB65-46C 3-85A5-49B 83C8EDDB1 (26GB) .BIn, .VSV files
D:\Hyper-V\Virtual Machines\DDB0D718-BCBA-43D 5-8668-369 6AEB1B394 (20MB)
Is it typical to see 26GB size folder in Virtual Machines folder?
Also somehow I need to remove 13GB file in Snapshots folder.
(1)
D:\Hyper-V\Snapshots has one folder with the size of 13.2GB:
E35C23F9-828B-458F-BA65-83
E35C23F9-828B-458F-BA65-83
(2)
D:\Hyper-V\Virtual Hard Disks has 4 vhdx files for 4 VMs: 436GB
FT1.vhdx (419GB),
XP1 (20GB),
XP2 (13GB),
XP3 (4GB)
(3)
D:\Hyper-V\Virtual Machines: 27.4GB. The breakdown of this folder is as follows:
D:\Hyper-V\Virtual Machines\3C42CC16-1DF4-492
D:\Hyper-V\Virtual Machines\80DD39C2-7E9C-43B
D:\Hyper-V\Virtual Machines\902E887F-E764-46C
D:\Hyper-V\Virtual Machines\CA4375D2-EB65-46C
D:\Hyper-V\Virtual Machines\DDB0D718-BCBA-43D
Is it typical to see 26GB size folder in Virtual Machines folder?
Also somehow I need to remove 13GB file in Snapshots folder.
ASKER
If the .AVHDX file is backed up then empty the Recycle Bin.
Any update on the backup story?
Leave the VM OFF.
Set up a new VM and use the option to choose existing storage. Choose the parent VHDX file. Set up the vRAM and vCPU settings along with the vNIC the same.
Boot the VM. Verify and set the network configuration then reboot.
If that works, then delete the original VM in Hyper-V Manager.
Any newer needed data would be found in the .AVHDX file on the external drive.
Any update on the backup story?
Leave the VM OFF.
Set up a new VM and use the option to choose existing storage. Choose the parent VHDX file. Set up the vRAM and vCPU settings along with the vNIC the same.
Boot the VM. Verify and set the network configuration then reboot.
If that works, then delete the original VM in Hyper-V Manager.
Any newer needed data would be found in the .AVHDX file on the external drive.
ASKER
(1) Having deleted RECYCLED folder, D drive has 20GB.
(2) D:\Hyper-V\Virtual Machines\CA4375D2-EB65-46C 3-85A5-49B 83C8EDDB1 folder has CA4375D2-EB65-46C3-85A5-49 B83C8EDDB1 .bin file 26GB in size. Is this normal? Does this have something to do with snapshot?
(3) Any update on the backup story? --> I have a complete backup of every single file in FT1 VM in an external USB HD.
(4) Set up a new VM --> I see what you are trying to do here. I think I may want to do that next weekend. At this point, FT1 is up and running and all workstation PCs can access files and folders from the server. So I like to keep it as it is, since Monday is tomorrow.
Can you shed some light on (2)?
(2) D:\Hyper-V\Virtual Machines\CA4375D2-EB65-46C
(3) Any update on the backup story? --> I have a complete backup of every single file in FT1 VM in an external USB HD.
(4) Set up a new VM --> I see what you are trying to do here. I think I may want to do that next weekend. At this point, FT1 is up and running and all workstation PCs can access files and folders from the server. So I like to keep it as it is, since Monday is tomorrow.
Can you shed some light on (2)?
ASKER
I rebooted Hyper-v server and all VMs are running like they used to and D drive has 18GB space left. We are good for now.
"Set up a new VM and use the option to choose existing storage. Choose the parent VHDX file. Set up the vRAM and vCPU settings along with the vNIC the same." --> I thought about this now and this makes sense. Since new virtual machine will be using existing good FT1.VHDX file as hard drive, no more drive space/additional RAM or CPU would be required as long as I turn off current FT1 VM, right?
"Set up a new VM and use the option to choose existing storage. Choose the parent VHDX file. Set up the vRAM and vCPU settings along with the vNIC the same." --> I thought about this now and this makes sense. Since new virtual machine will be using existing good FT1.VHDX file as hard drive, no more drive space/additional RAM or CPU would be required as long as I turn off current FT1 VM, right?
Answer to the .BIN question: Yes, this is normal. Hyper-V lays down a file that is the equivalent to the amount of virtual memory assigned to each VM. One needs to keep this in mind when planning storage.
And yes, the current VM FT1 would not consume any resources when off. So, the "new" version would be the one to do that.
ASKER
I will try to create a new VM for FT1 (and delete FT1 VM) in the next few days as it sounds safe and simple.
I will report back.
I will report back.
Hang on, but since the Hyper-V reboot the VMs are now running normally. Does that include FT1? If it does, then there's no point making any further changes?
ASKER
Yes FT1 is up and running. However it has a trace of failed snapshot (as seen in the hyper/v manager screenshot) and incomplete avhdx file is not even in the D drive as I moved that file into external USB HD.
So it is a kind of misinformation and nuisance. If I can somehow clear that, then I do not need to create a new VM.
So it is a kind of misinformation and nuisance. If I can somehow clear that, then I do not need to create a new VM.
This should show it:
Get-VMSnapshot -VMName FT1
This should delete it:
Get-VMSnapshot -VMName FT1 | Remove-VMSnapshot
ASKER
I can delete the Snapshot while FT1 VM is turned on?
I don't see why not. It's just a listing at this point with no actual data associated with it.
Was there any data in the .AVHDX file that needed to be copied back?
Was there any data in the .AVHDX file that needed to be copied back?
ASKER
"Was there any data in the .AVHDX file that needed to be copied back?" --> No. FT1.vhdx had everything up to the point & time when I, by mistake, clicked Snapshot option.
Good to hear.
ASKER
ASKER
Status?
ASKER
The cursor is still blinking at the same spot.
Let me wait until everyone is done for the day, re-open Powershell and try to run Get-VMSnapshot -VMName FT1 | Remove-VMSnapshot.
I will report back this evening.
Let me wait until everyone is done for the day, re-open Powershell and try to run Get-VMSnapshot -VMName FT1 | Remove-VMSnapshot.
I will report back this evening.
ASKER
Philip,
Since all VMs are running well, I am not going to bother with erasing snapshot for FT1. The reason is that since I have moved avhdx file to external usb HD, it is not going to find the file that it will try to delete.
I just performed the same test in my Test W2016 Hyper-v server where I created a snapshot(they call it checkpoint now), moved avhdx file to another folder and tried to delete the checkpoint. It displayed an error complaining about avhdx file not found.
So at this point, I like to wrap up this case and I like to thank you for your help.
Since all VMs are running well, I am not going to bother with erasing snapshot for FT1. The reason is that since I have moved avhdx file to external usb HD, it is not going to find the file that it will try to delete.
I just performed the same test in my Test W2016 Hyper-v server where I created a snapshot(they call it checkpoint now), moved avhdx file to another folder and tried to delete the checkpoint. It displayed an error complaining about avhdx file not found.
So at this point, I like to wrap up this case and I like to thank you for your help.
ASKER
Thanks Philip for your help and I really appreciate it, particularly, over the weekend!
No worries, glad to help.
ASKER
I have not seen any response so far. Did I select the right category?
Thanks