danindub
asked on
There is no more space for redo log
Hi All,
we have really weird issue. We have simple VMWare ESXi host with two datastores - one of 800 GB and one of 80. We have all virtual machines on the 800 GB one . Both datastores have around 35 GB free space according to VMWare Infrastructure Client.
I am trying to start up one of the machines (2 GB of RAM) and i'm getting "There is no more space for redo log...". I tried to commit snapshots overnight with no effect. I even deleted one of the old Virtual Machines releasing further 10 GB, but still it claims there's no space.
Any ideas?
we have really weird issue. We have simple VMWare ESXi host with two datastores - one of 800 GB and one of 80. We have all virtual machines on the 800 GB one . Both datastores have around 35 GB free space according to VMWare Infrastructure Client.
I am trying to start up one of the machines (2 GB of RAM) and i'm getting "There is no more space for redo log...". I tried to commit snapshots overnight with no effect. I even deleted one of the old Virtual Machines releasing further 10 GB, but still it claims there's no space.
Any ideas?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
All hypervisors have their share of issues :)
I think that if you have committed the snapshots then use the vmware converter.
-Convert the problem guest to a temp location.
-Delete the guest copy and files.
-Copy the new guest back and you should be fine.
I think that if you have committed the snapshots then use the vmware converter.
-Convert the problem guest to a temp location.
-Delete the guest copy and files.
-Copy the new guest back and you should be fine.
As my fellow expert nappy_d has stated you will need to use VMware Convertor to convert the machine to a new machine. This is the VMware Recommendation approach, when snapshots get out of hand. You shouldn't really run Production Machines on a Snapshotm it's not best practise as you've discovered.
For the conversion steps, read Bestway's article.
https://www.experts-exchange.com/Software/VMWare/A_3639-VMware-vConverter-P2V-for-Windows-Servers.html
For the conversion steps, read Bestway's article.
https://www.experts-exchange.com/Software/VMWare/A_3639-VMware-vConverter-P2V-for-Windows-Servers.html
My 1st suggestion in moving forward is, unless you have some critical 'need' to have snapshots, do NOT use them. They are NOT a b/u solution. Use them when apply app/OS/software updates if need be, but within a day, get rid of them. You can try to use cmd line to commit your snapshots if they are indeed still there; see:
http://kb.vmware.com/kb/1002310
Or, you can simply Clone your VM/VMs (if you have vCenter). If you don't have vCenter, then a conversion, as "nappy_d" points out, will do the trick.
Regards,
~coolsport00
http://kb.vmware.com/kb/1002310
Or, you can simply Clone your VM/VMs (if you have vCenter). If you don't have vCenter, then a conversion, as "nappy_d" points out, will do the trick.
Regards,
~coolsport00
give all of the VM's a full memory reservation (edit settings, resources, memory). that will prevent the VM's from needing a swap file.
Also, while you are in the settings, verify that each hard drive is on the 800GB datastore.
If you have really large snapshot files, the roll up process may still be running in the background.
http:/kb.vmware.com/kb/1015180 explains snapshots in case you hadn't read it yet.
can you give us a list of snapshots and their sizes so we have an idea of what you're up against?
Also, while you are in the settings, verify that each hard drive is on the 800GB datastore.
If you have really large snapshot files, the roll up process may still be running in the background.
http:/kb.vmware.com/kb/1015180 explains snapshots in case you hadn't read it yet.
can you give us a list of snapshots and their sizes so we have an idea of what you're up against?
Dannindub, any further updates?
ASKER
Hi All,
Ok, here's what i did.
vmware-cmd doesn't exist on ESXi 3.5 server and it looks like VMWare pulled Remote Command line or whatever it's called for it already.
I managed to clean up some space (about 30 GB). After that I created new snapshot for the machine, and then went "Delete all snapshots". I could see files being modified to I left it overnight just to wake up in the morning to same dozens of huge files and empty disk space (i'm guessing it tried to commit them but run out of space).
As there isn't much I could do at this stage, I deleted one of the disks, that gave me extra 20 GB. Machine worked, but I have to move it away. So i shut it down and fired up VMWare vCentre converted. I connected to vm host, chosen my machine. Unfortunately i can't go any further, as I'm getting (after good few minutes of "retrieving information about virtual machine") error Unable to obtain hardware information for the selected machine.
:(
Ok, here's what i did.
vmware-cmd doesn't exist on ESXi 3.5 server and it looks like VMWare pulled Remote Command line or whatever it's called for it already.
I managed to clean up some space (about 30 GB). After that I created new snapshot for the machine, and then went "Delete all snapshots". I could see files being modified to I left it overnight just to wake up in the morning to same dozens of huge files and empty disk space (i'm guessing it tried to commit them but run out of space).
As there isn't much I could do at this stage, I deleted one of the disks, that gave me extra 20 GB. Machine worked, but I have to move it away. So i shut it down and fired up VMWare vCentre converted. I connected to vm host, chosen my machine. Unfortunately i can't go any further, as I'm getting (after good few minutes of "retrieving information about virtual machine") error Unable to obtain hardware information for the selected machine.
:(
Yes...the vmware-cmd isn't in ESXi, the "command" for ESXi is vim-cmd. I recommend trying the cmd line way again, or look through BestWay's article posted by "hanccocka" above to work through your conversion errors.
Also...you mention you deleted a disk...what disk exactly? That could cause costly as, depending on what you deleted, you may not be able to restore your VM.
~coolsport00
Also...you mention you deleted a disk...what disk exactly? That could cause costly as, depending on what you deleted, you may not be able to restore your VM.
~coolsport00
ASKER
Ok, here's what I did.
I moved all other virtual machines (around 150 GB) away from this server to a temporary location which took about 7 - 8 hours.
I created one more snapshot in VMWare Infrastructure client, and clicked "Delete all" after that.
I could see heavy disk activity and files being modified. Slowly but surely, 14 hours later (!) all deltas were commited and removed, and amount of free space increased by over 500 GBs.
I am bringing other machines back onto the server. The machine that caused the problems is running like never (I would dare to say it's way faster as well).
Thanks all for help, still I believe it's really bad implementation of the feature and will be moving away towards Hyper-V. Fact that it's Windows alone makes it much easier to troubleshoot in situations like this one.
Thanks again.
I moved all other virtual machines (around 150 GB) away from this server to a temporary location which took about 7 - 8 hours.
I created one more snapshot in VMWare Infrastructure client, and clicked "Delete all" after that.
I could see heavy disk activity and files being modified. Slowly but surely, 14 hours later (!) all deltas were commited and removed, and amount of free space increased by over 500 GBs.
I am bringing other machines back onto the server. The machine that caused the problems is running like never (I would dare to say it's way faster as well).
Thanks all for help, still I believe it's really bad implementation of the feature and will be moving away towards Hyper-V. Fact that it's Windows alone makes it much easier to troubleshoot in situations like this one.
Thanks again.
Be careful of Hyper-V Windows tends to hog alot of memory(IMO) for this kind of environment. But all hypervisors have their own share of issues, as you will find out when you move to Hyper-V :)
ASKER
I have huge snapshot images. I tried to commit them, but apart from dissapearing from Snapshot Manager, files are still there. I tried command line last night, just to wake up this morning to finished operation, and all huge snapshot disks still there...
Going to switch to Hyper-V as this is driving me crazy, but for now I need to fire this machine up ...