datastore running out of storage ESXi 4.1

I have a LUN that is presented to ESXi v4.1 as a datastore.  Every few days it runs completely out of space.  I free'ed up 28 GBs but I have a feeling it's going to happen again.  What could be causing this?  I have an old backup job running the older backup API but I have it set to use an external drive at the guest OS level when running the backup jobs.. And.. I think it was already done.. I'll have to check the logs.  What else could be eating up the space?  Did I over-provision the drives maybe?
gopher_49Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
1. Snapshots

2. You have increased memory on VMs sharing this datastore, and the SWAP files have increased.

3. You have move more VMs onto this datastore.

28GB is not a very good, free space figure we work with is 25-30% of datastore space FREE!

28GB is very low, I've got flash drives which are larger!

4. Thin Provisioned disks?
0
pjamCommented:
Do you have snapshots enabled?  We ran out of space due to snapshots and servers went down, take this seriously.
sorry hanccocka, I am a slow typist.
0
gopher_49Author Commented:
We don't have snapshots running..  I'll double check.  But..  I do have a folder I back up to at the guest OS level that's stored on this datastore..  I'm using thin provisioning on almost all of my disks... But..  VMware wouldn't let me over provision I would think.
0
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
it will.

if you create

4 VMs with 1TB Thin Disks, on 1TB datastore.

if all VMs fill up, they will stop and fail, VMware ESXi will do nothing to prevent this!

So if you have thin provisioned, check ALL the VMs, and total up all the VMDKS, so at worse case, if all are fully provisioned, do you have enough storage space, on the datastore?
0
coolsport00Commented:
Backups typically create a snapshot on the same storage location that the VM's configuration file is located (.vmx file). So, regardless of where the backed up files are targeted to, the backup solution itself may be taking, and not removing, snaps that are used for the b/u job.

coolsport00
0
gopher_49Author Commented:
I'll disable the VMWare backups for now.. I have them backing up via the application level for these few VMs..  I found the job causing it..  I'm going to send the job to removable drive for now until get more storage.  I thought VMWare wouldn't let me over provision.
0
coolsport00Commented:
vSphere doesn't stop provisioning of storage. When you thin provision, for example, say you have 100GB datastore - you can configure 10VMs with 40GB thin provision vmdk's each. But wait (you say)... that is 400GB of storage! Well, that's thin provisioning. Even though you configured 40GB vmdk's, the VMs only use the storage amt that is written to disk. So, the danger is this ... if you don't have a Datastore alarm with a threshold set, yes... there is a potential to hit 0MB space left on the Datastore & thus prevent VMs from operating appropriately. (thus, over-provisioned)  The only way you can NOT provision another VM is if, when creating the VM, you allocate a certain vmdk size and vSphere 'sees' the storage size as not capable of the size wanting to configure. Hope that makes sense... it can be a bit confusing :)

Regards,
~coolsport00
0
gopher_49Author Commented:
I have the threshold alarms set but REALLY low.. For I needed the storage..  I disabled the backup that could of been eating up the storage for now.. I'll update the ticket once I deploy another ESXi host to move some VM's to.  In regards to moving.. Since my space is low is there a chance this will cause an issue?
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Yes, it's dangerous living with low space....especially if you get a snapshot situation, how do you backup Vms?
0
coolsport00Commented:
Veeam Backup & Replication. It takes snaps during the b/u process, but upon completion of the job, it removes the snap.

~coolsport00
0
gopher_49Author Commented:
I back 3 x VMs via the old VMware API's.. The other VM's are done at the application level.  In two weeks we're rolling out VEEAM once we get another host.
0
coolsport00Commented:
Ha...sorry.. I answered @hanccocka thinking it was you :P

To answer you, it depends. How are you moving the VMs? If you're low on space, you can power down the VMs and recover the space consumed by the VM's memory (vswp. file). If your VMs are on different Hosts than the b/u job sees those VMs on, those b/u jobs will fail.

~coolsport00
0
gopher_49Author Commented:
So..  The safest process is to shutdown the VM and move it, correct?  I plan to move the smallest disks/VM first.
0
coolsport00Commented:
Yes, solely to gain space that's utilized by the .vswp file, and it also prevents writes to disk.

~coolsport00
0
gopher_49Author Commented:
okay...  I'll migrate next week.  Does ESX v4.1 Vsphere Essentials allow me to migrate disks to different hosts?  Don't think I'll need to but just curious.
0
coolsport00Commented:
No.. that version doesn't allow Storage VMotion
0
coolsport00Commented:
If you do need to move disks, use vCenter Converter Standalone & V2V; or, you can power down the VM and move the VM folder from one Datastore to another via the Datastore Browser.

~coolsport00
0
gopher_49Author Commented:
Shouldn't I simply migrate via vcenter?  Or is that too risky due to storage issues?  I assume you recommend the VMware standalone converter for safer due to it copying and then uploading...  Right?
0
coolsport00Commented:
If you need to VMotion, you can do so ONLY if you have Essentials plus; otherwise, you can migrate with the VM powered off (as I'm suggesting to do anyway). You should be able to migrate disks as well with the VM powered off. Powered on (Storage VMotion) is not a feature of Essentials though. Sorry for the confusion.

~coolsport00
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
You can migrate via vCenter server, that works fine, and is easier.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
VMware

From novice to tech pro — start learning today.