Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

VMWare: How to prevent 'out of space' error making a backup with veeam?

Posted on 2010-11-24
11
4,091 Views
Last Modified: 2012-05-10
Hello everybody.

I have a VM of 300GB in a Datastore with a capacity of 305 GB, so there are only 5 GB free. I am using Veeam as Backup software.Its working directory is in another datastore with 300+ GB free.

I thought that having 5GB of free space in the VM's datastore was enough for the configuration files and that stuff, but maybe I was wrong. The problem happened when I found the following error last Monday:


[msg.hbacommon.outofspace] There is no more space for virtual disk myvm-000001.vmdk.  You
Nov 21 01:00:34.073: vcpu-0| may be able to continue this session by freeing disk space on
Nov 21 01:00:34.073: vcpu-0| the relevant partition, and clicking Retry. Otherwise,
Nov 21 01:00:34.073: vcpu-0| click Abort to terminate this session.

Taking a look at the logs, it seems that Veeam had just made a snapshot, but in the very same datastore of the VM, not in the working directory condfigured in the VM, where there is more than enough space free.

Why was that snapshot created in the VM's datastored when its working directory was configured to be in other datastore?


Nov 20 22:00:06.912: vmx| SnapshotVMX done with snapshot 'VEEAM BACKUP TEMPORARY SNAPSHOT': 27
Nov 20 22:00:06.912: vmx| Vix: [105074 vmxCommands.c:2353]: VMAutomationCreateSnapshotCallback: Got CreateSnapshot callback, snapshotErr = 0, UID = 27
Nov 20 22:00:07.241: vcpu-1| HBACommon: First write on scsi0:0.fileName='/vmfs/volumes/4beada54-cca8b737-8c6c-f4ce46bcaa39/myvm_1/myvm-000001.vmdk'
Nov 20 22:00:07.347: vcpu-1| DISKLIB-CHAIN : UpdateContentID: old = 0x1c0a9282, new = 0x7c25aa6d
Nov 20 22:00:07.347: vcpu-1| DISKLIB-DDB   : "longContentID" = "e605c5b301ba16c949cc573c7c25aa6d" (was "15037ad78fd1b288bd4bceba1c0a9282")
Nov 20 22:00:07.437: vcpu-1| DISKLIB-CHAIN : UpdateContentID: updated long content id
Nov 21 01:00:34.070: vcpu-0| Msg_Question:
Nov 21 01:00:34.073: vcpu-0| [msg.hbacommon.outofspace] There is no more space for virtual disk myvm-000001.vmdk.  You
Nov 21 01:00:34.073: vcpu-0| may be able to continue this session by freeing disk space on
Nov 21 01:00:34.073: vcpu-0| the relevant partition, and clicking Retry. Otherwise,
Nov 21 01:00:34.073: vcpu-0| click Abort to terminate this session.
Nov 21 01:00:34.073: vcpu-0| ----------------------------------------

How can we prevent this situation from happening again?

Thanks in advance,
    Gregorio Diaz
0
Comment
Question by:McGregor09
  • 3
  • 3
  • 2
  • +2
11 Comments
 
LVL 23

Expert Comment

by:Luciano Patrão
ID: 34205031
Hi

Veeam always create a temporarily snaphot for the backups. After the backups will delete the snapshots.

So yes you need to have Storage space to hold this temporarily snapshots for your backups.

And if you get out of storage space, the VM stop working.

Jail
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 34205293
Also keep in mind that the amount of memory in your VM is the amount that your VM will create a swap file if you don't reserver the memory. If you have plenty of memory try to reserve 4GB RAM for example, this will reduce your swap file by 4GB and provide this space for the snaphsot.  You should always leave at least 15GB of space on the datastores.
0
 

Author Comment

by:McGregor09
ID: 34205649
Hi Paulsolov,

The VM has always had 4GB of RAM assigned, no problem with that.
Now, what I have done is changed the configuration of the VM. Let's say that I have the VM in Datastore A. I have set the working directory to another datastore, B.  And, just in case this was not enough, I set the swapfile directory in another datastore C. Now, B and C have  about 300GB free space, and A just 5.

The thing is that shrinking the size of my VM will take me some days, and meanwhile I want to make sure there is plenty of space for everything.

As I have done this, do you think I have to worry about this error taking place again?
Are those 15GB really necessary? Which VMWare processes do take up that space?

Thanks again
0
Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

 
LVL 42

Expert Comment

by:paulsolov
ID: 34206252
15GB is best practice.  You lose 5% of storage for the vmdk wrapper around the OS volumes, if you thin provision it will kill you.  

If you're using snapshots you need min. of 10GB free space but it all depends on how much in changes will be done during the backup.  

0
 
LVL 23

Expert Comment

by:Luciano Patrão
ID: 34207596
Hi

Yes paulsolov is correct, but snapshots with the Veeam incremental will not take much Storage, but sill you will always need the extra space for snapshots backup and also for normal VM management(swap etc)

Jail
0
 
LVL 27

Expert Comment

by:Tolomir
ID: 34208191
As said in the soon  to be deleted question. A snapshot can grow to the original size of the harddisk.

Of cause you do the backup off hours still you should reserve some space paulsolov's 5% would be around 50 GB to be reserved.

We could discuss the point if it is possible to keep working files and / or snapshots in a separate datastore or if all those file have to be kept in one datastore.

Tolomir
0
 
LVL 27

Accepted Solution

by:
Tolomir earned 63 total points
ID: 34208316
Just as reference in this question 15% of free space was accepted.

http://www.experts-exchange.com/Software/VMWare/Q_26636453.html?cid=1576

0
 

Author Comment

by:McGregor09
ID: 34211009
Hi all,

thanks for the answers!

I see, so around 15% is necessary. Just to make it 110% clear, even in my case, I have set the working and swap directories in another two datastores, should I have that 15% free space in the VM's datastore?

0
 
LVL 42

Assisted Solution

by:paulsolov
paulsolov earned 62 total points
ID: 34212831
With the swap file on a different partition it gives you more space when you start up the VM (since it doesn't create the swap file on the same volume).  You still need space for the snapshots so 10-15 percent would be min. depending on how much your snapshots grow during backups.  Also ensure that you're not thin provisioning.
0
 
LVL 17

Expert Comment

by:Gerald Connolly
ID: 34213897
An old rule of thumb for snapshot sizing is that 80% of writes go to 20% of the disk, so you should plan on at least 20% but YMMV
0
 

Author Closing Comment

by:McGregor09
ID: 34217517
Thanks for the detailed information. I see I have to invest a little bit more on my system  :)
0

Featured Post

Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

If we need to check who deleted a Virtual Machine from our vCenter. Looking this task in logs can be painful and spend lot of time, so the best way to check this is in the vCenter DB. Just connect to vCenter DB(default DB should be VCDB and using…
In 2017, ransomware will become so virulent and widespread that if you aren’t a victim yourself, you will know someone who is.
This tutorial will walk an individual through locating and launching the BEUtility application to properly change the service account username and\or password in situation where it may be necessary or where the password has been inadvertently change…
This tutorial will show how to configure a new Backup Exec 2012 server and move an existing database to that server with the use of the BEUtility. Install Backup Exec 2012 on the new server and apply all of the latest hotfixes and service packs. The…

837 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question