Solved

VMWare: How to prevent 'out of space' error making a backup with veeam?

Posted on 2010-11-24
11
4,376 Views
Last Modified: 2012-05-10
Hello everybody.

I have a VM of 300GB in a Datastore with a capacity of 305 GB, so there are only 5 GB free. I am using Veeam as Backup software.Its working directory is in another datastore with 300+ GB free.

I thought that having 5GB of free space in the VM's datastore was enough for the configuration files and that stuff, but maybe I was wrong. The problem happened when I found the following error last Monday:


[msg.hbacommon.outofspace] There is no more space for virtual disk myvm-000001.vmdk.  You
Nov 21 01:00:34.073: vcpu-0| may be able to continue this session by freeing disk space on
Nov 21 01:00:34.073: vcpu-0| the relevant partition, and clicking Retry. Otherwise,
Nov 21 01:00:34.073: vcpu-0| click Abort to terminate this session.

Taking a look at the logs, it seems that Veeam had just made a snapshot, but in the very same datastore of the VM, not in the working directory condfigured in the VM, where there is more than enough space free.

Why was that snapshot created in the VM's datastored when its working directory was configured to be in other datastore?


Nov 20 22:00:06.912: vmx| SnapshotVMX done with snapshot 'VEEAM BACKUP TEMPORARY SNAPSHOT': 27
Nov 20 22:00:06.912: vmx| Vix: [105074 vmxCommands.c:2353]: VMAutomationCreateSnapshotCallback: Got CreateSnapshot callback, snapshotErr = 0, UID = 27
Nov 20 22:00:07.241: vcpu-1| HBACommon: First write on scsi0:0.fileName='/vmfs/volumes/4beada54-cca8b737-8c6c-f4ce46bcaa39/myvm_1/myvm-000001.vmdk'
Nov 20 22:00:07.347: vcpu-1| DISKLIB-CHAIN : UpdateContentID: old = 0x1c0a9282, new = 0x7c25aa6d
Nov 20 22:00:07.347: vcpu-1| DISKLIB-DDB   : "longContentID" = "e605c5b301ba16c949cc573c7c25aa6d" (was "15037ad78fd1b288bd4bceba1c0a9282")
Nov 20 22:00:07.437: vcpu-1| DISKLIB-CHAIN : UpdateContentID: updated long content id
Nov 21 01:00:34.070: vcpu-0| Msg_Question:
Nov 21 01:00:34.073: vcpu-0| [msg.hbacommon.outofspace] There is no more space for virtual disk myvm-000001.vmdk.  You
Nov 21 01:00:34.073: vcpu-0| may be able to continue this session by freeing disk space on
Nov 21 01:00:34.073: vcpu-0| the relevant partition, and clicking Retry. Otherwise,
Nov 21 01:00:34.073: vcpu-0| click Abort to terminate this session.
Nov 21 01:00:34.073: vcpu-0| ----------------------------------------

How can we prevent this situation from happening again?

Thanks in advance,
    Gregorio Diaz
0
Comment
Question by:McGregor09
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
  • 2
  • +2
11 Comments
 
LVL 23

Expert Comment

by:Luciano Patrão
ID: 34205031
Hi

Veeam always create a temporarily snaphot for the backups. After the backups will delete the snapshots.

So yes you need to have Storage space to hold this temporarily snapshots for your backups.

And if you get out of storage space, the VM stop working.

Jail
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 34205293
Also keep in mind that the amount of memory in your VM is the amount that your VM will create a swap file if you don't reserver the memory. If you have plenty of memory try to reserve 4GB RAM for example, this will reduce your swap file by 4GB and provide this space for the snaphsot.  You should always leave at least 15GB of space on the datastores.
0
 

Author Comment

by:McGregor09
ID: 34205649
Hi Paulsolov,

The VM has always had 4GB of RAM assigned, no problem with that.
Now, what I have done is changed the configuration of the VM. Let's say that I have the VM in Datastore A. I have set the working directory to another datastore, B.  And, just in case this was not enough, I set the swapfile directory in another datastore C. Now, B and C have  about 300GB free space, and A just 5.

The thing is that shrinking the size of my VM will take me some days, and meanwhile I want to make sure there is plenty of space for everything.

As I have done this, do you think I have to worry about this error taking place again?
Are those 15GB really necessary? Which VMWare processes do take up that space?

Thanks again
0
Edgartown IT Case Study

Learn about Edgartown's quest to ensure the safety and security of the entire town's employee and citizen data. Read the case study!

 
LVL 42

Expert Comment

by:paulsolov
ID: 34206252
15GB is best practice.  You lose 5% of storage for the vmdk wrapper around the OS volumes, if you thin provision it will kill you.  

If you're using snapshots you need min. of 10GB free space but it all depends on how much in changes will be done during the backup.  

0
 
LVL 23

Expert Comment

by:Luciano Patrão
ID: 34207596
Hi

Yes paulsolov is correct, but snapshots with the Veeam incremental will not take much Storage, but sill you will always need the extra space for snapshots backup and also for normal VM management(swap etc)

Jail
0
 
LVL 27

Expert Comment

by:Tolomir
ID: 34208191
As said in the soon  to be deleted question. A snapshot can grow to the original size of the harddisk.

Of cause you do the backup off hours still you should reserve some space paulsolov's 5% would be around 50 GB to be reserved.

We could discuss the point if it is possible to keep working files and / or snapshots in a separate datastore or if all those file have to be kept in one datastore.

Tolomir
0
 
LVL 27

Accepted Solution

by:
Tolomir earned 63 total points
ID: 34208316
Just as reference in this question 15% of free space was accepted.

http://www.experts-exchange.com/Software/VMWare/Q_26636453.html?cid=1576

0
 

Author Comment

by:McGregor09
ID: 34211009
Hi all,

thanks for the answers!

I see, so around 15% is necessary. Just to make it 110% clear, even in my case, I have set the working and swap directories in another two datastores, should I have that 15% free space in the VM's datastore?

0
 
LVL 42

Assisted Solution

by:paulsolov
paulsolov earned 62 total points
ID: 34212831
With the swap file on a different partition it gives you more space when you start up the VM (since it doesn't create the swap file on the same volume).  You still need space for the snapshots so 10-15 percent would be min. depending on how much your snapshots grow during backups.  Also ensure that you're not thin provisioning.
0
 
LVL 17

Expert Comment

by:Gerald Connolly
ID: 34213897
An old rule of thumb for snapshot sizing is that 80% of writes go to 20% of the disk, so you should plan on at least 20% but YMMV
0
 

Author Closing Comment

by:McGregor09
ID: 34217517
Thanks for the detailed information. I see I have to invest a little bit more on my system  :)
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Amazon Glacier backup 6 70
VMware - Migrating 1 50
ESXi vmnic Stand By Status 3 106
does Veeam backup only used space inside a virtual machine? 8 62
In this article, I will show you HOW TO: Perform a Physical to Virtual (P2V) Conversion the easy way from a computer backup (image).
This article outlines why you need to choose a backup solution that protects your entire environment – including your VMware ESXi and Microsoft Hyper-V virtualization hosts – not just your virtual machines.
In this Micro Tutorial viewers will learn how to restore their server from Bare Metal Backup image created with Windows Server Backup feature. As an example Windows 2012R2 is used.
This tutorial will walk an individual through the steps necessary to enable the VMware\Hyper-V licensed feature of Backup Exec 2012. In addition, how to add a VMware server and configure a backup job. The first step is to acquire the necessary licen…

733 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question