?
Solved

ESXI 5.1 Thin provisioning

Posted on 2013-11-20
19
Medium Priority
?
1,238 Views
Last Modified: 2013-11-29
Hello Experts,

I'm hoping you can clear up some issues I'm having with VMware/ESXI. We have an Exchange VM running on an ISCSI SAN using ESXI 5.1 and VMFS 5 using block sizes of 1MB. The Datastore where the volume is located is 4TB in size, the Exchange VMDK file is in thin provisioned mode a provisioned size of 2.02 TB with 216 GB in use and the data store has 3.67TB free.

The issue is occurring when we try to take a snap-shot using either VCentre Server or using the Veem backup tool, we get the following error; "File <unspecified filename> is larger than the maximum size supported by datastore <unspecified datastore>". I have looked at various internet posts and they indicate various reasons why this might occur. And it looks like the issue is either one of two things; because the provisioned size of the VMDK file which is just over 2TB, although 2TB was selected when it was setup, I have seen posts that suggest that the max size of the VMDK file needs to be slightly less than 2TB

The other reason might be because there isn’t enough disk space available to complete the snap-shot despite the datastore indicating that there is 3.67TB available, this might be because this reading is based on disk space used where the snap-shot calculation is based on the provisioned size, not the space physically used?

So in short, is the problem that there isn’t enough disk space because the "provisioned" size is too big for the snap shot to fit in the datastore, or is it because VMware allowed me to create a VMDK file just over the 2TB threshold.

At the moment we're not getting any snap-shots or backups which we need before pushing some Exchange 2013 updates through.
0
Comment
Question by:markfurey
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 10
  • 9
19 Comments
 
LVL 123
ID: 39662287
Just a note, that Microsoft do not support thin provisioned disks for Microsoft Exchange.

The issue that you are experiencing is your virtual machine disk plus the snapshot size, exceeds the maximum size of virtual disk (vmdk), which is 2TB-512bytes.

So, a fix, shrink the size of the exchange virtual disk to 500MB-1TB, and then the snapshot plus main virtual disk, will be supported.

Upgrade to ESXi 5.5, 63TB virtual machine disks are supported.

Add a new disk, and move the mailbox store to this new smaller disk.
0
 

Author Comment

by:markfurey
ID: 39662600
Hello Hanccocka,

Than you for your input but I have a couple of questions regarding your response. Regarding Exchange not supporting thin provisioned disks, I haven’t seen this anywhere and I'd assumed that the VMDK type would be transparent to the VM inside anyway?

Also, you say that the snapshot and the VM exceeds max size of the VMDK file, excuse my ignorance but I didn’t think a snapshot increased the size of the VMDK file, I'd assumed that the snapshot would generate a new file not increase the existing one? Does this mean that when creating a VM you must create a disk/VMDK file size that will always be smaller than 2TB with the additional size of a snapshot?

Also I'm not aware of any methods to "shrink" a thin provisioned disk, I'm guessing I would need to use a VMconverter and set the destination drive to the smaller size, but then I think the destination/cloned VMDK would then be "thick" provisioned which may then need yet another conversion to put it back to "thin".

Sorry about all the questions but alot of the VM support information is slightly ambiguous.
0
 
LVL 123
ID: 39662687
It's a Microsoft support statement. We have many clients using thin with Exchange, but if you need to escalate a support issue to Microsoft with Exchange, they may decide not to support your environment. This could have changed with the recent release of 2013.

Does this mean that when creating a VM you must create a disk/VMDK file size that will always be smaller than 2TB with the additional size of a snapshot?

This is correct, snapshot does not exceed the size of the parent disk, but at a moment in time, size of snapshot + size of parent must be less than 2TB-512 bytes.

Yes, you will need to use VMware vCenter Converter Standalone, this is the only supported, method, VMware vCenter Converter Standalone can create thick or thin options in the Wizard.

see my EE Article, Step by Step Tutorial Instructions with Screenshots

HOW TO:  P2V, V2V for FREE - VMware vCenter Converter Standalone 5.5


HOW TO:  Synchronize changes when completing a P2V or V2V with VMware vCenter Converter Standalone 5.1

HOW TO: Improve the transfer rate of a Physical to Virtual (P2V), Virtual to Virtual Conversion (V2V) using VMware vCenter Converter Standalone 5.0
0
What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

 

Author Comment

by:markfurey
ID: 39665220
Hello Hanccocka,

Again thanks for your input, but I would just like to clarify a point, we were under the assumption that the snapshot was failing because the datastore was 4TB in size and it has other smaller VM's occupying 300Gb or so, the Exchange VM is using 2TB thin provisioned and the datastore is reporting over 3TB of free diskspace, but when trying to run a snapshot there wasn’t enough diskspace because it was provisioned to 2TB, so the provisioned size of 2TB and the additional 300 Gb of the other VM's means there's less than the required 2TB available to snapshot the Exchange VM.

I think you’re saying that it’s the size of the VMDK combined with the snapshot that's over 2TB that's the issue? In that case how do you calculate the max size of a VMDK that will be able to perform snapshots?

Also regarding the converters, should you use the latest, or the one that matches the version of ESXI?
0
 
LVL 123
ID: 39665325
I think you’re saying that it’s the size of the VMDK combined with the snapshot that's over 2TB that's the issue? In that case how do you calculate the max size of a VMDK that will be able to perform snapshots?

This is correct. You would need to work out the rate of change of data, in your Exchange Server, during the backup Window, or reduce the size of the VMDK. e.g. 1TB.

Also regarding the converters, should you use the latest, or the one that matches the version of ESXI?

I would use the latest version 5.5, as it fixes a lot of bugs. It's compatible with ESXi 5.1.
0
 

Author Comment

by:markfurey
ID: 39666016
Thanks for your help Hanccocka, it's odd that I haven't seen this referenced anywhere else, it effectivley means that you shousld never allocate a VM diskspace of above or around 2TB as you wont be able to get a snapshot or in many cases a backup as vendors like Veem use this technology to run the backup. I will try the conversion in a lab environment before attempting it on a live system. One of my main concerns is the time it takes to run the conversion.

Regards,

Mark.
0
 
LVL 123
ID: 39666038
It's quite common, see my EE Article

HOW TO: VMware Snapshots :- Be Patient

also there is a VMware KB here:-

For ESXi 5.0 and 5.1, this error occurs when any individual flat .vmdk file exceeds 2,181,972,430,848 bytes. Try to keep the size of the disk below 2,181,972,430,848 bytes for normal snapshot operations. This is the size generated when adding a virtual disk with size of 1.984492366201720 TB is specified.

source
http://kb.vmware.com/kb/1012384

This is no longer valid for 5.5, because virtual disks, can be up to 63TB. (of course, it would still not be able to snapshot, if the size was set to 63TB).
0
 

Author Comment

by:markfurey
ID: 39674218
Hello Hanncocka,

Tried using the VM converter over the weekend but it wouldn’t run, as soon as I start the process it shows this error; "unable to obtain hardware information for the selected machine". I thought it might be because the VMDK file was hosted on an ISCSI SAN, so I tried using VM converter 5.5 as it has improved support for SAN's but just got the same error. I have tried it in a lab environment where the VM is located on and a SAN and it just works, any ideas? One thing different is that the live site has a VM infrastructure managed with VCenter Server and the VM conversion is being initiated from this Server, I'm wondering if there's a access/credentials issue here?
0
 
LVL 123
ID: 39674230
Have you installed Converter on the VM to be converted?
0
 

Author Comment

by:markfurey
ID: 39674269
No, I've installed VMconverter on the VCentrer Server as it already has access to the ESXI hosts and VM's within.
0
 
LVL 123

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 1500 total points
ID: 39674286
It's recommended and best practice, to install on the computer to be converted.
0
 

Author Comment

by:markfurey
ID: 39674327
Hello Hanncocka,

Sorry but I'm a little confused, when running a V2V VM conversion the VM is powered off.
0
 
LVL 123
ID: 39674336
Clearly if you turn if off, you cannot do the conversion if the software is installed on the computer to be converted, but many issues can be overcome, by installing converter on the VM to be converted, and converted Live or Hot.

To overcome permission issues, it's often recommended to install on the computer to be converted.

iSCSI or FC SAN should not make any difference, the VM is not aware of where it's hosted.
0
 

Author Comment

by:markfurey
ID: 39674585
Hello Hanncocka,

I'm guessing this would effectively mimic a P2V conversion from within the VM shell/OS? I have added another Windows 2012 VM to the same ESXI host and SAN volume and it looks like the Conversion tool can connect and extract it's hardware details without any issues so it looks like its something specific to this VM, as the other identical VM on the same SAN volume doesn’t appear to have any issues.
0
 
LVL 123
ID: 39674651
Converter, cannot tell if the computer is physical or virtual!
0
 

Author Comment

by:markfurey
ID: 39674755
OK, but I'm guessing it will take a longer time to convert a live machine that one that's powered off? Also, as previously mentioned, a test VM created in the same location/volume using the same Hw, ESXI host and OS is communicating with the converter so I think I'll pursue this line to get it working with the Exchange VM. I'm assuming there isnt any issues with drive size and the converter? Kinda ironic if there is!
0
 
LVL 123
ID: 39674820
No issues with drive size.

In our experience, it's quicker to convert Live than powered off!
0
 

Author Comment

by:markfurey
ID: 39684887
Hello Hanccocka,

Thank you for your assistance with this matter, I went with your suggestion and ran the conversion from within the VM, it completed successfully after about 7.5hrs and the new VM has been running fine on a live system for over two days now at its reduced, thin-provisioned size. Never got to the bottom of why the V2V conversion though, but its all sorted now anyway
0
 
LVL 123
ID: 39684891
Thanks for your feedback.
0

Featured Post

Plug and play, no additional software required!

The ATEN UE3310 USB3.1 Gen1 Extender Cable allows users to extend the distance between the computer and USB devices up to 10 m (33 ft). The UE3310 is a high-quality, cost-effective solution for professional environments such as hospitals, factories and business facilities.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this step by step tutorial with screenshots, we will show you HOW TO: Enable SSH Remote Access on a VMware vSphere Hypervisor 6.5 (ESXi 6.5). This is important if you need to enable SSH remote access for additional troubleshooting of the ESXi hos…
In this article, I will show you HOW TO: Create your first Windows Virtual Machine on a VMware vSphere Hypervisor 6.5 (ESXi 6.5) Host Server, the Windows OS we will install is Windows Server 2016.
Advanced tutorial on how to run the esxtop command to capture a batch file in csv format in order to export the file and use it for performance analysis. He demonstrates how to download the file using a vSphere web client (or vSphere client) and exp…
This Micro Tutorial walks you through using a remote console to access a server and install ESXi 5.1. This example is showing remote access and installation using a Dell server. The hypervisor is the very first component of your virtual infrastructu…
Suggested Courses

765 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question