Solved

ESXI 5.1 Thin provisioning

Posted on 2013-11-20
19
1,205 Views
Last Modified: 2013-11-29
Hello Experts,

I'm hoping you can clear up some issues I'm having with VMware/ESXI. We have an Exchange VM running on an ISCSI SAN using ESXI 5.1 and VMFS 5 using block sizes of 1MB. The Datastore where the volume is located is 4TB in size, the Exchange VMDK file is in thin provisioned mode a provisioned size of 2.02 TB with 216 GB in use and the data store has 3.67TB free.

The issue is occurring when we try to take a snap-shot using either VCentre Server or using the Veem backup tool, we get the following error; "File <unspecified filename> is larger than the maximum size supported by datastore <unspecified datastore>". I have looked at various internet posts and they indicate various reasons why this might occur. And it looks like the issue is either one of two things; because the provisioned size of the VMDK file which is just over 2TB, although 2TB was selected when it was setup, I have seen posts that suggest that the max size of the VMDK file needs to be slightly less than 2TB

The other reason might be because there isn’t enough disk space available to complete the snap-shot despite the datastore indicating that there is 3.67TB available, this might be because this reading is based on disk space used where the snap-shot calculation is based on the provisioned size, not the space physically used?

So in short, is the problem that there isn’t enough disk space because the "provisioned" size is too big for the snap shot to fit in the datastore, or is it because VMware allowed me to create a VMDK file just over the 2TB threshold.

At the moment we're not getting any snap-shots or backups which we need before pushing some Exchange 2013 updates through.
0
Comment
Question by:markfurey
  • 10
  • 9
19 Comments
 
LVL 117
ID: 39662287
Just a note, that Microsoft do not support thin provisioned disks for Microsoft Exchange.

The issue that you are experiencing is your virtual machine disk plus the snapshot size, exceeds the maximum size of virtual disk (vmdk), which is 2TB-512bytes.

So, a fix, shrink the size of the exchange virtual disk to 500MB-1TB, and then the snapshot plus main virtual disk, will be supported.

Upgrade to ESXi 5.5, 63TB virtual machine disks are supported.

Add a new disk, and move the mailbox store to this new smaller disk.
0
 

Author Comment

by:markfurey
ID: 39662600
Hello Hanccocka,

Than you for your input but I have a couple of questions regarding your response. Regarding Exchange not supporting thin provisioned disks, I haven’t seen this anywhere and I'd assumed that the VMDK type would be transparent to the VM inside anyway?

Also, you say that the snapshot and the VM exceeds max size of the VMDK file, excuse my ignorance but I didn’t think a snapshot increased the size of the VMDK file, I'd assumed that the snapshot would generate a new file not increase the existing one? Does this mean that when creating a VM you must create a disk/VMDK file size that will always be smaller than 2TB with the additional size of a snapshot?

Also I'm not aware of any methods to "shrink" a thin provisioned disk, I'm guessing I would need to use a VMconverter and set the destination drive to the smaller size, but then I think the destination/cloned VMDK would then be "thick" provisioned which may then need yet another conversion to put it back to "thin".

Sorry about all the questions but alot of the VM support information is slightly ambiguous.
0
 
LVL 117
ID: 39662687
It's a Microsoft support statement. We have many clients using thin with Exchange, but if you need to escalate a support issue to Microsoft with Exchange, they may decide not to support your environment. This could have changed with the recent release of 2013.

Does this mean that when creating a VM you must create a disk/VMDK file size that will always be smaller than 2TB with the additional size of a snapshot?

This is correct, snapshot does not exceed the size of the parent disk, but at a moment in time, size of snapshot + size of parent must be less than 2TB-512 bytes.

Yes, you will need to use VMware vCenter Converter Standalone, this is the only supported, method, VMware vCenter Converter Standalone can create thick or thin options in the Wizard.

see my EE Article, Step by Step Tutorial Instructions with Screenshots

HOW TO:  P2V, V2V for FREE - VMware vCenter Converter Standalone 5.5


HOW TO:  Synchronize changes when completing a P2V or V2V with VMware vCenter Converter Standalone 5.1

HOW TO: Improve the transfer rate of a Physical to Virtual (P2V), Virtual to Virtual Conversion (V2V) using VMware vCenter Converter Standalone 5.0
0
 

Author Comment

by:markfurey
ID: 39665220
Hello Hanccocka,

Again thanks for your input, but I would just like to clarify a point, we were under the assumption that the snapshot was failing because the datastore was 4TB in size and it has other smaller VM's occupying 300Gb or so, the Exchange VM is using 2TB thin provisioned and the datastore is reporting over 3TB of free diskspace, but when trying to run a snapshot there wasn’t enough diskspace because it was provisioned to 2TB, so the provisioned size of 2TB and the additional 300 Gb of the other VM's means there's less than the required 2TB available to snapshot the Exchange VM.

I think you’re saying that it’s the size of the VMDK combined with the snapshot that's over 2TB that's the issue? In that case how do you calculate the max size of a VMDK that will be able to perform snapshots?

Also regarding the converters, should you use the latest, or the one that matches the version of ESXI?
0
 
LVL 117
ID: 39665325
I think you’re saying that it’s the size of the VMDK combined with the snapshot that's over 2TB that's the issue? In that case how do you calculate the max size of a VMDK that will be able to perform snapshots?

This is correct. You would need to work out the rate of change of data, in your Exchange Server, during the backup Window, or reduce the size of the VMDK. e.g. 1TB.

Also regarding the converters, should you use the latest, or the one that matches the version of ESXI?

I would use the latest version 5.5, as it fixes a lot of bugs. It's compatible with ESXi 5.1.
0
 

Author Comment

by:markfurey
ID: 39666016
Thanks for your help Hanccocka, it's odd that I haven't seen this referenced anywhere else, it effectivley means that you shousld never allocate a VM diskspace of above or around 2TB as you wont be able to get a snapshot or in many cases a backup as vendors like Veem use this technology to run the backup. I will try the conversion in a lab environment before attempting it on a live system. One of my main concerns is the time it takes to run the conversion.

Regards,

Mark.
0
 
LVL 117
ID: 39666038
It's quite common, see my EE Article

HOW TO: VMware Snapshots :- Be Patient

also there is a VMware KB here:-

For ESXi 5.0 and 5.1, this error occurs when any individual flat .vmdk file exceeds 2,181,972,430,848 bytes. Try to keep the size of the disk below 2,181,972,430,848 bytes for normal snapshot operations. This is the size generated when adding a virtual disk with size of 1.984492366201720 TB is specified.

source
http://kb.vmware.com/kb/1012384

This is no longer valid for 5.5, because virtual disks, can be up to 63TB. (of course, it would still not be able to snapshot, if the size was set to 63TB).
0
 

Author Comment

by:markfurey
ID: 39674218
Hello Hanncocka,

Tried using the VM converter over the weekend but it wouldn’t run, as soon as I start the process it shows this error; "unable to obtain hardware information for the selected machine". I thought it might be because the VMDK file was hosted on an ISCSI SAN, so I tried using VM converter 5.5 as it has improved support for SAN's but just got the same error. I have tried it in a lab environment where the VM is located on and a SAN and it just works, any ideas? One thing different is that the live site has a VM infrastructure managed with VCenter Server and the VM conversion is being initiated from this Server, I'm wondering if there's a access/credentials issue here?
0
 
LVL 117
ID: 39674230
Have you installed Converter on the VM to be converted?
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 

Author Comment

by:markfurey
ID: 39674269
No, I've installed VMconverter on the VCentrer Server as it already has access to the ESXI hosts and VM's within.
0
 
LVL 117

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE) earned 500 total points
ID: 39674286
It's recommended and best practice, to install on the computer to be converted.
0
 

Author Comment

by:markfurey
ID: 39674327
Hello Hanncocka,

Sorry but I'm a little confused, when running a V2V VM conversion the VM is powered off.
0
 
LVL 117
ID: 39674336
Clearly if you turn if off, you cannot do the conversion if the software is installed on the computer to be converted, but many issues can be overcome, by installing converter on the VM to be converted, and converted Live or Hot.

To overcome permission issues, it's often recommended to install on the computer to be converted.

iSCSI or FC SAN should not make any difference, the VM is not aware of where it's hosted.
0
 

Author Comment

by:markfurey
ID: 39674585
Hello Hanncocka,

I'm guessing this would effectively mimic a P2V conversion from within the VM shell/OS? I have added another Windows 2012 VM to the same ESXI host and SAN volume and it looks like the Conversion tool can connect and extract it's hardware details without any issues so it looks like its something specific to this VM, as the other identical VM on the same SAN volume doesn’t appear to have any issues.
0
 
LVL 117
ID: 39674651
Converter, cannot tell if the computer is physical or virtual!
0
 

Author Comment

by:markfurey
ID: 39674755
OK, but I'm guessing it will take a longer time to convert a live machine that one that's powered off? Also, as previously mentioned, a test VM created in the same location/volume using the same Hw, ESXI host and OS is communicating with the converter so I think I'll pursue this line to get it working with the Exchange VM. I'm assuming there isnt any issues with drive size and the converter? Kinda ironic if there is!
0
 
LVL 117
ID: 39674820
No issues with drive size.

In our experience, it's quicker to convert Live than powered off!
0
 

Author Comment

by:markfurey
ID: 39684887
Hello Hanccocka,

Thank you for your assistance with this matter, I went with your suggestion and ran the conversion from within the VM, it completed successfully after about 7.5hrs and the new VM has been running fine on a live system for over two days now at its reduced, thin-provisioned size. Never got to the bottom of why the V2V conversion though, but its all sorted now anyway
0
 
LVL 117
ID: 39684891
Thanks for your feedback.
0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

Suggested Solutions

Create your own, high-performance VM backup appliance by installing NAKIVO Backup & Replication directly onto a Synology NAS!
HOW TO: Upload an ISO image to a VMware datastore for use with VMware vSphere Hypervisor 6.5 (ESXi 6.5) using the vSphere Host Client, and checking its MD5 checksum signature is correct.  It's a good idea to compare checksums, because many installat…
Teach the user how to use configure the vCenter Server storage filters Open vSphere Web Client:  Navigate to vCenter Server Advanced Settings: Add the four vCenter Server storage filters: Review the advanced settings: Modify the values of the four v…
Teach the user how to install and configure the vCenter Orchestrator virtual appliance Open vSphere Web Client: Deploy vCenter Orchestrator virtual appliance OVA file: Verify vCenter Orchestrator virtual appliance boots successfully: Connect to the …

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now