How do I create a RDM within ESXi 5.0 that is larger than 2TB?
In response to this article, "HOW TO: Add Local Storage (e.g. a SATA disk) as a Raw Disk Mapping (RDM) or Mapped RAW LUN to a virtual machine hosted on ESXi", I have several questions about utilizing RDM for my deployment. I am attempting to utilize a RAID volume that is larger than 2TB within my datastore and was hoping that RDM would be a stable enough solution. I am running ESXi 5.0 while following the mentioned article and have run into an error.
In step 3 of the guide, "Using the vmkfstools command to create the RDM", I am getting an error when I attempt to run the command to provision the RAID volume. I have located my RAID volume and assembled my command, but I am receiving the following error with the accompanying debug info:
# vmkfstools -r /vmfs/devices/disks/naa.XX
DISKLIB-LIB : CREATE: "/vmfs/volumes/datastore1/
DISKLIB-LIB : Only disks up to 2TB-512 are supported for 'vmfsRawDeviceMap' disk.
Failed to create virtual disk: The destination file system does not support large files (12).
In the head of this article, it is stated:
There is a limitation in VMware Hypervsior ESXi 4.x of a 2TB VMFS datastore limit, this can be overcome by using RDMs.
It has been also been stated that this limitation has been removed as of ESXi 5.0. Is there a necessary step to allow partitions greater than 2TB to be created as RDM disks? I was under the impression the change from MBR to GPT was what allowed this functionality, but I am not certain as to what that will require. For your 500GB drive there was no additional preparation, but to overcome the 2TB limitation, is there another step that I need to complete prior to running the vmkfstools command? Possibly creating a GUID Partition table for the drive?
Also, is there a way to use the remaining space on the installation drive for an RDM? For instance, with a large RAID 5 volume with ESX installed on it, the remaining space is partitioned automatically. Is it possible to use the remaining space as an RDM with a size of > 2TB within my datastore?
In step 3 of the guide, "Using the vmkfstools command to create the RDM", I am getting an error when I attempt to run the command to provision the RAID volume. I have located my RAID volume and assembled my command, but I am receiving the following error with the accompanying debug info:
# vmkfstools -r /vmfs/devices/disks/naa.XX
DISKLIB-LIB : CREATE: "/vmfs/volumes/datastore1/
DISKLIB-LIB : Only disks up to 2TB-512 are supported for 'vmfsRawDeviceMap' disk.
Failed to create virtual disk: The destination file system does not support large files (12).
In the head of this article, it is stated:
There is a limitation in VMware Hypervsior ESXi 4.x of a 2TB VMFS datastore limit, this can be overcome by using RDMs.
It has been also been stated that this limitation has been removed as of ESXi 5.0. Is there a necessary step to allow partitions greater than 2TB to be created as RDM disks? I was under the impression the change from MBR to GPT was what allowed this functionality, but I am not certain as to what that will require. For your 500GB drive there was no additional preparation, but to overcome the 2TB limitation, is there another step that I need to complete prior to running the vmkfstools command? Possibly creating a GUID Partition table for the drive?
Also, is there a way to use the remaining space on the installation drive for an RDM? For instance, with a large RAID 5 volume with ESX installed on it, the remaining space is partitioned automatically. Is it possible to use the remaining space as an RDM with a size of > 2TB within my datastore?
Who is Participating?
Experts Exchange Solution brought to you by
Enjoy your complimentary solution view.
Get every solution instantly with Premium.
Start your 7-day free trial.
I wear a lot of hats...
"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.
I have a VM that does media streaming that needs one physical storage device of about 5TBs to put its media files.
In my current situation I don't have external FC or iSCSI storage and so would love to be able to figure out a local RDM solution. I have a RAID5 with almost 6TBs of storage on a Dell Perc 700.
I don't want to use extents with multiple VMDKs, etc.
Has anyone tested this? Thanks.
In my current situation I don't have external FC or iSCSI storage and so would love to be able to figure out a local RDM solution. I have a RAID5 with almost 6TBs of storage on a Dell Perc 700.
I don't want to use extents with multiple VMDKs, etc.
Has anyone tested this? Thanks.
okay so you have a requirement for a larger than 2TB virtual disk.
Yes, we use here with SATA controllers, as mentioned RAID cards cause issues.
again, this is unsupported, and generally regarded as a hack or tweak
Yes, we use here with SATA controllers, as mentioned RAID cards cause issues.
again, this is unsupported, and generally regarded as a hack or tweak
Experts Exchange Solution brought to you by
Your issues matter to us.
Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.
Start your 7-day free trial
One question about the LUN which stores media files? Does it have read and writes or only reads? The physical RAID level at the volume group on the SAN can make a difference when selecting the same for the LUN.
Hi Arunraju,
We're talking about creating an RDM on local storage, not on an external SAN unit.
I'm not sure what you mean by read and writes. The storage in question is a local RAID 5 over 2TBs which we would like to use with RDM. We can write to it as we can create volumes up to 2TBs on it.
We're talking about creating an RDM on local storage, not on an external SAN unit.
I'm not sure what you mean by read and writes. The storage in question is a local RAID 5 over 2TBs which we would like to use with RDM. We can write to it as we can create volumes up to 2TBs on it.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
VMware
From novice to tech pro — start learning today.
Experts Exchange Solution brought to you by
Enjoy your complimentary solution view.
Get every solution instantly with Premium.
Start your 7-day free trial.
I've also seen similar issues, when trying to create unsupported RDMs in this fashion on RAID cards, rather than standard SATA controllers.
VMFS-5 in ESXi 5.0 no longer has a 2TB-512byte VMFS limit, so why not use VMFS-5 1MB Block size to support up to 64TB VMFS.
the vmfs limitation has been removed from ESXi 5.0, but its always been possible to create RDMs larger than 2TB, but not supported.
Note: As of ESXi 5.0, support for LUNs larger then 2TB is available with both VMFS5 datastores and Physical-mode RDMs. However, virtual disks used in virtual machines and Virtual-mode RDMs are still limited to 2TB - 512 bytes.
Source
VMware KB: ESX/ESXi 3.x/4.x hosts do not support 2-terabyte LUNs
kb.vmware.com/kb/3371739
as for your ESXi installation question, personally I would not install ESXi on disk, and use a USB or SD card for installation, in this way ALL the disk storage can be used for the datastore.