The Percona Live Open Source Database Conference Europe 2017 is the premier event for the diverse and active European open source database community, as well as businesses that develop and use open source database software.
COURSE of the MONTH
15 days, 4 hours left to enrollBecome a Premium Member and unlock a new, free course in leading technologies each month.
Solved
How do I create a RDM within ESXi 5.0 that is larger than 2TB?
Posted on 2012-03-30
In response to this article, "HOW TO: Add Local Storage (e.g. a SATA disk) as a Raw Disk Mapping (RDM) or Mapped RAW LUN to a virtual machine hosted on ESXi", I have several questions about utilizing RDM for my deployment. I am attempting to utilize a RAID volume that is larger than 2TB within my datastore and was hoping that RDM would be a stable enough solution. I am running ESXi 5.0 while following the mentioned article and have run into an error.
In step 3 of the guide, "Using the vmkfstools command to create the RDM", I am getting an error when I attempt to run the command to provision the RAID volume. I have located my RAID volume and assembled my command, but I am receiving the following error with the accompanying debug info:
# vmkfstools -r /vmfs/devices/disks/naa.XX
DISKLIB-LIB : CREATE: "/vmfs/volumes/datastore1/
DISKLIB-LIB : Only disks up to 2TB-512 are supported for 'vmfsRawDeviceMap' disk.
Failed to create virtual disk: The destination file system does not support large files (12).
In the head of this article, it is stated:
There is a limitation in VMware Hypervsior ESXi 4.x of a 2TB VMFS datastore limit, this can be overcome by using RDMs.
It has been also been stated that this limitation has been removed as of ESXi 5.0. Is there a necessary step to allow partitions greater than 2TB to be created as RDM disks? I was under the impression the change from MBR to GPT was what allowed this functionality, but I am not certain as to what that will require. For your 500GB drive there was no additional preparation, but to overcome the 2TB limitation, is there another step that I need to complete prior to running the vmkfstools command? Possibly creating a GUID Partition table for the drive?
Also, is there a way to use the remaining space on the installation drive for an RDM? For instance, with a large RAID 5 volume with ESX installed on it, the remaining space is partitioned automatically. Is it possible to use the remaining space as an RDM with a size of > 2TB within my datastore?
In step 3 of the guide, "Using the vmkfstools command to create the RDM", I am getting an error when I attempt to run the command to provision the RAID volume. I have located my RAID volume and assembled my command, but I am receiving the following error with the accompanying debug info:
# vmkfstools -r /vmfs/devices/disks/naa.XX
DISKLIB-LIB : CREATE: "/vmfs/volumes/datastore1/
DISKLIB-LIB : Only disks up to 2TB-512 are supported for 'vmfsRawDeviceMap' disk.
Failed to create virtual disk: The destination file system does not support large files (12).
In the head of this article, it is stated:
There is a limitation in VMware Hypervsior ESXi 4.x of a 2TB VMFS datastore limit, this can be overcome by using RDMs.
It has been also been stated that this limitation has been removed as of ESXi 5.0. Is there a necessary step to allow partitions greater than 2TB to be created as RDM disks? I was under the impression the change from MBR to GPT was what allowed this functionality, but I am not certain as to what that will require. For your 500GB drive there was no additional preparation, but to overcome the 2TB limitation, is there another step that I need to complete prior to running the vmkfstools command? Possibly creating a GUID Partition table for the drive?
Also, is there a way to use the remaining space on the installation drive for an RDM? For instance, with a large RAID 5 volume with ESX installed on it, the remaining space is partitioned automatically. Is it possible to use the remaining space as an RDM with a size of > 2TB within my datastore?
[X]
Welcome to Experts Exchange
Add your voice to the tech community where 5M+ people just like you are talking about what matters.
- Help others & share knowledge
- Earn cash & points
- Learn & ask questions
2-
2
5 Comments
Active today
Why do you need to use a RDM? when VMFS-5 supports large datastores?
I've also seen similar issues, when trying to create unsupported RDMs in this fashion on RAID cards, rather than standard SATA controllers.
VMFS-5 in ESXi 5.0 no longer has a 2TB-512byte VMFS limit, so why not use VMFS-5 1MB Block size to support up to 64TB VMFS.
the vmfs limitation has been removed from ESXi 5.0, but its always been possible to create RDMs larger than 2TB, but not supported.
Note: As of ESXi 5.0, support for LUNs larger then 2TB is available with both VMFS5 datastores and Physical-mode RDMs. However, virtual disks used in virtual machines and Virtual-mode RDMs are still limited to 2TB - 512 bytes.
Source
VMware KB: ESX/ESXi 3.x/4.x hosts do not support 2-terabyte LUNs
kb.vmware.com/kb/3371739
as for your ESXi installation question, personally I would not install ESXi on disk, and use a USB or SD card for installation, in this way ALL the disk storage can be used for the datastore.
I've also seen similar issues, when trying to create unsupported RDMs in this fashion on RAID cards, rather than standard SATA controllers.
VMFS-5 in ESXi 5.0 no longer has a 2TB-512byte VMFS limit, so why not use VMFS-5 1MB Block size to support up to 64TB VMFS.
the vmfs limitation has been removed from ESXi 5.0, but its always been possible to create RDMs larger than 2TB, but not supported.
Note: As of ESXi 5.0, support for LUNs larger then 2TB is available with both VMFS5 datastores and Physical-mode RDMs. However, virtual disks used in virtual machines and Virtual-mode RDMs are still limited to 2TB - 512 bytes.
Source
VMware KB: ESX/ESXi 3.x/4.x hosts do not support 2-terabyte LUNs
kb.vmware.com/kb/3371739
as for your ESXi installation question, personally I would not install ESXi on disk, and use a USB or SD card for installation, in this way ALL the disk storage can be used for the datastore.
I have a VM that does media streaming that needs one physical storage device of about 5TBs to put its media files.
In my current situation I don't have external FC or iSCSI storage and so would love to be able to figure out a local RDM solution. I have a RAID5 with almost 6TBs of storage on a Dell Perc 700.
I don't want to use extents with multiple VMDKs, etc.
Has anyone tested this? Thanks.
In my current situation I don't have external FC or iSCSI storage and so would love to be able to figure out a local RDM solution. I have a RAID5 with almost 6TBs of storage on a Dell Perc 700.
I don't want to use extents with multiple VMDKs, etc.
Has anyone tested this? Thanks.
Active today
okay so you have a requirement for a larger than 2TB virtual disk.
Yes, we use here with SATA controllers, as mentioned RAID cards cause issues.
again, this is unsupported, and generally regarded as a hack or tweak
Yes, we use here with SATA controllers, as mentioned RAID cards cause issues.
again, this is unsupported, and generally regarded as a hack or tweak
Featured Post
Question has a verified solution.
If you are experiencing a similar issue, please ask a related question
Fine Tune your automatic Updates for Ubuntu / Debian
Google Drive is extremely cheap offsite storage, and it's even possible to get extra storage for free for two years. You can use the free account 15GB, and if you have an Android device..when you install Google Drive for the first time it will give…
Advanced tutorial on how to run the esxtop command to capture a batch file in csv format in order to export the file and use it for performance analysis. He demonstrates how to download the file using a vSphere web client (or vSphere client) and exp…
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting.
This w…
Suggested Courses
Course of the Month15 days, 4 hours left to enroll
770 members asked questions and received personalized solutions in the past 7 days.
Join the community of 500,000 technology professionals and ask your questions.