Community Pick: Many members of our community have endorsed this article.
Editor's Choice: This article has been selected by our editors as an exceptional contribution.

HOW TO: Add Local Storage (e.g. a SATA disk) as a Raw Disk Mapping (RDM) or Mapped RAW LUN to a virtual machine hosted on ESXi

Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)VMware and Virtualization Consultant
CERTIFIED EXPERT
EE Fellow, MVE, Expert of the Year 2021,2017-11, Scribe 2016-2012, Author of the Year 2018-6,2013-2012 VMware vExpert Pro, vExpert 2022-2011
Published:
Updated:
In this article, I am going to expose some of the hidden, undocumented, unsupported features and functions of the VMware vSphere Hypervisor (ESXi).

VMware vSphere ESXi 4.x, and ESXi 5.0 does not officially support Raw Disk Mapping of Local Storage e.g SATA, SAS or SCSI disks.

Please use at your own risk, and if you do escalate a Support Request (SR) to VMware Support, do not be surprised if they state this is unsupported.


VMware's definition of an RDM
'Introduced with VMware ESX Server 2.5, raw device mapping allows a special file in a VMFS volume to act as a proxy for a raw device. The mapping file contains metadata used to manage and redirect disk accesses to the physical device. The mapping file gives you some of the advantages of a virtual disk in the VMFS file system, while keeping some advantages of direct access to physical device characteristics. In effect, it merges VMFS manageability with raw device access.'

Traditionally and VMware supported, a Raw Disk Mapping (RDM) which may be used to present a LUN directly to a virtual machine from a SAN. The SAN could be Fibre Channel or iSCSI.

Why would we want to create a Raw Disk Mapping (RDM) of a local storage device?

An RDM can offer improved performance, which can be used for more disk I/O intensive applications such as database servers.
You may already have a data partition on the disk you want to present directly to the VM.
There is a limitation in VMware Hypervsior ESXi 4.x of a 2TB VMFS datastore limit, this can be overcome by using RDMs.
VMware Hypervisor ESXi 4.x and VMware Hypervisor ESXi 5.0 have a 2TB virtual disk (VMDK) limit.
You may want to have access directly to local storage to create a JBOD, because you do not have a supported storage controller,
SCSI controller may not be supported by VMware Hypervisor ESXi.
Creating a VMFS datastore on a single SATA drive can be slow.
Creating a 500GB virtual disk on a single SATA drive can be slow.

A few valid reasons why you may want to create an RDM of a local storage device.

This solution creates a RDM virtual disk proxy mapping file on an existing mounted VMFS datastore. This virtual disk proxy mapping "file" is then added to a Virtual Machine. I would recommend that RDM disks are used for data storage only e.g. SQL databases, large file servers.

The following procedure can be used to create the RDM on a local storage device. In the following example, a Seagate ST3500320AS 7200k rpm 500GB SATA disk is going to be used to create an RDM.

1. Connect to the VMware Hypervisor ESXi server via SSH

Using PuTTY a free telnet and SSH client or another SSH client Login and Connect to the VMware Hypervisor ESXi server, using IP address or hostname of the VMware Hypervisor ESXi server, using root username and password credentials.

putty SSH terminal session
logged in as root to ssh terminal session

2. Obtain the physical disk path of the storage device


There are a number of methods that can be used to find the physical disk path, the vSphere Client can be used, check Storage Adaptors, but the easiest way to find the physical disk path of your storage device is to use the commands

esxcfg-mpath -l

output from esxcg-mpath
or

ls -al /vmfs/devices/disks

output from ls -al /vmfs/devices/disks
using the above commands, we can cut and paste the physical disk path, which when using SATA devices can be very long and difficult to type.

In the above sessions the physical disk path is

t10.ATA_____ST3500320AS_________________________________________XXXXXXXX

(i've obsecured the screenshots above, because it includes a serial number for the Seagate ST3500320AS 7200k rpm SATA disk.). I've also replaced the serial number with Xs above.

If using a SCSI storage controller the physical disk path could be
example vmhba4:1:0:0


3. Using the vmkfstools command to create the RDM


Using the command vmkfstools we can create a RDM linked to a local storage device.

the command syntax is

vmkfstools -r <source> (space) <destination>

vmkfstools -r  /vmfs/devices/disks/<physical disk path> (space) /vmfs/volumes/[b]<vmfs datastore name>[/b]/<folder name>/<virtual disk name>.vmdk

at the console prompt type

vmkfstools -r  /vmfs/devices/disks/t10.ATA_____ST3500320AS_________________________________________XXXXXXXX /vmfs/volumes/esxdev004_local1_vmfs3/SATA500G/SATA500G.vmdk

esxdev004_local1_vmfs3 is the friendly VMFS datastore name
SATA500G is a folder created on the VMFS datastore to store the proxy mapping (vmdk) file.
SATA500G.vmdk is the name of the proxy mapping file (false virtual disk - vmdk.)

I've replaced the serial number with Xs above.

After entering the above command, vmkfstools does not respond with any output this is normal. If you check the datastore, you will see that a virtual machine disk named SATA500G.vmdk has been created.

SATA500G.vmdk

4. Adding the RDM to a Virtual Machine


Using the VMware vSphere Client, Login and Connect to the ESXi server, using IP address or hostname of the ESXi server, using root username and password credentials. If you have a VMware vSphere vCenter server, you could also specify IP address or hostname of the vCenter server.

Using the VMware vSphere Client, Login and Connect to the ESXi server

5. Select an existing virtual machine


Select an existing virtual machine, you want to add the RDM to, and select Edit Settings. The new hard disk can be added whilst the virtual machine is powered on or off. Hotplug is supported on some Windows OS.

 Edit Settings of Virtual Machine
Click Add to add a new device

Add a new Device
Click Add Hard Disk followed by Next

Add Hard Disk
Select Use an existing Virtual Disk
(please do not select Raw Device Mapping)

Use An Existing Disk
Browse the datastore and select the VMDK file and click OK, followed by Next

Browse Datastore for VMDK
Select a SCSI Virtual Device Mode. Select Mode - Independent - Persistent. Followed by Next.

Select Mode - Independent - Persistent.
Review the selected options and Click Finish

Disk Configuration Finished
The virtual machine will report a Mapped Raw LUN has been added.

Mapped Raw LUN has been added
The "magic" has been completed and we have successfully added a Mapped Raw LUN (RDM), which is the local SATA 500GB hard disk. Check that the words Mapped Raw LUN are present.

Click OK and wait several seconds for the device to be added to the virtual machine.

6. Checking the hard disk in the Virtual Machine


Open a console and login to the virtual machine OS. In this example Windows XP Professional.

Select Windows Disk Management (My Computer, Properties, Manage, Disk Management).

Before Rescan Disk Managment in Windows XP
The above screenshot, is before we have used the Rescan Disks option, as the hard disk was hot added.

Right Click Disk Management - Rescan Disks

After Rescan Disk Managment in Windows XP
A New Volume E: NTFS has been added to the virtual machine. This was an existing formatted NTFS disk with existing data we required in the VM.

Just also to note the virtual machine disk file VMDK, SATA500G.vmdk created although this file appears to be 500GB on disk, it's a small proxy mapping file, which does not take up this amount of disk space on the datastore. In this example above the local datastore is too small to accomodate a 500GB vmdk file.

You have successully created an RDM and added it to a virtual machine.

Thank you for reading my article, please leave valuable feedback.
9
97,105 Views
Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)VMware and Virtualization Consultant
CERTIFIED EXPERT
EE Fellow, MVE, Expert of the Year 2021,2017-11, Scribe 2016-2012, Author of the Year 2018-6,2013-2012 VMware vExpert Pro, vExpert 2022-2011

Comments (12)

Commented:
Is there a solution for this?

DISKLIB-LIB   : Only disks up to 2TB-512 are supported for 'vmfsRawDeviceMap' disk.
Failed to create virtual disk: The destination file system does not support large files (12).

I have a 3TB drive with data on I wish to copy to a new vmware platform so can't reformat it.

Commented:
hanccocka:
I'm having an issue with a new SSHD 1 TB drive that's installed on the ESXi 4.1 host not showing the sata disk i.e. "ls -al /vmfs/devices/disks". Please see output below. Thanks!

login as: root
The time and date of this activation have been sent to the system logs.

VMware offers supported, powerful system administration tools.  Please
see www.vmware.com/go/sysadmintools for details.


~ # ls -al /vmfs/devices/disks
drwxr-xr-x    1 root     root                512 Oct  5 17:20 .
drwxr-xr-x    1 root     root                512 Oct  5 17:20 ..
-rw-------    1 root     root       146163105792 Oct  5 17:20 naa.6001c230cd19b5                                                                                                                               0019df6a6d377dcabd
-rw-------    1 root     root          939524096 Oct  5 17:20 naa.6001c230cd19b5                                                                                                                               0019df6a6d377dcabd:1
-rw-------    1 root     root         4293918720 Oct  5 17:20 naa.6001c230cd19b5                                                                                                                               0019df6a6d377dcabd:2
-rw-------    1 root     root       140925468672 Oct  5 17:20 naa.6001c230cd19b5                                                                                                                               0019df6a6d377dcabd:3
-rw-------    1 root     root            4177920 Oct  5 17:20 naa.6001c230cd19b5                                                                                                                               0019df6a6d377dcabd:4
-rw-------    1 root     root          262127616 Oct  5 17:20 naa.6001c230cd19b5                                                                                                                               0019df6a6d377dcabd:5
-rw-------    1 root     root          262127616 Oct  5 17:20 naa.6001c230cd19b5                                                                                                                               0019df6a6d377dcabd:6
-rw-------    1 root     root          115326976 Oct  5 17:20 naa.6001c230cd19b5                                                                                                                               0019df6a6d377dcabd:7
-rw-------    1 root     root          299876352 Oct  5 17:20 naa.6001c230cd19b5                                                                                                                               0019df6a6d377dcabd:8
lrwxrwxrwx    1 root     root                 36 Oct  5 17:20 vml.02000000006001                                                                                                                               c230cd19b50019df6a6d377dcabd504552432035 -> naa.6001c230cd19b50019df6a6d377dcabd
lrwxrwxrwx    1 root     root                 38 Oct  5 17:20 vml.02000000006001                                                                                                                               c230cd19b50019df6a6d377dcabd504552432035:1 -> naa.6001c230cd19b50019df6a6d377dca                                                                                                                               bd:1
lrwxrwxrwx    1 root     root                 38 Oct  5 17:20 vml.02000000006001                                                                                                                               c230cd19b50019df6a6d377dcabd504552432035:2 -> naa.6001c230cd19b50019df6a6d377dca                                                                                                                               bd:2
lrwxrwxrwx    1 root     root                 38 Oct  5 17:20 vml.02000000006001                                                                                                                               c230cd19b50019df6a6d377dcabd504552432035:3 -> naa.6001c230cd19b50019df6a6d377dca                                                                                                                               bd:3
lrwxrwxrwx    1 root     root                 38 Oct  5 17:20 vml.02000000006001                                                                                                                               c230cd19b50019df6a6d377dcabd504552432035:4 -> naa.6001c230cd19b50019df6a6d377dca                                                                                                                               bd:4
lrwxrwxrwx    1 root     root                 38 Oct  5 17:20 vml.02000000006001                                                                                                                               c230cd19b50019df6a6d377dcabd504552432035:5 -> naa.6001c230cd19b50019df6a6d377dca                                                                                                                               bd:5
lrwxrwxrwx    1 root     root                 38 Oct  5 17:20 vml.02000000006001                                                                                                                               c230cd19b50019df6a6d377dcabd504552432035:6 -> naa.6001c230cd19b50019df6a6d377dca                                                                                                                               bd:6
lrwxrwxrwx    1 root     root                 38 Oct  5 17:20 vml.02000000006001                                                                                                                               c230cd19b50019df6a6d377dcabd504552432035:7 -> naa.6001c230cd19b50019df6a6d377dca                                                                                                                               bd:7
lrwxrwxrwx    1 root     root                 38 Oct  5 17:20 vml.02000000006001                                                                                                                               c230cd19b50019df6a6d377dcabd504552432035:8 -> naa.6001c230cd19b50019df6a6d377dca                                                                                                                               bd:8
~ #

Open in new window

CERTIFIED EXPERT
Fellow
Expert of the Year 2021

Author

Commented:
@crjaq

Please post a question in the VMware Zone, for myself or other Experts to Answer.

Thanks
Hello

It is a pleasure to even ask you a question.

The file server itself is a virtual machine running on a VMware ESXi 5.1.0 host. The D: drive is a Mapped Raw LUN which resides on a HP Lefthand SAN cluster connected via an iSCSI network

The problem is intermittent, the drive is not accessible even from the virtual machine. I checked event viewer logs to get to know the problem, but there were no logs generated. Is there a way to find out why would it be not accessible.

Your advice would be a life saver.

Thanks
CERTIFIED EXPERT
Fellow
Expert of the Year 2021

Author

Commented:
@rauldeshmukh

Please post a question in the VMware Zone, for myself or other Experts to Answer.

Thanks

View More

Have a question about something in this article? You can receive help directly from the article author. Sign up for a free trial to get started.