Solved

Esx 2tb limit

Posted on 2010-11-22
23
1,107 Views
Last Modified: 2012-05-10
Hi all I've been browsing around but prbly I wasn't thorough    Enough to find the answer,for that I apologize in advance. My situation is like this I have a Dell r710 server with percent h700 and 6 disks x 2tb each. I need to install Vmware esx. My first attempt has failed since I created 1 raid 5 virtual disk of approximate 10tb. Esx wasnot able to handle it. The datastore was 2 tb max. I tried to expand but  I got errors saying that this wasn't possible. My question has 2 parts. Is it possible to bypass the 2tb limit with my current raid configuration ? And if not what should be my next step configuring the raid, to maximize the datastore size?
Many Thanx in Advance.
0
Comment
Question by:thanosgr
  • 8
  • 7
  • 7
  • +1
23 Comments
 

Author Comment

by:thanosgr
Comment Utility
Percent = PERC (damn iPhone autocorrect:)
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
Nope, you have to use extents.

The reason, so you know why to just give up ... is that FFFFFFFF hex blocks = 2.09TB.  This is the largest number of blocks that is addressable with the 10-byte SCSI command set which gives you 32 bits to address the block number.  

ESX still uses 10-byte SCSI commands, and expanding that is major rewrite.

0
 

Author Comment

by:thanosgr
Comment Utility
Extends having the same 10th virtual disk or using a different raid configuration?
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
But just be prepared for crappy performance, every write on every VM is going to generate I/O on EVERY disk you have in a RAID5.  You will probably be extremely unhappy with RAID5 performance.

I strongly suggest going with pairs of 2TB RAID1s.  Write performance will be probably 4X faster, and read, at least 2X faster then a RAID5.
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
I wouldn't use extends at all, unless you MUST have a single file that is > 2TB.   Just make pairs of RAID1s, and allocate as necessary.   Depending on specifics of the config, you could well be have even a greater hit then 4-8X then with a RAID5 you carve out in the RAID controller, then use VM extends to combine back.  Every I/O on every VM could hit every disk several times that way.

0
 
LVL 22

Expert Comment

by:Luciano Patrão
Comment Utility
Hi

This is the limits to ESX datastore sizes

http://kb.vmware.com/kb/1003565

and

http://www.vmware.com/pdf/vsphere4/r41/vsp_41_config_max.pdf

So you need to create LUNs and Datastore using this limits.

Block sizes is very important for creating the Datastore.

Can you explain why you need large Datastore? You can create 3 or 4 small(like 2Tb) and add your VMs into it.

I don't like to create large LUNs, I prefer create small ones for a group or VMs, than large Datastorare.

Regarding RAID, well RAID10 is the best for holding the VMs. Also RAID6 is a good solution. Just use one drive as hot-spare. This 2 is the best RAID.

Jail
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
it is a myth that RAID 10 is best.  it depends on the secific config.

RAID10 is only always best on single threaded, large block, sequential I/O.  2 RAID1s can deliver twice as many random IOPs as a single RAID10.

ESX is more of a random load:
0
 
LVL 3

Expert Comment

by:Virtalicious
Comment Utility
I would only slightly disagree Raid 10 performance is better there is no condition that Raid 5 would perform faster reads or writes, however thats not what you would use it.

I would recommend that you stay well below 2TB as the best performance to date is on 500GB Luns with a 1MB Block size.  I have tried many in many environments (many being far more than 100) in large scale environments and testing scenarios 500/1MB performs the best bar none.

-Virt
0
 
LVL 22

Expert Comment

by:Luciano Patrão
Comment Utility
Hi

One think I agree, small LUNs have a better performance. But in many environments 500Gb is low.

Jail
0
 

Author Comment

by:thanosgr
Comment Utility
Actually there is no need for the big datastore. But now I'm confused, raid 10 or 3 pairs of mirrors? The esx will initially host a SQL server for data warehousing, and a sharepoint server for 50 users, and another Cognos express server. So performance is an issue. Can u please decide 4 me? :)

Many Thanx for all ur replies
0
 
LVL 22

Expert Comment

by:Luciano Patrão
Comment Utility
Hi

In my opinion for that configuration I will create some LUNs(like 500Gb or 1Tb if you need more Data) with RAID10.

Sorry but I prefer this, than 2 RAID1.

Jail
0
Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 47

Expert Comment

by:dlethe
Comment Utility
2 x RAID1s will provide more I/Os per second  then a single RAID10, but it will be more difficult to manage.  The choice is yours, and what is "BEST" is a function of manageability vs. performance, and only you can make that determination.

I am just telling you that RAID10, due to the architecture can not possibly provide more I/Os per second then 2 x RAID1s.
0
 
LVL 22

Expert Comment

by:Luciano Patrão
Comment Utility
Hi

@dlethe I never tested that, so I cannot replicate that, but I believe that you have tested and as better performance. But still, I prefer a RAID10 than 2 RAID1 :)

But like you said, is more difficult to manage, and will "steal" more disk space than a RAID10.

But like I said before, this is just my personal opinion.

Jail
0
 

Author Comment

by:thanosgr
Comment Utility

pls excuse my stupidity.
the 3 mirrored pairs works just fine but when i created a raid 10 Virtual Disk in H700 (approx 6TB)  but vmware is only able to see only 2 tb ..and there is no Disk/LUN available when i try to extend or create another datasource what am i doing wrong?
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
That is because VMWARE can not handle a 6TB LUN when exposed from an external controller.  You simply can not expose a LUN > 2.09 TB to VMWARE as it won't be able to address any block higher than block # FFFFFFFF hex, which is the largest number of blocks addressable in 32 bits.  That is due to the 10-byte SCSI command limitation which only has 32-bits to stick in a block number.

VMWare uses 10-byte SCSI commands to communicate with the hardware.   Using extents is a work-around within vmware, where vmware does mapping internally ... but it still sends only 10byte SCSI commands to the physical or logical disks exposed by your storage JBOD or RAID controllers.

There is no work around.  VMWare needs to redo the kernel to support 16-byte SCSI commands to fix it, or to go with block sizes > 512 bytes (which the H700 doesn't support anyway, and this is brand-new up-and-coming stuff).
0
 
LVL 22

Expert Comment

by:Luciano Patrão
Comment Utility
Hi

You can create a 6Tb VD in the Storage side, but you need to create LUNs smaller than 2Tb for VMware recognize the LUN.

Jail
0
 

Author Comment

by:thanosgr
Comment Utility
This will drive me nuts. I have the big Vd , but how Do I create the smaller LUNS?
0
 
LVL 22

Expert Comment

by:Luciano Patrão
Comment Utility
Hi

I do no know you Storage. What Storage are you using?

Jail
0
 

Author Comment

by:thanosgr
Comment Utility
I have a Dell r710 server with percent h700 and 6 disks x 2tb each
0
 
LVL 22

Expert Comment

by:Luciano Patrão
Comment Utility
Hi

Ahh :(

Sorry I think you are using a Storage like SAN. In this case you cannot create a VD and create several LUNs. You can only create VD(one or more).

You have 6 disks with 2Tb each? 12Tb of Storage?

Jail
0
 

Author Comment

by:thanosgr
Comment Utility
yup thats exactly my case  

and on the same server that i have esx

0
 
LVL 47

Accepted Solution

by:
dlethe earned 500 total points
Comment Utility
so we are still at creating 3 x 2TB RAID1s in the controller.  Just do it and move on.  
0
 

Author Closing Comment

by:thanosgr
Comment Utility
ill definatelly move on, by installing XEN :))))

many thanx guyz
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Join & Write a Comment

Last article we focus in how to VMware: How to create and use VMs TAGs – Part 1 so before follow this article and perform the next tasks, you should read the first article how to create the TAG before using them in Veeam Backup Jobs.
In this article, I will show you HOW TO: Create your first Windows Virtual Machine on a VMware vSphere Hypervisor 6.5 (ESXi 6.5) Host Server, the Windows OS we will install is Windows Server 2016.
Teach the user how to delpoy the vCenter Server Appliance and how to configure its network settings Deploy OVF: Open VM console and configure networking:
This Micro Tutorial steps you through the configuration steps to configure your ESXi host Management Network settings and test the management network, ensure the host is recognized by the DNS Server, configure a new password, and the troubleshooting…

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now