Esx 2tb limit

Percent = PERC (damn iPhone autocorrect:)

Nope, you have to use extents.

The reason, so you know why to just give up ... is that FFFFFFFF hex blocks = 2.09TB.  This is the largest number of blocks that is addressable with the 10-byte SCSI command set which gives you 32 bits to address the block number.  

ESX still uses 10-byte SCSI commands, and expanding that is major rewrite.

Extends having the same 10th virtual disk or using a different raid configuration?

But just be prepared for crappy performance, every write on every VM is going to generate I/O on EVERY disk you have in a RAID5.  You will probably be extremely unhappy with RAID5 performance.

I strongly suggest going with pairs of 2TB RAID1s.  Write performance will be probably 4X faster, and read, at least 2X faster then a RAID5.

I wouldn't use extends at all, unless you MUST have a single file that is > 2TB.   Just make pairs of RAID1s, and allocate as necessary.   Depending on specifics of the config, you could well be have even a greater hit then 4-8X then with a RAID5 you carve out in the RAID controller, then use VM extends to combine back.  Every I/O on every VM could hit every disk several times that way.

Hi

This is the limits to ESX datastore sizes

http://kb.vmware.com/kb/1003565

and

http://www.vmware.com/pdf/vsphere4/r41/vsp_41_config_max.pdf

So you need to create LUNs and Datastore using this limits.

Block sizes is very important for creating the Datastore.

Can you explain why you need large Datastore? You can create 3 or 4 small(like 2Tb) and add your VMs into it.

I don't like to create large LUNs, I prefer create small ones for a group or VMs, than large Datastorare.

Regarding RAID, well RAID10 is the best for holding the VMs. Also RAID6 is a good solution. Just use one drive as hot-spare. This 2 is the best RAID.

Jail

it is a myth that RAID 10 is best.  it depends on the secific config.

RAID10 is only always best on single threaded, large block, sequential I/O.  2 RAID1s can deliver twice as many random IOPs as a single RAID10.

ESX is more of a random load:

I would only slightly disagree Raid 10 performance is better there is no condition that Raid 5 would perform faster reads or writes, however thats not what you would use it.

I would recommend that you stay well below 2TB as the best performance to date is on 500GB Luns with a 1MB Block size.  I have tried many in many environments (many being far more than 100) in large scale environments and testing scenarios 500/1MB performs the best bar none.

-Virt

Hi

One think I agree, small LUNs have a better performance. But in many environments 500Gb is low.

Jail

Actually there is no need for the big datastore. But now I'm confused, raid 10 or 3 pairs of mirrors? The esx will initially host a SQL server for data warehousing, and a sharepoint server for 50 users, and another Cognos express server. So performance is an issue. Can u please decide 4 me? :)

Many Thanx for all ur replies

Hi

In my opinion for that configuration I will create some LUNs(like 500Gb or 1Tb if you need more Data) with RAID10.

Sorry but I prefer this, than 2 RAID1.

Jail

2 x RAID1s will provide more I/Os per second  then a single RAID10, but it will be more difficult to manage.  The choice is yours, and what is "BEST" is a function of manageability vs. performance, and only you can make that determination.

I am just telling you that RAID10, due to the architecture can not possibly provide more I/Os per second then 2 x RAID1s.

Hi

@dlethe I never tested that, so I cannot replicate that, but I believe that you have tested and as better performance. But still, I prefer a RAID10 than 2 RAID1 :)

But like you said, is more difficult to manage, and will "steal" more disk space than a RAID10.

But like I said before, this is just my personal opinion.

Jail

pls excuse my stupidity.
the 3 mirrored pairs works just fine but when i created a raid 10 Virtual Disk in H700 (approx 6TB)  but vmware is only able to see only 2 tb ..and there is no Disk/LUN available when i try to extend or create another datasource what am i doing wrong?

That is because VMWARE can not handle a 6TB LUN when exposed from an external controller.  You simply can not expose a LUN > 2.09 TB to VMWARE as it won't be able to address any block higher than block # FFFFFFFF hex, which is the largest number of blocks addressable in 32 bits.  That is due to the 10-byte SCSI command limitation which only has 32-bits to stick in a block number.

VMWare uses 10-byte SCSI commands to communicate with the hardware.   Using extents is a work-around within vmware, where vmware does mapping internally ... but it still sends only 10byte SCSI commands to the physical or logical disks exposed by your storage JBOD or RAID controllers.

There is no work around.  VMWare needs to redo the kernel to support 16-byte SCSI commands to fix it, or to go with block sizes > 512 bytes (which the H700 doesn't support anyway, and this is brand-new up-and-coming stuff).

Hi

You can create a 6Tb VD in the Storage side, but you need to create LUNs smaller than 2Tb for VMware recognize the LUN.

Jail

This will drive me nuts. I have the big Vd , but how Do I create the smaller LUNS?

Hi

I do no know you Storage. What Storage are you using?

Jail

I have a Dell r710 server with percent h700 and 6 disks x 2tb each

Hi

Ahh :(

Sorry I think you are using a Storage like SAN. In this case you cannot create a VD and create several LUNs. You can only create VD(one or more).

You have 6 disks with 2Tb each? 12Tb of Storage?

Jail

yup thats exactly my case  

and on the same server that i have esx

ill definatelly move on, by installing XEN :))))

many thanx guyz