setting up dedicated NAS for ESX 4.x datastore Best practice

Hi fellow experts,

As long as I'm here I thought I might post a question as well.

We recently freed up one of out NAS devices, which I'm planning to use as a dedicated data store for our ESX (4.1) cluster, three servers at the moment.

What have I got:

A eurostor E-8700 which contains an Areca ARC-1261(firmware V1.42 2006-10-13) with 16 Samsung HD103SJ 1T disks, Motherboard is a Supermicro x7DB8 (BIOS R2.1a) with 2GB RAM and some dualcore Xeon (not sure which).

4 GB nics (2x Intel 80003ES2LAN Gigabit Ethernet Controller & Intel Corporation PRO/1000 PT Dual Port Server Adapter), going to bond them to one interface.

The device is running open-e DSS Model: Data Storage Server Version: 5.0.DB49000000.3278 (still have to upgrade to the latest version).

The ESX servers will each have 2 nic's connected to the storage network. The servers and the NAS will be on a separate switch, a Cisco Catalyst WS-C3560G-48TS-S with no other devices attached to it. Mayby later on I will add my ESX test cluster to it (also 3 servers).

What I would like you to share with me is the best practice to set this thing up as a dedicated NAS for my esx farm.

Two things I will do: use RAID6 with two hot spares and use NFS.

For the rest, tell me what you think is the best way, beginning with creating the volume(s) (volume size, stripe size, cache mode, etc) all the way through to connecting it to the network and the ESX machines (jumbo frames,etherchanneling, etc).

What do you think is the best way, what is recommended, what are your own experiences, any documents?

Let me know & share you knowledge :)
LVL 35
Ernie BeekExpertAsked:
Who is Participating?
markzzConnect With a Mentor Commented:
I would expect you'll runout of IO capability on the disks before you can fully utilize you potential 4Gb of Network bandwidth particularly if you use raid 6.
We are running NetApp 146GB Fiber HDD in raid 6 although I am always amazed at the storage IO we can attain over a 1Gb Storage IP connection we did see Disk IO bottle necks with extreamly radom IO work loads.
Still thinking about your raiding. It's quite obvious the best performance you can achieve is going to be with raid 10 but it's the least space efficient option.
If you use a combination of raid 10 raid 6 you could have the best of both. Useing the raid 10 for your high IO systems and raid 6 for the less demanding. Hey if you find the raid 6 delivers the required performance you can hot storage migrate the guests from raid 10 to the raid 6 NFS store and expand the raid 6..
Size wise.
NFS doesn't suffer the SCSI lock issues of traditional VMFS LUNS but I would still be cautious how large you make the LUNS.
Currently we limit our LUNS to 1TB although I expect we'll have to increase that to accomodate individual Server storage requirements. Until this becomes a requirement we won't.
Smaller LUNS add some complexity and administration over head but also limits the impact of a simgle rouge server running a NFS Store out of space..
1TB is my suggestion.
Enable jumbos is an obvious one.. Your switch is capable..
Consider buying another switch, eg. a 3560-24g. If I recall the 3560's are a single power supply switch.. There's a risk there..
I think the 37xx Series Cisco's that support MEC.. Effectively making them stackable so you can etherchanel accross multiple switches.
As you are on a single switch. I would bond (team) the 4 SAN NIC's into a single etherchanel group, but ensuring I connected the 4 ports accross at least 2 separate busses on the 3560. (typically each bus comprises 8 ports, it will be in the Cisco documentation, I'm sure someone here is more of a Cisco geek than me)

From the VMWare side again Jumbos and etherchanel using "Route based on IP Hash"

Once setup.
Run IO meter or one of the many offerings to stress test the environment.
If you can estimate you expected IO work loads you will have a much better idea of what  raid structure will best suit the environment.

Hope this helps
Sorry forgot the white pagers..
I would also suggest reading some of the Cisco produced papers. Although I can't find it there's an excellent paper from Cisco on Storage IP and VMware, if fact much of our design was influenced by one such Cisco paper..
Cloud Class® Course: C++ 11 Fundamentals

This course will introduce you to C++ 11 and teach you about syntax fundamentals.

Ernie BeekExpertAuthor Commented:

Thanks for your elaborate comment, I'll have some reading to do :-)

At the moment there's no change of buying any additional hardware I'm afraid, we'll have to do with what we have now.

And now for some reading, I'll be back.
One more VERY important point..
Set all port speeds hard so 1000/full. This has been and continues to be an issue with Cisco gear.
We need to set all port hard. NO AUTO
The only piece of advice I would have being involved in some of the largest VMware deployments in the country is that VMware likes a 500GB Lun with a 1MB block size.  You can tell me they support all types of volume sizes but benchmarks do not lie...
Ernie BeekExpertAuthor Commented:

Enlighten me!

Any benchmarks to show?

500GB feels like such a small thing, can't put many VM's on that (with nowadays 'minimum' (+10GB) specs).
Ernie BeekExpertAuthor Commented:
Thanks for the info.

Might anything else come to mind, let me know.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.