Go Premium for a chance to win a PS4. Enter to Win


setting up dedicated NAS for ESX 4.x datastore Best practice

Posted on 2010-11-19
Medium Priority
Last Modified: 2012-05-10
Hi fellow experts,

As long as I'm here I thought I might post a question as well.

We recently freed up one of out NAS devices, which I'm planning to use as a dedicated data store for our ESX (4.1) cluster, three servers at the moment.

What have I got:

A eurostor E-8700 which contains an Areca ARC-1261(firmware V1.42 2006-10-13) with 16 Samsung HD103SJ 1T disks, Motherboard is a Supermicro x7DB8 (BIOS R2.1a) with 2GB RAM and some dualcore Xeon (not sure which).

4 GB nics (2x Intel 80003ES2LAN Gigabit Ethernet Controller & Intel Corporation PRO/1000 PT Dual Port Server Adapter), going to bond them to one interface.

The device is running open-e DSS Model: Data Storage Server Version: 5.0.DB49000000.3278 (still have to upgrade to the latest version).

The ESX servers will each have 2 nic's connected to the storage network. The servers and the NAS will be on a separate switch, a Cisco Catalyst WS-C3560G-48TS-S with no other devices attached to it. Mayby later on I will add my ESX test cluster to it (also 3 servers).

What I would like you to share with me is the best practice to set this thing up as a dedicated NAS for my esx farm.

Two things I will do: use RAID6 with two hot spares and use NFS.

For the rest, tell me what you think is the best way, beginning with creating the volume(s) (volume size, stripe size, cache mode, etc) all the way through to connecting it to the network and the ESX machines (jumbo frames,etherchanneling, etc).

What do you think is the best way, what is recommended, what are your own experiences, any documents?

Let me know & share you knowledge :)
Question by:Ernie Beek
  • 4
  • 3

Accepted Solution

markzz earned 2000 total points
ID: 34173586
I would expect you'll runout of IO capability on the disks before you can fully utilize you potential 4Gb of Network bandwidth particularly if you use raid 6.
We are running NetApp 146GB Fiber HDD in raid 6 although I am always amazed at the storage IO we can attain over a 1Gb Storage IP connection we did see Disk IO bottle necks with extreamly radom IO work loads.
Still thinking about your raiding. It's quite obvious the best performance you can achieve is going to be with raid 10 but it's the least space efficient option.
If you use a combination of raid 10 raid 6 you could have the best of both. Useing the raid 10 for your high IO systems and raid 6 for the less demanding. Hey if you find the raid 6 delivers the required performance you can hot storage migrate the guests from raid 10 to the raid 6 NFS store and expand the raid 6..
Size wise.
NFS doesn't suffer the SCSI lock issues of traditional VMFS LUNS but I would still be cautious how large you make the LUNS.
Currently we limit our LUNS to 1TB although I expect we'll have to increase that to accomodate individual Server storage requirements. Until this becomes a requirement we won't.
Smaller LUNS add some complexity and administration over head but also limits the impact of a simgle rouge server running a NFS Store out of space..
1TB is my suggestion.
Enable jumbos is an obvious one.. Your switch is capable..
Consider buying another switch, eg. a 3560-24g. If I recall the 3560's are a single power supply switch.. There's a risk there..
I think the 37xx Series Cisco's that support MEC.. Effectively making them stackable so you can etherchanel accross multiple switches.
As you are on a single switch. I would bond (team) the 4 SAN NIC's into a single etherchanel group, but ensuring I connected the 4 ports accross at least 2 separate busses on the 3560. (typically each bus comprises 8 ports, it will be in the Cisco documentation, I'm sure someone here is more of a Cisco geek than me)

From the VMWare side again Jumbos and etherchanel using "Route based on IP Hash"

Once setup.
Run IO meter or one of the many offerings to stress test the environment.
If you can estimate you expected IO work loads you will have a much better idea of what  raid structure will best suit the environment.

Hope this helps

Expert Comment

ID: 34173701
Sorry forgot the white pagers..

I would also suggest reading some of the Cisco produced papers. Although I can't find it there's an excellent paper from Cisco on Storage IP and VMware, if fact much of our design was influenced by one such Cisco paper..

Expert Comment

ID: 34173718
Get your Conversational Ransomware Defense e‑book

This e-book gives you an insight into the ransomware threat and reviews the fundamentals of top-notch ransomware preparedness and recovery. To help you protect yourself and your organization. The initial infection may be inevitable, so the best protection is to be fully prepared.

LVL 35

Author Comment

by:Ernie Beek
ID: 34174122

Thanks for your elaborate comment, I'll have some reading to do :-)

At the moment there's no change of buying any additional hardware I'm afraid, we'll have to do with what we have now.

And now for some reading, I'll be back.

Expert Comment

ID: 34174630
One more VERY important point..
Set all port speeds hard so 1000/full. This has been and continues to be an issue with Cisco gear.
We need to set all port hard. NO AUTO

Expert Comment

ID: 34176949
The only piece of advice I would have being involved in some of the largest VMware deployments in the country is that VMware likes a 500GB Lun with a 1MB block size.  You can tell me they support all types of volume sizes but benchmarks do not lie...
LVL 35

Author Comment

by:Ernie Beek
ID: 34177095

Enlighten me!

Any benchmarks to show?

500GB feels like such a small thing, can't put many VM's on that (with nowadays 'minimum' (+10GB) specs).
LVL 35

Author Closing Comment

by:Ernie Beek
ID: 34252338
Thanks for the info.

Might anything else come to mind, let me know.

Featured Post

Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

If you’re involved with your company’s wide area network (WAN), you’ve probably heard about SD-WANs. They’re the “boy wonder” of networking, ostensibly allowing companies to replace expensive MPLS lines with low-cost Internet access. But, are they …
August and September have been big months for VMware—from VMworld last month to our new Course of the Month in VMware Professional - Data Center Virtualization. We reached out to Andrew Hancock, resident VMware vExpert, to have a more in-depth discu…
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …
This Micro Tutorial walks you through using a remote console to access a server and install ESXi 5.1. This example is showing remote access and installation using a Dell server. The hypervisor is the very first component of your virtual infrastructu…

877 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question