Solved

setting up dedicated NAS for ESX 4.x datastore Best practice

Posted on 2010-11-19
8
1,194 Views
Last Modified: 2012-05-10
Hi fellow experts,

As long as I'm here I thought I might post a question as well.

We recently freed up one of out NAS devices, which I'm planning to use as a dedicated data store for our ESX (4.1) cluster, three servers at the moment.

What have I got:

A eurostor E-8700 which contains an Areca ARC-1261(firmware V1.42 2006-10-13) with 16 Samsung HD103SJ 1T disks, Motherboard is a Supermicro x7DB8 (BIOS R2.1a) with 2GB RAM and some dualcore Xeon (not sure which).

4 GB nics (2x Intel 80003ES2LAN Gigabit Ethernet Controller & Intel Corporation PRO/1000 PT Dual Port Server Adapter), going to bond them to one interface.

The device is running open-e DSS Model: Data Storage Server Version: 5.0.DB49000000.3278 (still have to upgrade to the latest version).

The ESX servers will each have 2 nic's connected to the storage network. The servers and the NAS will be on a separate switch, a Cisco Catalyst WS-C3560G-48TS-S with no other devices attached to it. Mayby later on I will add my ESX test cluster to it (also 3 servers).

What I would like you to share with me is the best practice to set this thing up as a dedicated NAS for my esx farm.

Two things I will do: use RAID6 with two hot spares and use NFS.

For the rest, tell me what you think is the best way, beginning with creating the volume(s) (volume size, stripe size, cache mode, etc) all the way through to connecting it to the network and the ESX machines (jumbo frames,etherchanneling, etc).

What do you think is the best way, what is recommended, what are your own experiences, any documents?

Let me know & share you knowledge :)
0
Comment
Question by:Ernie Beek
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
8 Comments
 
LVL 8

Accepted Solution

by:
markzz earned 500 total points
ID: 34173586
I would expect you'll runout of IO capability on the disks before you can fully utilize you potential 4Gb of Network bandwidth particularly if you use raid 6.
We are running NetApp 146GB Fiber HDD in raid 6 although I am always amazed at the storage IO we can attain over a 1Gb Storage IP connection we did see Disk IO bottle necks with extreamly radom IO work loads.
Still thinking about your raiding. It's quite obvious the best performance you can achieve is going to be with raid 10 but it's the least space efficient option.
If you use a combination of raid 10 raid 6 you could have the best of both. Useing the raid 10 for your high IO systems and raid 6 for the less demanding. Hey if you find the raid 6 delivers the required performance you can hot storage migrate the guests from raid 10 to the raid 6 NFS store and expand the raid 6..
Size wise.
NFS doesn't suffer the SCSI lock issues of traditional VMFS LUNS but I would still be cautious how large you make the LUNS.
Currently we limit our LUNS to 1TB although I expect we'll have to increase that to accomodate individual Server storage requirements. Until this becomes a requirement we won't.
Smaller LUNS add some complexity and administration over head but also limits the impact of a simgle rouge server running a NFS Store out of space..
1TB is my suggestion.
Enable jumbos is an obvious one.. Your switch is capable..
Consider buying another switch, eg. a 3560-24g. If I recall the 3560's are a single power supply switch.. There's a risk there..
I think the 37xx Series Cisco's that support MEC.. Effectively making them stackable so you can etherchanel accross multiple switches.
As you are on a single switch. I would bond (team) the 4 SAN NIC's into a single etherchanel group, but ensuring I connected the 4 ports accross at least 2 separate busses on the 3560. (typically each bus comprises 8 ports, it will be in the Cisco documentation, I'm sure someone here is more of a Cisco geek than me)

From the VMWare side again Jumbos and etherchanel using "Route based on IP Hash"

Once setup.
Run IO meter or one of the many offerings to stress test the environment.
If you can estimate you expected IO work loads you will have a much better idea of what  raid structure will best suit the environment.

Hope this helps
0
 
LVL 8

Expert Comment

by:markzz
ID: 34173701
Sorry forgot the white pagers..

http://vmware.com/files/pdf/VMware_NFS_BestPractices_WP_EN.pdf
I would also suggest reading some of the Cisco produced papers. Although I can't find it there's an excellent paper from Cisco on Storage IP and VMware, if fact much of our design was influenced by one such Cisco paper..
0
 
LVL 8

Expert Comment

by:markzz
ID: 34173718
0
Connect further...control easier

With the ATEN CE624, you can now enjoy a high-quality visual experience powered by HDBaseT technology and the convenience of a single Cat6 cable to transmit uncompressed video with zero latency and multi-streaming for dual-view applications where remote access is required.

 
LVL 35

Author Comment

by:Ernie Beek
ID: 34174122
@markzz

Thanks for your elaborate comment, I'll have some reading to do :-)

At the moment there's no change of buying any additional hardware I'm afraid, we'll have to do with what we have now.

And now for some reading, I'll be back.
0
 
LVL 8

Expert Comment

by:markzz
ID: 34174630
One more VERY important point..
Set all port speeds hard so 1000/full. This has been and continues to be an issue with Cisco gear.
We need to set all port hard. NO AUTO
0
 
LVL 3

Expert Comment

by:Virtalicious
ID: 34176949
The only piece of advice I would have being involved in some of the largest VMware deployments in the country is that VMware likes a 500GB Lun with a 1MB block size.  You can tell me they support all types of volume sizes but benchmarks do not lie...
0
 
LVL 35

Author Comment

by:Ernie Beek
ID: 34177095
@Virtalicious

Enlighten me!

Any benchmarks to show?

500GB feels like such a small thing, can't put many VM's on that (with nowadays 'minimum' (+10GB) specs).
0
 
LVL 35

Author Closing Comment

by:Ernie Beek
ID: 34252338
Thanks for the info.

Might anything else come to mind, let me know.
0

Featured Post

Will your db performance match your db growth?

In Percona’s white paper “Performance at Scale: Keeping Your Database on Its Toes,” we take a high-level approach to what you need to think about when planning for database scalability.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article aims to explain the working of CircularLogArchiver. This tool was designed to solve the buildup of log file in cases where systems do not support circular logging or where circular logging is not enabled
Each year, investment in cloud platforms grows more than 20% (https://www.immun.io/hubfs/Immunio_2016/Content/Marketing/Cloud-Security-Report-2016.pdf?submissionGuid=a8d80a00-6fee-4b85-81db-a4e28f681762) as an increasing number of companies begin to…
This tutorial will walk an individual through the steps necessary to enable the VMware\Hyper-V licensed feature of Backup Exec 2012. In addition, how to add a VMware server and configure a backup job. The first step is to acquire the necessary licen…
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

628 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question