Solved

setting up dedicated NAS for ESX 4.x datastore Best practice

Posted on 2010-11-19
8
1,184 Views
Last Modified: 2012-05-10
Hi fellow experts,

As long as I'm here I thought I might post a question as well.

We recently freed up one of out NAS devices, which I'm planning to use as a dedicated data store for our ESX (4.1) cluster, three servers at the moment.

What have I got:

A eurostor E-8700 which contains an Areca ARC-1261(firmware V1.42 2006-10-13) with 16 Samsung HD103SJ 1T disks, Motherboard is a Supermicro x7DB8 (BIOS R2.1a) with 2GB RAM and some dualcore Xeon (not sure which).

4 GB nics (2x Intel 80003ES2LAN Gigabit Ethernet Controller & Intel Corporation PRO/1000 PT Dual Port Server Adapter), going to bond them to one interface.

The device is running open-e DSS Model: Data Storage Server Version: 5.0.DB49000000.3278 (still have to upgrade to the latest version).

The ESX servers will each have 2 nic's connected to the storage network. The servers and the NAS will be on a separate switch, a Cisco Catalyst WS-C3560G-48TS-S with no other devices attached to it. Mayby later on I will add my ESX test cluster to it (also 3 servers).

What I would like you to share with me is the best practice to set this thing up as a dedicated NAS for my esx farm.

Two things I will do: use RAID6 with two hot spares and use NFS.

For the rest, tell me what you think is the best way, beginning with creating the volume(s) (volume size, stripe size, cache mode, etc) all the way through to connecting it to the network and the ESX machines (jumbo frames,etherchanneling, etc).

What do you think is the best way, what is recommended, what are your own experiences, any documents?

Let me know & share you knowledge :)
0
Comment
Question by:Ernie Beek
  • 4
  • 3
8 Comments
 
LVL 8

Accepted Solution

by:
markzz earned 500 total points
Comment Utility
I would expect you'll runout of IO capability on the disks before you can fully utilize you potential 4Gb of Network bandwidth particularly if you use raid 6.
We are running NetApp 146GB Fiber HDD in raid 6 although I am always amazed at the storage IO we can attain over a 1Gb Storage IP connection we did see Disk IO bottle necks with extreamly radom IO work loads.
Still thinking about your raiding. It's quite obvious the best performance you can achieve is going to be with raid 10 but it's the least space efficient option.
If you use a combination of raid 10 raid 6 you could have the best of both. Useing the raid 10 for your high IO systems and raid 6 for the less demanding. Hey if you find the raid 6 delivers the required performance you can hot storage migrate the guests from raid 10 to the raid 6 NFS store and expand the raid 6..
Size wise.
NFS doesn't suffer the SCSI lock issues of traditional VMFS LUNS but I would still be cautious how large you make the LUNS.
Currently we limit our LUNS to 1TB although I expect we'll have to increase that to accomodate individual Server storage requirements. Until this becomes a requirement we won't.
Smaller LUNS add some complexity and administration over head but also limits the impact of a simgle rouge server running a NFS Store out of space..
1TB is my suggestion.
Enable jumbos is an obvious one.. Your switch is capable..
Consider buying another switch, eg. a 3560-24g. If I recall the 3560's are a single power supply switch.. There's a risk there..
I think the 37xx Series Cisco's that support MEC.. Effectively making them stackable so you can etherchanel accross multiple switches.
As you are on a single switch. I would bond (team) the 4 SAN NIC's into a single etherchanel group, but ensuring I connected the 4 ports accross at least 2 separate busses on the 3560. (typically each bus comprises 8 ports, it will be in the Cisco documentation, I'm sure someone here is more of a Cisco geek than me)

From the VMWare side again Jumbos and etherchanel using "Route based on IP Hash"

Once setup.
Run IO meter or one of the many offerings to stress test the environment.
If you can estimate you expected IO work loads you will have a much better idea of what  raid structure will best suit the environment.

Hope this helps
0
 
LVL 8

Expert Comment

by:markzz
Comment Utility
Sorry forgot the white pagers..

http://vmware.com/files/pdf/VMware_NFS_BestPractices_WP_EN.pdf
I would also suggest reading some of the Cisco produced papers. Although I can't find it there's an excellent paper from Cisco on Storage IP and VMware, if fact much of our design was influenced by one such Cisco paper..
0
 
LVL 8

Expert Comment

by:markzz
Comment Utility
0
 
LVL 35

Author Comment

by:Ernie Beek
Comment Utility
@markzz

Thanks for your elaborate comment, I'll have some reading to do :-)

At the moment there's no change of buying any additional hardware I'm afraid, we'll have to do with what we have now.

And now for some reading, I'll be back.
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 8

Expert Comment

by:markzz
Comment Utility
One more VERY important point..
Set all port speeds hard so 1000/full. This has been and continues to be an issue with Cisco gear.
We need to set all port hard. NO AUTO
0
 
LVL 3

Expert Comment

by:Virtalicious
Comment Utility
The only piece of advice I would have being involved in some of the largest VMware deployments in the country is that VMware likes a 500GB Lun with a 1MB block size.  You can tell me they support all types of volume sizes but benchmarks do not lie...
0
 
LVL 35

Author Comment

by:Ernie Beek
Comment Utility
@Virtalicious

Enlighten me!

Any benchmarks to show?

500GB feels like such a small thing, can't put many VM's on that (with nowadays 'minimum' (+10GB) specs).
0
 
LVL 35

Author Closing Comment

by:Ernie Beek
Comment Utility
Thanks for the info.

Might anything else come to mind, let me know.
0

Featured Post

What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

VM backup deduplication is a method of reducing the amount of storage space needed to save VM backups. In most organizations, VMs contain many duplicate copies of data, such as VMs deployed from the same template, VMs with the same OS, or VMs that h…
Data center, now-a-days, is referred as the home of all the advanced technologies. In-fact, most of the businesses are now establishing their entire organizational structure around the IT capabilities.
Teach the user how to use create log bundles for vCenter Server or ESXi hosts Open vSphere Web Client: Generate vCenter Server and ESXi host log bundle:  Open vCenter Server Appliance Web Management interface and generate log bundle: Open vCenter Se…
Teach the user how to use vSphere Update Manager to update the VMware Tools and virtual machine hardware version Open vSphere Client: Review manual processes for updating VMware Tools and virtual hardware versions: Create a new baseline group in vSp…

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now