Solved

setting up dedicated NAS for ESX 4.x datastore Best practice

Posted on 2010-11-19
8
1,191 Views
Last Modified: 2012-05-10
Hi fellow experts,

As long as I'm here I thought I might post a question as well.

We recently freed up one of out NAS devices, which I'm planning to use as a dedicated data store for our ESX (4.1) cluster, three servers at the moment.

What have I got:

A eurostor E-8700 which contains an Areca ARC-1261(firmware V1.42 2006-10-13) with 16 Samsung HD103SJ 1T disks, Motherboard is a Supermicro x7DB8 (BIOS R2.1a) with 2GB RAM and some dualcore Xeon (not sure which).

4 GB nics (2x Intel 80003ES2LAN Gigabit Ethernet Controller & Intel Corporation PRO/1000 PT Dual Port Server Adapter), going to bond them to one interface.

The device is running open-e DSS Model: Data Storage Server Version: 5.0.DB49000000.3278 (still have to upgrade to the latest version).

The ESX servers will each have 2 nic's connected to the storage network. The servers and the NAS will be on a separate switch, a Cisco Catalyst WS-C3560G-48TS-S with no other devices attached to it. Mayby later on I will add my ESX test cluster to it (also 3 servers).

What I would like you to share with me is the best practice to set this thing up as a dedicated NAS for my esx farm.

Two things I will do: use RAID6 with two hot spares and use NFS.

For the rest, tell me what you think is the best way, beginning with creating the volume(s) (volume size, stripe size, cache mode, etc) all the way through to connecting it to the network and the ESX machines (jumbo frames,etherchanneling, etc).

What do you think is the best way, what is recommended, what are your own experiences, any documents?

Let me know & share you knowledge :)
0
Comment
Question by:Ernie Beek
  • 4
  • 3
8 Comments
 
LVL 8

Accepted Solution

by:
markzz earned 500 total points
ID: 34173586
I would expect you'll runout of IO capability on the disks before you can fully utilize you potential 4Gb of Network bandwidth particularly if you use raid 6.
We are running NetApp 146GB Fiber HDD in raid 6 although I am always amazed at the storage IO we can attain over a 1Gb Storage IP connection we did see Disk IO bottle necks with extreamly radom IO work loads.
Still thinking about your raiding. It's quite obvious the best performance you can achieve is going to be with raid 10 but it's the least space efficient option.
If you use a combination of raid 10 raid 6 you could have the best of both. Useing the raid 10 for your high IO systems and raid 6 for the less demanding. Hey if you find the raid 6 delivers the required performance you can hot storage migrate the guests from raid 10 to the raid 6 NFS store and expand the raid 6..
Size wise.
NFS doesn't suffer the SCSI lock issues of traditional VMFS LUNS but I would still be cautious how large you make the LUNS.
Currently we limit our LUNS to 1TB although I expect we'll have to increase that to accomodate individual Server storage requirements. Until this becomes a requirement we won't.
Smaller LUNS add some complexity and administration over head but also limits the impact of a simgle rouge server running a NFS Store out of space..
1TB is my suggestion.
Enable jumbos is an obvious one.. Your switch is capable..
Consider buying another switch, eg. a 3560-24g. If I recall the 3560's are a single power supply switch.. There's a risk there..
I think the 37xx Series Cisco's that support MEC.. Effectively making them stackable so you can etherchanel accross multiple switches.
As you are on a single switch. I would bond (team) the 4 SAN NIC's into a single etherchanel group, but ensuring I connected the 4 ports accross at least 2 separate busses on the 3560. (typically each bus comprises 8 ports, it will be in the Cisco documentation, I'm sure someone here is more of a Cisco geek than me)

From the VMWare side again Jumbos and etherchanel using "Route based on IP Hash"

Once setup.
Run IO meter or one of the many offerings to stress test the environment.
If you can estimate you expected IO work loads you will have a much better idea of what  raid structure will best suit the environment.

Hope this helps
0
 
LVL 8

Expert Comment

by:markzz
ID: 34173701
Sorry forgot the white pagers..

http://vmware.com/files/pdf/VMware_NFS_BestPractices_WP_EN.pdf
I would also suggest reading some of the Cisco produced papers. Although I can't find it there's an excellent paper from Cisco on Storage IP and VMware, if fact much of our design was influenced by one such Cisco paper..
0
 
LVL 8

Expert Comment

by:markzz
ID: 34173718
0
The Eight Noble Truths of Backup and Recovery

How can IT departments tackle the challenges of a Big Data world? This white paper provides a roadmap to success and helps companies ensure that all their data is safe and secure, no matter if it resides on-premise with physical or virtual machines or in the cloud.

 
LVL 35

Author Comment

by:Ernie Beek
ID: 34174122
@markzz

Thanks for your elaborate comment, I'll have some reading to do :-)

At the moment there's no change of buying any additional hardware I'm afraid, we'll have to do with what we have now.

And now for some reading, I'll be back.
0
 
LVL 8

Expert Comment

by:markzz
ID: 34174630
One more VERY important point..
Set all port speeds hard so 1000/full. This has been and continues to be an issue with Cisco gear.
We need to set all port hard. NO AUTO
0
 
LVL 3

Expert Comment

by:Virtalicious
ID: 34176949
The only piece of advice I would have being involved in some of the largest VMware deployments in the country is that VMware likes a 500GB Lun with a 1MB block size.  You can tell me they support all types of volume sizes but benchmarks do not lie...
0
 
LVL 35

Author Comment

by:Ernie Beek
ID: 34177095
@Virtalicious

Enlighten me!

Any benchmarks to show?

500GB feels like such a small thing, can't put many VM's on that (with nowadays 'minimum' (+10GB) specs).
0
 
LVL 35

Author Closing Comment

by:Ernie Beek
ID: 34252338
Thanks for the info.

Might anything else come to mind, let me know.
0

Featured Post

Free eBook: Backup on AWS

Everything you need to know about backup and disaster recovery with AWS, for FREE!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

This article aims to explain the working of CircularLogArchiver. This tool was designed to solve the buildup of log file in cases where systems do not support circular logging or where circular logging is not enabled
This article outlines why you need to choose a backup solution that protects your entire environment – including your VMware ESXi and Microsoft Hyper-V virtualization hosts – not just your virtual machines.
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …

680 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question