?
Solved

VMware vSphere and Dell MD3200, and other shared storage options

Posted on 2014-10-03
36
Medium Priority
?
441 Views
Last Modified: 2016-11-23
Hi guys,

I've a project coming up with several options and I would like to make sure I get this right. Dell is very naughty with their shared storage hardware where it doesn't permit you to use non Dell disks, I've tried even the same model number of drives but because Dell puts their own firmware, you can't use any other drive then their own. They get a normal SAS drive, mark up 4 times the price and put out in the market.

So, cheap SSD for read only applications on the MD3200 is a no go as I can't use anything that is not sold by Dell. It is a shame, I have Citrix servers here that barely do any read and a couple of RAID10 arrays would do very well.

I could perhaps use Solarwinds iscsi software on a Windows server and share the storage, but this is scsi, hiher latency and gbit network. I have already a DAS device connected through SAS which is a very low latency device.

The virtual SAN from vmware seems quite expensive and not fully matured for proper production use, it would be good to have shared storage across the hosts that is replicated to other hosts so it works as a "shared" storage, but god knows if that works well.

What other options do you guys suggest?

Thanks!
0
Comment
Question by:Alex
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 19
  • 14
  • 2
  • +1
36 Comments
 
LVL 123
ID: 40359140
What is your budget ?

iSCSI and NFS are very popular, or you could use Nexenta Community Edition based on ZFS using SSD for Cache.

How many ESXi servers do you have?

What license, do you want to do, vMotion, HA, DRS, Storage vMotion ?
0
 

Author Comment

by:Alex
ID: 40359152
about 20-30k

iSCSI and NFS would first require 10gbit, second I don't like the latency. I've already got 2x MD3200 with dual controllers from dell, they are worth 15k. I've tested Nexenta not long ago, what a fucking frustrating piece of crap. You can't get support with community of trial, never got it to work properly, I won't pay thousands to find out is is no good for me. Diffidently don't recommend it.

I've 8 ESXi hosts but they will be split into two companies so each having its own DAS

I need vMotion and HA, so it seems the basic standard license will do it. I'm wondering now about being a cloud provider depending on license costs.
0
 
LVL 10

Expert Comment

by:Carlos Ijalba
ID: 40359159
EMC ViPR Software-Defined Storage is a mature SDS product (more than VSAN, anyway).
VMware VSAN will release their new version this year (supposedly).
FreeNAS.
Windows 2008 / 2012 iSCSI.
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:Alex
ID: 40359175
EMC ViPR seems interesting, but I assume it is all iSCSI too?

VSAN hopefully will get better but I don't wanna risk anything for these clients I have

Freenas unfortunatelly I think is not geared towards a high availability client like that

I heard some good stuff about 2012R2 iSCSI like multi level caching, zimilar to ZFS


What about the MD3200, do you guys have any ideas what I could do with them? I'm thinking about just putting 12x 600GB 15K SAS as I did for another client and a big RAID10 array. I would love to have a couple of SSDs but Dell selling each for £2k is asking too much! I have a T620 with 4x Intel 730 480GB in RAID10 and it is incredible, outperform the DAS with 12x 600GB 15K in every single aspect.
0
 
LVL 123
ID: 40359393
Why not use CACHING TECHNOLOGY in the HOST ESXi server with SSDs, so ALL traffic is cached before it hits the MD3200?

See here

http://www.pernixdata.com/

Put your SSDs where they need to be close to the CPU, and all read and write cache go through the cache before VMFS...

and you can turn on and off for specific VMs

We use it with great results on slow SANs!
0
 

Author Comment

by:Alex
ID: 40359405
that is just with the enterprise plus license so far I know, it is damn expensive, cheaper to buy a proper hardware SAN, isn't it?
0
 
LVL 123
ID: 40359544
VMware's offering is very poor by comparison to Pernixdata!

Even with a proper SAN, you may still not get the performance you require, and rather than adding more disks, putting SSDs in the Host near the CPU is the answer!
0
 

Author Comment

by:Alex
ID: 40359721
Have you used their software? These days pretty much everyone is developing their own iSCSI san and other accelerator things like solarwind so I'm a bit sceptical about throwing all my data into a software storage.

I might go to use the DAS with a big raid 10 container for the servers like exchange and DCs and local SSDs on each host for teh Citrix/remote desktop servers.

How far have you played with these software accelerators for VMs?
0
 
LVL 123
ID: 40359731
Yes, we use their software in Production, in-front of LARGE SANs to BOOST IOPS!
0
 

Author Comment

by:Alex
ID: 40359736
The SMB version is $9,999 for up to four hosts and 100 VMs

Hmm, quite pricey. It can do a lot but for this price I can put a lot of SSDs on each host, as they will run only citrix server if one goes down i have 10 left to load balance.

I think I'll go for the local SSD route due to costs.

It is funny this SMB version for VMware essentials that costs 10k, Essentials plus costs half this!

Perhaps when we get bigger we can use it but at this moment it is too pricey.
0
 
LVL 10

Expert Comment

by:Carlos Ijalba
ID: 40360378
EMC ViPR is iSCSI, NFS and CIFS to give storage and FC and the rest to get storage.

v1.0 is free to try, if you like it, then you can evaluate buying v2.0.

But then again, I don't know it's price.
0
 
LVL 62

Expert Comment

by:gheist
ID: 40362421
vmware vsan works very well, just like any software appliance.
still nowadays flash storage has very fast write wear, so it is best used as cache.
(it takes some esxcli to get vmware to understand them as flash cache)
We do use 32-64-128GB microsdh/xc UHS-1 SD cards as local flash cache. While it sounds like lightly retarded, in reality it does full spec 10k read IOPS and beats the 100IOPs you can get from <overstrike>old laptop drives rebranded as </overstike> premium server disks of the time.
0
 

Author Comment

by:Alex
ID: 40363298
The EMC solution is more towards a big enterprise sector, my area is more SBS and medium business, the costs are a major concern for us.
Gheist, how are you using these devices as cache? is in VSAN?
0
 
LVL 123
ID: 40363304
HP P2000 series ? iSCSI based?

Dell Equallogic ?

or are these too expensive ?
0
 

Author Comment

by:Alex
ID: 40363337
Yep, as we have already in place a couple of MD3200s, migrating to iSCSI doesn't make much sense in my opinion plus we add latency of the network instead of using SAS
0
 
LVL 123
ID: 40363350
So what are you wanting to achieve?

Faster performing SAN?

Faster performing local disk?

Do you have a need for shared storage /?
0
 

Author Comment

by:Alex
ID: 40363358
Fast shared storage, but if possible without the Dell proprietary disks. Is there any work around this requirement for the Dell MD3200?

Yes I need shared storage across 4 hosts, each of them with redundant connections to the DAS as it is now

I might use local SSDs in RAID1 for the Citrix servers though is it will be cheap
0
 
LVL 123
ID: 40363365
Fast Shared Storage keeping the Dell MD3200 in the configuration ?
0
 

Author Comment

by:Alex
ID: 40363469
Yes, I can't just throw it away to buy a physical iSCSI device, the costs I would "save" getting generic disks would go away with the purchase of the device. Cost is key here so I use the existing MD3200 or I believe the other simple option is to get in Windows 2012 R2 physical server and have the iSCSI on it as I heard it has improved a lot, very close to ZFS is now these days skipping the need for a RAID controller and good at managing cache
0
 

Author Comment

by:Alex
ID: 40363483
this might be more in line with what I'm looking for

http://www.aidanfinn.com/?p=15430

So, I'll still need a host that users a hardware raid controller, looks like software raid is still similar to what it used to be in 2008, not very reliable or good performer.

perhaps this with 10gbit network cards would be a good cost overall
0
 
LVL 123
ID: 40363486
Put SSDs in your hosts and trial

http://www.pernixdata.com/

Keep your old SAN and boost its performance using SSD close to the Host CPU, and then select the VMs you need the boost on or enable for ALL.
0
 

Author Comment

by:Alex
ID: 40363501
I've just sent them an email asking for costs first, doesn't help me to try, it works fine but the price is far from my client reality.
0
 
LVL 123
ID: 40363505
Most fast storage systems, not longer use RAID, and just use a JBOD of disks, and SSDs for CACHE (ZFS - Arc and ZIL), the issue with the MD3200 as an ISCSI solution, is you  can only connect it to iSCSI, so to speed up, you need Cache in front of it, if it's IOPS you are trying to BOOST, because you cannot insert or want to pay for Dell SSDs.
0
 

Author Comment

by:Alex
ID: 40363507
7500 dollar for a essentials version for 3 hosts

about £5000, not cheap, not too pricey if it does what is says

I'll evaluate it next month if my client go forward with the project
0
 

Author Comment

by:Alex
ID: 40363509
The MD3200 I use is SAS, I don't use the iSCSI version.

You mean most systems running freebsd/Solaris and variations that support ZFS, right?
0
 
LVL 123
ID: 40363512
You'll not get much else for £5k in terms of fast storage, unless you want to build something from scratch.

How much storage do you have in the MD3200 ?

Okay, if you have a SAS version, you could connect to a Solaris Server.....

add some SSD for ZIL and ARC, add 32GB of RAM, and then offer NFS or iSCSI to ESXi.

or you could look at the "Commerical" or "Community version of Nexenta" which is ZFS based.

It depends on how the MD3200 presents itself to a server, e.g. ignoring the RAID card, does it appear as a JBOD.

e.g. the MD3200 is not just an enclosure of disks!
0
 

Author Comment

by:Alex
ID: 40363535
I've got 12x 600K disks in RAID10

Solaris still pure command line to set up and maintain ZFS storage right? I remember the license costs were a bit high, I was playing with Solaris express a few year ago and ZFS but then ended up finding out is was not going to be possible to deploy to a client due to costs. It was then when I looked into Nexenta.

Regarding adding teh Solaris box in between DAS and the ESXi hosts, have you done something like this? I try always to leave things as simple as possible, so I'm wondering if this is a good option for a production environment.

Thanks!
0
 
LVL 123
ID: 40363549
We use ZFS on Solaris for production in our offices, we have 12TB and 24TB ZFS SANs!

Your issue, is the RAID.....because ZFS does it's own RAID in software, disks are presented to server as a JBOD!

ZFS has many implementations now, the latest versions are always on Oracle Solaris.

But ZFS version exist for FreeBSD, Linux.

You can use Napp-it

http://www.napp-it.org/

a front end GUI to configure if you like....
0
 

Author Comment

by:Alex
ID: 40365571
How much do you guys pay for the Solaris license to have it working as a proper ZFS box?

Yeah I'm away I used some Dell Perc H200 in IT mode to test ZFS, I used napp-it but it was still very early stages, lots of bugs and problems.
0
 

Author Comment

by:Alex
ID: 40454593
Hi Andrew,

What were the license costs involved with Solaris to use as a ZFS storage device?
0
 
LVL 123
ID: 40454720
if you use an open source version - free!
0
 

Author Comment

by:Alex
ID: 40454729
My concern is support in the future in case it is needed. We use a freepbx software one client we got on-board had already in place and when it was time to get some support for it when we ran out of ideas, it ended up being a pain.

Comparing to pfsense for example, you have a opensource project, but also can pay for commercial support if needed.

I would prefer Solaris to get it working, do you have any idea of what figures would I be looking for the licenses?
0
 
LVL 123
ID: 40454749
You could use any ZFS on any version of FreeBSD (FreeNAS), Linux, (Centos!).

or you could use Nexenta....(base on Oracle Solaris and use NFS)

http://www.nexentastor.org/


I'm afraid what we paid is irrelevant, because you will not be able to obtain the same price!
0
 

Author Comment

by:Alex
ID: 40454756
I tried Nexenta version 3 a few years ago and never got some performance problems sorted, tried both community and the trial. Freenas didn't alert if a physical drive failed as it was not included into the freebsd code back then, so it would be a risky to not know if a drive had failed, no notifications. Perhaps this has changed now?

I'll give Nexenta version 4 another go, as the community version and see if there are improvements (I hope there are)

As it was a long time ago since I last tried ZFS, things may have hanged a lot, so is the linux implementation mature enough for production these days in your opinion? Thanks!
0
 
LVL 123

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 2000 total points
ID: 40454767
We use different flavors of ZFS, Oracle Solaris flavor does perform better in our experience.

but this is mostly dependent upon the amount of memory you give a server! We give our servers, 32GB or 48GB

Oracle ZFS does always have the latest version of ZFS, with all the latest functionality (if required!)
0
 

Author Closing Comment

by:Alex
ID: 40454774
The explanations all together helped me to choose what do to next. I'll try again ZFS based on Andrew's advise.
0

Featured Post

On Demand Webinar: Networking for the Cloud Era

Did you know SD-WANs can improve network connectivity? Check out this webinar to learn how an SD-WAN simplified, one-click tool can help you migrate and manage data in the cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this article, I will show you HOW TO: Suppress Configuration Issues and Warnings Alert displayed in Summary status for ESXi 6.5 after enabling SSH or ESXi Shell.
What if you have to shut down the entire Citrix infrastructure for hardware maintenance, software upgrades or "the unknown"? I developed this plan for "the unknown" and hope that it helps you as well. This article explains how to properly shut down …
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …
This video shows you how easy it is to boot from ISO images for virtual machines with the ISO images stored on a local datastore on the ESXi host.
Suggested Courses

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question