Solved

Flavor of Dell SAN, or forgo SAN for less expensive NAS solution.

Posted on 2011-09-19
30
2,729 Views
Last Modified: 2013-03-30
I'm getting my feet wet on the concept of virtualization at our main data center (30 servers, not a heavy workload) and our colo (just a few servers, low usage).   While our needs are relatively simple, in that we have 30 full time system users and 40 terminal server users, and an I/O analysis has shown that we are not hitting our systems very hard, our business model is moving from the commercial sector to the residential sector and uptime, availability and dr is becoming more important.  Also, we require more flexibility than is allowed by individual physical hosts, which sometimes are only serving a handful of users for a fraction of the day.

I could use some guidance on whether an equilogic appliance, or the like, would be overkill for us.  Dell is pushing pretty hard for us to go that route and I'm hesitant to recommend to our corporate principals that we spend the money, even though system uptime and redundancy has been named as high priority.  On the other side, it has been suggested that we just create an iscsi connection from a physical host to a LUN on a netgear/NAS device and use those volumes for virtual hosts.    Is this scenario too simple and setting us up for too many single points of failure?   Is there a practical difference between a NAS and a Dell Powervault?

Lastly, I received information today that is making the project more urgent which in turn makes the factors of ease of setup and administration for an admin new to virtualization more important.  If you  have any thoughts on this in regards to hardware and software options, i would appreciate it.

I know these are general questions, but I tried to give you as much information as I could without writing a book.    Thanks in advance.
0
Comment
Question by:alexsupertramp
  • 8
  • 7
  • 6
  • +3
30 Comments
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
A Dell solution will give you more spindles, higher performance, redundant or dual controllers for dual paths. NetGear solutions are fine for Test Labs and Home Labs, small numbers of VMs, but try many VMs simultaneously and you will start to have performance problems.

But ranges are chaning all the time, Netgear, Drobo, Synology, Qnap - Dell solutions are very good enterprise performing SANs. Please feel free to discuss.
0
 
LVL 13

Assisted Solution

by:Greg Hejl
Greg Hejl earned 166 total points
Comment Utility
dell MD3200i, raid 10

dual iSCSI controllers,  1U servers,

NLB webservers, clustered file servers and cluster DB

two 3200 watt batt backup.

I run a fully expanded MD3000 - 8 Hosts - virtual servers

serve 1,000 clients, ~4,000 web users - i can take 25% of the system down for 18 hours a day for maintenance, upgrades, etc....

you could probably do you setup with 4 servers
0
 
LVL 8

Assisted Solution

by:gsmartin
gsmartin earned 100 total points
Comment Utility
My preference is SAN for overall performance in a virtualization platform vs. NAS. I would choose a flexible SAN platform the supports at least FC, iSCSI, and potentially FCOE in the future.  In my SAN environment I use both FC and iSCSI; FCOE hasn't matured enough at this time.  Based on my experiences in my virtualization environment 8Gb FC is a much better option than 10Gb iSCSI for your primary ESX LUNs, but will require a little more of an investment.  If cost is an issue than iSCSI is going to be best for you.  FYI... We built our virtualization environment on a Compellent SAN, which is now owned by Dell.

When you begin virtualize you don't realize how dependant you become on the virtualization platform.  You need to ensure you reduce or eliminate any single points of failure at all level in your virtualization architechture; I assume you already know this.  At some point, you will become fully dependant on your virtual environment.  I went from 1% virtual to 98% virtual within a year.  This will obviously vary from company to company, but virtualization is now the standard.  Given that you want to make sure to architect the environment from the ground up using the best hardware your company allows you to implement.  

For me, I went with a best of breed aproach on the hardware, from my view point.  HP C7000 Blade System, BL490 G7 Blade servers (Boot from SAN ESX configuration), Compellent SAN, and Foundry Networks/Brocade 10GB core Switch and Brocade 8Gb FC Switch Fabric.  Note everyone will have different opions on the what is the best hardware, I am not looking to get into the discussion or debate.  I am just trying to highlight what is important to consider for your virtualization platform in respect to the storage and other components.  

As you know, there are a number of ways to build out your virtualization environment, but I would say the SAN is the most crucial piece.  Plus you want to the SAN to be flexible where you can pool spindle resources even accross tiers of storage like with the Compellent SAN. Compellent is a very dynamic and flexible SAN much more than Equalogic, but bottom line is what you can get approved by the business.  Equalogic I am sure is still a very good candidate in that respect.

I hope this information helps.    
0
 
LVL 42

Assisted Solution

by:kevinhsieh
kevinhsieh earned 167 total points
Comment Utility
I can tell you that if I got my hands on an original 9 year old EqualLogic PS50E, I would load it with the firmware that matches what I am currently running and put it into my production cluster. Dell still supports. Them, and they don't charge through the nose for maintenance either. My first unit is nearly 5 years old, and I can see running it in production for the foreseeable future, just like my 10 year old Catalyst 6500 switches. With EqualLogic, you don't ever need to do a forklift upgrade, and all new features have been included with maintenance. (The only exception is unified storage, which requires purchase of FS7500 units. It is unclear if dedupe will be an add on hardware like the FS7500 which will work with every EqualLogic ever sold, or if they will add it to the controller firmware (free!), but that will mean that dedupe is only available for specific models with the necessary available horsepower.)

With EqualLogic, you have replication capability built-in, which will help with DR. The PS4000E, PS4100E, and 4100X are all excellent models to get started with. You can possibly run them for 10 years, and you can always add a PS6100 if you ever outgrow two PS4000 units. I just got a PS4100X to go with my PS400Es, and I am looking forward to the additional IOPS and capacity.
0
 
LVL 4

Author Comment

by:alexsupertramp
Comment Utility
Thank you all for great info.    Can anyone speak to the difference between the following models of san hardware:

Dell Equilogic 4100
Dell Powervault MD3200
HP P2000/P4000


0
 
LVL 117

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE) earned 67 total points
Comment Utility
They are all very good units, offering similar levels of performance, what makes a difference is the Disks in the Chassis. Both MD3200 and HP P2000 series are common in VMware installations, offering dual power supplies, dual controllers, ability to add additional enclosures.

What is becoming popular in the industry, at present is the Virtual SAN Array, using storage on your vSphere Servers as a vSAN, all the SANs you have listed are a Single Points of failure (even if you have dual controllers, dual power supplies).

vSANs now available from, one of the biggest benefits, is all these products use existing storage in your ESXi server, and Synchronous mirroring of data stores between stortage on ESX servers. Many forget that the SAN is a single point of failure.

and last year we had 3 clients with total failure of P2000s, backplane failures, resulting in total loss of SAN for 18 hours! requiring total restore and DR required, and 5 clients with MD3200 controllers dead.

All clients have now switched to VSAs, because of the single point of failure.

VMware vSphere VSA O(new product)
http://www.vmware.com/products/datacenter-virtualization/vsphere/vsphere-storage-appliance/overview.html

Starwind iSCSI Software VSA SAN
http://www.starwindsoftware.com/

StoreMagic SvSAN
http://www.stormagic.com/SvSAN.php

Datacore SanSymphony
http://www.datacore.com/

HP P4000 VSA Software
http://h18006.www1.hp.com/storage/highlights/lefthandsans.html

On an aside, you may also want to test and check, the Terminal Services, our experience has been, that Terminal Server usage under Hypervisor is poor, when compared to physical machines. So you may find you have to create more terminal servers in your farm, to cope with 40 concurrent users.

Any questions please ask.
0
 
LVL 42

Expert Comment

by:kevinhsieh
Comment Utility
Here's a thread comparing EqualLogic and LeftHand, which HP purchased and is now the P series line. Reading down to the newer posts which are after the HP acquisition, it sounds like the product has really gone downhill. I was shocked.

http://storagemojo.com/2009/10/21/ask-storagemojo-equallogic-vs-lefthand-more/

The Dell MD3200 line is a SAS interconnect direct attached storage, which means that it is limited in the number of hosts that you can connect, and you can't do any VM level HA clustering under Hyper-V; you require iSCSI. You should look at the MD32X0i and 36X0i line, which are iSCSI. I am not really familier with them, but it looks like the pricing of the MD32X0i is within maybe $5000 or so of a similar EqualLogic PS4100E, and the EqualLogic is a much more capable box. The EqualLogic PS line has snapshots, thin provisioning, thin clones, replication, active-active controllers, amazing performance from whatever drives you have, online upgrades, online expansion by adding additional units, automatic data tiering among units, and no incremental licensing costs.
0
 
LVL 13

Assisted Solution

by:Greg Hejl
Greg Hejl earned 166 total points
Comment Utility
thanks Kevin

i thought the MD3200i was the newer MD3000i line.  direct attached storage is certainly not the way to go for virtual.

one of the things I have discussed with Dell is the lack of SNMP monitoring on their MD units.  it can send traps for errors and also send emails

our next SAN will be the Compellent line from dell - it does some amazing stuff for the price point
0
 
LVL 42

Expert Comment

by:kevinhsieh
Comment Utility
The MD3200i line is newer than the MD3000i, and the MD3600i is even newer. The sales guy who sold me EqualLogic cashed out and tried to sell my Compellent, but I didn't buy because I already had EqualLogic and it didn't seem to make any sense for me to switch. The more I think about it, the more amazed I am to think that I would actually use a 9 year old piece of EqualLogic gear if I had it, and I think that I would keep using into the foreseeable future. I don't think that you can say that about any other kind of SAN storage. Everything else has planned obsolensence. I don't know if you will be able to run Compellent fully supported in production 9 years later.
0
 
LVL 4

Author Comment

by:alexsupertramp
Comment Utility
Kevin, can you please assure me that you don't work for Dell?   :)    The more I hear about it, the more it sounds like equalogic a good solution for me, a non-storage admin trying to accomplish the task of a storage admin.  Of course sticker shock is coming when i get the quote this Friday.

the storagemojo site is an excellent read.  thanks again.
0
 
LVL 42

Expert Comment

by:kevinhsieh
Comment Utility
I do not work for any technology company. I work in a private business enterprise. ;-)

I spent around $50K for my PS400E in 2006 with 750 GB drives. A PS4100E with 12 500 GB drives should be less than half that, and I would hope (though I don't know) that it would be closer to 20K, or even less.

I am just a Windows admin trying to keep my network running.
0
 
LVL 3

Expert Comment

by:gerdawg
Comment Utility
I don't work for Dell but wanted to relay my experiences as I have gone through a similiar upgrade as yours in the last few months. In effect I have moved all of my physical servers to virtual, including my domain controllers, exchange servers, dns servers, dhcp clusters, etc.

Since we have 2 seperate sites connected via a 10mb WAN I choose the following (mirrored) setup which has worked quite well for us.

2 - Dell R610 servers with redundant PSU's
1 - MD3200i (With 2TB of storage to start on 10K drives)
2 - Dell Power Connect 5424 iSCSI switches
1 - VMWare ESXi 4.1 vSphere Essentials plus licenses (Which gives you the High Availiablity, vMotion support)
1 - Veeam Backup Essentials license (Allows me to backup my entire virtual enviornment, supports hot restore, live recovery via backup, etc)

Again this is 2 setups of the above so we spend in total around 100K with exchange licensing through Dell which included clustering our exchange servers for disaster recovery/high avialiabllity using DAG.

The setup for us is working flawlessly, but a lot of that happened from doing the research, understanding the options and after making an approriate hardware choice, testing testing testing an rebuilding for the proper optimal configurations.

To give you some statistics on MD Series -

Dell MD1000 (Direct Attached Storage) I get about 14,000 IOPS. (This is a direct attach SAS)
Dell MD3000i (iSCSI) - I get about 7-8K IOPS with more spindles than my MD3200i
Dell MD3200i (iSCSI) - I'm getting about 10-12K IOPS

What's nice about the MD3200i SAN arrays is that they have dual controller cards in them in the even that one of the controllers die you are still up and running. The negative about this entire configuration is that we only have one storage array, so let's say something goes crazy and we lose the firmware in the array, this is still our single point of failure and will need to be rebuilt. Keeping in mind now that EVERYTHING is on this one array and it could be pretty hairy. This is the reason for the veeam backup server. Veeam gives us the ability to directly mount the backups off of one of my poweredge 2950 servers that i put 10TB of storage space into. While it might suck performance wise, it'll get the job done until the array is back up and running. With that in mind we are looking to get an array on the purchase list for next year to provide for fault tolerance on our array itself going down. Over the MD3000i I would definitely go with the MD3200i as it has more twice the throughput of the MD3000i from my own tests that I have seen.

On each R610 servers I'm getting around an 9-1 consolidation ratio....which is huge in my opinion. I feel that the only thing preventing us from getting higher benefits in the future will be VMware's new licensing change and how vRam is now an item to consider for licensing costs. (You'll have to do some reasearch of your own on this).

That said I'm a happy camper and our company has never had as much high availability as it does today. The only thing I would suggest is getting as many NICS in the server with 1GB connections that you can afford. They really do come in handy to have. Off the bat with the MD3000i you'd be looking at taking at least 4 of the NICS for iSCSI traffic to start with...if you are setting up a fully redundant multipathed connection to both controllers...so plan accordingly.

Good Luck!
0
 
LVL 3

Expert Comment

by:gerdawg
Comment Utility
Also on that note - (Since the last time I checked) -  If you are going with VMware either SAN or NAS will work. If due to the new licensing change of VMware your company finds a strong need to at least evaluate Hyper-V over VMware a NAS solution is not supported. The only thing supported on Hyper-V as far as I am aware is iSCSI, Fiber and DAS.
0
 
LVL 4

Author Comment

by:alexsupertramp
Comment Utility
I received an equillogic quote today that is going to net me 8tb of usable diskspace running raid 50, using 12x1tb spindles of 7.2rpm sata.  about $20k.   I am told a similar md3200 is going to run me $5-8k less.     Although I am told by everyone how great equalogic is, the differences between the two don't seem to justify, to me, a 25 to 40 percent cost difference.     Can anyone speak to this?      Thanks in advance, and thanks for all the great info i have already received from this post.
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
7.2rpm SATA - Disks make all the difference to performance. What are you quoting for in MD3200?
0
Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

 
LVL 4

Author Comment

by:alexsupertramp
Comment Utility
I have not received the md3200 quote yet, but i expect the disk configuration to be the same.  i was also quoted on a ps4100x (the quote above is a ps4100e) for $31k. 14.4TB raw, 10K SAS, 24x 600GB.  i will follow up when i get the md3200 quote.

i should also note what i understand to be the differences, although it's still vague to me:

Cross site dr:  not an included feature with md3200.  this is something we will want to implement at some point, but will it cost $5-8k to do on an md3200?

Snap shotting:  not as robust on the md3200.

MD3200: No management tools/software hosted on the appliance itself.

I must be missing something else.
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
Have you considered NetApp?
0
 
LVL 4

Author Comment

by:alexsupertramp
Comment Utility
No, i have not looked at netapp yet, but i understand that is one of the top three players.  
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
Very flexible options, and co-exists with VMware, like pancakes and maple syrup!

Very easy DR functions, to another NetApp Filer, via Snap Mirror additional license. Options of FC, NFS, iSCSI, CIFS, Backup Integration with vSphere, SAN Snapshots (no need for additional expensive Veeam of third party software).
0
 
LVL 42

Assisted Solution

by:kevinhsieh
kevinhsieh earned 167 total points
Comment Utility
The EqualLogic is a high performing unit that runs circles around similarly configured systems (number of spindles and drive type). Try to get a quote for the PS4000E. They might be selling at a discount because of the new PS4100 line, and the PS4000E has 16 drives instead of 12, so you will have more capacity and more IOPS.

One thing to be aware of is that EqualLogic uses a 15 MB internal page size, which is why they get such good performance.  The down side is that snapshots aren't as space effiecient as they would like you to believe. With snapshots and replication, I have 46% allocated for volumes.

Fundamentally, the MD line is aimed at departmental/workgroup. The EqualLogic line was developed for the mid-range market, like EMC Clariion. The difference is that EqualLogic doesn't nickel and dime you, or force you into an upgrade by jacking up maintenance and support costs like EMC and NetApp will. The EqualLogic you should be able to easily run for 5+ years. The original units are 9 years old and fully supported with the current firmware. I can't tell you that Dell will support them for another 5 years+, but EOL for those original units has not been announced.

I don't even know how you would do replication on the MD3000 line. It looks like you need host based replication such as Double-Take, which will probably run you about $5000 PER HOST (at least one host on each side, that's $10K+). It sounds like the next version of Hyper-V will have replication baked in.
0
 
LVL 13

Expert Comment

by:Greg Hejl
Comment Utility
i'll recommend again based on your system requirements....

you speak of 30 servers for the user base you have.  

thats a really small user base that many servers....Do you have cpu/ram intensive apps?

four 1u servers with two hi perf quad cores and 32gb ram or more with an MD3200 unit would suffice.

don't use less than 15k 300gb drives in a raid 10 config. in a virtual environment it's all about disk i/o.

you can expand the MD unit with more drives.
0
 
LVL 3

Expert Comment

by:gerdawg
Comment Utility
Put me in to agree with Greg....I'm in the same situation and have to agree that the MD's work fine for my requirements. I also believe my user base is greater than yours and currently I have plenty of room to expand.

That said the EqualLogic systems are GREAT...I just think they might likely be more than what you want to spend for the features that are provided.

Last I would like to say that getting anything with 7K drives for usage in multiple hosted virtual servers is going to be absolute murder. I'd at least recommend 10K SAS drives and if you can afford it the 15K drives.
0
 
LVL 42

Expert Comment

by:kevinhsieh
Comment Utility
I run about 80 server VMs for 600 users on two 14 drive EqualLogic PS400E units running RAID 50 on 7.2K SATA drives. That's 28 drives, 4 are hot spare, and it's RAID 50. SATA. My entire infrastructure is virtualized, and I have SQL, Exchange, and Blackberry, a 1 TB file server replicating with 12 other file servers. You know, the usual. The MD line is certainly a cheaper way to start and to get your foot in the door.

As for servers, I get the most RAM that is cost effective in a server. Dell shows very little price difference between using 8 and 16 GB DIMMS. A single processor box with 48 GB or 96 GB RAM using 3 or 6 16 GB DIMMS makes the most sense, IMHO. If you need greater density, add a second CPU and go to up to 192 GB RAM on a R710. I have servers with only 32 GB of RAM that I don't even use any more becuase I can't add any RAM to them, so I replaced them with a R710 with 144 GB RAM. I must have bought it when the 16 GB DIMMS were selling at a large premium over the 8 GB DIMMS - I guess I should have waited two months. Every server I have ever used for virtualization was maxed out on RAM. First 16, then 32, and then 72 GB of RAM on a single host - and I have always ended up needing more. The next time I need more RAM in my cluster I may end up pulling the 18 x 4 GB DIMMS from a R710 and replace them with 16 GB DIMMs

You should be able to fit all of your VMs on a single R710 if you put enough RAM in it. Buy two for HA clustering, Windows Datacenter to cover all of your procesors, and you're good to go once you buy the storage. I use Hyper-V. It looks like VMware will charge up the @$$ for running hosts with that much RAM under the new licensing scheme, so I do it for free.
0
 
LVL 4

Author Comment

by:alexsupertramp
Comment Utility
i know my user to server ratio is low, and actually i did a specific user count for the first time in a while and we have more potential users than i thought, but the amount of simultaneous users would commonly not exceed more than 100.  part of the high server count is due to small clusters of client users needing a specific server environment, hence my need to move away from physical hosts.   i would not say that our applications and users are very cpu or ram intensive, but i farm out a lot of applications through citrix and want to start moving into xen desktop as well.

so, it seems like i'm hearing opposing viewpoints on the hard drive configurations.   dell had me run an IOPS utility in our environment and came away with the idea that the 7.2k sata drives would suffice...and of course they are in sales so they also may be trying to keep me from running away with sticker shock.

maybe this is worth another post to the citrix experts, but in regards to the configurations i have mentioned, is there anything i am missing if i want to deploy xen desktop and terminal services to local and remote users?   this will be the most common use of the vm's.
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
But when you run the same utility in a VMware environment IOPS will drop in the VM, on the datastore. Also IOPS will drop because you are not using direct attached storage.

7.2k sata drives are shocking, here's another EE question, with 7.2k SATA drives, with performance issues.

http://www.experts-exchange.com/Software/VMWare/Q_27328394.html

Also ask the Dell SalesMan if they WILL YOU AN IRONCLAD GUARANTEE those IOPS in VMs, or your moneyback on their SANs from within a VM!

I've seen so many virtual projects fail, because of poor performance of 7.2k SATA drives!
0
 
LVL 13

Expert Comment

by:Greg Hejl
Comment Utility
i convinced my company that it was worth the cost of a new delivery van to provide service to our clients.

50-60k in hardware, new hardware, that works without flaw is very cheap to run a business on.  it's also cheaper than renting the equipment in the cloud.  the amortization rate, the resale value, etc make it a very cheap investment.

good luck!  it sounds like a fun project!
0
 
LVL 4

Author Comment

by:alexsupertramp
Comment Utility
copy that, hanccocka, on the 7.2k sata.  it's on my radar...although kevin seems to use it heavily without issue.    i am also intrigued by vsa for a fast and inexpensive setup at our colo, or maybe even at our main data center.  i'm open to any comments on that, otherwise i'll close this ticket soon.    thanks again.
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
vSphere VSA and HP VSA, are interesting products, but SvSAN and SanSymphony, are also now released, and I believe StarWind also have a VSA on the way!
0
 
LVL 3

Expert Comment

by:gerdawg
Comment Utility
rsync works really well for syncing datastores from production to a colo as a backup as well if that's what you're looking for.

In regards to the 7.2K drives I would just keep in mind that IOPS are going to be dependent on how many drives you stick into your array as the spindle count is a factor in determining IOPS. What I am getting at with this statement is at some point if you were to fill your entire MD3200i out of the gate with 7.2K drives, you might find that you are hitting the maximum IOPS that the controller itself can maintain. It's just food for thought. Personally I would go with the 10K spindles or the 15K but I believe there could be other scenarios where having a 7.2K might be advantageous as Kevin has found. The best way to determine that would be to ask your Dell rep.Since I wasn't filling up my MD3220i out of the gate and only put in about 3/8ths the total capacity 10K drives were right in line for me. I think results will vary based on your needs.

0
 
LVL 4

Author Closing Comment

by:alexsupertramp
Comment Utility
Great discussion.  Thank you.
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Last article we focus in how to VMware: How to create and use VMs TAGs – Part 1 so before follow this article and perform the next tasks, you should read the first article how to create the TAG before using them in Veeam Backup Jobs.
HOW TO: Connect to the VMware vSphere Hypervisor 6.5 (ESXi 6.5) using the vSphere (HTML5 Web) Host Client 6.5, and perform a simple configuration task of adding a new VMFS 6 datastore.
Teach the user how to delpoy the vCenter Server Appliance and how to configure its network settings Deploy OVF: Open VM console and configure networking:
Teach the user how to convert virtaul disk file formats and how to rename virtual machine files on datastores. Open vSphere Web Client: Review VM disk settings: Migrate VM to new datastore with a thick provisioned (lazy zeroed) disk format: Rename a…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now