SAN Storage Solutions

I'm looking to upgrade my enterprises storage solution to a more robust enterprise level solution.  Currently all my servers are using directly attached  storage (either on board or via RAID controller)

Currently I am looking at moving to a SAN based solution, and have realized there is a lot more out there than I originally thought.  I'm looking for information on what vendors have quality systems.  Currently I am looking at migrating some 10 servers (some of these are to be consolidated onto virtual machines as I am rolling out  VM with the SAN).  

I'm fairly new to SANs (I've never actually used one) but as a networking admin the technology doesn't seem that far off.  My main questions are regarding:
1. Quality of vendors
2. Hardware needs (as far as i can tell, I'd need an enclosure/controller, switch, connectors and PCI cards for my servers)  
3. Protocol - iSCSI, FC or FCoE

I've been doing research, but am always looking for suggestions and tips.  I've been looking at the Dell Compellent System, as well as their smaller FC solutions.  

My thought  at this time, is to be able to have a total of about 10TB of storage space, for maybe some 5 physical servers (one of which will be a VM host), with my drives preferably in RAID 10.  

One other thing I read, was that, looking at the dell system, it can house "SAS, NLSAS, SSD and Fiber Channel" - I was unaware that FC was a drive type (I though it was just a connection protocol) .  Am I confusing something here?

Any good resources would be highly appreciated.

Who is Participating?

Improve company productivity with a Business Account.Sign Up

CCUITAdminConnect With a Mentor Commented:
Go with EMC or any vendor that offers "thin provisioning". Thin provisioning is storage magic. Because it doesn't store white space between virtual machines, you can store many times your need in a very small space. So lets say you have a server for a department that might experience growth in data. You can create a 100 Gb vm, and only use 6 gig. The storage only store 6 gig until the server needs more. I am storing over 2 Tb of vms in less than 500 Gb of space.

As long as it does storage dedup and thin provisioning, its better to go with a small (4-5 tb) but fast storage for vms than large but slow storage.  We got our SAN from EMC, and are extremely happy with it.
Paul SolovyovskyConnect With a Mentor Senior IT AdvisorCommented:
For a green field environment you should look at iSCSI, especially for a smaller environment.  You'll on FC switches, HBAs, training overhead, etc..

Some SANs offer NAS functionality as well and act as your file server.  

Take a look at the EMC VMXe series and the Netapp 2000 series (both offer iscsi, fc, cifs, nfs)

For FC/iSCSI only take a look at HP P2000/4000 lines, the Dell MD series is an unknown at this point as Netapp purchased the company that rebrands it.
andyalderConnect With a Mentor Commented:
As far as fibre channel goes there are native fibre channel disks meaning that they have a fibre channel interface and run that protocol. There's no such thing as fiber channel.

For a controller to disk connection protocol it doesn't really have any benefit over SAS in a small array, you're not going to have the disks more than 2 metres away from the controller and you can't read and write disks at 6Gb, let alone 8Gb.
Improve Your Query Performance Tuning

In this FREE six-day email course, you'll learn from Janis Griffin, Database Performance Evangelist. She'll teach 12 steps that you can use to optimize your queries as much as possible and see measurable results in your work. Get started today!

Paul SolovyovskySenior IT AdvisorCommented:
@andyadler:  Any word on what Netapp will do with the IBM DS and Dell MD relationships?
JamesonJendreasAuthor Commented:
Just a quick FYI,
We are currently a 100% MS shop (although if I have my way we'll be throwing in some Linux boxes).  This is starting as a smaller scale project for my more non-mission critical data (essentially everything but my database)

One of the main things is we are looking to grow this system over time to become our main storage solution.  I'm looking for future growth, and to be able to keep my system up to date with the newest tech.  Looking at modular systems, where future disk/enclosures can be added.  I'm also currently looking at using a single storage device, and two switches (for redundancy) to start as a base.  

I'm looking at the moment at an FC fabric with 8Gbps (for future bandwidth concerns).  I'm OK with the learning curve involved (I've got a few books on SANs, so I'm not lost).  From what I can tell, FC is the major player in SANs at the moment.  Are there any benefits to looking at iSCSI?

Thanks for the recommendations, I'll take a look at those systems.
Paul SolovyovskySenior IT AdvisorCommented:
EMC and Netapp will do this in addition to acting as your file servers and let you access your shares through windows computer manager.
JamesonJendreasAuthor Commented:
Oh yeah, I meant fibre channel, not fiber channel in describing the disk, my bad :)
iSCSI will never compete for performance with fibre channel, you're basically encapsulating one protocol inside another which increases latency, 9 Gb FC beats 10Gb iSCSI. FCoE competes because it's just a different type of wire it runs over but it's not that mature yet.

You can always hold off on buying the FC switches for a while, you can connect a couple of hosts to the shared storage with point-to-point cabling assuming dual controllers and at least two host ports.

I'd start with something relatively cheap like a HP P2000 MSA, but do you have a preferred server vendor? That can really be the deciding factor on what SAN to buy because you don't have two different vendors to deal with if something goes wrong.

Don't forget your MS licensing costs when you have high availability, a cluster of 3 servers running 6 MS VMs = 3*6=18 standard Windows licenses, you soon need Enterprise (4VMs) or Datacentre (unlimited VMs per CPU) licensing model.
JamesonJendreasAuthor Commented:
Thanks Andy,
I'm fairly vendor agnostic, although we've used Dell for most of our server hardware (we had a stint with SuperMicro, but have had many issues with them).  Dell, HP, EMC etc are all viable options. I know that there are some compatibility issues between vendors, so I want to make sure whatever vendor I choose has the scalability needed.  FC is looking like the protocol I'm looking at.

MS licensing is something I'm working on as well (we have a bunch of licenses on order)  
Gerald ConnollyConnect With a Mentor Commented:
The main vendors in the storage area are HP, EMC, IBM, HDS and NetApps. These vendors typically cover the entire storage field with a plethora of smaller specialised vendors that cover point solutions. Just to make it more fun there has been a lot of consolidation and a lot of brands that are out there are now owned by these bigger players, i notice that BlueArc got gobbled up by HDS recently.

Do your research carefully, look at what you need in terms of connectivity, Capacity, MB/sec and IOPS and and planned growth in these areas. If your MB/sec requirement is high you might consider a FC SAN (or even Infinband) but for most people a iSCSI SAN is suitable (and a lot cheaper) but don't forget to provision your Network to cater for all the extra traffic!

Factor in your backup regime, and the extra bandwidth it might add, plus the extra costs your backup vendor is likely to add as well.

Oh and get the vendors reps to do a lot of this work for you.
JamesonJendreasAuthor Commented:
Thanks Conn -
I'm working with vendors on the SAN as well - I like having 3rd party advice (ya' know, not someone who's selling me the 60K+ system) , so I appreciate the feedback.

Re: iSCSI - I'd rather keep this traffic off my LAN - we have capacity issues as is with our system (another big project I'm working on).  Sadly, they want the SAN in prior to my re-vamping of the entire core of my network (at least I finally get some good hardware!).  

I'm thinking FC - With the capacity I'm looking at (about 10TB to start) and the money I have budgeted, it may be the way to go.  At this point I'm looking at a fairly basic setup:

HP StorageWorks P2000 G3 MSA Dual Controller 8Bbps, pre-packed with 24x600GB SAS
HP D2700 25-bay 2.5" enclosure w/ 10x600 GB drives
16-port FC 8Gbps Switch x 2 for redunancy
Qlogic 8Gb Dual Port FC PCIe 8 LC Multi-mode Optic HBA (4 to begin for 4 servers)

I haven't seen how the D2700 interfaces with the P2000 - I'm wondering if it needs a HBA or some kind of SAS\RAID connector
kevinhsiehConnect With a Mentor Commented:
I have been an EqualLogic customer for about 4 years now. I can tell you that it performs really, really well. For your needs, iSCSI is more than suitable and I see no reason why you would need FC. All you need is the Microsoft iSCSI initiator, gigabit Ethernet, and a good iSCSI SAN backend. You don't need iSCSI HBA, TCP offload, or even jumbo frames. The slowest part of your system will be reading from spinning rust. The connection protocol and interface doesn't make that much difference. IOPS and bandwidth are not the same thing, and unless you are doing video streaming or other very large sequential data transfers, you need IOPS, not bandwidth, so the differences between 1 GB Ethernet, 4 GB FC, 8 GB FC, 10 GB Ethernet, and 40 GB Infiniband and meaningless. From what I cantell, FC really makes the most sense if you already have an investment in FC. If you have a new installation and you think you really need FC, you should also be looking at Infiniband. Otherwise, iSCSI should be the default conenction of choice.

EqualLogic allows you to add as you grow, without ever needing to do a forklift upgrade. You can run the original PS50E in the same group (virtual cluster) as a brand new PS6100XV and use that as a single virtual pool of storage and have your data auto-tier between the 7.2K SATA, 10K SAS, and SSD drives. Every unit EqualLogic has ever sold is still supported and can run the current firmware. NetApp won't do that, and neither will EMC. They will force you to upgrade by jacking up their maintenance costs so high that it's cheaper to upgrade.

I suggest you look at the PS4100 line. They offer great capacity and IOPS per $ spent. The only downside is that Dell limits you to 2 PS 4XXX units in a storage group. Otherwise, you could just keep on buying the PS4XXX units and skip on the more expensive PS6XXX line completely. If you bought a PS4100X with the 900 GB 10K SAS drives (24 drives), that would keep you going for a long time. When you need to expand capacity, you can add another unit or do a on-line conversion to RAID 50. With EqualLogic all features and future upgrades and features are included if they are part of the firmware. There are no per host licensing costs, replication licenses, thin provisioning licenses, snapshot licenses, MPIO licenses, etc. Everything is included for a single understandable price. Maintenance is really cheap (I spend less than $3000 per unit on 3+ year old hardware).
If you buy 2 gigabit Ethernet switches instead of FC switches, you will have the iSCSI traffic off your LAN, and you can connect ALL of your physical and virtual machines, not just the ones you buy HBAs for. I am so glad that I am able to attach to my SAN everything that I want to, without worrying about buying and installing a FC HBA. It also makes replication to my DR site SOO much easier.
Gerald ConnollyCommented:
i agree with@kevinhsieh, try and stick with iSCSI unless your i/o requirements are very high.

the p2000 controller has a SAS backend, but can have a FC,SAS or iSCSI front-end, it has SAS ports to connect additional storage shelves to
davek101Connect With a Mentor Commented:
If you are going down the SAN route my recommendation would first be to ensure that SAN is the correct solution for your environment.  You may well find that a NAS device will meet your requirements potentially at a fraction of the overall cost.

Things to consider in any SAN/NAS solution

thin provisioning capability;
on-system backups (knowns as snapshots by some vendors);
multiprotocol (FC;iSCSI;CIFS;NFS);
support multiple disk types (SAS;SATA;ATA;FC);
can attach a backup device (FC tape drive as an example);
DR capabilities with added options and hardware.

If your environment if Windows servers and VMWare, I would look at a NAS device for your solution.  Use CIFS for Windows presentation and NFS datastores for VMWare.  If you do want to go down the SAN route, for the size of your environment I would recommend using iSCSI.  It takes advantage of your existing infrastructure without having the expense of FC switches and the associated licenses and support contracts.  It is true that FC may provide a faster solution but you have to ask yourself whether your environment needs all that capacity?  A well designed and implemented iSCSI SAN will provide sufficient performance.  A quick search will show a number of large companies running EXCHANGE; SQL; ORACLE and more over iSCSI without issue.

Hopefully this is of some use.
Gerald ConnollyCommented:
@davek101, since when have snapshots been known as "on-system backups".

Surely it should be snapshots (known as "on/off-system backup by one vendor")?                :-)

does that mean that snapshots on something like an EVA are "off-system backups"?
I think he's comparing them with remote snapshots for DR such as you could have with P2000 FC/iSCSI combo controller, and most midrange and above SANs.
Different vendors employ different methods of taking snapshots.  I can't say how each vendor implements this and which ones are better.  That is not a conversation for these forums and is best left to salesmen.

In my environment we use NetApp.  Multiple snapshots can be taken of any volume.  On our CIFS volumes we retain 30 days of snapshots.  These take up little space and give us the capability for almost immediate recovery.  These are what I call "on-system backups".  We also replicate these volumes and the associated snapshots to a DR system.  In addition to CIFS we also run snapshots of our VMWare, MS Exchange and MS SQL environments.

I know that EMC and other vendors have similar capabilities however, I'm not here to recommend one over the other.
On the subject of vendor quality, it seems that the HP purchase of LeftHand was a very bad thing for customers. Read the later comments about LeftHand after the HP purchase.

I don't have experience with Compellent, but I have been an EqualLogic customer since before the Dell purchase and I can say that Dell has maintained and improved upon the quality of the product and the support. I expect that they will do the same with Compellent.

As I have said over in , I would put a 9 year old, original EqualLogic PS100E into my production environment if I could get my hands on one. They don't seem to be available in the used marketplace because everyone keeps them running forever. I don't think you can say the same thing about other storage products.
Strange that you would introduce LeftHand this late into the therad. I don't think anyone introduced it before because it doesn't sound like they need quite the level of redundancy that it offers. We do sell some LeftHand but it's really for people like banks who want more than two copies of the data.
Gerald ConnollyCommented:
Different vendors employ different methods of taking snapshots.  I can't say how each vendor implements this and which ones are better.  That is not a conversation for these forums and is best left to salesmen.

My point was not meant to be vendor specific, it was purely about terminology!   And i know very well that different manufacturers implement them in different ways.
OP asked about quality of vendors, and there has been much more said about iSCSI vs FC than actual vendors and products. The HP LeftHand solution was recommended by paulsolov, I thought that I would point out that there seem to be some HP LeftHand customers that are very unhappy, which seems worth mentioning.

@JamesonJendreas, the Compellent solution looks like a traditional controller based architecture which will eventually require a forklift upgrade. On the other hand, as long as your controllers are up to it you can probably just keep adding disk shelves for a while to improve capacity and IOPS. The EqualLogic solution has proven over 9 years that you can add the newest technology unit to the first unit ever sold and run them as a single group of storage.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.