Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

NetApp SAN switch network design question

Posted on 2014-04-13
28
Medium Priority
?
941 Views
Last Modified: 2014-04-26
I have a NetApp, a switch and two ESXi servers in my rack. The switch in the rack is connected to the core switch for our LAN via ether channel. It looks like this:



[core switch]---------------------------[storage switch]--------------[NetApp]--------[ESXi 1 & ESXi2]  

My question: is this a typical design?  Do most environments have a separate storage switch sitting between their storage and ESXi servers or do they just connect everything to the core of the LAN?

What is the best practice approach?

Thanks
0
Comment
Question by:wayy2be
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 12
  • 9
  • 7
28 Comments
 
LVL 123

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 1000 total points
ID: 39997314
It could be, but we install seperate storage switches, usually two, in a STACK, and then comnnect at least two paths from every switch to each ESXi server, using Multipath.

(not teamed or ether channel connections, e.g. trunks)

Best Practice is Multipath, and enable jumbo frames make help.

HOW TO: Add an iSCSI Software Adaptor and Create an iSCSI Multipath Network in VMware vSphere Hypervisor ESXi 5.0

HOW TO: Enable Jumbo Frames on a VMware vSphere Hypervisor (ESXi 5.0) host server using the VMware vSphere Client
0
 
LVL 10

Expert Comment

by:Schuyler Dorsey
ID: 39997319
As Andrew mentioned, you certainly should have two storage switches stacked in this scenario as you have created a single point of failure. If that one storage switch goes down, all of your VMs will go down.
0
 

Author Comment

by:wayy2be
ID: 39997327
Absolutely I agree and that is what I am planning to do in the next few weeks. Budget was tight last year so we could only afford one switch.  So how does the storage switch stack get connected to the core LAN switch?  What is the best way?
0
When ransomware hits your clients, what do you do?

MSPs: Endpoint security isn’t enough to prevent ransomware.
As the impact and severity of crypto ransomware attacks has grown, Webroot fought back, not just by building a next-gen endpoint solution capable of preventing ransomware attacks but also by being a thought leader.

 
LVL 10

Expert Comment

by:Schuyler Dorsey
ID: 39997328
Typically a port-aggregation. You could do two ports (one from each storage switch) to the core switch.
0
 

Author Comment

by:wayy2be
ID: 39997345
I did have a vendor tell me to forget the storage switch and connect straight to the core with the ESXi and NetApp. However I did not feel that this was correct. What do you think of that?  The current storage switch is a Cisco 3750. Is that a good switch for this purpose or overkill?

Thanks for the replies guys
0
 
LVL 10

Expert Comment

by:Schuyler Dorsey
ID: 39997348
Actually.. that switch is BELOW what I normal spec. Cisco 3750's often have difficulty keeping up with storage due to the low cache buffer sizes. It will REALLY affect your switch is your storage environment does a burst.

If we are specc'ing Cisco, we do a MINIMUM of Cisco 3850. If we do HP, we spec HP 3800s.

As far as connecting directly to the core, I see that a lot at small businesses. So I suppose it would depend on the size of your network/business. It is common practice to use a separate switch for storage.
0
 

Author Comment

by:wayy2be
ID: 39997367
We have around 200 clients and 25 vm's.  What would you suggest in Brocade?  If I bought Brocade would they place nice with our Cisco core, which are also 3750's?
0
 
LVL 10

Expert Comment

by:Schuyler Dorsey
ID: 39997369
I can't speak on the Brocade side. I have only worked with their fiber switches. Those are okay but I'm not crazy about the mgmt side of them.

It should play nicely as long as you are not using Cisco proprietary protocols such as EIGRP.  In this case, you would have to add another routing protocol to your network such as OSPF.

Any reason you mentioned Brocade? The specific HP and Cisco switches I mentioned would do wonderfully. The HP one would actually be a better performer than the Cisco, has lifetime warranty, lifetime updates and would be much cheaper.
0
 

Author Comment

by:wayy2be
ID: 39997372
No reason, just looking at all the options. Would the HP play nice with the Cisco core?
0
 
LVL 10

Expert Comment

by:Schuyler Dorsey
ID: 39997375
Same with the Brocade.. you only have to consider if you are running any proprietary Cisco protocols like EIGRP. HP is an open standards based switch so it will support RIP and OSPF. Cisco DOES allow you to inject EIGRP routes into OSPF if needed.

But I have several environments with both HP and Cisco running without issue.
0
 
LVL 123
ID: 39997559
We use Brocade or Dell Power Connect 7024 Storage switches in a stack, just for storage.

These are not connected to any other networking, it's the storage network is isolated and runs Jumbo Frames.
0
 

Author Comment

by:wayy2be
ID: 39997908
Andrew:  So how are you connected to your LAN?
0
 
LVL 123
ID: 39997937
We have a seperate completely dedicated 2 to 4 storage switches, which ONLY NFS and iSCSI traffic passes from ESXi servers to SAN.

These are Storage Switches purchased for the purpose of high available NetApp Storage, we DO NOT use our Core, for the passing of storage traffic.

This gives us the high availability we need, if the Core is affected, our VMs and SAN are still running.

We use Cisco and HP for our Core and Edge switches to which are VMs are connected, for End User traffic, but ALL storage goes via seperate Storage Network.

There is no requirement to connect iSCSI and SAN to Core.
0
 
LVL 10

Assisted Solution

by:Schuyler Dorsey
Schuyler Dorsey earned 1000 total points
ID: 39998068
Andrew is completely correct.

However, if you are a smaller business and don't have the capacity to physically separate them, you can still logically separate them.

E.g. Say you have two Core switches stacked. You can have two "storage" switches stacked which connects your NetApp to your servers. How CAN pass DATA through those switches and uplink those switches to your Core without an issue. The key is putting data and storage traffic on separate VLANS.

So if you vlan them out, the switches will NOT pass storage traffic to your core. You wouldn't even have to tag storage traffic on the uplink port and this would prevent traffic from going to the core for sure.

For smaller businesses, this is a perfectly acceptable solution. I actually see this case more often than not in enterprise networks too.

Though I do have a question for Andrew:

If you do not connect your storage switches to the network in any way, how do you manage them? Seems that would limit you to console only unless you put a pc on that vlan, on that switch.
0
 

Author Comment

by:wayy2be
ID: 39998171
Very interesting.  What I have now is one 3750 as a "Storage Switch". It has vlans for IP storage for my NFS traffic and then vlans for my ESXi hosts. It is also connected to the core so users can access the vm's. I am not sure if I explained this properly before. So I just need to add another switch, yes?  What else can be done to make it better?  Thanks guys
0
 
LVL 123
ID: 39998327
Add another switch, add and configure Jumbo Frames, switch configuration to multipath.

Storage Switches do not need to be connected to Core, so users can access VMs!

ESXi Servers needed to be connected to core.
0
 

Author Comment

by:wayy2be
ID: 39998768
I am not familar with multipath. Where is that configured and how?  Thanks
0
 
LVL 123
ID: 39998771
I posted in my first posting above here in my EE Article http:#a39997314
0
 

Author Comment

by:wayy2be
ID: 40012815
Sorry guys got pulled off in another direction.  Here is a diagram of what I have.  What do you think?
vmware.jpg
0
 
LVL 123
ID: 40013244
looks okay, if just using NFS.

So no iSCSI.

Also enable jumbo frames.
0
 

Author Comment

by:wayy2be
ID: 40013515
So the storage switch is my primary concern, should it go down, everything vm goes down with it.
0
 
LVL 123
ID: 40013528
Yes, that's correct, if you have a single switch, all your VMs will go down.

the same as if you SAN goes down, all your VMs will go down.

two single points of failure!

Add another switch, and/or SAN!

Availability/Resilience/Redundancy is all amount, how much money do you throw at a solution!

How much downtime, can the business afford, how long would it take to get another switch.

(you could have one on the shelf, ready configured!)

Also, do the switches, have one PSU or two, aer they connected to different UPS.
0
 
LVL 10

Expert Comment

by:Schuyler Dorsey
ID: 40014765
Does your NetApp only have one controller? If it does, I would at least get a second controller for it.

As far as the switch, I would recommend getting a second switch and stacking the two and connecting your environment to both switches. So if one switch fails, everything continues working seamlessly.
0
 

Author Comment

by:wayy2be
ID: 40016564
I have two controllers in a HA configuration.  There is only one point of failure, the switch. We have another SAN at our DR site that we snap mirror all of our data to every hour. I am looking at getting 2 - 24 port Brocade 6450 switches, stack them and cross cable them.
0
 
LVL 123
ID: 40016877
Another switch will resolve the single point of failure issue.
0
 

Author Comment

by:wayy2be
ID: 40023878
Thanks guys for your replies and time!
0
 

Author Comment

by:wayy2be
ID: 40023887
Andrew,

What model Brocade do you normally use?  I have been looking at the 6610 for this environment.
0
 
LVL 123
ID: 40024399
Dell PowerConnect 7024 (re-badged Brocade switches by Dell!)
0

Featured Post

Optimum High-Definition Video Viewing and Control

The ATEN VM0404HA 4x4 4K HDMI Matrix Switch supports 4K resolutions of UHD (3840 x 2160) and DCI (4096 x 2160) with refresh rates of 30 Hz (4:4:4) and 60 Hz (4:2:0). It is ideal for applications where the routing of 4K digital signals is required.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The following article is comprised of the pearls we have garnered deploying virtualization solutions since Virtual Server 2005 and subsequent 2008 RTM+ Hyper-V in standalone and clustered environments.
The business world is becoming increasingly integrated with tech. It’s not just for a select few anymore — but what about if you have a small business? It may be easier than you think to integrate technology into your small business, and it’s likely…
How to install and configure Citrix XenApp 6.5 - Part 1. In this video tutorial we have explained step by step installation of Citrix XenApp 6.5 Server on Windows Server 2008 R2 is explained in this video. We have explained the difference between…
This course is ideal for IT System Administrators working with VMware vSphere and its associated products in their company infrastructure. This course teaches you how to install and maintain this virtualization technology to store data, prevent vuln…

664 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question