Solved

Blades and Convergent Network Switches

Posted on 2014-09-10
12
280 Views
Last Modified: 2016-11-23
I'm looking to deploy a blade solution in our network.  It is, however, going to be my first time with them and I'd like to verify my understanding what I've read about it.  We're a VMware shop, so I'll be referencing it specifically to how it'd be used.

Since the Dell website is the easiest to load out a full enclosure, that's what I'm going to use as a demo unit for this question.

Dell M1000E Enclosure:
	8 x M520 Blade Servers:
		1 x Broadcom 5720 Quad NIC in slot A
		2 x Brocade 1741M-K Dual Port 10GbE CNA in slots B and C
------------------------------------------------
	1 pair of M6348 Ethernet Switch in slot A
	2 pairs of M8428-K Converged Network Switch in slots B and C

Open in new window


I have an existing fibre channel storage array which plugs into a pair of SAN switches, which presently feeds a number of rack mounted servers (to be replaced by the blades), as well as a pair of GigE switches operating as distribution layer switches.

Assuming blade CNAs were numbered and converged switches were lettered, would it make sense that I could:
Connect CNAs to Storage Array, offering FC adapters to VMware vSphere hosts (blades)
	Switch B-1 to extern Storage array Controller 1 port 1 via fibre
	Switch B-2 to extern Storage array Controller 2 port 1 via fibre
	Switch C-1 to extern Storage array Controller 1 port 2 via fibre
	Switch C-2 to extern Storage array Controller 2 port 2 via fibre
Connect CNAs to Ethernet distribution switching, offering 10GbE network access (negotiating down to 1GbE if extern switches don't support 10GbE):
	Switch B-1 to extern GbE switch 1
	Switch B-2 to extern GbE switch 2
	Switch C-1 to extern GbE switch 1
	Switch C-2 to extern GbE switch 2
Connect 1GbE switch to Ethernet distribution switching:
	Switch A-1 to extern GbE Switch 1
	Switch A-1 to extern GbE Switch 2
	Switch A-2 to extern GbE Switch 1
	Switch A-2 to extern GbE Switch 2

Open in new window

And that the connections as described above would offer:
4 paths to the FC storage
4 10GbE paths to the network
4 1GbE paths to the network

Would I also be able to connect the converged switches to each other for increased availability?  For example, if they were not connected and both Converged Switch B1 and Storage Array Controller 2 were to fail, I would ultimately only have 1 path to the storage array (C1 -> SA C1P2), where if they were interconnected, I would also be able to utilize C2 -> SA C1P2, providing room for one of the Blade slot C mez cards to fail as well.
0
Comment
Question by:lunanat
  • 6
  • 5
12 Comments
 
LVL 20

Accepted Solution

by:
SelfGovern earned 500 total points
Comment Utility
You look to be on the right track.
Would you like me to recommend an HP reseller who can help you put together a system that meets your needs using the HP blade system?  (or even perhaps a reseller for another vendor, but I'm partial to the HP systems)
0
 
LVL 1

Author Comment

by:lunanat
Comment Utility
Thanks SelfGovern, I have a sizable list of resellers from previous business dealings (covering all the major manufacturers), as well as a published request for proposal when the time comes.  At this point I still need to see if my budget request will be approved (fingers crossed, we're pretty much at full capacity now and we still have 3 months till I'd be able to make the purchase!)
0
 
LVL 55

Expert Comment

by:andyalder
Comment Utility
If you go with HP I'd skip the convergent networking part to avoid the very expensive switches that can handle FCoE to FC and just put FC mezzanines in the blades and Brocade switches in the I/O slots of the C7000 and connect those FC switches to your current ones.  Convergent networking is still in its infancy and therefore isn't cheap.
0
 
LVL 1

Author Comment

by:lunanat
Comment Utility
Hi Andyalder,

The idea is to get ahead of the curve, historically I have been either at the curve or behind and we've suffered for it a few years after our purchase when we end up at capacity.  Time to learn from past mistakes.  From a VMware perspective, I would arrange my networking as follows:
A:  2 active/2 standby 1GbE on a vSwitch with vmotion and vmkernel
B/C: 2 active / 2 standby 10GbE on a vSwitch with assorted vlans and vms for vm traffic
B/C: 4 active paths to the storage array

I am also considering arranging the 10GbE vSwitch to 4 active as well, but I'd need to look at the load balancing to ensure I didn't have any mac flapping or similar issues going on.  For the time being that would be a "to consider" solution.  Will be fun to test :)

Ultimately, I end up with a dedicated nic for system usage (vmk for example) which although is a SPOF, isn't a big one - the VMs will continue running, even if the cluster loses contact with a host, which allows me to gracefully resolve the problem after hours.

I also end up with 2 converged adapters doing the work of 4 purpose-built... I have 4 active paths to my storage, and either 2/2 or 4/0 for VM traffic at aggregated 10GbE speeds.  My network would certainly bottleneck the service out to PCs, but cross-host VM to VM communication would be blisteringly fast by our present standards.

I certainly understand what you're saying, but that's my logic :)
0
 
LVL 55

Expert Comment

by:andyalder
Comment Utility
What do you have to convert the FCoE from the CNAs into FC to connect to your SAN?
0
 
LVL 1

Author Comment

by:lunanat
Comment Utility
It is my expectation that the converged switches will be able to accept FC inputs, likely via an adapter such as a SFP.
0
What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 55

Expert Comment

by:andyalder
Comment Utility
They are, I was just asking what switches you were buying.
0
 
LVL 1

Author Comment

by:lunanat
Comment Utility
Ah,  right.  My plan is to plug the storage array controllers right into the blade fibre switches on a 1:1 basis (4 controller FC ports, 4 switches).  Nothing else is yet plugging into the storage fabric but my hosts, so no reason to have additional FC SAN switches.

If I end up connecting all my FC together as I mentioned in my OP, I would use existing FC SAN switching for that... HP branded Brocade switches.
0
 
LVL 55

Expert Comment

by:andyalder
Comment Utility
So you are getting convergent network switches that have FC support for your blade enclosure? Hmmm, Dell M8428-k price $13K each from one supplier but another is claiming 1/10th of that price - obviously a typo. It'll certainly work but you're looking at €40K+ for the 4 switches.
0
 
LVL 1

Author Comment

by:lunanat
Comment Utility
Yup, super expensive.  But this time I can get ahead of the curve, get some wicked fast access to my storage.  I'm only going to put 8 of the possible 16 blades in it, which means I've got room for a very cost-effective 100% increase in capacity (the blades themselves are pretty inexpensive, and the switching infrastructure will be in place already).

It'll stay in production for between 5 and 10 years (unless something insane happens, but then I get more new toys so no complaints), then it'll go to my colo for another 5 to 10 years, and it will then get relegated down to non-virtualized tasks.  The cost over the lifetime is quite low on a year by year basis - I just need to front the capital.

When it's time to replace my storage array, I will be in a position to leverage the latest and greatest in throughputs, while actually being able to handle that kind of IO on the fabric.
0
 
LVL 55

Expert Comment

by:andyalder
Comment Utility
Nice to know it'll keep up with the 16Gb ports on your future storage array.

I fail to understand how it will be faster than having a normal LAN plus additional FC infrastructure in your enclosure. I do understand the bit about new toys though. If you can afford $1000 per port now why wait for the price to drop.
0
 
LVL 1

Author Comment

by:lunanat
Comment Utility
Okay, so maybe not "the" latest and greatest... but it's a big step from our present 4Gb ports ;)

The speed increase will be predominantly noticed in cross-VM communication... for example, an application server talking to a backend server.  We have a few line of business apps which are pretty heavily scaled out, and giving them a faster means of communication should improve their performance now and allow for it to continue to grow in the future.  It also means that I can leverage our existing disaster recovery solution faster.... muuuch faster :)

If the budget doesn't get approved in full I'll scale down to dedicated FC and eth slots.
0

Featured Post

How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

Join & Write a Comment

Suggested Solutions

This is an issue that we can get adding / removing permissions in the vCSA 6.0. We can also have issues searching for users / groups in the AD (using your identify sources). This is how one of the ways to handle this issues and fix it.
This article will show you how to create an ISO CD-ROM/DVD-ROM image (*.iso), and MD5 checksum signature, for use with VMware vSphere Hypervisor 6.5 (ESXi 6.5). It's a good idea to compare checksums, because many installations fail because of a corr…
Teach the user how to install ESXi 5.5 and configure the management network System Requirements: ESXi Installation:  Management Network Configuration: Management Network Testing:
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now