cfan73
asked on
Switching/routing design recommendations for hyper-converged environments
I have a (hopefully very simple question) regarding routing in a hyper-converged environment. In the simplest form, let's say we have a single HC node (HyperFlex, SimpliVity, etc.), and the VMs being hosted on this node are within a typical Web/App/DB hierarchy - so, all in different VLANs/subnets, virtual firewalls in-between, etc.
Given the above, would communication between the different tiers (W/A/DB) require exiting the HC node to an external routing device, or would the virtualization hypervisor somehow be able to handle this? (Let's assume NSX is off the table.) This would seem to be a "normal" deployment for any HC environment, in that the more condensed the environment, the higher demand for internetwork connectivity.
I'm trying to get to whether a Layer 3 physical switch is "normal" for these environments to handle this inter-tier traffic, or if there are more efficient options.
Thank you
Given the above, would communication between the different tiers (W/A/DB) require exiting the HC node to an external routing device, or would the virtualization hypervisor somehow be able to handle this? (Let's assume NSX is off the table.) This would seem to be a "normal" deployment for any HC environment, in that the more condensed the environment, the higher demand for internetwork connectivity.
I'm trying to get to whether a Layer 3 physical switch is "normal" for these environments to handle this inter-tier traffic, or if there are more efficient options.
Thank you
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
A Kepler-47 setup, as originally configured by Dan Lovinger, requires nothing more than the Gigabit ports on the server board for a shared virtual switch with the host OS.
There are plenty of examples out there.
They usually use direct-connect 10GbE RDMA via RoCE (Mellanox) or can use 25GbE direct-connect if there's a throughput need. Cost wise, they are not that expensive.
The solution can scale-out from there into a pair of switches and more nodes to add both compute and storage into the mix.
The technology that makes it all work is called Storage Spaces Direct and it's built-in to Windows Server 2016 Datacenter (SPLA the license).
There are plenty of examples out there.
They usually use direct-connect 10GbE RDMA via RoCE (Mellanox) or can use 25GbE direct-connect if there's a throughput need. Cost wise, they are not that expensive.
The solution can scale-out from there into a pair of switches and more nodes to add both compute and storage into the mix.
The technology that makes it all work is called Storage Spaces Direct and it's built-in to Windows Server 2016 Datacenter (SPLA the license).
ASKER
Good input, folks - thank you. One more follow-up, pertaining to the selection of switches for these 10-Gig storage links.
We're a Cisco shop, and while Cisco will always lead with their "enterprise-class" products (Catalyst for campus, Nexus for DC, etc.), In a multi-node environment where we'd need to have 10-Gig switching for storage/federation traffic between nodes, could we get by with something much less expensive (or providing more of limited port density required)? It's my understanding that HC nodes (HyperFlex, SimpliVity, etc.) are extremely efficient in their replication traffic, so would there be any risk in using fairly low-cost switching to handle this?
As an example, I could position a 12-port 10-Gbps/SFP+ Catalyst 3850, or on the opposite side of the spectrum I could use something from the Cisco "Small Business Suite", such as a SG550X-24. The latter would fit the port speeds/feeds requirement, and based on datasheets, its performance capacity, memory/table sizes, etc. seem to be somewhat equal to the Catalyst family. This switch would come in at 5-6x lower cost than the 3850 12-port SFP. I'm asking if there are any reasons NOT to recommend those for such a specific use case, where it's likely the 10-Gig links would be very underutilized and thus not subject to buffering issues, overruns, etc.
Thanks again
We're a Cisco shop, and while Cisco will always lead with their "enterprise-class" products (Catalyst for campus, Nexus for DC, etc.), In a multi-node environment where we'd need to have 10-Gig switching for storage/federation traffic between nodes, could we get by with something much less expensive (or providing more of limited port density required)? It's my understanding that HC nodes (HyperFlex, SimpliVity, etc.) are extremely efficient in their replication traffic, so would there be any risk in using fairly low-cost switching to handle this?
As an example, I could position a 12-port 10-Gbps/SFP+ Catalyst 3850, or on the opposite side of the spectrum I could use something from the Cisco "Small Business Suite", such as a SG550X-24. The latter would fit the port speeds/feeds requirement, and based on datasheets, its performance capacity, memory/table sizes, etc. seem to be somewhat equal to the Catalyst family. This switch would come in at 5-6x lower cost than the 3850 12-port SFP. I'm asking if there are any reasons NOT to recommend those for such a specific use case, where it's likely the 10-Gig links would be very underutilized and thus not subject to buffering issues, overruns, etc.
Thanks again
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
We deploy SG500x series switches for all endpoints. Their Small Business Pro 10GbE has enough RJ45 ports for small setups and also TOR duties with the SFP ports.
For larger deployments check out the NETGEAR XS716T for a reasonable cost 10GbE switch. Ubiquiti also has one but we've seen some firmware issues with their switches.
If RDMA needs to be in the mix then the Mellanox MSX1012X 10GbE switch is a good place to start.
For larger deployments check out the NETGEAR XS716T for a reasonable cost 10GbE switch. Ubiquiti also has one but we've seen some firmware issues with their switches.
If RDMA needs to be in the mix then the Mellanox MSX1012X 10GbE switch is a good place to start.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
@all
I agree that the "non-DC" options I've mentioned up to now (SG550, Catalyst options, etc.) wouldn't be positioned for primary data center switches due to all of the reasons already mentioned. The specific use case here is traffic between hyper-converged nodes (again, SimpliVity in this instance), and I've had a couple folk pretty familiar with that solution (but not the networking piece) talk to me about how uber-efficient the storage/replication traffic is.
I agree that the "non-DC" options I've mentioned up to now (SG550, Catalyst options, etc.) wouldn't be positioned for primary data center switches due to all of the reasons already mentioned. The specific use case here is traffic between hyper-converged nodes (again, SimpliVity in this instance), and I've had a couple folk pretty familiar with that solution (but not the networking piece) talk to me about how uber-efficient the storage/replication traffic is.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks everyone for your input on this one.
ASKER
@Aaron Tomosky - cost. This is a fairly small deployment, they don't have Enterprise vSphere (or even vCenter), and the entry point to NSX (as well as the services/effort to turn it up) would be out of reach.
Thanks again