Building a redundant network.

modsiw used Ask the Experts™
My small company needs to scale up, and with that, our SLA is also going up. I'm in a bit over my head on building a fully redundant network. (there must be no single point of failure). Load balancing is not a major concern as we do not have that much traffic. This is all about trying to acheive 99.999% or 99.9999% up time.

We have to connections to the world coming in. One from a Teir1 provider and another form a Tier2.
We have a dozen hosts behind this network, each host has dual nic.

We will be setting up a mirror location, geographically far apart, within six months.

I imagine that I will need two Border Gateway Protocol boxes, one for each of our connections? Beyond that, I really don't have much of an idea. Any guidance in the right direct will be greatly appreciated.
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
What specific network layer are you talking about?  99.9999 is 31 seconds of down time in a year.  You cannot do this without a rather huge amount of cash...  The whole path from server to Internet will need to be looked at and your developers better be real good.  If this is really your goal hire some expensive consultants.  But what specific area of the network are you looking at, only Internet connection?


6 9's is a bit much, 5 is more realistic. We are also allowed 1hr of downtime / week per schedule. We're in the process of sorting out our SLA at the moment; I believe the 6 9's are probably just there to scare us. That doesn't stop the business people from barking down the orders though :(

We're going to try to do this in-house if reasonably possible, but we are considering hiring the data center to put everything together. The idea being, that as we scale the TCO stays low.

We've had issues with broken lines and poor routing from our current provider. When this happens mid-day, it costs us dearly. We're looking to overcome this weakness. We are already looking to replace this provider, but we may just be trading apples.

Also, in our current setup, we have single points of failure at our firewall and our switch. We'd like to get rid of this issue also.

Another note:
We like to go with openBSD boxes instead of cisco as that is where our current knowledge base is.


The idea being, that as we scale the TCO stays low if you don't have to hire consultants constantly*
Success in ‘20 With a Profitable Pricing Strategy

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

Ok 5 9s is still expensive and will have to work the layers.  Question: it appears you are mixing apples and oranges.  openBSD is a server platform, cisco is a network specific platform.  Why do you want to replace openBSD for Cisco?  I have never heard of a openBSD box acting as a switch or border router.


I'd rather stay in a nix environment because I'm comfortable with it. I choose openBSD in particular because of  .

I haven't had to many opportunities to use cisco equip, so, if possible I would prefer to avoid it here.

Perhaps I should restructure my question. Currently, we have several hosts connected to a switch, which is in turned connected to a linux box managing our vlan, inturn connected to a single teir 1 provider.
We're aiming at redundant everything from the hosts up in this project. What is the best way to achieve this?

Is there anything aside from BGP capable of making use of both uplinks concurrently?
If you have a specifc set of publci IP addresses that you want accessed over the Internet then the answer is no.  BGP is the standard and will need to be used to access your IPs from two different ISPs.


What would be the ideal topology to connect my vlan to the BGP box(es) while avoiding single points of failure?


I'm turning in for the night. Than you for your answers thus far, and I look forward to continuing this tomorrow.
Pretty much you're looking to double up, but I must say I'd recommend Cisco due to the fact that there is more to go wrong on a BSD box. I know of some people using QUAGGA on BSD boxes in place of Cisco's but they eventually move to Cisco.

Drawing attached


Achieving a highly reliable and available network is much about the design and the management policies you wrap around the network/infrastructure. You can design a network with no single points of failure, however, in the end if you do not have a trained and committed staff armed with standard operating procedures that define how to manage this network you will never achieve anywhere near 3/4/5 - 9s.  The network design is the easy part. 99.96 is approx 3.5 hours downtime a year !. 3 - 9s is approx 9 hours and 2-9s is 88 hours of downtime. There is a science behind achieving multple 9s, 24x7 staffing, sparring strategies, physical path diversity for connectivity as well as electrical sources, backup power, cooling, physical security, virtual security, the list goes on ......

Depending on your business requirements I would try to achieve 3-9s, this still includes no single points of failure and all the technology features to make you available as much as possible. However, a 3-9s commitment allows you to develop all of the necessary policies and procedures to achieve future higher goals.


Thanks. Here is a little about the rest of our setup. Please let me know if anything looks off, or if you just have general comments.

Our equip is in a locked racked, which only 3 of our employs know the combo to. That rack is behind a biometric scanner which only datacenter staff and 8 other customers (that operate the other 8 racks in that area) can access. There is another layer preventing access to the data part of the data center, the data center building, and the data center property. There are no less than 2 armed guards at any given moment at the datacenter. This isnt perfect, but it's about the best we can reasonably do.

Power runs into our rack from two grids. Each grid has inline batteries for continuous power in the event of a disruption. Generators are on site for each grid. There is enough fuel to run them for two days. We have enough battery backup in our rack to run our servers for 30mins under a complete power failure (though, i'm sure in such a catastrophe, the datacenter it's self would be shut down.). Each device except the switches has redundant PSU, with a PSU plugged into each grid.

The datacenter is cooled through several independent HVAC? (look like really big regular air conditioners) systems. There is also a newer cooling system that runs massive amounts of water to cool radiators; it doesn't seem to really do that much cooling when you stand next to one, but they are there none the less.

Virtual security is probably our weakest point with our in house code. We're improving this now and will eventually have a third party take a look.

We are about an hour away from our data center; however, we have a support contract with the datacenter to respond within 15minuits. They also take a daily look at our equip to check for warning lights and such.

No matter how many 9s we promise, we will be allowed to take the system offline for 1hr if we provided sufficient notice, no more than once per week, should be we need to maintain something that we can't do while online.


What would be the minimum cisco equip for this? 5mbit transfer, bursting to 20mbit.

Are special switches required for the bonding?
Switches: No special switches are needed, you are just going to create a port channel.
Router: For Internet routers running BGP, recommend 7200 series router with at least 512 Mb mem
Did you get your question answered?



Sorry for disappearing.

I've ordered a pair of dell servers and have been reading up on vyatta.

I'll go ahead and close this out, though I'm sure I'll have more specific questions once I get to start testing.
Have fun building.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial