Multihomed Site Edge - Design Verification

I need help in verifying the legitimacy of a design for a multihomed Internet-facing site.  I have attached a Layer 3 diagram that has four routers with red labels and a fault-tolerant ASA 5515 (two ASAs configured in failover).  I have also attached the relevant configuration for each device.  I need someone to verify that this design will work as expected.

I plan on having two ISPs that I will advertise my public /24 subnet to.  Each ISP will advertise only a default route into my AS.  I want to achieve fault tolerance at all levels and avoid routing loops and "black holes".  I present two optional sections in the configs that pertain to load balancing.  One option is to use the round robin load balancing feature in the ASA to balance between both outbound routes.  The other option, mentioned in the edge configs, uses the weighted GLBP load balancing, allowing for weights, simpler ASA operation, and the use of a single, virtual gateway IP.  I'm liking this design over the ASA load balancer method.  Any opinions on this decision?  I also plan on running iBGP between the edge routers.

If it needs more explaining, let me know.
Who is Participating?
pgstephanConnect With a Mentor Commented:
Hi mate,

1) The BGP neighborship towards your ISP better not use your loopback as source IP address. Because that means your loopback will need to have a public IP address that is routable on the Internet, or even with a static route on your adjacent internet router at the provider. I would just delete the line of "update-source loopback" and just peer with your IP address on your outgoing WAN interface (as the source for the BGP neighborship).
2) You cannot really publish your public /24 range to the 2 internet service provider equally because you will run into an assymetric routing issue. What I would propose is to either run with an active/passive scenario between the 2 internet service providers where you will advertise the /24 to both providers but AS-Prepend on the secondary ISP you chose. The better solution will be to split the /24 into 2 /25 and advertise them to both providers and again play with AS-Prepend to make the 2 prefixes (one primary and the other secondary on each router) and so on.
The problem here will be that you will need to source nat on your routers instead of doing source based routing on your firewalls.
3) Now you need to chose a way forward to automate your failover between the 2 providers:
a) one way is to run an OSPF area behind the internet routers and with the firewall where you will redistribute the default route you receive from the carriers so the firewall will know which one to fail-to when it loose connectivity.
b) the alternative will be to use 2 HSRP groups on the LAN interface of the routers, which will track your BGP neighborship towards the service providers.
4) You need to configure an access-list on your WAN inbound interface on the routers to deny RFC 1918 prefixes + packets sourced from your public /24 range that you're advertising, and if you want to enable uRFP for antispoofing.
What you can also do is have the first line in your access-list to have a pin-hole access-list that allows communication between your service provider WAN ip address and your WAN address (only for TCP 179 (BGP). And deny ALL other traffic destined to your WAN interface to deny any DOS attack to your WAN router.
5) You may want to add a route-map on your BGP inbound to only receive a default-route from your carriers. If for any reason your internet service provider pushed you down the full internet routing table, and your routers may crash if they don't have enough RAM (each BGP peer requires 128MB RAM for the full table).
6) If you enable OSPF or another protocol between the routers and the firewall, you will not need the static route you configured between the peer for the iBGP neighborship.
7) You really need to think about the iBGP function in this scenario. You really may not need it if you will expect traffic to be active on 1 single ISP at any point in time. In this scenario you can use HSRP which will track the BGP neighborhood relationship, and will drop priority of your primary router once the BGP fails.
8) If you want to use load balancing on your outgoing firewall configuration, then you will certainly need to split the 2 /24 range into 2 and do source NATting on your outgoing interfaces according to the route the traffic will take. Otherwise you will be really able to load balance your outgoing traffic but you will certainly run into assymetric routing for the incoming traffic which is not really recommended especially if your clients have dual upstreams and have uRPF enabled.

I'm simulating the configuration I'm proposing and will post it here shortly.
marrjAuthor Commented:
I failed to mention that the two edge routers will be Cisco 2921 ISRs.
marrjAuthor Commented:
Thank you very much, pgstephan.  Here are my responses.

1. Ok, I have taken the loopbacks out of the neighborships with the ISP's.

2. I'm really not a huge fan of running NAT from my edge routers.  I think I will settle for a active/standby topology.  I do like the concept of splitting the /24 into two /25's, but I'm not sure I can sell it to my administration.  I like the idea of leveraging BGP prepend, but I'm reading some reports that claim it may not always work, depending on what your ISP will do with BGP path attributes like local pref.  What is your opinion on conditional advertisement?   I like the concept, but I'm having trouble wrapping my mind around how it would work with two edge routers.

3. Since I've settled for an active/standby config, I'm more than happy leveraging HSRP with tracking for failover.

4. How would such an access list look?  
(Block all traffic to WAN interface except TCP 179)
#access-list 101 permit tcp eq 179
#interface gi0/0
#ip access-group 101 in

5. I can handle a default-route-only route map.  Could I essentially do the same thing with a prefix list?
(Block all inbound BGP-learned prefixes except default route to avoid RAM overload)
#ip prefix-list only-default seq 5 permit
#neighbor prefix-list only-default in

6. Do I still need OSPF since I've settled for an active/standby?  I'd rather not complicate things if I don't have to.

7. Ok.  Makes sense.

8. I think I like the looks of active/standby and simplifying things with HSRP.  My main question at this point is how do I keep my prefix from being advertised to the standby ISP.

Once again, thank you very much for your input.
Improve Your Query Performance Tuning

In this FREE six-day email course, you'll learn from Janis Griffin, Database Performance Evangelist. She'll teach 12 steps that you can use to optimize your queries as much as possible and see measurable results in your work. Get started today!

marrjAuthor Commented:

What do you thing about this guy's setup for two edge routers?
2. You're right AS-Prepend doesn't always work, because it really depends on your upstream providers. Instead of conditional advertisement, what you can also do is advertise to your primary provider 2 /25 blocks and a single /24 block to your secondary provider. That way the /25 will be always preferred over the /25 advertised by the secondary provider.
I had a bit of a read to your link about conditional advertisement, my problem with it is that you will only start advertising the prefix via BGP when you lose your primary connection. That's not quick enough, so I'd prefer the summary way (above).
You can also talk to  your secondary carrier, some carriers can apply a certain policy by which when you advertise them a prefix with a specific community, they would only advertise it when they lose receiving your prefix via the primary provider.
4. Here is what you can use:
access-list 101 permit tcp host host eq 179 log
access-list 101 permit tcp host host eq 179 log
access-list 101 deny ip any host
access-list 101 deny ip any
access-list 101 deny ip any
access-list 101 deny ip any
access-list 101 permit ip any any

In the first couple of lines in the access-list, apply it for a couple of second and see how the BGP session is flowing then delete the redundant line (reason being is that the BGP session between the 2 neighbors may start from your router or your carrier's) (and delete the log from the line too).

5. Yes
6. No
marrjAuthor Commented:
I'm curious now that you mentioned that conditional advertising may not be fast enough.  About how long does it take to propagate a prefix with BGP?

I'm going to research BGP communities now.
marrjAuthor Commented:
I've attached revised configs for both E1 and E2, with changes per your recommendations.  I have decided to change from load sharing to active/standby with E1 and ISP1 being the active route.  I'm going to ask my ISPs about their use of local pref and BGP communities in their AS.  I hope to get by with as-prepend.

Would you mind taking a look at the new configs to make sure I'm still on the right track?

Thank you.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.