Solved

Cisco Route Maps, IP SLA and Failover

Posted on 2013-06-30
19
1,190 Views
Last Modified: 2014-04-26
Hi there,

I have a Cisco 2911 and 2 broadband connections entering my LAB (1 BT and 1 Virgin).  Both WAN connections connect to the 2911 and work independantly.  Schematic attached.

What I'm trying to do now is use each WAN connection for different purposes.  The LAN addressing scheme is 192.168.7.0/24 and I have some statically assigned LAN IPs (servers) in the following ranges:
192.168.7.1 - 91
192.168.7.192 - 254
I want the servers to use the BT broadband connection, and all other devices to use the VIRGIN connection.  But in the even of failure of either circuit I'd like all traffic to use the remaining circuit.

I've setup an ACL to seperate out the static IPs (STATIC_DEVICE_RANGE) and have had a go at setting up route maps as follows:

!
track 10 ip sla 1 reachability
 delay down 1 up 1
!
track 20 ip sla 2 reachability
 delay down 1 up 1
!
route-map PBR permit 10
 match ip address STATIC_DEVICE_RANGE
 set ip next-hop verify-availability 81.134.112.1 1 track 10
!
route-map PBR permit 30
 match ip address INTERNAL_ADDRESSES
 set ip next-hop verify-availability 92.239.102.1 2 track 20
!
ip sla 1
 icmp-echo 81.134.112.1
 threshold 100
 timeout 500
 frequency 1
ip sla schedule 1 life forever start-time now
!
ip sla 2
 icmp-echo 92.239.102.1
 threshold 100
 timeout 500
 frequency 1
ip sla schedule 2 life forever start-time now
!
ip route 0.0.0.0 0.0.0.0 81.134.112.1 track 10
ip route 0.0.0.0 0.0.0.0 92.239.102.1 track 20
!
!
!
!
route-map VIRGIN_ROUTE permit 10
 match ip address INTERNAL_ADDRESSES
 match interface GigabitEthernet0/1
!
route-map BT_ROUTE permit 10
 match ip address STATIC_DEVICE_RANGE
 match interface Dialer0
!
ip nat inside source route-map BT_ROUTE interface Dialer0 overload
ip nat inside source route-map VIRGIN_ROUTE interface GigabitEthernet0/1 overload
!
!
!
!

The route maps seem to work.  Both the servers and laptops can access the internet, if I complete a trace route from a server it shows routing via BT, and a trace route from a laptop shows routing via VIRGIN.

Up to this point is great, but what I can't do is get the routes to failover.  If I run continuous pings from a server and disconnect the BT WAN cable from the router the ping will stop (even though the router shows the link and tracking going down).  The same happens for laptops if I disconnect the VIRGIN WAN cable.

*Jun 30 15:26:10.711: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/2, changed state to down
*Jun 30 15:26:11.711: %LINK-3-UPDOWN: Interface GigabitEthernet0/2, changed state to down
ROUTER-1#
*Jun 30 15:26:12.939: %TRACKING-5-STATE: 10 ip sla 1 reachability Up->Down

Interestingly, when I pull either WAN connection, both default routes remain in the routing table.  I'm guessing this is because both default gateways (next hop IPs) are accessible via both interfaces.  I'm guessing that this config is designed for leased lines.

Any ideas on the best way to rectify this?

I'm thinking either an ACL on each WAN interface to stop the BT interface reaching the VIRGIN gateway (and visa-cersa), or somehow modifying the config to make the router understand that it's lost the 'ideal' route to each gateway.

Sanitised config attached.

Cheers, Andy
Router-1.txt
LAN---WAN-connections.jpg
0
Comment
Question by:andrewprouse
  • 10
  • 7
  • 2
19 Comments
 
LVL 17

Expert Comment

by:surbabu140977
ID: 39288488
Can you try using the "rtr" command please? (IPbase image will not have this, you will need advance services)

rtr 10
type echo protocol ipIcmpEcho  81.134.112.1
timeout 900
frequency 1
rtr schedule 1 life forever start-time now

rtr 20
type echo protocol ipIcmpEcho 92.239.102.1
timeout 900
frequency 1
rtr schedule 2 life forever start-time now

track 101 rtr 10 reachability
track 102 rtr 20 reachability

ip route 0.0.0.0 0.0.0.0 81.134.112.1 track 101
ip route 0.0.0.0 0.0.0.0 92.239.102.1 track 102

The only change in my gns3 was: I was using AD in the static route alongside the track command in a primary/backup scenario.

Best,
0
 

Author Comment

by:andrewprouse
ID: 39289099
I was trying to use the 'rtr' command when I was setting up the policy based routing but as you say, my router doesn't support it. I'm running an 'ipbase' license and a 'security' license.  I also don't have a support contract for this router so getting my hands on the 'data' license would be difficult.

Any alternatives?
0
 

Author Comment

by:andrewprouse
ID: 39297506
Any help would be much appreciated...
0
 
LVL 17

Expert Comment

by:surbabu140977
ID: 39297810
The logical problem happening here is :you want failover but your routes are having the same administrative distance. If you need failover,  basic rule of thumb is administrative distance should vary.

Track is seeing that one guy is going down, but since both are having same AD, nothing happens.

In normal testing, if you give 2 static routes with same AD, then also same behaviour will happen. Router will not divert traffic to the other.

But in that case, your route map won't work. Also, policy routing stating that xxxx host need 92.239.102.1 as gateway won't do since unavailability of that gateway will stop those hosts getting to internet.

There should be a solution. Will try in the lab.

Best,
0
 

Author Comment

by:andrewprouse
ID: 39297816
ok cheers
0
 
LVL 17

Accepted Solution

by:
surbabu140977 earned 500 total points
ID: 39297874
access-list 10 permit <IP addresses> [your permitted IP's goes here]

route-map VICTORY permit 1
match ip address <acl>
set ip next-hop verify-availability <IP address> <Seq> <track number>

interface <name>
ip policy route-map VICTORY

The keyword I missed was the "next-hop veryfy-availability" command. It will only keep on doing policy based routing as long as your Ip is reachable. Then it will stop and your default route will take over.

You can now add the static routes as,

ip route 0.0.0.0 0.0.0.0 81.134.112.1 track 101
ip route 0.0.0.0 0.0.0.0 92.239.102.1 20

I hope you may have got the logic.
0
 

Author Comment

by:andrewprouse
ID: 39302241
Thanks for the info but I don't quite understand.

If I create route-map VICTORY as detailed above then surely route-map PBR will no longer take effect.  Route-map PBR shapes the traffic flow depending upon source address (static devices down BT link, DHCP devices down VIRGIN link), route-map VICTORY doesn't seem to do this.

Also, I'm already using the 'verify-availability' statement within the PBR route-maps.

Regarding static routes, I already have 2 static routes, both with the same metric but with different track numbers.  Is this where I'm going wrong?

Basically, I can't just implement route-map VICROTY as you've suggested otherwise I loose my split route functionality.

Cheers, Andy
0
 
LVL 17

Expert Comment

by:surbabu140977
ID: 39302798
logic is:

data coming from user in router interface---> VICTORY comes into picture--> route map checks acl-->acl says if data is from that source then allows--> Victory allows---> data is thrown to the desired hop address-->END

if one gateway not working....
data coming from user in router interface---> VICTORY comes into picture--> route map checks acl-->acl says if data is from that source then allows--> Victory allows--->BUT next hop is not reachable--->router looks in routing table-->data is thrown out.

Route-map should have 2 permits for 2 destinations with 2 ACL's.

You overlooked the acl part. ACL will define from which source IP data is coming. verify-availability with track number I don't think you are using. Please read each line carefully and try to understand the logic. Hopefully solution will come out.


Best,
0
 

Author Comment

by:andrewprouse
ID: 39303872
Ok, so with a mixture of you recommendations and some more google bashing I'm now in a position where failover and failback is working (yay)! Config attached. The main issues I had was that I had to keep manually entering CLEAR IP NAT TRANS * to get the failover / failback to work, I've now entered this into an event manager applet.

The question I have now is regarding NAT.  As the servers prefer to use the BT line (which is where static NAT (port forwarding) is configured, if the BT line goes down and the servers failover to the VIRGIN line, how do I then re-write the NAT rules in real-time?

Failing over allows the servers to get out to the internet on the VIRGIN line, but wont allow requests to come back in.

Any ideas?
router-config.txt
0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 
LVL 17

Expert Comment

by:surbabu140977
ID: 39304201
We usually run a script (windows or shell) which runs as a cron or batch and keeps pinging the next hop and as soon as it's unreachable, execute the relevant "NO" form of static NAT's and paste the new ones.

Sometimes it won't work for some hops since they will be reachable via our alternate links. In those cases we used to define in the PBR that traffic from that PC should always be diverted through that particular interface only.

You have to manually put back the original config once it's up.

Best,
0
 

Author Comment

by:andrewprouse
ID: 39304280
So there's literally no way to 'automatically' update / re-write the NAT config ?

Seems a bit of a shame that Cisco go to all of the effort of creating SLAs, PBRs & tracking to only fall at the last hurdle.
0
 
LVL 17

Expert Comment

by:surbabu140977
ID: 39304947
Nope, but  TCL option can be explored too. I am not an expert in TCL though. In any case, scripting would be required. So, I always choose the easier ones. What you did in the PBR we used to write shell script and execute.(in place of ip sla)  :)

Best,
0
 

Author Comment

by:andrewprouse
ID: 39309598
Umm, I've just been told that we've got major issues with external workers accessing our internal PPTP VPN server (192.168.7.2).

I've added TCP 1723 statements and have tried adding GRE tratements to the ACLs but have had no joy.

Updated config attached.

Any ideas?
ROUTER-CONF.txt
0
 

Author Comment

by:andrewprouse
ID: 39311808
Ok, so here's a bit more info (I've literally wasted days on this now)...

SLA 1 - BT line (primary for servers - STATIC ACL)
SLA 2 - Virgin line (primary for DHCP devices - DHCP ACL)

PPTP connections from outside to in only work when SLA 2 goes DOWN (strange)
Pings from 192.168.7.4 consistantly work to 4.2.2.2
Pings from 192.168.7.3 consistantly work to 4.2.2.2
Pings from 192.168.7.2 consistantly work to 4.2.2.2
Pings from 192.168.7.1 DON'T work to 4.2.2.2 unless only one SLA is up (doesn't matter which SLA)

Also, when SLA 2 goes down I get the following messages on the router:

Jul  9 18:53:02.430: %IP_VFR-3-COALESCE_ERROR: Unable to coalesce fragments arriving through GigabitEthernet0/0 - datagram 4017 byt
es is too big. UDP packet 192.168.7.2:53 -> 217.172.182.22:25345 has ident: 3471 flags:0x0 and 3 fragments

Please help !!
0
 
LVL 17

Expert Comment

by:surbabu140977
ID: 39313479
IP_VFR-3-COALESCE_ERROR messages are typically due to ios bug..... you need an upgrade or a downgrade of your ios which will not have this bug.......
0
 

Author Comment

by:andrewprouse
ID: 39313505
I'm not so bothered about the bug at the moment, I really need a way to get the route maps, failover & PPTP passthrough working correctly.  I'm also struggling with the concept that in a failed over state server services (PPTP VPN, Exchange, etc) won't be available to external clients due to the NAT rules.

There must be a better way to do this.  I'm actually better reverting to my BT Business Hub and Virgin Superhub running in parrallel if this is the case.
0
 

Author Comment

by:andrewprouse
ID: 39327494
I got that annoyed with this issue that I've had the VIRGIN ISP interface shutdown for the past week.  Having just re-enabled the interface I found out that external DNS lookups were failing from my internal MS DNS server.  Strange because pings from the DNS server to 4.2.2.2 are fine and a tracert shows that it's using the BT WAN link so I'm completely stumped.

As soon as I shut down int Gi0/1 (the virgin link) DNS starts working straight away.
0
 
LVL 2

Expert Comment

by:bbwonders
ID: 39331890
That is actually incorrect, I have three ISP's and they all fail over to each other and i do not require a "cron or shell script". We are talking about Cisco routers, this is not a $19.99 no name brand. As long as your IOS is upto date you can do this.

When the link fails over you can "re-write" the NAT translations. Here is a sample config, use this with your config, should work fine.

!Configure Standard NAT to match all NAT traffic
ip access-list standard NAT_ACL
permit any
!
!Configure Track object for BT internet
track 1 rtr 1 reachability
delay down 1 up 1
!

!Configure IP SLA monitor for BT Internet
ip sla 1
icmp-echo 81.134.112.1 source-interface Dialer0
timeout 5000
threshold 5000
frequency 5
ip sla schedule 1 life forever start-time now
!

!Configure Track object for Virgin internet
track 2 rtr 2 reachability
delay down 1 up 1
!

!Configure IP SLA monitor for Virgin Internet
ip sla 2
icmp-echo 92.239.102.1 source-interface GigabitEthernet0/1
timeout 5000
threshold 5000
frequency 5
ip sla schedule 2 life forever start-time now
!
!Build route map for BT Internet Policy
!
route-map INTERNET_POLICY permit 10
 match ip address STATIC
 set ip next-hop verify-availability 81.134.112.1 1 track 1
 set ip next-hop verify-availability 92.239.102.1 2 track 2
!

!Build route map for BT NAT
!
route-map BT_NAT permit 10
 match ip address NAT_ACL
 match interface Dialer0
!

!Add source NAT for BT internet
ip nat inside source route-map BT_NAT interface Dialer0 overload
!

!Build route map for Virgin internet
route-map INTERNET_POLICY permit 20
match ip address INTERNAL_ADDRESSES
set ip next-hop verify-availability 92.239.102.1 1 track 2
set ip next-hop verify-availability 81.134.112.1 2 track 1
!
!Build NAT mapping for VIRGIN NAT
route-map VIRGIN_NAT permit 10
match ip address NAT_ACL
match interface GigabitEthernet0/1
!

!Add source NAT for VIRGIN Internet
ip nat inside source route-map VIRGIN_NAT interface GigabitEthernet0/1 overload

Open in new window


As long as your ACL statements are correct and you have applied this PBR to the interface, you should have no problems.
0
 
LVL 2

Expert Comment

by:bbwonders
ID: 39331951
If I was you, I would put users and servers in different VLAN's.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Suggested Solutions

The Cisco RV042 router is a popular small network interfacing device that is often used as an internet gateway. Network administrators need to get at the management interface to make settings, change passwords, etc. This access is generally done usi…
In the world of WAN, QoS is a pretty important topic for most, if not all, networks. Some WAN technologies have QoS mechanisms built in, but others, such as some L2 WAN's, don't have QoS control in the provider cloud.
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

759 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now