We help IT Professionals succeed at work.

Avoiding redundant rules for source based routing.

We use imagestream routers. We have two bandwidth circuits. One is for business customers, one is for residential customers. The resi customers all go out the default route. we have a table 100 for the business customers that sends their traffic out the business circuit with a source rule.

We had alot of issues with communications between the business and residential customers, as well as the business to business customers, however the residential customers could communicate fine amongst themselves.

We were seeing alot of loops.

It turned out that we had routes TO the business customers listed in the default ruleset. So the outside world could communicate with them no problem, but the internal routing wasnt happening.

We ended up duplicating the routes to the business customers instide the table 100 rules, and that got them communicating just fine amongst each other. However we still had loop between residential and business, which required duplicating all the routes to the residential customers inside of table 100.

Without static routes like this the traffic would loop between us and our upstream.

Now the setup we have, with both rulesets duplicated works, but seems labor intensive to mainatain and rather cumbersome.

There has to be a more appropriate solution (non dynamic) for the syntax in the rulesets here.

I hope I made myself clear.
Comment
Watch Question

Steve JenningsSr Manager Cloud Networking Ops

Commented:
Post what you have. Too difficult to understand what's going on without seeing your rules.

Good luck,
SteveJ

Author

Commented:
x.x IPs are the public subnets we have. x.x.229.0/24 is the primary business subnet that goes out our upstream gateway of 10.26.254.1 (I believe this is now routed to us from them as an entire class c via our 10.26.254.2)
y.y IPs are on an internal subnet for our router communication, everything is routed via these IPs internally. Each of the interior y.y routers has a NAT rule on our residential circuit for or residential NAT customers.
z.z IPs are the upstream assigned routing IPs z.z.88.1 is our residential bandwidth provider gateway x.x.228.0/24 is routed in this circuit.

when we are in failover for the residential circuit out the buisiness circuit, all residential users are NAT behind a single IP. Currently, we have no business failover to the residential bandwidth, but then again that circuit rarely has any issue.

The 10.x.x.x  subnets are on the routers out on our network, some are NATs, and some are strictly internally routed management IPs for CPE devices.


We use table 100 for the source based routes (business) as you can see, we had to duplicate the default (residential) entries into table 100 to get the communication to happen between the customers we also had to add routes to all of the business IPs

before we did this, all traffic from a business customer to a resi customer would loop between our core router and our business upstream

we also had to duplicate our business (table 100) entries into the default (residential) list, to get residential to communicate to the business circuit.

Its almost like there are two separate routers

we are still having some issues with subnet communication. I dont know if my lack of experience has found a workaround that causes long term harm, because this seems overly complicated and dangerously large.

I have to stress that right now, dynamic routing is not an option. I am too inexperienced to safely implement something I dont control.

!
version 2.00
!
interface Ethernet0
 description BUSINESS Gateway Interface
 ip address 10.26.254.2 255.255.255.0
 ip address 10.26.254.3 255.255.255.0 secondary
 ip address 10.26.254.4 255.255.255.0 secondary
!
interface Ethernet1
 description Residential Gateway Interface
 ip address z.z.88.3 255.255.255.248
 ip address z.z.88.2 255.255.255.248 secondary
 ip address x.x.228.5 255.255.255.192 secondary
 ip address x.x.228.6 255.255.255.192 secondary
 ip address x.x.228.7 255.255.255.192 secondary
 ip address x.x.228.8 255.255.255.192 secondary
 ip address x.x.228.9 255.255.255.192 secondary
 ip address x.x.228.10 255.255.255.192 secondary
 ip address x.x.228.11 255.255.255.192 secondary
 ip address x.x.228.12 255.255.255.192 secondary
 ip address x.x.228.13 255.255.255.192 secondary
 ip address x.x.228.14 255.255.255.192 secondary
 ip address x.x.228.15 255.255.255.192 secondary
 ip address x.x.228.16 255.255.255.192 secondary
 ip address x.x.228.17 255.255.255.192 secondary
 ip address x.x.228.18 255.255.255.192 secondary
 ip address x.x.228.19 255.255.255.192 secondary
 ip address x.x.228.20 255.255.255.192 secondary
 ip address x.x.228.21 255.255.255.192 secondary
 ip address x.x.228.22 255.255.255.192 secondary
 ip address x.x.228.23 255.255.255.192 secondary
 ip address x.x.228.24 255.255.255.192 secondary
 ip address x.x.228.25 255.255.255.192 secondary
 ip address x.x.228.26 255.255.255.192 secondary
 ip address x.x.228.26 255.255.255.192 secondary
 ip address x.x.228.27 255.255.255.192 secondary
 ip address x.x.228.28 255.255.255.192 secondary
 ip address x.x.228.29 255.255.255.192 secondary
 ip address x.x.228.30 255.255.255.192 secondary
!
interface Ethernet2
!
interface Ethernet3
 description Primary LAN interface facing the wireless cloud
 ip address y.y.0.1 255.255.255.0
 ip address x.x.229.25 255.255.255.252 secondary 

#########
# Routes#
#########

# Default Route through Residential REM OUT when Residential is DOWN
 ip route add default via z.z.88.1
##### Residential Failover Route, REM OUT when Residential is UP #####
# ip route add default via 10.26.254.1


# Define Interface subnets
 ip route add 10.26.254.0/24 dev eth0
 ip route add 65.163.88.0/29 dev eth1
 ip route add x.x.228.0/26 dev eth1
# ip route add 10.26.253.0/24 dev eth2
 ip route add y.y.0.0/24 dev eth3
 ip route add x.x.229.24/30 dev eth3

# Policy Routes for BUSINESS netblocks
# From Rules
 ip rule add from x.x.230.0/24 table 100
 ip rule add from x.x.231.0/24 table 100
 ip rule add from x.x.229.0/24 table 100

# Define routes
 ip route add x.x.228.0/26 dev eth1 table 100
# ip route add x.x.229.24/30 dev eth3 table 100
# ip route add x.x.229.24/30 via x.x.229.25 table 100
 ip route add y.y.0.0/24 dev eth3 table 100
# ip route add 10.26.253.0/24 dev eth2 table 100
 ip route add 10.26.254.0/24 dev eth0 table 100
# ip route add x.x.229.0/29 via 10.26.253.2 table 100
 ip route add x.x.230.0/24 via y.y.0.4 table 100
 ip route add x.x.231.0/24 via y.y.0.4 table 100
# Forwarded Static Customers
 ip route add x.x.228.64/29 via y.y.0.7 table 100
 ip route add x.x.228.128/25 via y.y.0.4 table 100
 ip route add x.x.229.120/29 via y.y.0.4 table 100
 ip route add x.x.229.88/29 via y.y.0.4 table 100
 ip route add x.x.229.80/29 via y.y.0.27 table 100
 ip route add x.x.229.72/29 via y.y.0.27 table 100
# ip route add x.x.229.128/25 via y.y.0.4 table 100
 ip route add x.x.229.128/30 via y.y.0.4 table 100
 ip route add x.x.229.132/30 via y.y.0.4 table 100
 ip route add x.x.229.136/30 via y.y.0.4 table 100
 ip route add x.x.229.140/30 via y.y.0.4 table 100
 ip route add x.x.229.144/30 via y.y.0.4 table 100
 ip route add x.x.229.148/30 via y.y.0.4 table 100
 ip route add x.x.229.152/30 via y.y.0.4 table 100
 ip route add x.x.229.156/30 via y.y.0.4 table 100
 ip route add x.x.229.160/30 via y.y.0.4 table 100
 ip route add x.x.229.164/30 via y.y.0.4 table 100
 ip route add x.x.229.168/30 via y.y.0.4 table 100
 ip route add x.x.229.172/30 via y.y.0.4 table 100
 ip route add x.x.229.176/30 via y.y.0.4 table 100
 ip route add x.x.229.180/30 via y.y.0.4 table 100
 ip route add x.x.229.184/30 via y.y.0.4 table 100
 ip route add x.x.229.188/30 via y.y.0.27 table 100
 ip route add x.x.229.192/30 via y.y.0.4 table 100
 ip route add x.x.229.196/30 via y.y.0.4 table 100
 ip route add x.x.229.200/30 via y.y.0.4 table 100
 ip route add x.x.229.204/30 via y.y.0.4 table 100
 ip route add x.x.229.208/30 via y.y.0.4 table 100
 ip route add x.x.229.212/30 via y.y.0.4 table 100
 ip route add x.x.229.216/30 via y.y.0.4 table 100
 ip route add x.x.229.220/30 via y.y.0.4 table 100
 ip route add x.x.229.224/30 via y.y.0.4 table 100
 ip route add x.x.229.228/30 via y.y.0.4 table 100
 ip route add x.x.229.232/30 via y.y.0.4 table 100
 ip route add x.x.229.236/30 via y.y.0.25 table 100
 ip route add x.x.229.240/30 via y.y.0.4 table 100
 ip route add x.x.229.244/30 via y.y.0.4 table 100
 ip route add x.x.229.248/30 via y.y.0.7 table 100
 ip route add x.x.229.252/30 via y.y.0.4 table 100

 ip route add x.x.229.96/29 via y.y.0.4 table 100
 ip route add x.x.229.104/29 via y.y.0.4 table 100
 ip route add x.x.229.112/29 via y.y.0.4 table 100
 ip route add x.x.229.64/29 via y.y.0.4 table 100
 ip route add x.x.228.168/29 via y.y.0.4 table 100
 ip route add x.x.228.72/29 via y.y.0.4 table 100
 ip route add x.x.228.96/29 via y.y.0.4 table 100
 ip route add x.x.228.136/29 via y.y.0.16 table 100
 ip route add x.x.228.192/29 via y.y.0.25 table 100
 ip route add x.x.228.200/29 via y.y.0.4 table 100
 ip route add x.x.228.208/29 via y.y.0.6 table 100
 ip route add x.x.228.248/29 via y.y.0.4 table 100
 ip route add x.x.228.120/29 via y.y.0.14 table 100
# Default route
 ip route add table 100 default via 10.26.254.1

# Forward subnets to Rebel BMU
 ip route add x.x.230.0/24 via y.y.0.4
 ip route add x.x.231.0/24 via y.y.0.4

# Forwarded 228/29 subnets to BMUs
 ip route add x.x.228.64/29 via y.y.0.7
 ip route add x.x.228.72/29 via y.y.0.4
 ip route add x.x.228.80/29 via y.y.0.4
 ip route add x.x.228.88/29 via y.y.0.4
 ip route add x.x.228.96/29 via y.y.0.4
 ip route add x.x.228.104/29 via y.y.0.4
 ip route add x.x.228.112/29 via y.y.0.4
 ip route add x.x.228.120/29 via y.y.0.14
 ip route add x.x.228.128/29 via y.y.0.4
 ip route add x.x.228.136/29 via y.y.0.16
 ip route add x.x.228.144/29 via y.y.0.4
 ip route add x.x.228.152/29 via y.y.0.4
 ip route add x.x.228.160/29 via y.y.0.4
 ip route add x.x.228.168/29 via y.y.0.4
 ip route add x.x.228.176/29 via y.y.0.4
 ip route add x.x.228.184/29 via y.y.0.4
 ip route add x.x.228.192/29 via y.y.0.25
 ip route add x.x.228.200/29 via y.y.0.4
 ip route add x.x.228.208/29 via y.y.0.27
 ip route add x.x.228.216/29 via y.y.0.28
 ip route add x.x.228.248/29 via y.y.0.4

# Forward x.x.229.128/25 to Envoy1 BMU - SOME TO BMU1
# ip route add x.x.229.128/25 via y.y.0.4
 ip route add x.x.229.128/30 via y.y.0.4
 ip route add x.x.229.132/30 via y.y.0.4
 ip route add x.x.229.136/30 via y.y.0.4
 ip route add x.x.229.140/30 via y.y.0.4
 ip route add x.x.229.144/30 via y.y.0.4
 ip route add x.x.229.148/30 via y.y.0.4
 ip route add x.x.229.152/30 via y.y.0.4
 ip route add x.x.229.156/30 via y.y.0.4
 ip route add x.x.229.160/30 via y.y.0.4
 ip route add x.x.229.164/30 via y.y.0.4
 ip route add x.x.229.168/30 via y.y.0.4
 ip route add x.x.229.172/30 via y.y.0.4
 ip route add x.x.229.176/30 via y.y.0.4
 ip route add x.x.229.180/30 via y.y.0.4
 ip route add x.x.229.184/30 via y.y.0.4
 ip route add x.x.229.192/30 via y.y.0.4
 ip route add x.x.229.196/30 via y.y.0.4
 ip route add x.x.229.200/30 via y.y.0.4
 ip route add x.x.229.204/30 via y.y.0.4
 ip route add x.x.229.208/30 via y.y.0.4
 ip route add x.x.229.212/30 via y.y.0.4
 ip route add x.x.229.216/30 via y.y.0.4
 ip route add x.x.229.220/30 via y.y.0.4
 ip route add x.x.229.224/30 via y.y.0.4
 ip route add x.x.229.228/30 via y.y.0.4
 ip route add x.x.229.232/30 via y.y.0.4
 ip route add x.x.229.236/30 via y.y.0.25
 ip route add x.x.229.240/30 via y.y.0.4
 ip route add x.x.229.244/30 via y.y.0.4
 ip route add x.x.229.248/30 via y.y.0.7
 ip route add x.x.229.252/30 via y.y.0.4

# Business Route
 ip route add x.x.229.96/29 via y.y.0.4
# Business Route
 ip route add x.x.229.104/29 via y.y.0.4
 ip route add x.x.229.112/29 via y.y.0.4
# Business Route
 ip route add x.x.229.120/29 via y.y.0.4
# Business Route
 ip route add x.x.229.88/29 via y.y.0.4
# Business Route Business Route
 ip route add x.x.229.80/29 via y.y.0.27
# Business Route
 ip route add x.x.229.72/29 via y.y.0.27
# Unassigned
 ip route add x.x.229.64/29 via y.y.0.4
# Business Route
# ip route add x.x.229.220/30 via y.y.0.4
# Business Route
# ip route add x.x.229.216/30 via y.y.0.4
# Business Route
 ip route add x.x.229.188/30 via y.y.0.27

## Routes to BMU Subnets ## 

ip route add 10.3.2.0/24 via y.y.0.4	
ip route add 10.3.14.0/24 via y.y.0.4	
ip route add 10.3.17.0/24 via y.y.0.25	
ip route add 10.0.52.0/24 via y.y.0.27	
ip route add 10.2.52.0/24 via y.y.0.27	
ip route add 10.3.19.0/24 via y.y.0.27	
ip route add 10.0.2.0/24 via y.y.0.4	
ip route add 10.5.0.0/24 via y.y.0.4	
ip route add 10.0.53.0/24 via y.y.0.28	
ip route add 10.2.53.0/24 via y.y.0.28	
ip route add 10.3.20.0/24 via y.y.0.28	
ip route add 10.3.1.0/24 via y.y.0.7	
ip route add 10.3.10.0/24 via y.y.0.16	
ip route add 10.0.39.0/24 via y.y.0.16	
ip route add 10.2.39.0/24 via y.y.0.16	
ip route add 10.0.29.0/24 via y.y.0.16	
ip route add 10.2.29.0/24 via y.y.0.16	
ip route add 10.0.33.0/24 via y.y.0.4	
ip route add 10.2.33.0/24 via y.y.0.4	
ip route add 10.0.38.0/24 via y.y.0.4	
ip route add 10.0.35.0/24 via y.y.0.4	
ip route add 10.2.38.0/24 via y.y.0.4	
ip route add 10.2.35.0/24 via y.y.0.4	
ip route add 10.0.42.0/24 via y.y.0.4	
ip route add 10.2.42.0/24 via y.y.0.4	
ip route add 10.0.90.0/24 via y.y.0.4	
ip route add 10.0.41.0/24 via y.y.0.4	
ip route add 10.2.41.0/24 via y.y.0.4	
ip route add 10.0.47.0/24 via y.y.0.4	
ip route add 10.2.47.0/24 via y.y.0.4	
ip route add 10.0.31.0/24 via y.y.0.4	
ip route add 10.2.31.0/24 via y.y.0.4	
ip route add 10.0.46.0/24 via y.y.0.4	
ip route add 10.2.46.0/24 via y.y.0.4	
ip route add 10.0.30.0/24 via y.y.0.4	
ip route add 10.2.30.0/24 via y.y.0.4	
ip route add 10.0.44.0/24 via y.y.0.4	
ip route add 10.2.44.0/24 via y.y.0.4	
ip route add 10.0.34.0/24 via y.y.0.4	
ip route add 10.2.34.0/24 via y.y.0.4	
ip route add 10.0.32.0/24 via y.y.0.4	
ip route add 10.2.32.0/24 via y.y.0.4	
ip route add 10.0.40.0/24 via y.y.0.4	
ip route add 10.2.40.0/24 via y.y.0.4	
ip route add 10.0.36.0/24 via y.y.0.4	
ip route add 10.2.36.0/24 via y.y.0.4	
ip route add 10.0.48.0/24 via y.y.0.4	
ip route add 10.2.48.0/24 via y.y.0.4	
ip route add 10.0.50.0/24 via y.y.0.25	
ip route add 10.2.50.0/24 via y.y.0.25	
ip route add 10.0.51.0/24 via y.y.0.4	
ip route add 10.2.51.0/24 via y.y.0.4	
ip route add 10.0.49.0/24 via y.y.0.4	
ip route add 10.2.49.0/24 via y.y.0.4	
ip route add 10.2.37.0/24 via y.y.0.14	
ip route add 10.0.37.0/24 via y.y.0.14	
ip route add 10.0.45.0/24 via y.y.0.7	
ip route add 10.3.21.0/24 via y.y.0.4	
ip route add 10.3.5.0/24 via y.y.0.4	
ip route add 10.2.55.0/24 via y.y.0.30
ip route add 10.0.55.0/24 via y.y.0.30
ip route add 10.3.22.0/24 via y.y.0.30
ip route add 10.3.8.0/24 via y.y.0.14
!
end

Open in new window

Author

Commented:
A side request here would be proper router formatting. I like to follow established standards. Im talking about where best to place entries, where to have spaces, appropriate use of #s,. I like to be thorough, but clean, clear and concise (with the exception of requests for help)

Author

Commented:
anyone?
Steve JenningsSr Manager Cloud Networking Ops

Commented:
Sorry . . . on vacation and wife gets cranky when I get on this site from the hotel. Back in the grind next week.

Good luck,
SteveJ

Author

Commented:
thanks tell wife sorry

Author

Commented:
bump
Commented:
Found a solution at another forum. Need to provide a source route to the subnets from 0/0 to subnet via router, this eliminates the need to redundantly state the routes in every table.