Wan and Ethernet routing help

cb_it
cb_it used Ask the Experts™
on
We have 5 offices. A main office and 4 branch offices. We are all conected via a MPLS Wan provided by Verizon, running BGP. Last year we added a Ethernet line from branch A to the main office. Router at the main office and at the branch have a static route pointing to each other. Problem is the other branches see that static route and are sending traffic to the main office via the Ethernet line, and not the wan. Ethernet line goes down, everything goes down basically. Ethernet line is supposed to be used for disaster recovery, replication of servers from main office to branch A, not "regular" day-to-day traffic.

Any way to set this up so that only certain traffic goes through the Ethernet line?
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Setup a floating static route. You are having two paths to the same network. The route corresponding to the ethernet line might be having a lower metric value than the WAN.

Setup the route corresponding to the ethernet link with a higher metric value than the route corresponding to the WAN. So packets would be forwarded through the link with the lower metric value.

Author

Commented:
I agree with this. But would this mean that my replication traffic going from the main office to branch A would also use the wan, and not the Ethernet line. Ethernet line would kick in if the wan goes down?? I'm not a router guy at all, but is there a way so that certain servers can send traffic using the Ethernet line?

A separate side question. I have a basic ping tool to let me know when routers or servers go down, or the wan is down. Any specific way to monitor both the wan and Ethernet lines. If I add in the static route with a higher metric and the wan goes down how would I ever know?
I should think so.
The issue would seem to be.  Presumably each site has its own subnet, yes?
So, how are the interfaces being done to the Ethernet link and the MPLS links?
Typically there'd be a router for an interface at each site.
Typically there'd be routes for the MPLS that gets packets to the right destination.
And, now, there'd be routes just between the two sites using the Ethernet link.

Oh!  But now you have 2 links between those 2 sites?  One MPLS and one Ethernet?
Why would you use MPLS when you have Ethernet anyway?  I probably wouldn't do that.
If you have both implemented and active then you have to have some mechanism to use one or another.

I'm puzzled why ANY traffic between the other sites would go through the Ethernet link.  That seems a mistake in routing.  

I will assume that there are two routers at each end.  One for Ethernet and one for MPLS.
Router inside/outside addresses for Ethernet:
192.168.1.201 / 10.0.1.201
192.168.2.202 / 10.0.1.202
Router inside/outside addresses for MPLS:
192.168.1.251 / 10.1.1.201
192.168.2.252 / 10.1.1.202

Presumably, the default (MPLS) routes would have been (in shorthand involving 2 routers each):
At site "1":
Destination 192.168.2.xxx >192.168.1.251>10.1.1.202>192.168.2.0 the subnet
At site "2":
Destination 192.168.1.xxx >192.168.2.252>10.1.1.201>192.168.1.0 the subnet
...and then add as many site subnets similarly
So, this provides a route between the sites that also have the Ethernet link, right?

Now, if you add the Ethernet there will be a conflict in routing because you have to point the Ethernet like this:

At site "1":
Destination 192.168.2.0 >192.168.1.201>10.0.1.202>192.168.2.0
At site "2":
Destination 192.168.1.0 >192.168.2.2092>10.0.1.201>192.168.1.0

So, between those two sites, unless the metric is set differently perhaps, there wouldn't be two routes to the same subnet.

You might set particular routes though which because they are more specific would be used.  Say you want to communicate between two servers over Ethernet.
Server addresses:
192.168.1.91
192.168.2.92
At site "1":
Destination 192.168.2.92 >192.168.1.201>10.0.1.202>192.168.2.92
At site "2":
Destination 192.168.1.91 >192.168.2.202>10.0.1.201>192.168.1.91

This should take precedence over the more general 192.168.2.0 subnet range in each case.

I still don't get why inter-site comm's that aren't between sites "1" and "2", say "3" and "4" would hit the Ethernet link.  So there's something strange there.

What's the router setup / topology?
Exploring SQL Server 2016: Fundamentals

Learn the fundamentals of Microsoft SQL Server, a relational database management system that stores and retrieves data when requested by other software applications.

Author

Commented:
Each site has it's own subnet.
1 router at each site. Main office and branch a routers both have additional ethernet wic.
Yes, there is 2 links now between main office and branch A, MPLS wan and Ethernet.
Static route setup on branch A router. Points traffic to main office.
Someone told me that branch A router is advertising this static route to other branches.
Branch B wants to send data to main office and sends it to branch A, then off to main office. Routes at branch A below. Anything destined for the main office (.156) will go through the Ethernet interface.

S       10.100.156.0/24 [1/0] via 172.16.1.2

B       10.100.153.0/24 [20/0] via 152.176.74.100
B       10.100.154.0/24 [20/0] via 152.176.74.100
B       10.100.157.0/24 [20/0] via 152.176.74.100
B       10.100.158.0/25 [20/0] via 152.176.74.100

Author

Commented:
To add some more info that maybe wasn't too clear.

I'd like branch A (.155) to use the Ethernet as it's primary link to the main office, and the other branch offices to use the wan as it's primary link to the main office.

If the wan goes down, then the branches would switch over to the Ethernet line. If the Ethernet goes down then branch A could use the wan. Not sure if this is even possible.

I've read about distance, metrics, floating static routes, and more recently IP SLA that will ping each route and switch them if one goes down. I'm not sure what to do, thanks.
Well, it appears that you have something like RIP turned on.  In such a simple case like this I'd not think that you'd need it.  And, you surely don't want it because you want more control over the routes.

Author

Commented:
How do I know if RIP is turned on?
I didn't notice the MPLS router model number.  So, I'm not sure where you might look.  

And, rather than RIP, maybe it's part of your BGP implementation.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial