Solved

2 Inbound ISPs how to setup some sort of routing

Posted on 2008-06-22
19
780 Views
Last Modified: 2012-05-05
What the setup is:

On the machine their is eth0 (10.10.9.1) which is set up on the router to always go out ISP A (will say the IP is 1.1.1.1).
On the machine their is also eth1 (10.10.9.4) which is set up on the router to always go out ISP B (will say the IP is 2.2.2.2).

If ISP A goes down I want to somehow make the traffic go out ISP B while still keeping eth0 up for local connections (via the 10.0.0.0 network) and when both are up those connections coming into eth0 should always go out eth0 and those coming from eth1 should always go out eth1.

I was looking into fwmark and that may be my solution but need more details.
0
Comment
Question by:starwarp11
  • 8
  • 8
  • 3
19 Comments
 
LVL 39

Expert Comment

by:noci
ID: 21850032
To answer part two of the question first.

Network traffic always leaves (answer on TCP) through the interface the inbound request come in on.
(This also applies to [NATTED]/FORWARDED packets...)
So far so good. UDP might be more difficult you might need to bind to specific interfaces.

If the link to provider A goes down you need to tell the world they need a different address.
If DNS is involved, please use a RR (DNS resource record) with a short timeout, and update your zone file when things change
(remove a reference to an interface that goes down).During the short period that the updates are not spread yet (i.e.  remote systems ask the name again, due to chaed record that were removed when the timeout occurred). connections may fail.

Outgoing traffic might be easier if you have some routing protocol installed, OSPF is a good candidate as it can assign different cost to interfaces that are up or down, convergence of the routing is fast. you might loose one or maybe two  ping packets.

Your system should act with a routing daemon on the OSPF announcements of the router(s). (quagga http://quagga.net/) is a good candidate.
You also might need some extra routes for the nets of the ISP's themselves (in your example: 1.0.0.0/8 to ISP A's router and
2.0.0.0/8 ti ISP B' modem.) As your ISP might not allow certain traffic f.e. only POP from withing the providers network.
0
 
LVL 3

Expert Comment

by:dextermain
ID: 21850070
Hi,

I have a few questions.

1. What tipe of router do you have?
2. Is links to ISP A and B on static IP's?
3. See if you can setup a route on the Router with a metric.

EG on cisco

ip route 0.0.0.0 0.0.0.0 1.1.1.1    (Default gateway )
ip route 0.0.0.0 0.0.0.0 2.2.2.2 metric 100 (Default gateway if interface of 1.1.1.1 is down )

Please let me know.

l;)
0
 

Author Comment

by:starwarp11
ID: 21855848
Link A is static, Link B is Dynamic (But will hold the IP long enough for VOIP registration and transfer)
NETGEAR ProSafe VPN Firewall FVX538 Dual WAN  but I want to use the Load Balancing (for the rest of my network) and not failover WAN1->WAN2
Since the traffic is SIP it will always be UDP.
No need to update DNS as the SIP registration packet will update our VOIP providers servers as to the new IP every 30 seconds.

So on the router I bind eth0 to WAN 1 and eth1 to WAN 2.

Now I just need a way to say ok bind all traffic to eth0 until WAN1 goes down, once WAN1 is down leave eth0 for local traffic (ie: voip phones) then open eth1 for outbound internet traffic and corresponding inbound traffic.
0
 

Author Comment

by:starwarp11
ID: 21855887
There is a way in webmin to run a command when a ping fails, so I was thinking disable eth0 and enable eth1 if ping fails and vice versa for other way, but I will not be able to keep the phones registered as they are registering to eth0, so maybe could I change the default gw or something like that?
0
 
LVL 3

Expert Comment

by:dextermain
ID: 21858452
Add 2 routes to your server.
1. Default gw

2 other route eg "route add -net 10.0.0.0 netmask 255.0.0.0 gw 2.2.2.2 metric 100"

When default gateway is not available the metric 100 route will be used.

0
 
LVL 39

Expert Comment

by:noci
ID: 21860982
If you can see the line up/down on your gateway system (with both eth0&eth1)
If it is either WAN1 or WAN2 then you need a route with cost (=metric)  like dextermain suggests.

If your system cannot see if the link goes down or not (like when you have a router between your gateway system with both links attached)
then you need to be told that the link is up or down. (i.e a router update protocol, preferred is OSPF in this case because it converges fast.
RIP is widely used but it can take up to two minutes to converge  => notice a link goin down and comunicate it to your central system).

Load balancing is more tricky, you need to be able to select a route based on remaining capacity.
This article describes howto do loadbalancing using iptables. http://www.sysresccd.org/Sysresccd-networking_en_Iptables-and-netfilter-load-balancing-using-connmark
0
 

Author Comment

by:starwarp11
ID: 21861584
Unfortunately both IPs are on internal network and both ISP links are on the same router ie: same gateway IP address.  Load balancing is being handled by the router and if WAN 1 goes down the router will stop using it, but will the linux box still send out eth0 even though it cannot communicate?  Can I add metrics to the actual interfaces?  Lets say WAN 1 goes down will the linux box still communicate to local LAN through eth0 and switch to eth1 since it cannot reach second hop for internet traffic?  If so would there be times that if udp traffic comes in eth0 would it go out eth1 and not eth0?
0
 
LVL 39

Expert Comment

by:noci
ID: 21863582
Now let's draw a picture.....
Does your network look like this??? (see attached file)

If so all handling needs to be done on the LEFT router...
You can set the routes on the left linux system but the
routing dissision for ISP1/ISP2 will be made by the left one.
0
 
LVL 39

Expert Comment

by:noci
ID: 21863603
Now the attached file did upload...
tmp-ee.png
0
Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

 
LVL 3

Expert Comment

by:dextermain
ID: 21867423
Hi

What type of router do you have?

0
 

Author Comment

by:starwarp11
ID: 21871703
NETGEAR ProSafe VPN Firewall FVX538 Dual WAN

Yes that is a good picture.  The routing is handling it, but whats to stop stuff coming in 1.1.1.1 to eth0 (defined by router) going out from eth1 to 2.2.2.2 (bound by router)?  and if 1.1.1.1 goes down will the local UDP traffic going to 10.10.9.1 (eth0) go out eth1?  Those are the problems I am trying to stop
0
 

Author Comment

by:starwarp11
ID: 21871719
Here I am going to make this simpler.  Let's forget the router and the IP addresses.

Things I want to accomplish:
1. UDP traffic coming in eth0 goes out eth0
2. UDP traffic coming in eth1 goes out eth1
3. a ping to the first hop going out eth0 fails route all outbound traffic through eth1 and leave eth0 open for local traffic in and out.
4. a ping to the first hop going out eth1 fails disable eth1 and check every 5 minutes or so if it is back up.
0
 
LVL 39

Expert Comment

by:noci
ID: 21872387
The NAT& CONNECTION tracking should take care of pt. 1&2 (If answer are getting returned quick enough < 1 minute)
Ping normaly follows routing rules so that would complicate things.

How about the following:
Your concerns are failure of an interface...

Why not bundle them into a bridge
br0 uses (eth0 & eth1)
Then use br0:0 for one interface and br0:1 fo the other interface, this should give you your required failover, without the hassle of routing transformations.

if either interface fails then the other takes over. (transparent for upper layers).
0
 

Author Comment

by:starwarp11
ID: 21877331
nocci if I use your way, will traffic slip out of eth1 when eth0 is still up?
0
 

Author Comment

by:starwarp11
ID: 21877340
and I am concerned with failing ISP link so eth0 will still show as up when ISP A is down.  and vice versa with eth1 and ISP B.
0
 
LVL 39

Expert Comment

by:noci
ID: 21878545
The interface will only go down if the cable gets unplugged or the equipment ont the other end of the cable goes powerdown.

With using the bridge on two ethernet adapters you ensure that a single interface failure will never bring your system down (the router is still a SPOF though).

All routing desissions need to be taken in the NETGEAR router.
You can still use different source addresses to help routing (if source routing works) by using alias addresses on the linux box (br0:0 & br0:1).

There will never be a physical link between ISPA direct to eth0 unless you plug the cable into your linux box, then link down will mean interface down.
(BTW is that an option?, linux has a more than capable firewall on board, has a pretty good routing engine .....?)

And why should eth0 do down if ISP1 goes down?
0
 

Author Comment

by:starwarp11
ID: 21896480
To answer yoru last question I figured it would be best to not send traffic out it.

To answer your question about using linux as a router/firewall...it may be an idea for the future, but we are switching over to VOIP this thursday, would just like to get this part working first.  Plus the computer that is hosting the VOIP is a couple years old and no RAID or any kind of redudancy (boss is cheap).

Yes router is SPOF, but we limited budget, but then again how often do routers go down, and would probably take me 10 minutes to switch it out with our onsite cold spare.

Ok so if I use the "bridge" erthernet will UDP traffic coming in through br0:0  be guaranteed to go out br0:0 and same with br0:1 in and out.
0
 
LVL 39

Accepted Solution

by:
noci earned 500 total points
ID: 21897026
To answer your question about using linux as a router/firewall...it may be an idea for the future, but we are switching over to VOIP this thursday, would just like to get this part working first.  Plus the computer that is hosting the VOIP is a couple years old and no RAID or any kind of redudancy (boss is cheap).
OK that's a choice, raid is cheap now adays (You can have softraid by just adding a disk to you system and configure MDADM. The price is:
One disk of equal or greater capacity in your system.... (and having it on a (preferably) different controler. (so not as disk hda/hdb, but hda/hdc f.e.)

Yes router is SPOF, but we limited budget, but then again how often do routers go down, and would probably take me 10 minutes to switch it out with our onsite cold spare.
Fair enough.

Ok so if I use the "bridge" erthernet will UDP traffic coming in through br0:0  be guaranteed to go out br0:0 and same with br0:1 in and out.
Yes even UDP will do that as long as linux ip kernel engine knows about it. Entries are cached for about 3 minute for UDP, if connection tracking modules have been activated for SIP then those entries will be kept for about 1 hour. tcp is default 2 hours.
An entry will get dropped from the table when af this timer expires. The timer is reset when one packet travels through though.

This can be seen with the iptstate tool: http://www.phildev.net/iptstate/
(it doesn't show the interface, but it is part of the connection descriptions. The kernel part that does this is called connection tracking and is part of the linux firewall/routing engine.)
0
 
LVL 39

Expert Comment

by:noci
ID: 21897063
To answer this one too:

To answer yoru last question I figured it would be best to not send traffic out it.

The problem might be that if you do ifconfig eth0 down on a system where you bind you application to an address
that the application will get an exception as its socket get closed. (if down -> ip address removal => specific bound sockets get closed).
This might buy you more trouble than it save you.
(But this depends on application configuration.)
0

Featured Post

Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

Join & Write a Comment

I have seen several blogs and forum entries elsewhere state that because NTFS volumes do not support linux ownership or permissions, they cannot be used for anonymous ftp upload through the vsftpd program.   IT can be done and here's how to get i…
Note: for this to work properly you need to use a Cross-Over network cable. 1. Connect both servers S1 and S2 on the second network slots respectively. Note that you can use the 1st slots but usually these would be occupied by the Service Provide…
Sending a Secure fax is easy with eFax Corporate (http://www.enterprise.efax.com). First, Just open a new email message.  In the To field, type your recipient's fax number @efaxsend.com. You can even send a secure international fax — just include t…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now