Multiple public IP addresses (different ranges) and only 1 gateway

It doesn't seem to work. Let me restate that both IP addresses are in a completely different range. Each range has a gateway, however if I configure both (on the same interface), network services effectively go down. They only listen to 1 IP address at a time.

I think you should configure routeback with ip route 2. If you simply configure two gateways, Linux will load balance between them. This means traffic entering either IP will exit through either of them again, not the one through which it entered.

I always use shorewall to accomplish this, so I'm not too sure about the exact commands, but the theory behind it is:

1) Install iproute2
2) Create two routing tables in /etc/iproute2/rt_tables
3) Add the default gateway for ip 1 to routing table 1
4) Add the default gateway for ip 2 to routing table 2
5) Configure an ip routing rule so that traffic entering though IP 1, looks at routing table 1, and the traffic entereing through IP 2 looks at routing table 2

That way traffic entering though the one IP will look at a different routing table and thus have a different default gateway than traffic entering though the second IP. In this way, if it enters through IP 1, it exits through IP 1 again, and the other way around.

Have a look at the Linux Advanced Routing and Traffic control howto, found here:

http://lartc.org/

for more detail on how to do this.

Hope this helps

Specifically, this page:

http://lartc.org/howto/lartc.rpdb.multiple-links.html

If your servers are configured to listen to all ip addresses on the host,  the multiple paths shouldn't matter. You can check it using "netstat -an | grep 80" replacing "80" for whatever port your app is listening to. If it's listening to a single IP address, you need to reconfigure your app.

if there's a gateway in the picture (hopefully it's a firewall), why are your servers dealing with the public IP addresses? It might be simpler to delegate this responsibilty to the gateway, and let the servers deal with single private IP addresses.

Not to be hard arsed about it, but to my knowledge: It does matter.

Lets say you have (excuse the fake ip addresses) ip1=1.1.1.1, gateway=1.1.1.2 and ip2=2.2.2.1, gateway=2.2.2.2.2.

If you simply add two gateways, 1.1.1.2 and 2.2.2.2, Linux will choose either to reply. If you connect to, say apache, on 1.1.1.1:80, Linux might still reply through 2.2.2.2 and the other way around. You have to configure a routing rule as per http://lartc.org/howto/lartc.rpdb.multiple-links.html to make Linux force replies for packets that entered through 1.1.1.2 out through 1.1.1.2 again, and a rule that makes replies to packets that entered through 2.2.2.2 exit through 2.2.2.2 again.

Else it would work for the people that came in at the right time (while it was the correct gateway's turn in the load balancing), and not work for people that came in at the wrong time (while it was the wrong interface's turn).

I should check my old notes more thoroughly. You're right. It does matter when the multiple paths are on the host. I was thinking of redundancy in the network (multiple paths).

What makes it worse is that I recently helped a friend who pointed me to policy routing, so I should know better.

It is more complicated to set up, and unfortunately not well supported by normal graphical tools, including my favorite Webmin. In my friends case, he wanted to have application-to-application traffic go over a slower backup link, while having his user interface over the faster primary. Policy routing was the only practical way for him. Very cool stuff.

It may be necessary in this case as well, but it's not clear from the question if that's the simplest solution.

I cannot seem to get it to work.. I now have the following in /etc/network/interfaces

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
        address 1.1.1.1
        netmask 255.255.254.0
        network 1.1.1.0
        broadcast 1.1.1.255
        gateway 1.1.1.2

auto eth0:0
iface eth0:0 inet static
        address 2.2.2.1
        netmask 255.255.255.0
        network 2.2.2.0
        broadcast 2.2.2.255
        gateway 2.2.2.2

But I'm not sure how to add the routing rules (i've read the page) or where to place them. I'm also not sure if it is possible using only 1 physical interface. Where should i place the routing rules?

Oops, remove the gateway lines from the interfaces file.

And the T1 in the second line should be 1, like so:

        ip route add 1.1.1.0/24 dev eth0 src 1.1.1.1 table 1
        ip route add default via 1.1.1.2 table 1
        ip route add 2.2.2.0/24 dev eth0 src 2.2.2.1 table 2
        ip route add default via 2.2.2.2 table 2

Thank you..that works...i have poored it into a script for others to use in the future and easy usage accross servers. (see below). If i run the script with hard-coded values it works, however when i enter variables at the top i get the following error message:

Error: either "to" is duplicate, or "1.1.1.1" is a garbage.
Error: either "to" is duplicate, or "2.2.2.1" is a garbage.
Error: either "to" is duplicate, or "1.1.1.1" is a garbage.
Error: either "to" is duplicate, or "2.2.2.1" is a garbage.

# Setup variables, make sure that the tablenamers are in
# /etc/iproute2/rt_table
tablename1="1"
tablename2="2"
 
ip1="1.1.1.1"
network_ip1="1.1.1.0"
gateway_ip1="1.1.1.2"
interace_ip1="eth0"
 
ip2="2.2.2.1"
network_ip2="2.2.2.0"
gateway_ip2="2.2.2.2"
interace_ip2="eth0:0"
 
# Setup the routing table for route 1
ip route add ${network_ip1} dev ${interface_ip1} src ${ip1} table ${tablename1}
ip route add default via ${gateway_ip1} table ${tablename1}
 
ip route add ${network_ip2} dev ${interface_ip2} src ${ip2} table ${tablename2}
ip route add default via ${gateway_ip2} table ${tablename2}
# -----------------------------------------------------------------
 
# Setup the main routing table
ip route add ${network_ip1}  dev ${interface_ip1} src ${ip1}
ip route add ${network_ip2}  dev ${interface_ip2} src ${ip2}
 
# -----------------------------------------------------------------
# Add default gateway
ip route add default via ${gateway_ip1}
# -----------------------------------------------------------------
 
# Specify which routing table to use based on IP
ip rule add from ${ip1} table ${tablename1}
ip rule add from ${ip2} table ${tablename2}
# -----------------------------------------------------------------

                                              

1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:

I think you left out the subnet mask. Add a netmask_ip1 and netmask_ip2 and then do:

ip route add ${network_ip1}/${netmask_ip1} dev .... etc

Adding that to:

network_ip1="1.1.1.0/24"

Unfortunately didn't work. The error message is still there. However if i dont use variables, the error message does not appear?

My bad.. the problem was a mis-spelled variable name interace instead of interface (see above). :) Thanks for the help!

Thanks :)

Is it not duplicates by any change?

Try

ip route show table 1

if your routes are already in there, add a:

ip route flush table ${tablename1}
ip route flush table ${tablename2}

before adding the rules

:) Glad to help

Flushing the tables seem like a good idea. I have added it near the top of the script, or at least under the variable declarations. Do you know why the tablenames also have to be in /etc/iproute2/rt_tables ?

Actually I'm not sure whether its required. It seems you can add routes without the tables being in there. But all the documentation seems to indicate that you must put it in there too. You can test without it in there to see if it still works. Personally I have always added it because I was told to do so :)

Got it:

From their documentation:

Although we can work with pure numbers, it's far easier if we add our tables to /etc/iproute2/rt_tables.

So rt_tables is only if you want to use:

ip route list table isp1

instead of:

ip route list table 1

So you can ommit it.