Adding static routes / dual default gateways to a windows 2003 server

You are correct in that windows server 2003 only allows one interface to have a default gateway. It will however provide routes for your other interfaces if they exist.
So if you have any machines on the 10.0.0.0/24 network, they should currently work, but if you have a firewall nat'ing that address from another, the firewall will need to have a default route for that segment, and then that would work for the other segment.'
Hope that helps.

Why do both NICs need to get to the Internet?  Are you trying to separate out your Internet bandwidth?

If you're not actually trying to get to the internet but rather to various other internal networks, this is pretty easy. If you know that you have 3 networks internally, and you'd like your server to send these out NIC 2, then static routes are the right way to go:

I'm setting up a scenario:
Corporate Networks:
10.1.0.0
10.2.0.0
Directly connected Internal network:
192.168.1.0

Open up a command prompt and here's the command to use:
route add 10.1.0.0 MASK 255.255.0.0 192.168.1.1
route add 10.2.0.0 MASK 255.255.0.0 192.168.1.1

type "route " from a cmd prompt and you will see some of the syntax needed . what you will want to do is set up a static route for NIC 2 , for expamle you would want to find out what the ip range of your backup device is say it is 10.100.1.1 you would create a route statment the would be


from a command prompt "route add 10.100.1.1 255.255.255.0 10.0.0.1" this would be saying to the server "if you are looking for something at this address 10.100.1.1 then go out thru this interface 10.0.0.1"

I will defer to others who may come after me on this . I am rusty on my routing skills and currently refreshing them as I am working on recertifying my CCNA.

I forgot one piece.  You will likely want to add a -p once you get everything working right because that will make the routes stick even after a reboot.  Otherwise, these static routes get taken away after a reboot.  The command would look like this:
route -p add 10.1.0.0 MASK 255.255.0.0 192.168.1.1

I agree with franked it . would he have to put only one route statement in ? this would be for the second nic ? would he also have to enter in a default route at the end  to direct any of the non backup traffic to  nic #1 ?

There has to be one route entry for each "destination" network.  At this level (host) you could also safely "summarize" networks.  For example, you could say all 10.x.x.x traffic should go through the NIC 2 gateway even if you're not using the whole 10.x.x.x address space.  This assumes that you don't have to reach any 10.x.x.x network through NIC 1.  This should be true since the 10.x.x.x address space can't be routed over the Internet.

Hi Guys,

Thanks for this - looks worrying as the server is in a datacentre and we have very limited access and have to use their engineers . . . if we run one of these DOS comands and it doesn't work, how do we remove them/get on?!!!

To clarify though: We are effectively trying to split our internet bandwidth if you like. The data centre has given us two external IP's conneted to the same Gateway. The objective is get all users to log onto NIC1 and then to have webmail and an external cross WAN back-up from NIC2 via NAT.

NIC1 is external facing (no NAT) and software firewall, NIC2 is behind a hardware firewall whch is 'natting' down to the LAN. The reason for splitting the traffic is:

a) Main switch/NIC dies in data centre, users can still contact server on webmail (related IP but not in the same 24 but subnet)

b) Back-up does not affect main users on NIC1 and is effectively independant

So what I need to know is if it is possible, again the commands to run and perhaps most significantly how I undo then (or instruct the data centre to undo them) if it doesn't work and we lose contact!!!

Thanks and take care

Mark

How do you get to their LAN on the 10.0.0. network to rdp in? VPN? internet IP address?
Or are you in the dataCenter itself? with a 10.0.0. address yourself?
Please explain the topology a little better for a specific answer.
OK, NIC1 has a direct connection to the internet and has only software firewall..
But NIC2 has a private address connected via hardware firewall to LAN..
But then where does it go? to the internet? then it should have an external IP address also?
Did you get 1 ip or 2?

Hi,

O.K. NIC 2 is on a LAN IP we created of 10.0.0.254 and it is sitting behind the Firewall which
has an external IP and NAT's all traffic. The remote back-up is going to work as follows:

Remote data centre has static VPN between the two LAN's (say 10.0.0.254 and 10.0.1.254)
both of which are natted through a conventional firewall arrangement.

We then install a remote back-up agent on the server and heh presto we can back it up across
the VPN 'LAN to LAN'

Main traffic for users still comes in on NIC 1 with sofware firewall and so should be un-affected.

At present we have achieved:

a) NIC1 external IP and software firewall working completely
b) Firewall available on external related IP to above working fine

We have had to disable NIC 2 though as it seems to interfere with NIC1 even though we're setting
it IP at 10.0.0.254 and its gateway at 10.0.0.1. From what I can see, Windows will not accept two
network cards with different gateways to the internet whether through a Firewall or straight out,
probably because it wouldn't know which one to port traffic through at any given time . . .

Am I right and is there any way to achieve this?

Thanks and take care

Mark

The subnet masks of your two internal networks are going to be key.  It is much easier if there is no overlap of IP address space between the two networks.

First off, remove the gateway configured in Windows networking for NIC 2.

Is the VPN connection setup by the data center server we've been talking about?  If so, you'll likely need first to setup a static route to the VPN server:

route add 204.12.32.47 MASK 255.255.255.255 10.0.0.1

Next, if there is no overlap of IP address space, you can create a static route to the specific backup server to pass through the VPN interface.  Then you may want to create a static route like this:

route add 10.1.1.1 MASK 255.255.255.255 10.0.0.254

10.1.1.1 would be the host IP of the backup server.  the Mask is 255.255.255.255 to match just that host.  The 10.0.0.254 would be the local gateway out NIC 2. The interface should not have a gateway assigned in the Windows network control panel.  You can add as many host routes as you'd like.  To determine the gateway to use here, you may want to disable NIC 1, make sure the VPN is up, and traceroute to the backup server.

If you perform these commands, a reboot will wipe out the routes.  You can also perform a:

route delete 10.1.1.1 MASK 255.255.255.255 to manually remove the route.

Once you've tested the routes, and are happy with them, apply them by adding the "-p" flag between "route" and "add" to keep them across reboots.

If you run "route" by itself on the command line, you'll have the documentation on how to use the command.

I don't think you need static routes, so I'm not going to send them through, there is enough docs out there already...
Now the VPN may be a different matter....

Hi Buddy,

Very thorough answer however the VPN tunnel is going to be created between two external firewalls we manage and own, a Cisco at one end and a Watchguard at the other. As a result, the VPN link and all the routing will take place far and away before either Windows servers, and as with all VPN's if you ran a ping 10.0.1.1 out it would go from the main gateway, arrive at the external firewall which would pass it on appropriately.

Now here's the rub (I think), our server appears not to like having two network cards enabled, one purely on a LAN (10.0.0.254 say) and one on the WAN (87.135.123.123 say). Both are connected to the
same physical internet feed - one via the Firewall and one direct. The gateway we set on the LAN NIC is obviously the Firewall and the Gateway on the main NIC is the external gateway. However, we get an error message warning us about multiple gateways are for redundancy etc. etc. and if you accept it,
you can lose all connectivity!!

What I need to know is why and how we get around it!!! Your help is greatly appreciated - wish I had half your knowledge!!

Thanks

Mark

Its quite a warning message isn't it!!

What makes you think it wouldn't work without multiple default routes?

If you try to configure two default gateways, then Windows will use the NIC binding order to determine the default gateway to use.  This will not be based on the destination of the packet, but strictly by the NIC binding.  Unfortunately, in this case, you want to have the packets routed out the internal interface for a specific destination.  So you can enable multiple default gateways if you want, that won't help route the packets correctly.  

Here's where I would go with this as a first stab for the remote backup:
1 - Configure the NIC binding order in Windows.  You can do this by going to Control Panel -> Network connections.  Then click the Advanced menu item, and choose "Advanced Settings...".  Make sure that NIC 1 is at the top of the list in "Connections:" and that it's followed by NIC 2.  This means that Windows will prefer to use the gateway out NIC 1.  The exception to this rule is that packets destined for the directly connected network on NIC 2 will go out NIC 2.

2 - Configure both NICs as appropriate for their IP subnet.  Include both Default gateways, and ignore the warning, you shouldn't lose all connectivity.  I suggest allowing the second default gateway as redundancy if the primary Internet link goes down.

3 - Configure a static route so that packets for a specific destination use the default gateway you want them to.   For example, create a static route to the remote backup server, and specify the default gateway to reach the VPN tunnel.  

The default metric assigned by this command should be lower than the default route.  You can verify this by typing "route print" into the command line.  The lower the metric, the more preferred the route.  So you'll see a route indicated by Destination 0.0.0.0 which is your default route.  By default, Windows assigns this route a metric of 20.  When I add a host-specific route, I see the metric assigned as 1 so this specific route is preferred over the default route.   From the information you've provided, here's what I gather your command to be for the remote backup:

"route add xx.xx.xx.xx MASK 255.255.255.255 10.0.0.1"

This will add a new route in the route table.  If a packet is destined for xx.xx.xx.xx (remote backup server's IP address) then use the default gateway 10.0.0.1 (The near-end internal IP address of the gateway learing to the VPN tunnel).  So packets destined for xx.xx.xx.xx should hit the VPN device and be routed by the VPN device according to the VPN device's route table and, all things being configured correctly on the VPN device, passing over the VPN to the remote network.


Now...The web mail will be more difficult, as you don't know the far end IP address.  My first attempt to solve this one would be to setup the web server to listen on the NIC 2 IP address only, and not on NIC 1.  In IIS, this can be done by right clicking on the "Default Website" or the appropriate web site in IIS manager, and specifying the IP assignment.  Then, tell your users, DNS, or the firewall to use the NIC 2 IP address.  This can be achieved via port-forwarding, manual DNS entry, or typing in the 10.0.0.254 address into a web browser.  Since the web server is only on that IP address, it should be sending responses back out NIC 2's interface.


As far as the remote management and back out in case of failure, the IIS setup should not pose any problem.  You can perform all the IIS tasks, as well as the NIC binding tasks remotely through RDP or another remote control protocol.  The static route could cause problems, but if you perform the route add command listed above, you can have the on-site engineer reboot the server or use any remote power functions the data center provides.  Upon reboot, the static route you defined will be gone.

Once you've got the static route working correctly, either remove, or modify the static route and add the "-p" flag between "route" and "add" while otherwise running the same route add command that worked.

Make constant use of "route print" to check the routing table, and see the effect of the NIC binding, and the static route.  Also use the output of "route print" to think through any issues you run into.  Follow the trail of the routing table to determine which route gets used.  You can also use "tracert" command to see what route the server chooses for any given destination IP.  You can test this before your static route, and after to see the difference.

Hi Buddy,

When we enabled it on the secondary network card and got the multiple gateway error message, we lost all connectvity on the primary NIC (the one which was connected to the outside world direct). If we disabled the secondary card the primary came back!

We tried various Gateways on both cards, i.e. external - external gateway, internal - firewall, and external - external gateway and internal to external gateway (the latter's a bit mad as it effectively obsoletes the firewall!) but whatever we chose, Windows didn't seem to want 2 separate (unteamed) NIC's connecting to the internet at the same time albeit through different routes. I suspect it is because Windows can't then decide which card to route traffic through to the outside world.

What do you reckon?

Thanks

Mark

Here's a PDF to show what I understand your scenario to be.
SimpleScenario.pdf

What a stunning answer!! I'll try it and get back to you - thanks for the very thorough instructions.

Take care

Mark

I'm hoping that the NIC binding order is in reverse from what you need.  Where are you remoting in from?  A remote LAN or the Internet?

Given the setup I've suggested so far, putting the Internet NIC in priority means that the server will route any traffic it doesn't otherwise have a route for through the Internet gateway.  So, similar to the remote backup server, if you're on another internal IP network, you may lose connectivity since you want the server to reach you through the internal gateway.  You could add a static your for your computer similar to the remote backup static route...get's a little iffy you may want to be able to connect through the Internet, at least during testing; ie, let terminal services (RDP) through the software firewall on the Internet NIC.

You'll have to choose which connection you want to be the default, then make exceptions for the others.

Hi Buddy,

So in summary:

a) We need to specify THE default gateway on say the primary NIC (Control Panel -> Network connections -> Advanced menu items ->Advanced SettingsNIC 1 is at the top of the list in "Connections)
this will be the external interface with the software firewall.

b) We need to manually (and in DOS) add a static route to the second Gateway basically saying that any traffic bound for the back-up server goes through this gateway (route add xx.xx.xx.xx MASK 255.255.255.255 10.0.0.1

How do we apply this only to the 2nd NIC though or is this automatic as the 2nd NIC is on that subnet?

c) Webmail, configure llS to only listen for port 80/81 traffic on the second NIC (I know how to assign IP's to specific sites)

d) If it all works then make step b) permenant by adding
(route -p add xx.xx.xx.xx MASK 255.255.255.255 10.0.0.1

Critical is stage a) where we must define the Primary NIC as THE default gateway .  .

Thanks Mark

a) You set the default gateway in the NIC settings, and you choose which NIC is primary, therefore, which gateway is the primary on the box.  Yep, you got it.

b) Yes, this is also correct.  Although, after I wrote my last comment last night, I realized that you could probably do even better than that.  I'm assuming that all 10.x.x.x addresses you want the server to reach will be internal, and shouldn't go out the NIC 1.  In this case, you can adjust the address and mask accordingly:
route add 10.0.0.0 MASK 255.0.0.0 10.0.0.1

This is added to the whole computer's routing table, similarly to using the advanced settings.  It does not need to be applied to a specific NIC, in fact you don't really want it to.  No matter what NICs are connected, it will try to send data destined for 10.x.x.x through the interface that is directly connected to the gateway specified in the route table, in this case: 10.0.0.1.

c)  Yep, you got it.  You just have to make sure that the traffic hits that IP address.

d) yep, you got it.

In theory, based on what I understand of your setup, this should work.  Of course, we're talking about a computer, and even more computers, which mean they are complicated systems and something could inhibit one of our processes, but I'm 80% confident this will be the solution for you.  There may need to be some tweaking of the routes, and that's why I suggest not using the -p flag until they are 100% correct.

Cool! I'll ry it this weekend and let you know

Take care

Mark

Hi Mate,

Failed on point one!! Both NIC's enabled now with appropriate IP ranges but went to
Start - Settings - Control Panel - Network Connections but then I merely get a Window
with both NIC's showing. There is of course an Advanced tab on both but nothing like
you're suggesting!!!

Can you elaborate a little further on where the binding NIC order is to be found?

Thanks and take care

Mark

Take it bacK!! Found it in the menu mate - derrr!! Fingers crossed, the secondary network card
was dominant . . .

Hi!

Well good news really, your instructions for making the primary NIC domainant and the subsequent
static routes worked a treat. But there are two remaining problems. Firstly, although I can contact
the Firewall itself from the 'remote end', I cannot communicate to the server itself. (In other words
I have a one way street - ping to the remote server but no return from the remote server.

Also (and may be related) we seem unable to get the webmail to work from the second NIC although
I changed llS as suggested and I'm sure that end's O.K.

Any further ideas?

Thanks and take care

Mark

Ahh. . . we have our old problem . . . rebooted the server (to remove the static routes)
and also disabled the second NIC before doing so to be sure. Server came up fine.

Then enabled the second NIC and it shut down NIC1's access to the outside world.
This meant that all traffic was passing through NIC2 and of course webmail and
everything worked (proving the Firewall is not the problem). Problem is that NIC1
is where all the users hosts files point to!!

Took a deep breadth and disabled NIC2 and NIC1 came back on line - phew . . .
NIC1 is definitely the primary NIC according to the advanced tab, so how come
just enabling NIC2 after a reboot killed it?

Thanks and take care

Mark

Hi,

Well continued playing around and noticed your comment about metrics. Moved the primary NICS
metrics to 1 and the second NIC to 5, rebooted and NIC1 remained dominant!! Great.

Getting ping traffic both ways on the tunnel too so the bulk of my work is done - thanks!!!

Last problem, the webmail . . . for some reason despite specifying the internal IP address for
the webmail site to listen on - it doesn't work. From earlier (and when NIC2 was dominant)
it did work fine so the hardware Firewall can't be in the picture. Any ideas why we can't get
the webmail or for that matter RDC through NIC2, the firewall is forwarding both correctly.

Thanks and take care

Mark

On the webmail, what error do you get?

What network is the client on when trying to get to the webmail?  
What address are they using?

Can you ping the correct address for webmail from the client location?

On the webmail server, can you perform a "netstat -an | find 80" from the command line?  This will list all the open ports, and show only the ones listening on port 80 (The traditional web server port).  You should see the correct IP listed, or a 0.0.0.0:80 which would mean listening on all IP addresses.

You may also want to check and make sure your software firewall isn't blocking the required port.

How's the webmail?  Were you able to discover the trouble?

Hi,

No the problem remains but let me re-cap:

a) When we make NIC 2 the dominant NIC the webmail works from it as does everything else

This means the software firewall (and even the hardware firewall) can't be the problem

b) When we return to NIC 1 the webamil and everything works fine on that NIC (as above)
    AND the static route you showed us works both ways on the secondary NIC

c) When you have reverted to the primary NIC you cannot ping or connect to the webmail
    via the secondary NIC whatever you do but you can ping to and from the static route

d) When you switch over to NIC2 as dominant, we alter no settings on the server and the webmail
    transferrs and works fine - this also eliminates the possibility that the webmail is not listening on
    all ports and IP's

Now we're stumped!!!

Thanks and take care

Mark

yeah, you've got me stumped too!

On your server with webmail, try this command line:
netstat -an | find "80"

Try it both with NIC1 primary, and NIC2 primary.

This should list all IPs and ports listening on port 80.  If your webmail is configured for another port, then use that port number instead.  It'd be great to see the result of that command here if you can copy/paste it.

Also, if you can set up the server so NIC1 is primary, and then run tracert and capture that output here too, that would be helpful.  Try the reverse as well, whatever your client IP address is, try the tracert from the server to that IP address.  This should show us if there's a routing loop, and also what gateway the server and client are trying to use, and whether those are correct or not.

Hi my friend,

The primary card is listening on port 80 from the netstat command but the secondary card is not . . .
Primary                            
TCP     0.0.0.0:80               0.0.0.0                     LISTENING
TCP     0.0.0.0:1080           0.0.0.0                     LISTENING
TCP     x.x.102.87:80         x.x.66.66:40260      ESTABLISHED
TCP     x.x.102.87:80         x.x.66.66:42981      ESTABLISHED
TCP     x.x.102.87:80         x.x.66.66:43212      ESTABLISHED
TCP     x.x.102.87:1025     x.x.241.178:58780  ESTABLISHED
TCP     x.x.102.87:1025     x.x.241.178:58805  ESTABLISHED
TCP     x.x.102.87:44043   x.x.224.9:80            TIME_WAIT
TCP     x.x.102.87:44045   x.x.224.9:80            TIME_WAIT
TCP     x.x.102.87:44047   x.x.224.9:80            TIME_WAIT

The Static external IP of NIC on is: x.x.102.87 and the static external IP of the
Firewall is: x.x.103.44 with a static route to NIC 2 on  the LAN at 10.0.55.254

It appears that the LAN card is indeed not listening on port 80 - I think the other
IP's are clients connected to Exchange. Does that help?

Thanks Mark








     

Thanks Frank - the main bit worked!