We help IT Professionals succeed at work.

Exchange 2003 Server adding routes in route table

I built a new Server 2003 box from scratch, installed Exchange 2003 clean, and did a mailbox move from the old Exch 2003 server (referenced in the related post below).

The new Exchange 2003 server is doing the *SAME THING*!

I can't figure out why these 2 Exch 2003 servers are adding routes to the routing table for the SMTP connections they are receiving.

I don't know if it is limited to just SMTP-related routes, because these 2 server only receive SMTP traffic from the Internet, and only from a mail-scrubbing service, so the number of external hosts connecting is limited.

I've checked other Exchange 2003 servers I manage, and none of them have these outrageous routing tables.

Referring to my earlier post on this (never resolved)
http://www.experts-exchange.com/OS/Microsoft_Operating_Systems/Server/2003_Server/Q_26166887.html
Comment
Watch Question

Top Expert 2007

Commented:
could it be a router issue rather than a server issue ?

I hope this helps !

Author

Commented:
Thanks for the reply.

How would a router be forcing Windows 2003/Exchange 2003 to add routes to the Windows routing table?

I understand that the router is throwing route information around, but the server would have to actively add it to its own route table.  There are 10 other Windows 2003 servers on this network and none of them are adding these routes.

Top Expert 2007

Commented:
Is there any port forwarding to this specific server ?

Syed Mutahir Alibinarybonsai

Commented:
IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0      192.168.1.2     192.168.1.68     20
        127.0.0.0        255.0.0.0         On-link         127.0.0.1    306
        127.0.0.1  255.255.255.255         On-link         127.0.0.1    306
  127.255.255.255  255.255.255.255         On-link         127.0.0.1    306
      192.168.1.0    255.255.255.0         On-link      192.168.1.68    276
     192.168.1.68  255.255.255.255         On-link      192.168.1.68    276
    192.168.1.255  255.255.255.255         On-link      192.168.1.68    276
     192.168.56.0    255.255.255.0         On-link      192.168.56.1    276
     192.168.56.1  255.255.255.255         On-link      192.168.56.1    276
   192.168.56.255  255.255.255.255         On-link      192.168.56.1    276
        224.0.0.0        240.0.0.0         On-link         127.0.0.1    306
        224.0.0.0        240.0.0.0         On-link      192.168.56.1    276
        224.0.0.0        240.0.0.0         On-link      192.168.1.68    276
  255.255.255.255  255.255.255.255         On-link         127.0.0.1    306
  255.255.255.255  255.255.255.255         On-link      192.168.56.1    276
  255.255.255.255  255.255.255.255         On-link      192.168.1.68    276
===========================================================================

Does it resembles to the above ?

can you, ON the Exchange server Box, open up a command prompt, type : "route print > c:\routeprint.txt" without quotes and then attach the text file here ?

Author

Commented:
Rather than attach a file, this is a *SMALL* snippet of the output showing the anomalies in the route table.  I've put dots where I've deleted 100+ entries (twice).  Notice that the routes are all host-based (i.e., /32).

It is acting like every incoming SMTP connection gets a /32 route in the route table.

I made a batch file to "route delete" most of the entries, but I have to "route print > x.txt" then edit that to remove the needed routes and the extraneous crap - so I have to do that process manually, and the routes come back little by little.

Yeah...makes no sense to me either.

H:\>route print

IPv4 Route Table
===========================================================================
Interface List
0x1 ........................... MS TCP Loopback interface
0x2 ...6a 0b d4 61 11 06 ...... Citrix PV Ethernet Adapter
===========================================================================
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0      192.168.1.1    192.168.1.250     10
      8.14.161.83  255.255.255.255      192.168.1.2    192.168.1.250      1
   24.179.160.230  255.255.255.255      192.168.1.2    192.168.1.250      1
    24.213.17.130  255.255.255.255      192.168.1.2    192.168.1.250      1
    24.230.155.26  255.255.255.255      192.168.1.2    192.168.1.250      1
   32.160.147.185  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.161.17.49  255.255.255.255      192.168.1.2    192.168.1.250      1
   32.161.255.177  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.162.23.22  255.255.255.255      192.168.1.2    192.168.1.250      1
    32.163.25.159  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.163.44.32  255.255.255.255      192.168.1.2    192.168.1.250      1
   32.163.157.108  255.255.255.255      192.168.1.2    192.168.1.250      1
   32.163.180.155  255.255.255.255      192.168.1.2    192.168.1.250      1
   32.163.202.221  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.163.208.4  255.255.255.255      192.168.1.2    192.168.1.250      1
   32.163.231.100  255.255.255.255      192.168.1.2    192.168.1.250      1
   32.163.232.200  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.164.0.174  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.164.11.53  255.255.255.255      192.168.1.2    192.168.1.250      1
    32.164.52.160  255.255.255.255      192.168.1.2    192.168.1.250      1
    32.164.61.100  255.255.255.255      192.168.1.2    192.168.1.250      1
    32.164.66.207  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.164.72.56  255.255.255.255      192.168.1.2    192.168.1.250      1
    32.164.77.164  255.255.255.255      192.168.1.2    192.168.1.250      1
    32.164.88.223  255.255.255.255      192.168.1.2    192.168.1.250      1
   32.164.103.216  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.164.107.2  255.255.255.255      192.168.1.2    192.168.1.250      1
   32.164.109.137  255.255.255.255      192.168.1.2    192.168.1.250      1
   32.164.199.184  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.165.8.143  255.255.255.255      192.168.1.2    192.168.1.250      1
    32.165.58.108  255.255.255.255      192.168.1.2    192.168.1.250      1
   32.165.102.103  255.255.255.255      192.168.1.2    192.168.1.250      1
    32.165.173.48  255.255.255.255      192.168.1.2    192.168.1.250      1
   32.165.216.220  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.166.40.98  255.255.255.255      192.168.1.2    192.168.1.250      1
    32.166.40.207  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.166.54.19  255.255.255.255      192.168.1.2    192.168.1.250      1
    32.166.126.29  255.255.255.255      192.168.1.2    192.168.1.250      1
   32.166.204.181  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.167.43.28  255.255.255.255      192.168.1.2    192.168.1.250      1
    32.167.50.205  255.255.255.255      192.168.1.2    192.168.1.250      1
   32.167.232.102  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.167.250.9  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.168.13.10  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.168.92.28  255.255.255.255      192.168.1.2    192.168.1.250      1
   32.168.209.121  255.255.255.255      192.168.1.2    192.168.1.250      1
     32.171.30.14  255.255.255.255      192.168.1.2    192.168.1.250      1
    32.171.36.194  255.255.255.255      192.168.1.2    192.168.1.250      1
    32.171.48.212  255.255.255.255      192.168.1.2    192.168.1.250      1
   63.210.234.146  255.255.255.255      192.168.1.2    192.168.1.250      1
      64.20.60.99  255.255.255.255      192.168.1.2    192.168.1.250      1
    64.199.224.76  255.255.255.255      192.168.1.2    192.168.1.250      1
      69.97.55.66  255.255.255.255      192.168.1.2    192.168.1.250      1
     69.97.66.203  255.255.255.255      192.168.1.2    192.168.1.250      1
      69.97.67.58  255.255.255.255      192.168.1.2    192.168.1.250      1
     69.97.67.219  255.255.255.255      192.168.1.2    192.168.1.250      1
     69.97.76.131  255.255.255.255      192.168.1.2    192.168.1.250      1
     69.97.78.199  255.255.255.255      192.168.1.2    192.168.1.250      1
      69.97.80.98  255.255.255.255      192.168.1.2    192.168.1.250      1
     69.97.83.130  255.255.255.255      192.168.1.2    192.168.1.250      1
      69.97.84.90  255.255.255.255      192.168.1.2    192.168.1.250      1
     69.97.84.150  255.255.255.255      192.168.1.2    192.168.1.250      1
     69.97.85.116  255.255.255.255      192.168.1.2    192.168.1.250      1
     69.99.165.76  255.255.255.255      192.168.1.2    192.168.1.250      1
    69.99.171.114  255.255.255.255      192.168.1.2    192.168.1.250      1
    69.99.188.252  255.255.255.255      192.168.1.2    192.168.1.250      1
     70.13.48.252  255.255.255.255      192.168.1.2    192.168.1.250      1
    70.200.128.76  255.255.255.255      192.168.1.2    192.168.1.250      1
   70.201.246.243  255.255.255.255      192.168.1.2    192.168.1.250      1
   70.201.254.170  255.255.255.255      192.168.1.2    192.168.1.250      1
.
.
.
   75.210.54.152  255.255.255.255      192.168.1.2    192.168.1.250      1
    75.210.58.121  255.255.255.255      192.168.1.2    192.168.1.250      1
    75.210.60.159  255.255.255.255      192.168.1.2    192.168.1.250      1
    75.210.60.228  255.255.255.255      192.168.1.2    192.168.1.250      1
    75.210.62.105  255.255.255.255      192.168.1.2    192.168.1.250      1
    75.210.66.205  255.255.255.255      192.168.1.2    192.168.1.250      1
    75.210.105.37  255.255.255.255      192.168.1.2    192.168.1.250      1
   75.210.116.224  255.255.255.255      192.168.1.2    192.168.1.250      1
   75.210.122.255  255.255.255.255      192.168.1.2    192.168.1.250      1
   75.210.143.225  255.255.255.255      192.168.1.2    192.168.1.250      1
.
.
.
   166.222.44.201  255.255.255.255      192.168.1.2    192.168.1.250      1
    166.222.46.18  255.255.255.255      192.168.1.2    192.168.1.250      1
    166.222.75.93  255.255.255.255      192.168.1.2    192.168.1.250      1
  166.222.103.233  255.255.255.255      192.168.1.2    192.168.1.250      1
  166.222.113.245  255.255.255.255      192.168.1.2    192.168.1.250      1
   166.222.135.61  255.255.255.255      192.168.1.2    192.168.1.250      1
   166.222.187.88  255.255.255.255      192.168.1.2    192.168.1.250      1
  166.222.188.113  255.255.255.255      192.168.1.2    192.168.1.250      1
  166.224.110.215  255.255.255.255      192.168.1.2    192.168.1.250      1
   166.224.150.50  255.255.255.255      192.168.1.2    192.168.1.250      1
   166.228.11.249  255.255.255.255      192.168.1.2    192.168.1.250      1
   166.228.24.141  255.255.255.255      192.168.1.2    192.168.1.250      1
    166.232.217.7  255.255.255.255      192.168.1.2    192.168.1.250      1
   166.233.146.21  255.255.255.255      192.168.1.2    192.168.1.250      1
  166.234.144.128  255.255.255.255      192.168.1.2    192.168.1.250      1
   166.234.204.71  255.255.255.255      192.168.1.2    192.168.1.250      1
   166.235.58.120  255.255.255.255      192.168.1.2    192.168.1.250      1
  166.235.129.227  255.255.255.255      192.168.1.2    192.168.1.250      1
   166.235.148.42  255.255.255.255      192.168.1.2    192.168.1.250      1
     174.51.49.72  255.255.255.255      192.168.1.2    192.168.1.250      1
   174.124.72.221  255.255.255.255      192.168.1.2    192.168.1.250      1
  174.156.175.141  255.255.255.255      192.168.1.2    192.168.1.250      1
      192.168.1.0    255.255.255.0    192.168.1.250    192.168.1.250     10
    192.168.1.250  255.255.255.255        127.0.0.1        127.0.0.1     10
    192.168.1.255  255.255.255.255    192.168.1.250    192.168.1.250     10
   192.168.100.42  255.255.255.255      192.168.1.2    192.168.1.250      1
   192.168.100.43  255.255.255.255      192.168.1.2    192.168.1.250      1
   207.155.253.91  255.255.255.255      192.168.1.2    192.168.1.250      1
    216.250.24.64  255.255.255.255      192.168.1.2    192.168.1.250      1
        224.0.0.0        240.0.0.0    192.168.1.250    192.168.1.250     10
  255.255.255.255  255.255.255.255    192.168.1.250    192.168.1.250      1
Default Gateway:       192.168.1.1
===========================================================================
Persistent Routes:
  None

H:\>
Syed Mutahir Alibinarybonsai

Commented:
hi Snowdog_2112,

Can you, on the command prompt again :

do a "netstat -na > c:\netstat-na.txt" without quotes

and attach the file here ?

You can also do the following and please attach the text files - that will be easier.

netstat -tan > netstat-tan.txt
---

You can also monitor via netstat -na | more to see where these IPs are connecting to and to what port on the server

---

Can you check your firewall / router to see which ports are being forwarded to this Exchange Server ? I hope it is not in a DMZ on the firewall, make sure ONLY PORT 25 and 3389 RDP (if required) are open
---

On your server check for any other application loaded in memory (taskmanager) to which may be these connections are coming.

--

What sort of internet connection you have ?
Your firewall brand / model ?

---

Just trying to figure out, as I just checked my personal home box and it only lists IP addresses which are connected
to it.

Author

Commented:
I'll get that info, but as I mentioned, those are all from SMTP connections coming from address ranges allowed by the firewall.  There are several large ranges used by the mail scrubbing service (an external provider), and the firewall only allows SMTP from those ranges.  The 192.168.3.x/32 routes are from another remote branch connected by way of MPLS cloud.  

The firewall is a Cisco ASA5510

(Not to confuse matters any more, but I'm *only* seeing routes from the 192.168.3.x location, not 192.168.2.0/24 or .4/24 - the other 2 remote locations - this would seem odd in itself, but the .3/24 location actually has 2 paths in, the MPLS like the others or by VPN tunnel through a separate Internet connection which connects...drum roll...to the same firewall where these routes are coming from.  To me this seems a clue suggesting the ASA5510, but none of the other servers with NAT mappings and port forwards are exhibiting the behavior)

I should get to that output sometime today, but thought I'd provide some detail now in case that helps.

Thanks again!!

Author

Commented:
Followup - it appears any server 2003 box (the are vm's, not phys boxes) that has a nat mapp in the cisco 5510 is adding /32 routes for any external host making a connection.

Again, this seems to be firewall related, but I don't know how the firewall is getting the servers to add routes to the server's own routing table.

I've checked on other networks that do not have a Cisco ASA as a firewall - none of them are getting these extraneous routes.
ccomleyDirector

Commented:
The thing which stands out to me from your route table is that the DEFAULT route is to 192.168.1.1 but all the /32 routes being added are to 192.168.1.2.

What are these two addresses?

In an average network I would expect the default route to point to the LAN address of a/the router which traffic uses to exit the local network to the wider world. This is typically 192.168.1.1... but what is at 192.168.1.2 ?

If 192.168.1.2 is a valid default destination for non-local traffic, have you tried making THAT your "default gateway" and not 1.1 ?

Commented:
The default gateway for the network is likely a router.  You should place the command "no ip redirects" in the configuration of the interface on the router that is the default gateway interface for the LAN.  If it is not a Cisco, then you need to disable "ICMP Redirects".  The ASA does not do ICMP redirects so it's unlikely that it is originating them.  The ICMP redirects are what is causing the /32 routes to show up on the local servers.

Author

Commented:
ccomley - there is an MPLS network on 192.168.1.1 connecting some remote branches.  192.168.1.2 is the internet gateway.  The default gateway on the internal hosts points to the MPLS gateway since that is the bulk of the traffic.  The 192.168.1.1 router (it's a cisco 2801 router) has a route to the Internet by way of the asa5510 on 192.168.1.2.

gavving - the 192.168.1.1 router was recently modified (by someone else) to do bgp with the other MPLS routers.  Would "no ip redirects" break the BGP config?  Would that have been changed when BGP was configured?

Thanks!
Commented:
Putting 'no ip redirects' on the ethernet interface facing the internal network (i.e. the 192.168.1.1 interface) will not break any BGP functionality.  It will only stop the router from issuing ICMP redirect packets whenever a device routes IP traffic to it but is destined for the Internet. This setting will cause the BGP router to route the Internet traffic in and out it's ethernet interface as it forwards the traffic along to the firewall.  The 2801 should be able to handle this as long as we're not talking about 100mbit/sec+ worth of internet traffic.
As for what was changed when BGP was configured I can't say.  But if this router was added recently and made to be the default gateway then that's when the problem started happening.  
Note you can also disable ICMP redirects on the Windows server:
http://www.windowsreference.com/security/disable-icmp-redirects-in-vistaxp20032000/
That will cause it to stop listening to the redirect packets being sent by the router and adding the routes to the route table.
 
ccomleyDirector

Commented:
OK, it's making sense now.

AS you have MORE THAN ONE router on your home network, both provding valid routes OFF the network (but in different directions). what happens is any machine ON your home network needs to know the name of ONE router to act as default gateaway.

Normally, if the *default* gateway is not the *correct* gateway for any given destination, then either by using a routing protocol (e.g. OSPF) the routers learn from each other what routes are avaiable, or by using statiic routing, the network manager tells them - either way, so long as the router given as "default" knows routes it can reach via *either* it's own WAN connections *or* via other router(s) on the home network, then the traffic gets through. Traffic to the internet goes to the internet gateway. Traffic to your private destinations is forwarded by the internet router to the VPN router.  Job done.

BUT as they are trying to increase network efficiency the routers also tell any other router that they knew a better route to <chosen destination> and as an windows server *is* a router (it has a route table, it can learn routes do places other than the "defaul" destination) clearly it's picking up on these advertised "better" routes and adding them to its routing table.

This is not a problem, it just means the server learns loads of routes where an *average* server knows only one. But it's designed to handle this routing information so unless it literally learns zillions of routes, it shouldn't be an issue.

From the sample table you give, it appears that the DEFAULT gateway the server has listed is the MLPS router NOT the internet router. So it has one route (the default) which covers every MPLS destination on your VPN, and adds a new route entry for everywhere else on teh internet that it speaks to. IF YOU REVERSE this and make the *internet* router at 192.168.1.2 the "default gateway", it should then add to teh route table only the routes it learns which are better served by the MPLS router at 192.168.1.1, which will presumably be only half a dozen or so!!

Or, as gavving says, you can tell it to ignore these re-direct packets which are teaching it the "better" routes.

But it will reduce traffic on your network, and load on your MPLS router, if you *don't* send every internet packet to the MPLS router first. It only has to forward the packet to the Internet router. So I would say either (a) set the default gateway to the internet router or (b) leave the redirects enabled, so the server CAN learn the most efficient routes.  Or, indeed, both. :)

Commented:
Just FYI ccomley's option A will not work with the Cisco ASA in it's default configuration.  The Cisco ASA does not issue ICMP redirects by design, and it does not by default route traffic in and out the same interface.  Thus unless you make specific configuration changes setting the default route to the Cisco ASA will break routing connectivity to the sites connected through the internal MPLS router.  With just about every other firewall I've worked with it would work, but with the ASA - it won't normally.

ccomleyDirector

Commented:
Gavving - noted for future consumption. Trust Cisco to break it...

Though that said it would not HAVE to issue redirects for teh system to keep working with IT as the default gateway, it would just have routing info for the MPLS destinations and forward the traffic normally to teh other router. This it certainly shoud do if it's exchanging network data with the MPLS router.  

Commented:
Actually Cisco ASA's and PIX's do NOT route traffic in and out the same interface by default.   It wasn't until version 7.x code that they added a command 'same-security-traffic permit intra-interface', that it allowed this type of traffic.  But this command is not enabled by default.  

As a side note every other firewall I've ever worked on would route the traffic correctly as you describe by default.  Just not the Cisco PIX/ASA.

Cisco document:
http://www.cisco.com/en/US/products/ps6120/products_tech_note09186a0080734db7.shtml
ccomleyDirector

Commented:
Crikey - how primative. Well it explains why the default route was pointed at the MPLS router I guess.

In that case, the answer is probably - "LET the system work the way it is at the moment, it's not a problem, it's just a little unusual."

Author

Commented:
Thanks for the tip - the ICMP redirect was the trick.

http://www.windowsreference.com/security/disable-icmp-redirects-in-vistaxp20032000/

I didn't see the article reference it, but this change *DOES REQUIRE A REBOOT*.