AD/ Exchange 2010 Mail routing query

jasonwatts1972
jasonwatts1972 used Ask the Experts™
on
Hi

We are running Exchange 2010. Our AD topology for site links is as below:

All of the three below are AD sites configured in AD Sites and Services

New York -- Los Angeles -- Miami

That is:

NY-LA: Cost 10
LA-Miami: Cost 10

There is NO site link between NY and Miami.

Each AD site has Exchange 2010 Mailbox servers, Hub and CAS servers.

How does this relate to Exchange routing? If an email was sent from a NY user to  Miami user, would the email go straight from NY Hub > Miami Hub? Or, because there is no AD Site link between NY and Miami, does it go via the LA Hub server?

I know that it's possible to configure Exchange costs on AD site links, but is it possible to create an Exchange site link purely for mail routing purposes?
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Commented:
Exchange will take the best route without going back to a lower version of Exchange.  It uses AD sites to group servers. Because Exchange Transport handles the routing, it will go directly from NY to Miami.

Take this example.

London - 2007
NY - 2003
China - 2007

There is no link or route to China from London.  Because of this email would queue up in London trying to go directly to China.  If NY had upgraded to 2007 also then email would have flowed.  In the end they created a route so that at a network level there was a connection and email started flowing.

Source: This was on one of my consulting jobs in London.
tigermattSite Reliability Engineer
Most Valuable Expert 2011

Commented:
Nice info there about Exchange version details, UnConn! That's frequently missed...

I hate to send you to another website to find your answer, and UnConn has already givne you the info you need, but I just thought I would point out this great article at TechNet, which really goes into routing in spectacular detail: http://technet.microsoft.com/en-us/library/aa998825.aspx

The important part is the transitivity of site links. In a hub and spoke topology with LA as the hub and NY/Miami as the spokes, it is inferred that there is also a direct connection from NY to Miami, in this case, with a total cost of 20.

The crucial part to remember is that site links should not represent the path taken by physical network packets. However, they really should relate to the underlying network connectivity and represent network speed, bandwidth and indeed bandwidth cost in some shape or form.

So, yes, a connection from the NY to Miami HT servers will be attepted directly, unless the LA Hub Transport servers are configured as a "Hub site" in Exchange. In that casem messages will queue in LA and be relayed via those Hub servers.

There's also the requirement that a hub site lie along the least-cost routing path for a message to be relayed via the hub site's servers. In this case, this is not a concern for you, as there is only one path between your sites, so that is by its very nature the least cost path! Worth mentioning for completeness, though.

-Matt

(Seems odd to an outsider like me that there is a connection from NY to LA and LA to Miami but no direct AD Site Link from NY to Miami, considering they're both on the East coast! But then, maybe you have some awesome amount of bandwidth coming out of LA or something!)

Author

Commented:
Hi Matt

Thanks for answering!

Ok, so since we have a transitive link between NY and Miami of cost 20 (but less hops), that connection wil be used, am I correct?
tigermattSite Reliability Engineer
Most Valuable Expert 2011

Commented:
Sorry for the delay in getting back to you.

>> Ok, so since we have a transitive link between NY and Miami of cost 20 (but less hops), that connection wil be used, am I correct?

Ah! Yes, if there is a direct site link from NY to Miami with a cost either equal to or lower than the route via LA, then Exchange would use that route in its routing diagram preferentially to the NY -> LA -> Miami route.

Do remember that the physical links between the sites need not represent how the data actually flows between the sites. AD does not know how your underlying routing topology works. You could still configure your routers in NY to forward all packets destined for Miami via the routers in LA. It wouldn't matter; once a packet is issued and destined for an IP address, it's just down to the routers to get that packet to its final destination.

-Matt

Author

Commented:
Hi Matt

"Ah! Yes, if there is a direct site link from NY to Miami with a cost either equal to or lower than the route via LA, then Exchange would use that route in its routing diagram preferentially to the NY -> LA -> Miami route."

So, just to confirm - there is no direct site link from NY to Miami. The Site links are as below:

NY-LA: Cost 10
LA-Miami: Cost 10

However, as I understand, there is a transitive link automatically created between NY and Miami? Therefore a message from NY to Miami would use the transitive link (although not strictly defined in AD sites and services) as it has less hops with the same cost as going from NY-LA-Miami?

Commented:
"Active Directory IP site links define logical paths between Active Directory sites. Exchange 2010 references the IP site link objects to determine the least-cost routing path of remote Active Directory sites."
From: http://technet.microsoft.com/en-us/library/aa998825.aspx

So if there is no AD site link created between NY and Miami it will have to go through LA, no matter what the cost is.  Not sure that can be disputed.

And this information was alluded to in the other place where you also asked this question:
http://social.technet.microsoft.com/Forums/en-US/winserverDS/thread/77adc617-741c-4a17-ac8d-68919b9d64be

Author

Commented:
Hi there

So the query is whether Exchange will actually route the message via LA or not. If you read the link you sent it also states this:

Site B  - Site A - Site C


"It's important to note that Exchange uses site links only when determining the least-cost path, but will always try to deliver messages directly to the destination Hub Transport server. For example, if a user in Site B [NY] in the topology shown in the preceding figure sends a message to another user in Site C [Miami], the Hub Transport server in Site B will connect directly to the Hub Transport server in Site C. If you want to force messages to go through Site A,[LA] you must enable that site as a hub site"

That's where my confusion is.
Site Reliability Engineer
Most Valuable Expert 2011
Commented:
Hi Jason,

Okay - I see where you are coming from now. Sorry, I read one of your earlier updates and thought you were now adding a site link between NY and Miami, but that was just me misinterpreting it. :)

Yes, effectively the description you have given is accurate. I'm not sure that Exchange "uses" the transitive link per se (that link does not really exist physically in AD, so it does not see it). The process for actually defining the routing table is probably slightly different, but the end result is correct.

I think it's crucial to define here exactly what the site links represent: the underlying network infrastructure which is physically linking those sites together and allowing packets to flow between them. That is all. They do not represent how the servers actually talk to each other (it does not imply that all the traffic goes into the LA servers and gets re-transmitted).

So, your links would suggest there is no direct route for network traffic to pass from NY to Miami or vice-versa. (You don't have a VPN between those sites, for example). The link strategy you have given suggests all the network traffic goes via routers in LA, but that is the most which can be implied.

So, Exchange sees a site link from NY to LA. From LA, it sees a link to Miami. As far as it is concerned, this is a 2-hop path from NY to Miami which goes via LA. The cost is 20, which is obtained by summing the cost of the site links. Since this is the only site link, Exchange uses it for its NY -> Miami communication.

Once it evaluates the site links and figures out it can talk directly to Miami. The total connection between NY and Miami has 2 hops, at a cost of 20, you can forget the site links. The NY servers will issue packets destined directly for the Miami servers. If you did a packet trace on the packets leaving NY, they would bear the IP address of your Miami Hub Transport servers. How your network infrastructure handles this is up to you. Your site links suggest the router in NY would pass the packet to a router in LA, which immediately passes the packet back out to Miami (without sending it to any of the servers in LA).

Now, just by way of example, let's say you were to physically add a site link between NY and Miami, and you have the following IP address subnets for the Hub Transport server in each site:

NY HT Server: 10.0.0.1/24
LA HT Server: 10.0.1.1/24
Miami HT Server: 10.0.2.1/24

With the new site link, there would then be a 1-hop method of getting from NY to Miami and vice-versa. The behaviour now is exactly the same. Exchange in NY (10.0.0.1) will still be issuing packets into the network destined for 10.0.2.1.

Now, suppose you did not re-configure your network. Therefore, based on the site links:

The NY site's routers has a route for 10.0.1.0/24 and 10.0.2.0/24 which pushes all that data down the VPN to LA.

The LA site's routers have a route for 10.0.0.0/24 and 10.0.2.0/24. It pushes the former down the VPN tunnel to NY. It pushes the latter down the tunnel to Miami.

The Miami routers have the reverse of NY: a route for 10.0.0.0/24 and 10.0.1.0/24, both being pushed down a VPN tunnel to LA.

With a single site link between NY and Miami, you are suggesting that the NY site should also have a VPN tunnel to Miami, and a route for 10.0.2.0/24 which sends data down that tunnel instead.

So, regardless of the AD sites configuration, it is really down to the network infrastructure to route the packets accordingly. In the example given, NY Exchange will always try to talk direct to Miami Exchange. The LA Hub Transport Servers will not come into such a communication. The routers might, but that's not up to Exchange. As far as it is concerned, it will issue packets for 10.0.2.1 into your internal "cloud", if you will, and then wait until it gets a response.

Now, what does matter along the least-cost routing path is whether any sites are configured as Hub Sites. A Hub Site is effectively a site which gets in the way of the SMTP traffic. If the NY Exchange Servers are using the NY -> LA -> Miami site links to evaluate their route to Miami, and if LA is configured as a Hub Site, then Exchange will no longer issue packets directly for Miami. It will send packets to LA's Hub Transport server 10.0.1.1, which will receive them and then re-transmit to Miami's Hub Transport server 10.0.2.1.

I hope that makes a little more sense. This really is a confusing topic to try to adequately explain. Your question has spurred me to publish an EE article on it!

-Matt

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial