Exchange 2010 - Internal/External mail not being delivered/routed after legacy server shutdown.

Hi,

I am in the process of migrating our Exchange Organisation from 2010 SP1 to 2010 SP3 CU9, this has involved building two new 2010 SP3 CU9 servers and a new DAG due to a very unhealthy 2010 SP1 installation. For reference, we have:

EX1 - 2010 SP1 - in DAG with EX2
EX2 - 2010 SP1 - in DAG with EX1
EX3 - 2010 SP3 - in DAG with EX4
EX4 - 2010 SP3 - in DAG with EX3

We have migrated all archive, arbitration and user mailboxes from the old DAG mailbox databases and moved the public folder replicas over to EX3/EX4, which works fine.

As we are coming from 2010 SP1, we must "force" the Outlook clients to update their CAS address by making the old EX1/EX2 servers unavailable, forcing them to auto discover the new CAS array address. This is where the problem lays.

We removed EX1 and EX2 from all Send Connectors, remove their DNS records and shut the servers down so for all intents and purposes, the clients think they are not accessible.

Once the servers were shut down, no email was delivered - not internally or inbound from external. The messages were showing in the Queue Viewer as waiting, and shadow redundancy was showing the messages as queuing to the old, shut down servers (as expected), but the message was not being delivered - even know the destination mailbox was on EX3 or EX4, even mail which was destined for mailboxes on the same mailbox database was not being delivered.

EX3 didn't show any errors, however EX4 showed "Local loop detected", but nothing else of help was listed in the Event Viewer.

Am I missing something, or should this work as I expect it to? Any assistance is greatly appreciated.

Many thanks,
crgbt.
crgbtAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Amit KumarCommented:
have you installed SP3 with Latest CU on Hub and CAS servers?

Local loop detected for which queue?

Are mails going externally?
0
Will SzymkowskiSenior Solution ArchitectCommented:
We removed EX1 and EX2 from all Send Connectors, remove their DNS records and shut the servers down so for all intents and purposes, the clients think they are not accessible.

You cannot simply just shutdown your Exchange servers, without properly decommissioning them. When you shut them down they are still being referenced in Active Directory and if there are dependancies on these servers that have not been removed this is where you are experiencing problems. I would recommend just uninstalling the 2 servers that are no longer needed.

Upon doing this removes the Exchange Objects from active directory and they will not be referenced.

Will.
0
crgbtAuthor Commented:
Thanks for the replies Amit & Will, I'll answer you two in turn:

Amit
- SP3 cannot be installed on EX1 and EX2 as they have an unhealthy install of Exchange. Attempting to perform a SP update causes Exchange to become inoperable with many errors in the event log - we have previously tried this and always ended up restoring from backup. SP3 is on EX3 and EX4.
- Local loop was showing as an error for every queue on EX4. EX3 had no errors.
- Emails were not being sent externally either.

Will
I understand we cannot simply shut down the servers to decommission them, this was a temporary shut down to force the Outlook clients to update their CAS address due to the CAS address not automatically updating, see these links for an explanation:

http://clintboessen.blogspot.co.uk/2011/10/outlook-2007-sp2-does-not-redirect-to.html
https://frederickdicaire.wordpress.com/2013/08/29/outlook-profiles-dont-update-when-you-change-cas-server/
http://www.experts-exchange.com/Software/Server_Software/Email_Servers/Exchange/Q_27509885.html

Shutting EX1 and EX2 down (and removing their A record) should not have caused mail to not be delivered, especially as the mailbox databases are on EX3 and EX4.

For clarity; all servers are running HT, CAT and MBX roles. EX3 and EX4 are in the correct Send Connectors.

Thanks for the assistance so far.
crgbt.
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

Simon Butler (Sembee)ConsultantCommented:
Shutting down the servers will NOT trigger an update of the CAS role server that is being used by the clients.
The only way that "should" trigger it is the server going away completely - that means removing Exchange using add/remove programs.

Of course you cannot test that, so you have to basically cross your fingers that it works.
Once the servers have been removed properly, drop them from the domain. You could then change the DNS records for the old server names to point to the new ones, which should catch the traffic - but I only do that once it is seen that Autodiscover is not redirecting the clients.

Shutting down Exchange servers doesn't really do anything other than cause problems, as Exchange expects all servers to be available. If you have removed all services from the servers, remove them using add/remove programs, then restart the live servers. That should update Exchange correct for routing and client access functionality.

Simon.
0
Will SzymkowskiSenior Solution ArchitectCommented:
I understand we cannot simply shut down the servers to decommission them, this was a temporary shut down to force the Outlook clients to update their CAS address due to the CAS address not automatically updating, see these links for an explanation:

There should be no reason to do this. Active Directory will still see these servers so if they are not properly removed then AD will continue to send requests to it.

Will.
0
crgbtAuthor Commented:
OK, thank you for the comments all. Can anyone suggest an alternative way to test mail delivery without actually installing Exchange from EX1 and EX2? Paul Cunningham @ ExchangeServerPro recommends stopping the HT service on the legacy servers (http://exchangeserverpro.com/exchange-server-2010-2013-migration-removing-legacy-servers/) - would this work?
0
Simon Butler (Sembee)ConsultantCommented:
Stopping the Microsoft Exchange Transport Service should show that email can flow around the servers correctly. Ultimately there is only so far that you can go with testing because of the way Exchange is designed to work. It is designed to use all servers in the site.

Simon.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
crgbtAuthor Commented:
OK, many thanks for the assistance all.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Exchange

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.