Solved

Exchange 2010 mailbox server crashes when other MB server reboots

Posted on 2014-03-05
4
2,550 Views
Last Modified: 2014-03-10
Issue: I have two servers in a DAG. When I move all active database copies to server B & reboot server A, all's fine.  When I move all active database copies to server A and reboot server B, all mailbox databases dismount.  They come back online as soon as Server B is back online.  

Environment:
- 2 Exchange 2010 SP3 mailbox servers in a DAG, 2 CAS/HT servers in NLB cluster
- Windows 2008 R2 Enterprise servers
- Running as VMs on two separate Windows 2012 Hyper-V Hosts
- Primary Witness Server is one CAS/HT server, Secondary Witness Server is the other CAS/HT server.

All health checks make it look like everything's in good working order (server health, replication, etc.)
---------------------------------------------
Errors:
Insight Manager (HP utility to monitor server health): [DAG] System is unreachable.
---------------------------------------------
CAS/HT server:

Warning 1022: MSExchange Transport
"The connection between the Client Access server and Mailbox server "[ServerB]" failed...

Microsoft.Exchange.Data.Storage.ConnectionFailedTransientException: Cannot open mailbox [mailboxname]. ---> Microsoft.Mapi.MapiExceptionLogonFailed: MapiExceptionLogonFailed: Unable to make connection to the server. (hr=0x80040111, ec=-2147221231)
Diagnostic context:"
---------------------------------------------
Critical Error 1016: MSExchange ActiveSync

Exchange ActiveSync has encountered repeated failures when it tries to access data on Mailbox server [ServerB]. It will temporarily stop making requests to the Mailbox server for [60] seconds to reduce load on that server. This delay may occur if the Mailbox server is overloaded. If this event is logged frequently, review the Application log on this server and the Mailbox server noted above for other events that could indicate the root cause of performance problems.
---------------------------------------------
Errors on ServerB:

Critical Error 4066: MSExchangeRepl

An error occurred while trying to write to the cluster database. Error: ClusterRegBatchClose failed with error 1726.

---------------------------------------------
Critical error 4082: MSExchangeRepl

The replication network manager encountered an error while monitoring events. Error: Microsoft.Exchange.Cluster.Replay.AmClusterApiException: An Active Manager operation failed. Error An error occurred while attempting a cluster operation. Error: Cluster API '"OpenCluster(ServerB) failed with 0x6d9. Error: There are no more endpoints available from the endpoint mapper"' failed.. ---> System.ComponentModel.Win32Exception: There are no more endpoints available from the endpoint mapper
   --- End of inner exception stack trace ---
   at Microsoft.Exchange.Cluster.Replay.NetworkManager.DriveMapRefresh()
   at Microsoft.Exchange.Cluster.Replay.NetworkManager.TryDriveMapRefresh()
---------------------------------------------

The DAG was created without issue, although it pre-existed on two physical servers.  We added ServerA to the DAG, retired a physical, added ServerB, retired 2nd physical.

The DAG has a static IP address which pings from both nodes.

Anyone have any ideas?  I'm quite concerned that if ServerA goes down I'm going to be dead in the water.
0
Comment
Question by:CHR3800
  • 2
  • 2
4 Comments
 
LVL 38

Expert Comment

by:Adam Brown
ID: 39907891
1. With two Nodes, you should only have one Witness server in the configuration. Having 4 results in an even number of votes, which can cause problems.
2. Before rebooting server B, you'll want to make sure that all of the databases are in a healthy state. Run get-mailboxdatabase | get-mailboxdatabasecopystatus to view the status of all copies. If any of the database copies are in a state other than Healthy or Mounted, the database will enter a failed state when the server with the healthy copy fails.
3. Check Cluster services to make sure that each server has a vote in the quorum and that both servers are set as possible owners. This can also cause what you're seeing.
0
 

Accepted Solution

by:
CHR3800 earned 0 total points
ID: 39908102
Thanks for the response.

My issue ended up being that “The Alternate Witness Server itself does not provide any redundancy for the Witness Server, and DAGs do not dynamically switch witness servers, nor do they automatically start using the Alternate Witness Server in the event of a problem with the Witness Server.”  

So, in the Organization Config I'd defined primary & alternate witness servers, believing that when the primary went down the alternate would take over.  Apparently it doesn't work that way.  So, because I have the primary witness server on the same VM host as one of the mailbox servers, there was no way to establish a quorum when I took both down to patch the host.  The solution for me will be to create a primary witness server on a server that's not part of the Exchange VMs in any way.
0
 
LVL 38

Expert Comment

by:Adam Brown
ID: 39908200
"because I have the primary witness server on the same VM host as one of the mailbox servers" is something you should have mentioned, btw :D
0
 

Author Closing Comment

by:CHR3800
ID: 39917029
I'm accepting my own comment as the solution because it's the right one, which I'd found on my own before having it confirmed by another tech on another site. The one other response wasn't here wasn't helpful
0

Featured Post

[Webinar] Disaster Recovery and Cloud Management

Learn from Unigma and CloudBerry industry veterans which providers are best for certain use cases and how to lower cloud costs, how to grow your Managed Services practice in IaaS clouds, and how to utilize public cloud for Disaster Recovery

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Disabling the Directory Sync Service Account in Office 365 will stop directory synchronization from working.
MS Outlook is a world-class email client application that is mainly used for e-communication globally.  In this article, we will discuss the basic idea about MS Outlook, its advanced features, and types of MS Outlook File formats.
Familiarize people with the process of utilizing SQL Server views from within Microsoft Access. Microsoft Access is a very powerful client/server development tool. One of the SQL Server objects that you can interact with from within Microsoft Access…
how to add IIS SMTP to handle application/Scanner relays into office 365.

862 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

27 Experts available now in Live!

Get 1:1 Help Now