Solved

Exchange 2010 mailbox server crashes when other MB server reboots

Posted on 2014-03-05
4
2,513 Views
Last Modified: 2014-03-10
Issue: I have two servers in a DAG. When I move all active database copies to server B & reboot server A, all's fine.  When I move all active database copies to server A and reboot server B, all mailbox databases dismount.  They come back online as soon as Server B is back online.  

Environment:
- 2 Exchange 2010 SP3 mailbox servers in a DAG, 2 CAS/HT servers in NLB cluster
- Windows 2008 R2 Enterprise servers
- Running as VMs on two separate Windows 2012 Hyper-V Hosts
- Primary Witness Server is one CAS/HT server, Secondary Witness Server is the other CAS/HT server.

All health checks make it look like everything's in good working order (server health, replication, etc.)
---------------------------------------------
Errors:
Insight Manager (HP utility to monitor server health): [DAG] System is unreachable.
---------------------------------------------
CAS/HT server:

Warning 1022: MSExchange Transport
"The connection between the Client Access server and Mailbox server "[ServerB]" failed...

Microsoft.Exchange.Data.Storage.ConnectionFailedTransientException: Cannot open mailbox [mailboxname]. ---> Microsoft.Mapi.MapiExceptionLogonFailed: MapiExceptionLogonFailed: Unable to make connection to the server. (hr=0x80040111, ec=-2147221231)
Diagnostic context:"
---------------------------------------------
Critical Error 1016: MSExchange ActiveSync

Exchange ActiveSync has encountered repeated failures when it tries to access data on Mailbox server [ServerB]. It will temporarily stop making requests to the Mailbox server for [60] seconds to reduce load on that server. This delay may occur if the Mailbox server is overloaded. If this event is logged frequently, review the Application log on this server and the Mailbox server noted above for other events that could indicate the root cause of performance problems.
---------------------------------------------
Errors on ServerB:

Critical Error 4066: MSExchangeRepl

An error occurred while trying to write to the cluster database. Error: ClusterRegBatchClose failed with error 1726.

---------------------------------------------
Critical error 4082: MSExchangeRepl

The replication network manager encountered an error while monitoring events. Error: Microsoft.Exchange.Cluster.Replay.AmClusterApiException: An Active Manager operation failed. Error An error occurred while attempting a cluster operation. Error: Cluster API '"OpenCluster(ServerB) failed with 0x6d9. Error: There are no more endpoints available from the endpoint mapper"' failed.. ---> System.ComponentModel.Win32Exception: There are no more endpoints available from the endpoint mapper
   --- End of inner exception stack trace ---
   at Microsoft.Exchange.Cluster.Replay.NetworkManager.DriveMapRefresh()
   at Microsoft.Exchange.Cluster.Replay.NetworkManager.TryDriveMapRefresh()
---------------------------------------------

The DAG was created without issue, although it pre-existed on two physical servers.  We added ServerA to the DAG, retired a physical, added ServerB, retired 2nd physical.

The DAG has a static IP address which pings from both nodes.

Anyone have any ideas?  I'm quite concerned that if ServerA goes down I'm going to be dead in the water.
0
Comment
Question by:CHR3800
  • 2
  • 2
4 Comments
 
LVL 38

Expert Comment

by:Adam Brown
ID: 39907891
1. With two Nodes, you should only have one Witness server in the configuration. Having 4 results in an even number of votes, which can cause problems.
2. Before rebooting server B, you'll want to make sure that all of the databases are in a healthy state. Run get-mailboxdatabase | get-mailboxdatabasecopystatus to view the status of all copies. If any of the database copies are in a state other than Healthy or Mounted, the database will enter a failed state when the server with the healthy copy fails.
3. Check Cluster services to make sure that each server has a vote in the quorum and that both servers are set as possible owners. This can also cause what you're seeing.
0
 

Accepted Solution

by:
CHR3800 earned 0 total points
ID: 39908102
Thanks for the response.

My issue ended up being that “The Alternate Witness Server itself does not provide any redundancy for the Witness Server, and DAGs do not dynamically switch witness servers, nor do they automatically start using the Alternate Witness Server in the event of a problem with the Witness Server.”  

So, in the Organization Config I'd defined primary & alternate witness servers, believing that when the primary went down the alternate would take over.  Apparently it doesn't work that way.  So, because I have the primary witness server on the same VM host as one of the mailbox servers, there was no way to establish a quorum when I took both down to patch the host.  The solution for me will be to create a primary witness server on a server that's not part of the Exchange VMs in any way.
0
 
LVL 38

Expert Comment

by:Adam Brown
ID: 39908200
"because I have the primary witness server on the same VM host as one of the mailbox servers" is something you should have mentioned, btw :D
0
 

Author Closing Comment

by:CHR3800
ID: 39917029
I'm accepting my own comment as the solution because it's the right one, which I'd found on my own before having it confirmed by another tech on another site. The one other response wasn't here wasn't helpful
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

Follow this checklist to learn more about the 15 things you should never include in an email signature from personal quotes, animated gifs and out-of-date marketing content.
This article explains in simple steps how to renew expiring Exchange Server Internal Transport Certificate.
In this video we show how to create an Accepted Domain in Exchange 2013. We show this process by using the Exchange Admin Center. Log into Exchange Admin Center.: First we need to log into the Exchange Admin Center. Navigate to the Mail Flow >> Ac…
The basic steps you have just learned will be implemented in this video. The basic steps are shown to configure an Exchange DAG in a live working Exchange Server Environment and manage the same (Exchange Server 2010 Software is used in a Windows Ser…

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now