Solved

Issues joining a new Mailbox/Hub server to the Database Availability Group

Posted on 2013-06-13
4
1,223 Views
Last Modified: 2013-07-22
I have two servers currently in the DAG and I'm trying to add a 3rd.   I'm getting the error  below when trying to add that 3rd server. I'm seeing answers of all kinds on Forums, I've tried changing some NIC properties, I do have the same OS on all 3 servers (Win 2008 Ent.), I can't think of other things I've tried at this moment..   Hopefully someone else has run into this issue and fixed it.  Here is the error:

Summary: 1 item(s). 0 succeeded, 1 failed.
Elapsed time: 00:06:23


SEMEX06
Failed

Error:
A server-side database availability group administrative operation failed. Error: The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed. [Server: SEMEX05.sug.com]

An Active Manager operation failed. Error An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed..

This operation returned because the timeout period expired
Click here for help... http://technet.microsoft.com/en-US/library/ms.exch.err.default(EXCHG.140).aspx?v=14.2.342.0&t=exchgf1&e=ms.exch.err.ExC9C315

Warning:
The operation wasn't successful because an error was encountered. You may find more details in log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2013-06-13_13-23-54.076_add-databaseavailabiltygroupserver.log".


Exchange Management Shell command attempted:
Add-DatabaseAvailabilityGroupServer -MailboxServer 'SEMEX06' -Identity 'ExchDag'

Elapsed Time: 00:06:23
0
Comment
Question by:jonasdurst
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
4 Comments
 

Author Comment

by:jonasdurst
ID: 39244695
log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2013-06-13_13-23-54.076_add-databaseavailabiltygroupserver.log"
----------------------------------------------------------------------------------------------------------------------------------------

add-databaseavailabiltygroupserver started on machine SEMCAS01.
[2013-06-13T13:23:54] add-dagserver started
[2013-06-13T13:23:54] commandline:         $scriptCmd = {& $wrappedCmd @PSBoundParameters }
[2013-06-13T13:23:54] Option 'Identity' = 'ExchDag'.
[2013-06-13T13:23:54] Option 'MailboxServer' = 'SEMEX06'.
[2013-06-13T13:23:54] Option 'DatabaseAvailabilityGroupIpAddresses' = ''.
[2013-06-13T13:23:54] Option 'WhatIf' = ''.
[2013-06-13T13:23:54] Process: w3wp w3wp.exe:3248.
[2013-06-13T13:23:54] User context = 'NT AUTHORITY\SYSTEM'.
[2013-06-13T13:23:54]   Member of group 'Everyone'.
[2013-06-13T13:23:54]   Member of group 'BUILTIN\Users'.
[2013-06-13T13:23:54]   Member of group 'NT AUTHORITY\SERVICE'.
[2013-06-13T13:23:54]   Member of group 'CONSOLE LOGON'.
[2013-06-13T13:23:54]   Member of group 'NT AUTHORITY\Authenticated Users'.
[2013-06-13T13:23:54]   Member of group 'NT AUTHORITY\This Organization'.
[2013-06-13T13:23:54]   Member of group 'BUILTIN\IIS_IUSRS'.
[2013-06-13T13:23:54]   Member of group 'LOCAL'.
[2013-06-13T13:23:54]   Member of group 'IIS APPPOOL\MSExchangePowerShellAppPool'.
[2013-06-13T13:23:54]   Member of group 'BUILTIN\Administrators'.
[2013-06-13T13:23:54] Updated Progress 'Validating the parameters.' 2%.
[2013-06-13T13:23:54] Working
[2013-06-13T13:23:54] Mailbox server: value passed in = SEMEX06, mailboxServer.Name = SEMEX06, mailboxServer.Fqdn = SEMEX06.sug.com
[2013-06-13T13:23:54] LogClussvcState: clussvc is Stopped on SEMEX06.sug.com. Exception (if any) = none
[2013-06-13T13:23:54] The IP addresses for the DAG are (blank means DHCP): 10.1.5.162
[2013-06-13T13:23:54] Looking up IP addresses for ExchDag.
[2013-06-13T13:23:54]   ExchDag = [ 10.1.5.162 ].
[2013-06-13T13:23:54] Looking up IP addresses for semex06.
[2013-06-13T13:23:54]   semex06 = [ 10.1.5.160 ].
[2013-06-13T13:23:54] Looking up IP addresses for SEMEX06.sug.com.
[2013-06-13T13:23:54]   SEMEX06.sug.com = [ 10.1.5.160 ].
[2013-06-13T13:23:54] DAG ExchDag has 2 servers:
[2013-06-13T13:23:54] DAG ExchDag contains server SEMEX05.
[2013-06-13T13:23:54] DAG ExchDag contains server SEMEX04.
[2013-06-13T13:23:54] Updated Progress 'Checking if Mailbox server 'SEMEX06' is in a database availability group.' 4%.
[2013-06-13T13:23:54] Working
[2013-06-13T13:23:54] GetRemoteCluster() for the mailbox server failed with exception = An Active Manager operation failed. Error An error occurred while attempting a cluster operation. Error: Cluster API '"OpenCluster(SEMEX06.sug.com) failed with 0x6d9. Error: There are no more endpoints available from the endpoint mapper"' failed... This is OK.
[2013-06-13T13:23:54] Ignoring previous error, as it is acceptable if the cluster does not exist yet.
[2013-06-13T13:23:54] DumpClusterTopology: Opening remote cluster ExchDag.
[2013-06-13T13:23:54] Dumping the cluster by connecting to: ExchDag.
[2013-06-13T13:23:54] The cluster's name is: ExchDag.
[2013-06-13T13:23:54] Groups
[2013-06-13T13:23:54]     group: Available Storage [not a CMS]
[2013-06-13T13:23:54]         OwnerNode: SEMEX04.sug.com
[2013-06-13T13:23:54]         State: Offline
[2013-06-13T13:23:54]     group: Cluster Group [Cluster Main Group]
[2013-06-13T13:23:54]         OwnerNode: SEMEX05.sug.com
[2013-06-13T13:23:54]         State: Online
[2013-06-13T13:23:54]             Resource: IPv4 Static Address 1 (Cluster Group) [Online, type = IP Address, PossibleOwners = SEMEX04,SEMEX05 ]
[2013-06-13T13:23:54]                 Address = [10.1.5.162]
[2013-06-13T13:23:54]                     EnableDhcp = [0]
[2013-06-13T13:23:54]                     Network = [Cluster Network 1]
[2013-06-13T13:23:54]             Resource: Cluster Name [Online, type = Network Name, PossibleOwners = SEMEX04,SEMEX05 ]
[2013-06-13T13:23:54]                 NetName = [EXCHDAG]
[2013-06-13T13:23:54]                     RequireKerberos = 1. Computer account created on DC ''.
[2013-06-13T13:23:54]             Resource: File Share Witness (\\semfs01.sug.com\ExchDag.sug.com) [Online, type = File Share Witness, PossibleOwners = SEMEX04,SEMEX05 ]
[2013-06-13T13:23:54] Nodes
[2013-06-13T13:23:54]     node: SEMEX04.sug.com [ state = Up ]
[2013-06-13T13:23:54]     node: SEMEX05.sug.com [ state = Up ]
[2013-06-13T13:23:54] Subnets
[2013-06-13T13:23:54]     Name(Cluster Network 1), Mask(10.1.0.0/16), Role(ClusterNetworkRoleInternalAndClient)
[2013-06-13T13:23:54]         NIC 10.1.5.159 on Node SEMEX05 in State=Up
[2013-06-13T13:23:54]         NIC 10.1.5.158 on Node SEMEX04 in State=Up
[2013-06-13T13:23:54]     Name(Cluster Network 3), Mask(10.150.0.0/16), Role(ClusterNetworkRoleInternalAndClient)
[2013-06-13T13:23:54]         NIC 10.150.0.120 on Node SEMEX04 in State=Up
[2013-06-13T13:23:54]         NIC 10.150.0.122 on Node SEMEX05 in State=Up
[2013-06-13T13:23:54] Opening the cluster on nodes [semex05, semex04].
[2013-06-13T13:23:54] Other mailbox servers in the DAG are already members of cluster 'ExchDag'
[2013-06-13T13:23:54] The server SEMEX06 does not belong to a cluster, and the other servers belong to ExchDag.
[2013-06-13T13:23:54] Successfully resolved the servers based on the stopped servers list.
[2013-06-13T13:23:54] The following servers are in the StartedServers list (The list is the StartedServers property of the DAG in AD):
[2013-06-13T13:23:54] The following servers are in the StoppedServers list:
[2013-06-13T13:23:54] Verifiying that the members of database availability group 'ExchDag' are also members of the cluster.
[2013-06-13T13:23:54] Verifying that the members of cluster 'ExchDag' are also members of the database availability group.
[2013-06-13T13:23:54] According to GetNodeClusterState(), the server SEMEX06 is NotConfigured.
[2013-06-13T13:23:54] The CNO is currently Online.
[2013-06-13T13:23:54] InternalValidate() done.
[2013-06-13T13:23:54] Updated Progress 'Adding server 'SEMEX06' to database availability group 'ExchDag'.' 6%.
[2013-06-13T13:23:54] Working
[2013-06-13T13:23:54] Updated Progress 'Adding server 'SEMEX06' to the cluster.' 8%.
[2013-06-13T13:23:54] Working
[2013-06-13T13:30:17] The following log entry comes from a different process that's running on machine 'SEMEX05.sug.com'. BEGIN
[2013-06-13T13:30:17] [2013-06-13T13:23:54] Opening a local AmCluster handle.
[2013-06-13T13:23:54] Updated Progress 'Adding server 'semex06' to database availability group 'ExchDag'.' 2%.
[2013-06-13T13:23:54] Working
[2013-06-13T13:23:54] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:23:54] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:54] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseGettingCurrentMembership, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 25, szObjectName = ExchDag, dwStatus = 0x0 )
[2013-06-13T13:23:55] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseGettingCurrentMembership, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 25, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:55] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseAddNodeToCluster, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 37, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseAddNodeToCluster, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 37, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNetft, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 50, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNetft, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 50, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateClusDisk, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 62, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateClusDisk, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 62, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseConfigureClusSvc, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 75, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:23:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseConfigureClusSvc, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 75, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseStartingClusSvc, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 87, szObjectName = SEMEX06.sug.com, dwStatus = 0x0 )
[2013-06-13T13:23:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseStartingClusSvc, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 87, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseNodeUp, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:26:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseNodeUp, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseFatal, dwPercentComplete = 100, szObjectName = SEMEX06, dwStatus = 0x5b4 )
[2013-06-13T13:26:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:30:17] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = , dwStatus = 0x0 )

[2013-06-13T13:30:17] The preceding log entry comes from a different process running on computer 'SEMEX05.sug.com'. END
[2013-06-13T13:30:17] The operation wasn't successful because an error was encountered. You may find more details in log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2013-06-13_13-23-54.076_add-databaseavailabiltygroupserver.log".
[2013-06-13T13:30:17] WriteError! Exception = Microsoft.Exchange.Cluster.Replay.DagTaskOperationFailedException: A server-side database availability group administrative operation failed. Error: The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed. ---> Microsoft.Exchange.Cluster.Replay.AmClusterApiException: An Active Manager operation failed. Error An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed.. ---> System.ComponentModel.Win32Exception: This operation returned because the timeout period expired
   --- End of inner exception stack trace ---
   at Microsoft.Exchange.Cluster.ClusApi.AmCluster.AddNodeToCluster(AmServerName nodeName, IClusterSetupProgress setupProgress, IntPtr context, Exception& errorException, Boolean throwExceptionOnFailure)
   at Microsoft.Exchange.Cluster.Replay.DagHelper.AddDagClusterNode(AmServerName mailboxServerName, String& verboseLog)
   --- End of inner exception stack trace (Microsoft.Exchange.Cluster.Replay.AmClusterApiException) ---
   at Microsoft.Exchange.Cluster.Replay.DagHelper.ThrowDagTaskOperationWrapper(Exception exception)
   at Microsoft.Exchange.Cluster.Replay.DagHelper.AddDagClusterNode(AmServerName mailboxServerName, String& verboseLog)
   at Microsoft.Exchange.Cluster.ReplayService.ReplayRpcServer.<>c__DisplayClass34.<RpcsAddNodeToCluster>b__33()
   at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.RunRpcServerOperation(String databaseName, RpcServerOperation rpcOperation)
   --- End of stack trace on server (SEMEX05.sug.com) ---
   at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.ClientRethrowIfFailed(String databaseName, String serverName, RpcErrorExceptionInfo errorInfo)
   at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunRpcOperationDbName(AmServerName serverName, String databaseName, Int32 timeoutMs, IHaRpcExceptionWrapper rpcExceptionWrapperInstance, InternalRpcOperation rpcOperation)
   at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunRpcOperation(AmServerName serverName, Nullable`1 dbGuid, Int32 timeoutMs, IHaRpcExceptionWrapper rpcExceptionWrapperInstance, InternalRpcOperation rpcOperation)
   at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunAddNodeToCluster(AmServerName serverName, AmServerName newNode, String& verboseLog)
   at Microsoft.Exchange.Management.SystemConfigurationTasks.AddDatabaseAvailabilityGroupServer.JoinNodeToCluster()
[2013-06-13T13:30:17] Updated Progress 'Done!' 100%.
[2013-06-13T13:30:17] COMPLETED
add-databaseavailabiltygroupserver explicitly called CloseTempLogFile().
0
 
LVL 4

Assisted Solution

by:Alexander Kireev
Alexander Kireev earned 500 total points
ID: 39258678
Hello,

Do your DAG members use two network adapters (first for DAG, second for data)?
Do you sure that you haven't issues with ip-addresses? Could you send a screenshot of your DAG networks (Organization... - Mailbox - DAG - Networks) and ipconfig of new server?

Are SEMEX06 and other DAG members has same Mailbox/HUB role or different roles?
Is SEMEX06 has the same Exchange version (include RU)?

Send me a result of cmdlt "Get-DatabaseAvailabilityGroup | fl".

http://exchangeserverpro.com/exchange-server-2010-database-availability-group-installation-step-by-step/
0
 

Accepted Solution

by:
jonasdurst earned 0 total points
ID: 39327976
Since we are going with an entirely new domain I'm rebuilding the exchange environment from the ground up, no longer have a need to add the additional MDB.   I have physical servers, going to build two CAS/HUB servers and then two MDB servers.
0
 

Author Closing Comment

by:jonasdurst
ID: 39345319
Since I've avoided the issue buy going down another road I technically didn't solve the issue
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

MS Outlook is a world-class email client application that is mainly used for e-communication globally.  In this article, we will discuss the basic idea about MS Outlook, its advanced features, and types of MS Outlook File formats.
This article explains how to install and use the NTBackup utility that comes with Windows Server.
Viewers will learn the different options available in the Backstage view in Excel 2013.
The view will learn how to download and install SIMTOOLS and FORMLIST into Excel, how to use SIMTOOLS to generate a Monte Carlo simulation of 30 sales calls, and how to calculate the conditional probability based on the results of the Monte Carlo …
Suggested Courses

623 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question