Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

Issues joining a new Mailbox/Hub server to the Database Availability Group

Posted on 2013-06-13
4
Medium Priority
?
1,262 Views
Last Modified: 2013-07-22
I have two servers currently in the DAG and I'm trying to add a 3rd.   I'm getting the error  below when trying to add that 3rd server. I'm seeing answers of all kinds on Forums, I've tried changing some NIC properties, I do have the same OS on all 3 servers (Win 2008 Ent.), I can't think of other things I've tried at this moment..   Hopefully someone else has run into this issue and fixed it.  Here is the error:

Summary: 1 item(s). 0 succeeded, 1 failed.
Elapsed time: 00:06:23


SEMEX06
Failed

Error:
A server-side database availability group administrative operation failed. Error: The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed. [Server: SEMEX05.sug.com]

An Active Manager operation failed. Error An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed..

This operation returned because the timeout period expired
Click here for help... http://technet.microsoft.com/en-US/library/ms.exch.err.default(EXCHG.140).aspx?v=14.2.342.0&t=exchgf1&e=ms.exch.err.ExC9C315

Warning:
The operation wasn't successful because an error was encountered. You may find more details in log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2013-06-13_13-23-54.076_add-databaseavailabiltygroupserver.log".


Exchange Management Shell command attempted:
Add-DatabaseAvailabilityGroupServer -MailboxServer 'SEMEX06' -Identity 'ExchDag'

Elapsed Time: 00:06:23
0
Comment
Question by:jonasdurst
  • 3
4 Comments
 

Author Comment

by:jonasdurst
ID: 39244695
log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2013-06-13_13-23-54.076_add-databaseavailabiltygroupserver.log"
----------------------------------------------------------------------------------------------------------------------------------------

add-databaseavailabiltygroupserver started on machine SEMCAS01.
[2013-06-13T13:23:54] add-dagserver started
[2013-06-13T13:23:54] commandline:         $scriptCmd = {& $wrappedCmd @PSBoundParameters }
[2013-06-13T13:23:54] Option 'Identity' = 'ExchDag'.
[2013-06-13T13:23:54] Option 'MailboxServer' = 'SEMEX06'.
[2013-06-13T13:23:54] Option 'DatabaseAvailabilityGroupIpAddresses' = ''.
[2013-06-13T13:23:54] Option 'WhatIf' = ''.
[2013-06-13T13:23:54] Process: w3wp w3wp.exe:3248.
[2013-06-13T13:23:54] User context = 'NT AUTHORITY\SYSTEM'.
[2013-06-13T13:23:54]   Member of group 'Everyone'.
[2013-06-13T13:23:54]   Member of group 'BUILTIN\Users'.
[2013-06-13T13:23:54]   Member of group 'NT AUTHORITY\SERVICE'.
[2013-06-13T13:23:54]   Member of group 'CONSOLE LOGON'.
[2013-06-13T13:23:54]   Member of group 'NT AUTHORITY\Authenticated Users'.
[2013-06-13T13:23:54]   Member of group 'NT AUTHORITY\This Organization'.
[2013-06-13T13:23:54]   Member of group 'BUILTIN\IIS_IUSRS'.
[2013-06-13T13:23:54]   Member of group 'LOCAL'.
[2013-06-13T13:23:54]   Member of group 'IIS APPPOOL\MSExchangePowerShellAppPool'.
[2013-06-13T13:23:54]   Member of group 'BUILTIN\Administrators'.
[2013-06-13T13:23:54] Updated Progress 'Validating the parameters.' 2%.
[2013-06-13T13:23:54] Working
[2013-06-13T13:23:54] Mailbox server: value passed in = SEMEX06, mailboxServer.Name = SEMEX06, mailboxServer.Fqdn = SEMEX06.sug.com
[2013-06-13T13:23:54] LogClussvcState: clussvc is Stopped on SEMEX06.sug.com. Exception (if any) = none
[2013-06-13T13:23:54] The IP addresses for the DAG are (blank means DHCP): 10.1.5.162
[2013-06-13T13:23:54] Looking up IP addresses for ExchDag.
[2013-06-13T13:23:54]   ExchDag = [ 10.1.5.162 ].
[2013-06-13T13:23:54] Looking up IP addresses for semex06.
[2013-06-13T13:23:54]   semex06 = [ 10.1.5.160 ].
[2013-06-13T13:23:54] Looking up IP addresses for SEMEX06.sug.com.
[2013-06-13T13:23:54]   SEMEX06.sug.com = [ 10.1.5.160 ].
[2013-06-13T13:23:54] DAG ExchDag has 2 servers:
[2013-06-13T13:23:54] DAG ExchDag contains server SEMEX05.
[2013-06-13T13:23:54] DAG ExchDag contains server SEMEX04.
[2013-06-13T13:23:54] Updated Progress 'Checking if Mailbox server 'SEMEX06' is in a database availability group.' 4%.
[2013-06-13T13:23:54] Working
[2013-06-13T13:23:54] GetRemoteCluster() for the mailbox server failed with exception = An Active Manager operation failed. Error An error occurred while attempting a cluster operation. Error: Cluster API '"OpenCluster(SEMEX06.sug.com) failed with 0x6d9. Error: There are no more endpoints available from the endpoint mapper"' failed... This is OK.
[2013-06-13T13:23:54] Ignoring previous error, as it is acceptable if the cluster does not exist yet.
[2013-06-13T13:23:54] DumpClusterTopology: Opening remote cluster ExchDag.
[2013-06-13T13:23:54] Dumping the cluster by connecting to: ExchDag.
[2013-06-13T13:23:54] The cluster's name is: ExchDag.
[2013-06-13T13:23:54] Groups
[2013-06-13T13:23:54]     group: Available Storage [not a CMS]
[2013-06-13T13:23:54]         OwnerNode: SEMEX04.sug.com
[2013-06-13T13:23:54]         State: Offline
[2013-06-13T13:23:54]     group: Cluster Group [Cluster Main Group]
[2013-06-13T13:23:54]         OwnerNode: SEMEX05.sug.com
[2013-06-13T13:23:54]         State: Online
[2013-06-13T13:23:54]             Resource: IPv4 Static Address 1 (Cluster Group) [Online, type = IP Address, PossibleOwners = SEMEX04,SEMEX05 ]
[2013-06-13T13:23:54]                 Address = [10.1.5.162]
[2013-06-13T13:23:54]                     EnableDhcp = [0]
[2013-06-13T13:23:54]                     Network = [Cluster Network 1]
[2013-06-13T13:23:54]             Resource: Cluster Name [Online, type = Network Name, PossibleOwners = SEMEX04,SEMEX05 ]
[2013-06-13T13:23:54]                 NetName = [EXCHDAG]
[2013-06-13T13:23:54]                     RequireKerberos = 1. Computer account created on DC ''.
[2013-06-13T13:23:54]             Resource: File Share Witness (\\semfs01.sug.com\ExchDag.sug.com) [Online, type = File Share Witness, PossibleOwners = SEMEX04,SEMEX05 ]
[2013-06-13T13:23:54] Nodes
[2013-06-13T13:23:54]     node: SEMEX04.sug.com [ state = Up ]
[2013-06-13T13:23:54]     node: SEMEX05.sug.com [ state = Up ]
[2013-06-13T13:23:54] Subnets
[2013-06-13T13:23:54]     Name(Cluster Network 1), Mask(10.1.0.0/16), Role(ClusterNetworkRoleInternalAndClient)
[2013-06-13T13:23:54]         NIC 10.1.5.159 on Node SEMEX05 in State=Up
[2013-06-13T13:23:54]         NIC 10.1.5.158 on Node SEMEX04 in State=Up
[2013-06-13T13:23:54]     Name(Cluster Network 3), Mask(10.150.0.0/16), Role(ClusterNetworkRoleInternalAndClient)
[2013-06-13T13:23:54]         NIC 10.150.0.120 on Node SEMEX04 in State=Up
[2013-06-13T13:23:54]         NIC 10.150.0.122 on Node SEMEX05 in State=Up
[2013-06-13T13:23:54] Opening the cluster on nodes [semex05, semex04].
[2013-06-13T13:23:54] Other mailbox servers in the DAG are already members of cluster 'ExchDag'
[2013-06-13T13:23:54] The server SEMEX06 does not belong to a cluster, and the other servers belong to ExchDag.
[2013-06-13T13:23:54] Successfully resolved the servers based on the stopped servers list.
[2013-06-13T13:23:54] The following servers are in the StartedServers list (The list is the StartedServers property of the DAG in AD):
[2013-06-13T13:23:54] The following servers are in the StoppedServers list:
[2013-06-13T13:23:54] Verifiying that the members of database availability group 'ExchDag' are also members of the cluster.
[2013-06-13T13:23:54] Verifying that the members of cluster 'ExchDag' are also members of the database availability group.
[2013-06-13T13:23:54] According to GetNodeClusterState(), the server SEMEX06 is NotConfigured.
[2013-06-13T13:23:54] The CNO is currently Online.
[2013-06-13T13:23:54] InternalValidate() done.
[2013-06-13T13:23:54] Updated Progress 'Adding server 'SEMEX06' to database availability group 'ExchDag'.' 6%.
[2013-06-13T13:23:54] Working
[2013-06-13T13:23:54] Updated Progress 'Adding server 'SEMEX06' to the cluster.' 8%.
[2013-06-13T13:23:54] Working
[2013-06-13T13:30:17] The following log entry comes from a different process that's running on machine 'SEMEX05.sug.com'. BEGIN
[2013-06-13T13:30:17] [2013-06-13T13:23:54] Opening a local AmCluster handle.
[2013-06-13T13:23:54] Updated Progress 'Adding server 'semex06' to database availability group 'ExchDag'.' 2%.
[2013-06-13T13:23:54] Working
[2013-06-13T13:23:54] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:23:54] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:54] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseGettingCurrentMembership, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 25, szObjectName = ExchDag, dwStatus = 0x0 )
[2013-06-13T13:23:55] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseGettingCurrentMembership, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 25, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:55] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseAddNodeToCluster, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 37, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseAddNodeToCluster, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 37, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNetft, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 50, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNetft, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 50, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateClusDisk, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 62, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateClusDisk, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 62, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseConfigureClusSvc, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 75, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:23:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseConfigureClusSvc, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 75, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseStartingClusSvc, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 87, szObjectName = SEMEX06.sug.com, dwStatus = 0x0 )
[2013-06-13T13:23:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseStartingClusSvc, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 87, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseNodeUp, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:26:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseNodeUp, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseFatal, dwPercentComplete = 100, szObjectName = SEMEX06, dwStatus = 0x5b4 )
[2013-06-13T13:26:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:30:17] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = , dwStatus = 0x0 )

[2013-06-13T13:30:17] The preceding log entry comes from a different process running on computer 'SEMEX05.sug.com'. END
[2013-06-13T13:30:17] The operation wasn't successful because an error was encountered. You may find more details in log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2013-06-13_13-23-54.076_add-databaseavailabiltygroupserver.log".
[2013-06-13T13:30:17] WriteError! Exception = Microsoft.Exchange.Cluster.Replay.DagTaskOperationFailedException: A server-side database availability group administrative operation failed. Error: The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed. ---> Microsoft.Exchange.Cluster.Replay.AmClusterApiException: An Active Manager operation failed. Error An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed.. ---> System.ComponentModel.Win32Exception: This operation returned because the timeout period expired
   --- End of inner exception stack trace ---
   at Microsoft.Exchange.Cluster.ClusApi.AmCluster.AddNodeToCluster(AmServerName nodeName, IClusterSetupProgress setupProgress, IntPtr context, Exception& errorException, Boolean throwExceptionOnFailure)
   at Microsoft.Exchange.Cluster.Replay.DagHelper.AddDagClusterNode(AmServerName mailboxServerName, String& verboseLog)
   --- End of inner exception stack trace (Microsoft.Exchange.Cluster.Replay.AmClusterApiException) ---
   at Microsoft.Exchange.Cluster.Replay.DagHelper.ThrowDagTaskOperationWrapper(Exception exception)
   at Microsoft.Exchange.Cluster.Replay.DagHelper.AddDagClusterNode(AmServerName mailboxServerName, String& verboseLog)
   at Microsoft.Exchange.Cluster.ReplayService.ReplayRpcServer.<>c__DisplayClass34.<RpcsAddNodeToCluster>b__33()
   at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.RunRpcServerOperation(String databaseName, RpcServerOperation rpcOperation)
   --- End of stack trace on server (SEMEX05.sug.com) ---
   at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.ClientRethrowIfFailed(String databaseName, String serverName, RpcErrorExceptionInfo errorInfo)
   at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunRpcOperationDbName(AmServerName serverName, String databaseName, Int32 timeoutMs, IHaRpcExceptionWrapper rpcExceptionWrapperInstance, InternalRpcOperation rpcOperation)
   at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunRpcOperation(AmServerName serverName, Nullable`1 dbGuid, Int32 timeoutMs, IHaRpcExceptionWrapper rpcExceptionWrapperInstance, InternalRpcOperation rpcOperation)
   at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunAddNodeToCluster(AmServerName serverName, AmServerName newNode, String& verboseLog)
   at Microsoft.Exchange.Management.SystemConfigurationTasks.AddDatabaseAvailabilityGroupServer.JoinNodeToCluster()
[2013-06-13T13:30:17] Updated Progress 'Done!' 100%.
[2013-06-13T13:30:17] COMPLETED
add-databaseavailabiltygroupserver explicitly called CloseTempLogFile().
0
 
LVL 4

Assisted Solution

by:Alexander Kireev
Alexander Kireev earned 1500 total points
ID: 39258678
Hello,

Do your DAG members use two network adapters (first for DAG, second for data)?
Do you sure that you haven't issues with ip-addresses? Could you send a screenshot of your DAG networks (Organization... - Mailbox - DAG - Networks) and ipconfig of new server?

Are SEMEX06 and other DAG members has same Mailbox/HUB role or different roles?
Is SEMEX06 has the same Exchange version (include RU)?

Send me a result of cmdlt "Get-DatabaseAvailabilityGroup | fl".

http://exchangeserverpro.com/exchange-server-2010-database-availability-group-installation-step-by-step/
0
 

Accepted Solution

by:
jonasdurst earned 0 total points
ID: 39327976
Since we are going with an entirely new domain I'm rebuilding the exchange environment from the ground up, no longer have a need to add the additional MDB.   I have physical servers, going to build two CAS/HUB servers and then two MDB servers.
0
 

Author Closing Comment

by:jonasdurst
ID: 39345319
Since I've avoided the issue buy going down another road I technically didn't solve the issue
0

Featured Post

Fill in the form and get your FREE NFR key NOW!

Veeam is happy to provide a FREE NFR server license to certified engineers, trainers, and bloggers.  It allows for the non‑production use of Veeam Agent for Microsoft Windows. This license is valid for five workstations and two servers.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The main intent of this article is to make you aware of ‘Exchange fail to mount’ error, its effects, causes, and solution.
This month, Experts Exchange sat down with resident SQL expert, Jim Horn, for an in-depth look into the makings of a successful career in SQL.
The viewer will learn how to simulate a series of coin tosses with the rand() function and learn how to make these “tosses” depend on a predetermined probability. Flipping Coins in Excel: Enter =RAND() into cell A2: Recalculate the random variable…
A short tutorial showing how to set up an email signature in Outlook on the Web (previously known as OWA). For free email signatures designs, visit https://www.mail-signatures.com/articles/signature-templates/?sts=6651 If you want to manage em…
Suggested Courses
Course of the Month7 days, 8 hours left to enroll

824 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question