Solved

Issues joining a new Mailbox/Hub server to the Database Availability Group

Posted on 2013-06-13
4
1,151 Views
Last Modified: 2013-07-22
I have two servers currently in the DAG and I'm trying to add a 3rd.   I'm getting the error  below when trying to add that 3rd server. I'm seeing answers of all kinds on Forums, I've tried changing some NIC properties, I do have the same OS on all 3 servers (Win 2008 Ent.), I can't think of other things I've tried at this moment..   Hopefully someone else has run into this issue and fixed it.  Here is the error:

Summary: 1 item(s). 0 succeeded, 1 failed.
Elapsed time: 00:06:23


SEMEX06
Failed

Error:
A server-side database availability group administrative operation failed. Error: The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed. [Server: SEMEX05.sug.com]

An Active Manager operation failed. Error An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed..

This operation returned because the timeout period expired
Click here for help... http://technet.microsoft.com/en-US/library/ms.exch.err.default(EXCHG.140).aspx?v=14.2.342.0&t=exchgf1&e=ms.exch.err.ExC9C315

Warning:
The operation wasn't successful because an error was encountered. You may find more details in log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2013-06-13_13-23-54.076_add-databaseavailabiltygroupserver.log".


Exchange Management Shell command attempted:
Add-DatabaseAvailabilityGroupServer -MailboxServer 'SEMEX06' -Identity 'ExchDag'

Elapsed Time: 00:06:23
0
Comment
Question by:jonasdurst
  • 3
4 Comments
 

Author Comment

by:jonasdurst
Comment Utility
log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2013-06-13_13-23-54.076_add-databaseavailabiltygroupserver.log"
----------------------------------------------------------------------------------------------------------------------------------------

add-databaseavailabiltygroupserver started on machine SEMCAS01.
[2013-06-13T13:23:54] add-dagserver started
[2013-06-13T13:23:54] commandline:         $scriptCmd = {& $wrappedCmd @PSBoundParameters }
[2013-06-13T13:23:54] Option 'Identity' = 'ExchDag'.
[2013-06-13T13:23:54] Option 'MailboxServer' = 'SEMEX06'.
[2013-06-13T13:23:54] Option 'DatabaseAvailabilityGroupIpAddresses' = ''.
[2013-06-13T13:23:54] Option 'WhatIf' = ''.
[2013-06-13T13:23:54] Process: w3wp w3wp.exe:3248.
[2013-06-13T13:23:54] User context = 'NT AUTHORITY\SYSTEM'.
[2013-06-13T13:23:54]   Member of group 'Everyone'.
[2013-06-13T13:23:54]   Member of group 'BUILTIN\Users'.
[2013-06-13T13:23:54]   Member of group 'NT AUTHORITY\SERVICE'.
[2013-06-13T13:23:54]   Member of group 'CONSOLE LOGON'.
[2013-06-13T13:23:54]   Member of group 'NT AUTHORITY\Authenticated Users'.
[2013-06-13T13:23:54]   Member of group 'NT AUTHORITY\This Organization'.
[2013-06-13T13:23:54]   Member of group 'BUILTIN\IIS_IUSRS'.
[2013-06-13T13:23:54]   Member of group 'LOCAL'.
[2013-06-13T13:23:54]   Member of group 'IIS APPPOOL\MSExchangePowerShellAppPool'.
[2013-06-13T13:23:54]   Member of group 'BUILTIN\Administrators'.
[2013-06-13T13:23:54] Updated Progress 'Validating the parameters.' 2%.
[2013-06-13T13:23:54] Working
[2013-06-13T13:23:54] Mailbox server: value passed in = SEMEX06, mailboxServer.Name = SEMEX06, mailboxServer.Fqdn = SEMEX06.sug.com
[2013-06-13T13:23:54] LogClussvcState: clussvc is Stopped on SEMEX06.sug.com. Exception (if any) = none
[2013-06-13T13:23:54] The IP addresses for the DAG are (blank means DHCP): 10.1.5.162
[2013-06-13T13:23:54] Looking up IP addresses for ExchDag.
[2013-06-13T13:23:54]   ExchDag = [ 10.1.5.162 ].
[2013-06-13T13:23:54] Looking up IP addresses for semex06.
[2013-06-13T13:23:54]   semex06 = [ 10.1.5.160 ].
[2013-06-13T13:23:54] Looking up IP addresses for SEMEX06.sug.com.
[2013-06-13T13:23:54]   SEMEX06.sug.com = [ 10.1.5.160 ].
[2013-06-13T13:23:54] DAG ExchDag has 2 servers:
[2013-06-13T13:23:54] DAG ExchDag contains server SEMEX05.
[2013-06-13T13:23:54] DAG ExchDag contains server SEMEX04.
[2013-06-13T13:23:54] Updated Progress 'Checking if Mailbox server 'SEMEX06' is in a database availability group.' 4%.
[2013-06-13T13:23:54] Working
[2013-06-13T13:23:54] GetRemoteCluster() for the mailbox server failed with exception = An Active Manager operation failed. Error An error occurred while attempting a cluster operation. Error: Cluster API '"OpenCluster(SEMEX06.sug.com) failed with 0x6d9. Error: There are no more endpoints available from the endpoint mapper"' failed... This is OK.
[2013-06-13T13:23:54] Ignoring previous error, as it is acceptable if the cluster does not exist yet.
[2013-06-13T13:23:54] DumpClusterTopology: Opening remote cluster ExchDag.
[2013-06-13T13:23:54] Dumping the cluster by connecting to: ExchDag.
[2013-06-13T13:23:54] The cluster's name is: ExchDag.
[2013-06-13T13:23:54] Groups
[2013-06-13T13:23:54]     group: Available Storage [not a CMS]
[2013-06-13T13:23:54]         OwnerNode: SEMEX04.sug.com
[2013-06-13T13:23:54]         State: Offline
[2013-06-13T13:23:54]     group: Cluster Group [Cluster Main Group]
[2013-06-13T13:23:54]         OwnerNode: SEMEX05.sug.com
[2013-06-13T13:23:54]         State: Online
[2013-06-13T13:23:54]             Resource: IPv4 Static Address 1 (Cluster Group) [Online, type = IP Address, PossibleOwners = SEMEX04,SEMEX05 ]
[2013-06-13T13:23:54]                 Address = [10.1.5.162]
[2013-06-13T13:23:54]                     EnableDhcp = [0]
[2013-06-13T13:23:54]                     Network = [Cluster Network 1]
[2013-06-13T13:23:54]             Resource: Cluster Name [Online, type = Network Name, PossibleOwners = SEMEX04,SEMEX05 ]
[2013-06-13T13:23:54]                 NetName = [EXCHDAG]
[2013-06-13T13:23:54]                     RequireKerberos = 1. Computer account created on DC ''.
[2013-06-13T13:23:54]             Resource: File Share Witness (\\semfs01.sug.com\ExchDag.sug.com) [Online, type = File Share Witness, PossibleOwners = SEMEX04,SEMEX05 ]
[2013-06-13T13:23:54] Nodes
[2013-06-13T13:23:54]     node: SEMEX04.sug.com [ state = Up ]
[2013-06-13T13:23:54]     node: SEMEX05.sug.com [ state = Up ]
[2013-06-13T13:23:54] Subnets
[2013-06-13T13:23:54]     Name(Cluster Network 1), Mask(10.1.0.0/16), Role(ClusterNetworkRoleInternalAndClient)
[2013-06-13T13:23:54]         NIC 10.1.5.159 on Node SEMEX05 in State=Up
[2013-06-13T13:23:54]         NIC 10.1.5.158 on Node SEMEX04 in State=Up
[2013-06-13T13:23:54]     Name(Cluster Network 3), Mask(10.150.0.0/16), Role(ClusterNetworkRoleInternalAndClient)
[2013-06-13T13:23:54]         NIC 10.150.0.120 on Node SEMEX04 in State=Up
[2013-06-13T13:23:54]         NIC 10.150.0.122 on Node SEMEX05 in State=Up
[2013-06-13T13:23:54] Opening the cluster on nodes [semex05, semex04].
[2013-06-13T13:23:54] Other mailbox servers in the DAG are already members of cluster 'ExchDag'
[2013-06-13T13:23:54] The server SEMEX06 does not belong to a cluster, and the other servers belong to ExchDag.
[2013-06-13T13:23:54] Successfully resolved the servers based on the stopped servers list.
[2013-06-13T13:23:54] The following servers are in the StartedServers list (The list is the StartedServers property of the DAG in AD):
[2013-06-13T13:23:54] The following servers are in the StoppedServers list:
[2013-06-13T13:23:54] Verifiying that the members of database availability group 'ExchDag' are also members of the cluster.
[2013-06-13T13:23:54] Verifying that the members of cluster 'ExchDag' are also members of the database availability group.
[2013-06-13T13:23:54] According to GetNodeClusterState(), the server SEMEX06 is NotConfigured.
[2013-06-13T13:23:54] The CNO is currently Online.
[2013-06-13T13:23:54] InternalValidate() done.
[2013-06-13T13:23:54] Updated Progress 'Adding server 'SEMEX06' to database availability group 'ExchDag'.' 6%.
[2013-06-13T13:23:54] Working
[2013-06-13T13:23:54] Updated Progress 'Adding server 'SEMEX06' to the cluster.' 8%.
[2013-06-13T13:23:54] Working
[2013-06-13T13:30:17] The following log entry comes from a different process that's running on machine 'SEMEX05.sug.com'. BEGIN
[2013-06-13T13:30:17] [2013-06-13T13:23:54] Opening a local AmCluster handle.
[2013-06-13T13:23:54] Updated Progress 'Adding server 'semex06' to database availability group 'ExchDag'.' 2%.
[2013-06-13T13:23:54] Working
[2013-06-13T13:23:54] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:23:54] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:54] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseGettingCurrentMembership, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 25, szObjectName = ExchDag, dwStatus = 0x0 )
[2013-06-13T13:23:55] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseGettingCurrentMembership, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 25, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:55] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseAddNodeToCluster, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 37, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseAddNodeToCluster, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 37, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNetft, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 50, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNetft, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 50, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateClusDisk, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 62, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateClusDisk, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 62, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:56] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseConfigureClusSvc, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 75, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:23:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseConfigureClusSvc, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 75, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseStartingClusSvc, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 87, szObjectName = SEMEX06.sug.com, dwStatus = 0x0 )
[2013-06-13T13:23:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseStartingClusSvc, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 87, szObjectName = , dwStatus = 0x0 )
[2013-06-13T13:23:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseNodeUp, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:26:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseNodeUp, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseFatal, dwPercentComplete = 100, szObjectName = SEMEX06, dwStatus = 0x5b4 )
[2013-06-13T13:26:57] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = SEMEX06, dwStatus = 0x0 )
[2013-06-13T13:30:17] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = , dwStatus = 0x0 )

[2013-06-13T13:30:17] The preceding log entry comes from a different process running on computer 'SEMEX05.sug.com'. END
[2013-06-13T13:30:17] The operation wasn't successful because an error was encountered. You may find more details in log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2013-06-13_13-23-54.076_add-databaseavailabiltygroupserver.log".
[2013-06-13T13:30:17] WriteError! Exception = Microsoft.Exchange.Cluster.Replay.DagTaskOperationFailedException: A server-side database availability group administrative operation failed. Error: The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed. ---> Microsoft.Exchange.Cluster.Replay.AmClusterApiException: An Active Manager operation failed. Error An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed.. ---> System.ComponentModel.Win32Exception: This operation returned because the timeout period expired
   --- End of inner exception stack trace ---
   at Microsoft.Exchange.Cluster.ClusApi.AmCluster.AddNodeToCluster(AmServerName nodeName, IClusterSetupProgress setupProgress, IntPtr context, Exception& errorException, Boolean throwExceptionOnFailure)
   at Microsoft.Exchange.Cluster.Replay.DagHelper.AddDagClusterNode(AmServerName mailboxServerName, String& verboseLog)
   --- End of inner exception stack trace (Microsoft.Exchange.Cluster.Replay.AmClusterApiException) ---
   at Microsoft.Exchange.Cluster.Replay.DagHelper.ThrowDagTaskOperationWrapper(Exception exception)
   at Microsoft.Exchange.Cluster.Replay.DagHelper.AddDagClusterNode(AmServerName mailboxServerName, String& verboseLog)
   at Microsoft.Exchange.Cluster.ReplayService.ReplayRpcServer.<>c__DisplayClass34.<RpcsAddNodeToCluster>b__33()
   at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.RunRpcServerOperation(String databaseName, RpcServerOperation rpcOperation)
   --- End of stack trace on server (SEMEX05.sug.com) ---
   at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.ClientRethrowIfFailed(String databaseName, String serverName, RpcErrorExceptionInfo errorInfo)
   at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunRpcOperationDbName(AmServerName serverName, String databaseName, Int32 timeoutMs, IHaRpcExceptionWrapper rpcExceptionWrapperInstance, InternalRpcOperation rpcOperation)
   at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunRpcOperation(AmServerName serverName, Nullable`1 dbGuid, Int32 timeoutMs, IHaRpcExceptionWrapper rpcExceptionWrapperInstance, InternalRpcOperation rpcOperation)
   at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunAddNodeToCluster(AmServerName serverName, AmServerName newNode, String& verboseLog)
   at Microsoft.Exchange.Management.SystemConfigurationTasks.AddDatabaseAvailabilityGroupServer.JoinNodeToCluster()
[2013-06-13T13:30:17] Updated Progress 'Done!' 100%.
[2013-06-13T13:30:17] COMPLETED
add-databaseavailabiltygroupserver explicitly called CloseTempLogFile().
0
 
LVL 4

Assisted Solution

by:Alexander Kireev
Alexander Kireev earned 500 total points
Comment Utility
Hello,

Do your DAG members use two network adapters (first for DAG, second for data)?
Do you sure that you haven't issues with ip-addresses? Could you send a screenshot of your DAG networks (Organization... - Mailbox - DAG - Networks) and ipconfig of new server?

Are SEMEX06 and other DAG members has same Mailbox/HUB role or different roles?
Is SEMEX06 has the same Exchange version (include RU)?

Send me a result of cmdlt "Get-DatabaseAvailabilityGroup | fl".

http://exchangeserverpro.com/exchange-server-2010-database-availability-group-installation-step-by-step/
0
 

Accepted Solution

by:
jonasdurst earned 0 total points
Comment Utility
Since we are going with an entirely new domain I'm rebuilding the exchange environment from the ground up, no longer have a need to add the additional MDB.   I have physical servers, going to build two CAS/HUB servers and then two MDB servers.
0
 

Author Closing Comment

by:jonasdurst
Comment Utility
Since I've avoided the issue buy going down another road I technically didn't solve the issue
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
Add a SafeSender to Exchange 6 22
Import Cert issue 15 39
Exchange 2016 anti-spam 3 30
cached or not 5 34
Technology opened people to different means of presenting information, but PowerPoint remains to be above competition. Know why PPT still works today.
Marketers need statistics and metrics like everybody else needs oxygen. In this article we explain how to enable marketing campaign statistics for Microsoft Exchange mail.
Viewers will learn how to maximize accessibility options in an Excel workbook for users with accessibility issues.
The viewer will learn how to use a discrete random variable to simulate the return on an investment over a period of years, create a Monte Carlo simulation using the discrete random variable, and create a graph to represent the possible returns over…

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

6 Experts available now in Live!

Get 1:1 Help Now