Link to home
Start Free TrialLog in
Avatar of cdeblois
cdebloisFlag for United States of America

asked on

Exchange 2013 DAG Automatic failover

I have two Exchange 2013 Servers in the same site and one Witness server also in the same site.  I have configured the DAG and replication is working.  When I manage the Cluster from the msc and simulate a failure I get an error message and automatic failover does not occur.  I believe I am missing something in my configuration but can't seems to pinpoint it.  The error message I receive is "Cluster resource 'Cluster Name' of type 'Network Name' in clustered role 'Cluster Group' failed.

Based on the failure policies for the resource and role, the cluster service may try to bring the resource online on this node or move the group to another node of the cluster and then restart it.  Check the resource and group state using Failover Cluster Manager or the Get-ClusterResource Windows PowerShell cmdlet."  

Event ID 1069  
Both Exchange servers appear online.  Both are running Exchange 2013 on Windows 2012 Standard.  The Witness server is running Windows 2008 R2 enterprise
Avatar of Will Szymkowski
Will Szymkowski
Flag of Canada image

Is the DAG VIP IP and Name pingable? When you open failover Clustering manager are all of the services "up"?

If you run the command
Move-activemailboxdatabase mxb1 -activeonServer server2

Does the command complete successfully?

Also run the command...
Get-databaseavailibilitygroup -status

Check the active server for the database and also does it show both servers under operational servers?

Things to consider for DAG
- DAG VIP needs to be pingable
- DAG replication network needs to be up
- Failover clustering needs to have all services "up"

Will.
Avatar of cdeblois

ASKER

Hi Will Thank you so much for responding.  In answer to your questions I can ping the DAG VIP by both IP and by name and it responds.  the replication network is up.  When you say failover clustering needs to have all services up, What services are you referring to.  When I set up the DAG I did no configuration on the Failover Cluster management console it was all there.  Is there something I should be adding?
Emily
You are correct, you shouldn't have had to do anything. I was just saying to check the Failover clustering and make sure that all of the Exchange services are UP based on the cluster.

Will.
User generated imageYes services are up and running.  When I remote to the DAG I get the active server as well.  I have installed and configured clustering with Windows 2008 and Exchange 2007 and it's so very different.  What I see on the Cluster console is quite different as well.  I am including a picture of the Current console for this network for your review.  To me it seems like I'm missing something.  Hopefully this will help some.  My concern is that although the DB's are replicating with no problem when simulating a failure on the Cluster console I get the following error and mail stops flowing:

" Cluster resource 'Cluster Name' of type 'Network Name' in clustered role 'Cluster Group' failed.

Based on the failure policies for the resource and role, the cluster service may try to bring the resource online on this node or move the group to another node of the cluster and then restart it.  Check the resource and group state using Failover Cluster Manager or the Get-ClusterResource Windows PowerShell cmdlet."  

When I run Get-ClusterResource on PS I get the following error:
" Get-ClusterResource : You do not have administrative privileges on the cluster. Contact your network admini
request access.
    Access is denied
At line:1 char:1
+ Get-ClusterResource
+ ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : AuthenticationError: (:) [Get-ClusterResource], ClusterCmdletException
    + FullyQualifiedErrorId : ClusterAccessDenied,Microsoft.FailoverClusters.PowerShell.GetResourceCommand"
ASKER CERTIFIED SOLUTION
Avatar of Will Szymkowski
Will Szymkowski
Flag of Canada image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Update on this question:  
Ok I fixed the permission issue. This is what the current scenario is. 2 Exchange Servers. Exchange 1 has all active copies of DB's except 1, Exchange 2 has 1 active copy. When I stop information store on Exchange 1, none of the DB's mount on exchange 2, if I stop the information store on exchange 2 the DB mounts on exchange one. Get-DatabaseAvailabilityGroup | fl provides the following:

RunspaceId : 1b3139f2-e133-4a47-8695-b8a71886bbf8
Name : ExchDAG
Servers : {CHVMEXCHANGE2, CHVMEXCHANGE1}
WitnessServer : sapserver.xxx.local
WitnessDirectory : C:\DAGFileShareWitnesses\ExchDAG.xxx.local
AlternateWitnessServer :
AlternateWitnessDirectory :
NetworkCompression : InterSubnetOnly
NetworkEncryption : InterSubnetOnly
ManualDagNetworkConfiguration : False
DatacenterActivationMode : DagOnly
StoppedMailboxServers : {}
StartedMailboxServers : {CHVMExchange1.xxx.local, CHVMExchange2.xxx.local}
DatabaseAvailabilityGroupIpv4Addresses : {192.168.112.86}
DatabaseAvailabilityGroupIpAddresses : {192.168.112.86}
AllowCrossSiteRpcClientAccess : False
OperationalServers :
PrimaryActiveManager :
ServersInMaintenance :
ServersInDeferredRecovery :
ThirdPartyReplication : Disabled
ReplicationPort : 64327
NetworkNames : {}
WitnessShareInUse :
DatabaseAvailabilityGroupConfiguration :
AutoDagSchemaVersion : 1.0
AutoDagDatabaseCopiesPerDatabase : 1
AutoDagDatabaseCopiesPerVolume : 1
AutoDagTotalNumberOfDatabases : 0
AutoDagTotalNumberOfServers : 0
AutoDagDatabasesRootFolderPath : C:\ExchangeDatabases
AutoDagVolumesRootFolderPath : C:\ExchangeVolumes
AutoDagAllServersInstalled : False
AutoDagAutoReseedEnabled : True
AutoDagDiskReclaimerEnabled : True
ReplayLagManagerEnabled : False
AdminDisplayName :
ExchangeVersion : 0.10 (14.0.100.0)
DistinguishedName : CN=ExchDAG,CN=Database Availability Groups,CN=Exchange Administrative Group
(FYDIBOHF23SPDLT),CN=Administrative Groups,CN=xxx,CN=Microsoft
Exchange,CN=Services,CN=Configuration,DC=xxx,DC=local
Identity : ExchDAG
Guid : b67e4a1b-eee7-40c4-bf08-37a17d04c8e1
ObjectCategory : xxx.local/Configuration/Schema/ms-Exch-MDB-Availability-Group
ObjectClass : {top, msExchMDBAvailabilityGroup}
WhenChanged : 1/12/2014 8:16:06 PM
WhenCreated : 1/2/2014 12:53:21 PM
WhenChangedUTC : 1/13/2014 1:16:06 AM
WhenCreatedUTC : 1/2/2014 5:53:21 PM
OrganizationId :
OriginatingServer : CHVMADS1.xxx.local
IsValid : True
ObjectState : Unchanged
The databases that reside on Exchange 1 do you have them located on separate drives (i.e. logs = F:\ data = G:)? If this is true you need to setup the same directory strucutre on the Exchange 2 server. If you do not have this it will not work.

You will also need to make sure that you have actually set the Exchange 2 server as a copy for the mailboxes on Exchange 1. See below screenshot...
Adding Mailbox Copy to Another server in the DAG.
User generated image
Will.
Hi Will.  Both of these servers are identical.  They are vitual machines and the exchange logs and DB are both situated on the C Drive in the default location.  The DAG contains both servers and when I ran test-replicationhealth | fl I got the following:
[PS] C:\Windows\system32>test-replicationhealth | ft

Server          Check                      Result     Error
------          -----                      ------     -----
CHVMEXCHANGE1   ClusterService             Passed
CHVMEXCHANGE1   ReplayService              Passed
CHVMEXCHANGE1   ActiveManager              Passed
CHVMEXCHANGE1   TasksRpcListener           Passed
CHVMEXCHANGE1   TcpListener                Passed
CHVMEXCHANGE1   ServerLocatorService       Passed
CHVMEXCHANGE1   DagMembersUp               Passed
CHVMEXCHANGE1   ClusterNetwork             Passed
CHVMEXCHANGE1   QuorumGroup                Passed
CHVMEXCHANGE1   FileShareQuorum            Passed
CHVMEXCHANGE1   DatabaseRedundancy         Passed
CHVMEXCHANGE1   DatabaseAvailability       Passed
CHVMEXCHANGE1   DBCopySuspended            Passed
CHVMEXCHANGE1   DBCopyFailed               Passed
CHVMEXCHANGE1   DBInitializing             Passed
CHVMEXCHANGE1   DBDisconnected             Passed
CHVMEXCHANGE1   DBLogCopyKeepingUp         Passed
CHVMEXCHANGE1   DBLogReplayKeepingUp       Passed
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
On Get-DatabaseAvailabilityGroup -Status |fl I get:
RunspaceId                             : 213d11e1-9cb8-480c-baab-b093673ce055
Name                                   : ExchDAG
Servers                                : {CHVMEXCHANGE2, CHVMEXCHANGE1}
WitnessServer                          : sapserver.commlogik.local
WitnessDirectory                       : C:\DAGFileShareWitnesses\ExchDAG.commlogik.local
AlternateWitnessServer                 :
AlternateWitnessDirectory              :
NetworkCompression                     : InterSubnetOnly
NetworkEncryption                      : InterSubnetOnly
ManualDagNetworkConfiguration          : False
DatacenterActivationMode               : DagOnly
StoppedMailboxServers                  : {}
StartedMailboxServers                  : {CHVMExchange1.commlogik.local, CHVMExchange2.commlogik.local}
DatabaseAvailabilityGroupIpv4Addresses : {192.168.112.86}
DatabaseAvailabilityGroupIpAddresses   : {192.168.112.86}
AllowCrossSiteRpcClientAccess          : False
OperationalServers                     : {CHVMEXCHANGE1, CHVMEXCHANGE2}
PrimaryActiveManager                   : CHVMEXCHANGE2
ServersInMaintenance                   : {}
ServersInDeferredRecovery              : {}
ThirdPartyReplication                  : Disabled
ReplicationPort                        : 64327
NetworkNames                           : {MapiDagNetwork, ReplicationDagNetwork01}
WitnessShareInUse                      : Primary
DatabaseAvailabilityGroupConfiguration :
AutoDagSchemaVersion                   : 1.0
AutoDagDatabaseCopiesPerDatabase       : 1
AutoDagDatabaseCopiesPerVolume         : 1
AutoDagTotalNumberOfDatabases          : 0
AutoDagTotalNumberOfServers            : 0
AutoDagDatabasesRootFolderPath         : C:\ExchangeDatabases
AutoDagVolumesRootFolderPath           : C:\ExchangeVolumes
AutoDagAllServersInstalled             : False
AutoDagAutoReseedEnabled               : True
AutoDagDiskReclaimerEnabled            : True
ReplayLagManagerEnabled                : False
AdminDisplayName                       :
ExchangeVersion                        : 0.10 (14.0.100.0)
DistinguishedName                      : CN=ExchDAG,CN=Database Availability Groups,CN=Exchange Administrative Group
                                         (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=Commlogik,CN=Microsoft
                                         Exchange,CN=Services,CN=Configuration,DC=commlogik,DC=local
Identity                               : ExchDAG
Guid                                   : b67e4a1b-eee7-40c4-bf08-37a17d04c8e1
ObjectCategory                         : commlogik.local/Configuration/Schema/ms-Exch-MDB-Availability-Group
ObjectClass                            : {top, msExchMDBAvailabilityGroup}
WhenChanged                            : 1/12/2014 8:16:28 PM
WhenCreated                            : 1/2/2014 12:53:21 PM
WhenChangedUTC                         : 1/13/2014 1:16:28 AM
WhenCreatedUTC                         : 1/2/2014 5:53:21 PM
OrganizationId                         :
OriginatingServer                      : HERA.commlogik.local
IsValid                                : True
ObjectState                            : Unchanged

On Get-mailboxdatabase I get:
Name                           Server          Recovery        ReplicationType
----                           ------          --------        ---------------
MDB-2013_Corp                  CHVMEXCHANGE1   False           Remote
MDB-2013_VIP                   CHVMEXCHANGE1   False           Remote
MDB-2013_Col                   CHVMEXCHANGE1   False           Remote
MDB-2013_Mex                   CHVMEXCHANGE1   False           Remote
MDB-2013_Ecu                   CHVMEXCHANGE1   False           Remote
MDB_2013_Arg                   CHVMEXCHANGE1   False           Remote
Mailbox Database 0651537422    CHVMEXCHANGE2   False           Remote

All servers appear in the first.  I can't test the move at this moment as these are production servers and I don't want to stop users from working.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Thank you Will I will keep you informed.  Thank you!!
Hi Will sorry for the delay in responding.  I wound up calling Microsoft on this issue and after several ours of troubleshooting they found that the network had a DC that was improperly removed and it was causing the replication errors and the failover errors as a result.  Once the Dc was removed properly, I updated Exchange 2 to CU2 so that both servers were running the same version.  It was at this point that failover was successful.  Thank you so much for your help on this issue :)
Perfect, gald to have helped. In the mean time you can use the above commands to perform all of your testing/verification of services.

Will.
LOl Trust me I have an excel sheet loaded with all those goodies and some I got from the microsoft call as well :)  Again thank you!