Solved

Exchange 2013 DAG Automatic failover

Posted on 2014-01-08
15
3,654 Views
Last Modified: 2014-01-15
I have two Exchange 2013 Servers in the same site and one Witness server also in the same site.  I have configured the DAG and replication is working.  When I manage the Cluster from the msc and simulate a failure I get an error message and automatic failover does not occur.  I believe I am missing something in my configuration but can't seems to pinpoint it.  The error message I receive is "Cluster resource 'Cluster Name' of type 'Network Name' in clustered role 'Cluster Group' failed.

Based on the failure policies for the resource and role, the cluster service may try to bring the resource online on this node or move the group to another node of the cluster and then restart it.  Check the resource and group state using Failover Cluster Manager or the Get-ClusterResource Windows PowerShell cmdlet."  

Event ID 1069  
Both Exchange servers appear online.  Both are running Exchange 2013 on Windows 2012 Standard.  The Witness server is running Windows 2008 R2 enterprise
0
Comment
Question by:cdeblois
  • 8
  • 7
15 Comments
 
LVL 53

Expert Comment

by:Will Szymkowski
ID: 39767106
Is the DAG VIP IP and Name pingable? When you open failover Clustering manager are all of the services "up"?

If you run the command
Move-activemailboxdatabase mxb1 -activeonServer server2

Does the command complete successfully?

Also run the command...
Get-databaseavailibilitygroup -status

Check the active server for the database and also does it show both servers under operational servers?

Things to consider for DAG
- DAG VIP needs to be pingable
- DAG replication network needs to be up
- Failover clustering needs to have all services "up"

Will.
0
 

Author Comment

by:cdeblois
ID: 39769467
Hi Will Thank you so much for responding.  In answer to your questions I can ping the DAG VIP by both IP and by name and it responds.  the replication network is up.  When you say failover clustering needs to have all services up, What services are you referring to.  When I set up the DAG I did no configuration on the Failover Cluster management console it was all there.  Is there something I should be adding?
Emily
0
 
LVL 53

Expert Comment

by:Will Szymkowski
ID: 39769480
You are correct, you shouldn't have had to do anything. I was just saying to check the Failover clustering and make sure that all of the Exchange services are UP based on the cluster.

Will.
0
 

Author Comment

by:cdeblois
ID: 39769686
Cluster ConsoleYes services are up and running.  When I remote to the DAG I get the active server as well.  I have installed and configured clustering with Windows 2008 and Exchange 2007 and it's so very different.  What I see on the Cluster console is quite different as well.  I am including a picture of the Current console for this network for your review.  To me it seems like I'm missing something.  Hopefully this will help some.  My concern is that although the DB's are replicating with no problem when simulating a failure on the Cluster console I get the following error and mail stops flowing:

" Cluster resource 'Cluster Name' of type 'Network Name' in clustered role 'Cluster Group' failed.

Based on the failure policies for the resource and role, the cluster service may try to bring the resource online on this node or move the group to another node of the cluster and then restart it.  Check the resource and group state using Failover Cluster Manager or the Get-ClusterResource Windows PowerShell cmdlet."  

When I run Get-ClusterResource on PS I get the following error:
" Get-ClusterResource : You do not have administrative privileges on the cluster. Contact your network admini
request access.
    Access is denied
At line:1 char:1
+ Get-ClusterResource
+ ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : AuthenticationError: (:) [Get-ClusterResource], ClusterCmdletException
    + FullyQualifiedErrorId : ClusterAccessDenied,Microsoft.FailoverClusters.PowerShell.GetResourceCommand"
0
 
LVL 53

Accepted Solution

by:
Will Szymkowski earned 500 total points
ID: 39771240
It seems to me that your account only has read-only cluster access, related to permission issue. Take a look at the following link which will also help with troubleshooting this specific issue.

Failover Clustering Read-Only - http://blogs.msdn.com/b/clustering/archive/2009/08/15/9870675.aspx?Redirected=true

Also run test-replicationhealth | ft

See if you get any error messages.

Will.
0
 

Author Comment

by:cdeblois
ID: 39775607
Update on this question:  
Ok I fixed the permission issue. This is what the current scenario is. 2 Exchange Servers. Exchange 1 has all active copies of DB's except 1, Exchange 2 has 1 active copy. When I stop information store on Exchange 1, none of the DB's mount on exchange 2, if I stop the information store on exchange 2 the DB mounts on exchange one. Get-DatabaseAvailabilityGroup | fl provides the following:

RunspaceId : 1b3139f2-e133-4a47-8695-b8a71886bbf8
Name : ExchDAG
Servers : {CHVMEXCHANGE2, CHVMEXCHANGE1}
WitnessServer : sapserver.xxx.local
WitnessDirectory : C:\DAGFileShareWitnesses\ExchDAG.xxx.local
AlternateWitnessServer :
AlternateWitnessDirectory :
NetworkCompression : InterSubnetOnly
NetworkEncryption : InterSubnetOnly
ManualDagNetworkConfiguration : False
DatacenterActivationMode : DagOnly
StoppedMailboxServers : {}
StartedMailboxServers : {CHVMExchange1.xxx.local, CHVMExchange2.xxx.local}
DatabaseAvailabilityGroupIpv4Addresses : {192.168.112.86}
DatabaseAvailabilityGroupIpAddresses : {192.168.112.86}
AllowCrossSiteRpcClientAccess : False
OperationalServers :
PrimaryActiveManager :
ServersInMaintenance :
ServersInDeferredRecovery :
ThirdPartyReplication : Disabled
ReplicationPort : 64327
NetworkNames : {}
WitnessShareInUse :
DatabaseAvailabilityGroupConfiguration :
AutoDagSchemaVersion : 1.0
AutoDagDatabaseCopiesPerDatabase : 1
AutoDagDatabaseCopiesPerVolume : 1
AutoDagTotalNumberOfDatabases : 0
AutoDagTotalNumberOfServers : 0
AutoDagDatabasesRootFolderPath : C:\ExchangeDatabases
AutoDagVolumesRootFolderPath : C:\ExchangeVolumes
AutoDagAllServersInstalled : False
AutoDagAutoReseedEnabled : True
AutoDagDiskReclaimerEnabled : True
ReplayLagManagerEnabled : False
AdminDisplayName :
ExchangeVersion : 0.10 (14.0.100.0)
DistinguishedName : CN=ExchDAG,CN=Database Availability Groups,CN=Exchange Administrative Group
(FYDIBOHF23SPDLT),CN=Administrative Groups,CN=xxx,CN=Microsoft
Exchange,CN=Services,CN=Configuration,DC=xxx,DC=local
Identity : ExchDAG
Guid : b67e4a1b-eee7-40c4-bf08-37a17d04c8e1
ObjectCategory : xxx.local/Configuration/Schema/ms-Exch-MDB-Availability-Group
ObjectClass : {top, msExchMDBAvailabilityGroup}
WhenChanged : 1/12/2014 8:16:06 PM
WhenCreated : 1/2/2014 12:53:21 PM
WhenChangedUTC : 1/13/2014 1:16:06 AM
WhenCreatedUTC : 1/2/2014 5:53:21 PM
OrganizationId :
OriginatingServer : CHVMADS1.xxx.local
IsValid : True
ObjectState : Unchanged
0
 
LVL 53

Expert Comment

by:Will Szymkowski
ID: 39775629
The databases that reside on Exchange 1 do you have them located on separate drives (i.e. logs = F:\ data = G:)? If this is true you need to setup the same directory strucutre on the Exchange 2 server. If you do not have this it will not work.

You will also need to make sure that you have actually set the Exchange 2 server as a copy for the mailboxes on Exchange 1. See below screenshot...
Adding Mailbox Copy to Another server in the DAG.
screenshot add database
Will.
0
Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

 

Author Comment

by:cdeblois
ID: 39776496
Hi Will.  Both of these servers are identical.  They are vitual machines and the exchange logs and DB are both situated on the C Drive in the default location.  The DAG contains both servers and when I ran test-replicationhealth | fl I got the following:
[PS] C:\Windows\system32>test-replicationhealth | ft

Server          Check                      Result     Error
------          -----                      ------     -----
CHVMEXCHANGE1   ClusterService             Passed
CHVMEXCHANGE1   ReplayService              Passed
CHVMEXCHANGE1   ActiveManager              Passed
CHVMEXCHANGE1   TasksRpcListener           Passed
CHVMEXCHANGE1   TcpListener                Passed
CHVMEXCHANGE1   ServerLocatorService       Passed
CHVMEXCHANGE1   DagMembersUp               Passed
CHVMEXCHANGE1   ClusterNetwork             Passed
CHVMEXCHANGE1   QuorumGroup                Passed
CHVMEXCHANGE1   FileShareQuorum            Passed
CHVMEXCHANGE1   DatabaseRedundancy         Passed
CHVMEXCHANGE1   DatabaseAvailability       Passed
CHVMEXCHANGE1   DBCopySuspended            Passed
CHVMEXCHANGE1   DBCopyFailed               Passed
CHVMEXCHANGE1   DBInitializing             Passed
CHVMEXCHANGE1   DBDisconnected             Passed
CHVMEXCHANGE1   DBLogCopyKeepingUp         Passed
CHVMEXCHANGE1   DBLogReplayKeepingUp       Passed
0
 
LVL 53

Assisted Solution

by:Will Szymkowski
Will Szymkowski earned 500 total points
ID: 39776665
Run the following command

Get-DatabaseAvailibilityGroup -status | fl

Under Operational Servers do you see all of the servers?

Then run get-mailboxdatabase (verify the server your DAG is currently on)

Then run move-ActiveMailboxDatabase <DAG Name> -ActivateOnServer <DAG Member>

See if it fails over this way.

Will.
0
 

Author Comment

by:cdeblois
ID: 39776748
On Get-DatabaseAvailabilityGroup -Status |fl I get:
RunspaceId                             : 213d11e1-9cb8-480c-baab-b093673ce055
Name                                   : ExchDAG
Servers                                : {CHVMEXCHANGE2, CHVMEXCHANGE1}
WitnessServer                          : sapserver.commlogik.local
WitnessDirectory                       : C:\DAGFileShareWitnesses\ExchDAG.commlogik.local
AlternateWitnessServer                 :
AlternateWitnessDirectory              :
NetworkCompression                     : InterSubnetOnly
NetworkEncryption                      : InterSubnetOnly
ManualDagNetworkConfiguration          : False
DatacenterActivationMode               : DagOnly
StoppedMailboxServers                  : {}
StartedMailboxServers                  : {CHVMExchange1.commlogik.local, CHVMExchange2.commlogik.local}
DatabaseAvailabilityGroupIpv4Addresses : {192.168.112.86}
DatabaseAvailabilityGroupIpAddresses   : {192.168.112.86}
AllowCrossSiteRpcClientAccess          : False
OperationalServers                     : {CHVMEXCHANGE1, CHVMEXCHANGE2}
PrimaryActiveManager                   : CHVMEXCHANGE2
ServersInMaintenance                   : {}
ServersInDeferredRecovery              : {}
ThirdPartyReplication                  : Disabled
ReplicationPort                        : 64327
NetworkNames                           : {MapiDagNetwork, ReplicationDagNetwork01}
WitnessShareInUse                      : Primary
DatabaseAvailabilityGroupConfiguration :
AutoDagSchemaVersion                   : 1.0
AutoDagDatabaseCopiesPerDatabase       : 1
AutoDagDatabaseCopiesPerVolume         : 1
AutoDagTotalNumberOfDatabases          : 0
AutoDagTotalNumberOfServers            : 0
AutoDagDatabasesRootFolderPath         : C:\ExchangeDatabases
AutoDagVolumesRootFolderPath           : C:\ExchangeVolumes
AutoDagAllServersInstalled             : False
AutoDagAutoReseedEnabled               : True
AutoDagDiskReclaimerEnabled            : True
ReplayLagManagerEnabled                : False
AdminDisplayName                       :
ExchangeVersion                        : 0.10 (14.0.100.0)
DistinguishedName                      : CN=ExchDAG,CN=Database Availability Groups,CN=Exchange Administrative Group
                                         (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=Commlogik,CN=Microsoft
                                         Exchange,CN=Services,CN=Configuration,DC=commlogik,DC=local
Identity                               : ExchDAG
Guid                                   : b67e4a1b-eee7-40c4-bf08-37a17d04c8e1
ObjectCategory                         : commlogik.local/Configuration/Schema/ms-Exch-MDB-Availability-Group
ObjectClass                            : {top, msExchMDBAvailabilityGroup}
WhenChanged                            : 1/12/2014 8:16:28 PM
WhenCreated                            : 1/2/2014 12:53:21 PM
WhenChangedUTC                         : 1/13/2014 1:16:28 AM
WhenCreatedUTC                         : 1/2/2014 5:53:21 PM
OrganizationId                         :
OriginatingServer                      : HERA.commlogik.local
IsValid                                : True
ObjectState                            : Unchanged

On Get-mailboxdatabase I get:
Name                           Server          Recovery        ReplicationType
----                           ------          --------        ---------------
MDB-2013_Corp                  CHVMEXCHANGE1   False           Remote
MDB-2013_VIP                   CHVMEXCHANGE1   False           Remote
MDB-2013_Col                   CHVMEXCHANGE1   False           Remote
MDB-2013_Mex                   CHVMEXCHANGE1   False           Remote
MDB-2013_Ecu                   CHVMEXCHANGE1   False           Remote
MDB_2013_Arg                   CHVMEXCHANGE1   False           Remote
Mailbox Database 0651537422    CHVMEXCHANGE2   False           Remote

All servers appear in the first.  I can't test the move at this moment as these are production servers and I don't want to stop users from working.
0
 
LVL 53

Assisted Solution

by:Will Szymkowski
Will Szymkowski earned 500 total points
ID: 39776780
Everything looks fine from what i can see. What I would recommend is creating a test Database or perform the move off hours.

Let me know if you run into any issues.

Will.
0
 

Author Comment

by:cdeblois
ID: 39776850
Thank you Will I will keep you informed.  Thank you!!
0
 

Author Comment

by:cdeblois
ID: 39783619
Hi Will sorry for the delay in responding.  I wound up calling Microsoft on this issue and after several ours of troubleshooting they found that the network had a DC that was improperly removed and it was causing the replication errors and the failover errors as a result.  Once the Dc was removed properly, I updated Exchange 2 to CU2 so that both servers were running the same version.  It was at this point that failover was successful.  Thank you so much for your help on this issue :)
0
 
LVL 53

Expert Comment

by:Will Szymkowski
ID: 39783629
Perfect, gald to have helped. In the mean time you can use the above commands to perform all of your testing/verification of services.

Will.
0
 

Author Comment

by:cdeblois
ID: 39783637
LOl Trust me I have an excel sheet loaded with all those goodies and some I got from the microsoft call as well :)  Again thank you!
0

Featured Post

Wish Marketing would stop bothering you?

Is your marketing department constantly asking for new email signature updates? Are they requesting a different design for every department? Do they need yet another banner added? Don’t let it get you down! There is an easy way to manage all of these requests...

Join & Write a Comment

Suggested Solutions

Find out how to use Active Directory data for email signature management in Microsoft Exchange and Office 365.
Not sure what the best email signature size is? Are you worried about email signature image size? Follow this best practice guide.
To show how to create a transport rule in Exchange 2013. We show this process by using the Exchange Admin Center. Log into Exchange Admin Center.: First we need to log into the Exchange Admin Center. Navigate to the Mail Flow >> Rules tab.:  To cr…
The basic steps you have just learned will be implemented in this video. The basic steps are shown to configure an Exchange DAG in a live working Exchange Server Environment and manage the same (Exchange Server 2010 Software is used in a Windows Ser…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now