DAG Failed

Posted on 2013-12-06
Last Modified: 2013-12-20
We were facing problem to take exchange bakup for last couple of days, for that logged a case to backup vendor, and as per their suggestion, we had to reboot one of the DAG member server sitting in the secondary site. We put that server in maintenace mode, and performed reboot, and after reboot put that server out of maintenance mode. Found that every DB copy across the DAG is health state, but found cluster name DAGBL become failed,after opening failover cluster I have found that one of the IP address become offline, while i tried to bring it online receiving the following error.
Error "The operation has failed" An error occured while trying to bring the resource "IPV4 static address 1(Cluster Group) online. While i clicked details found the error code: Ox80071397.
The operation failed because either the specified cluster node is not the owner of the resource or the node is not a possible owner of the resource.

I have checked failover cluster properties that from for possbile owner four servers are selected like server01, server02,server03,server04

i am using exchannge 2010 rollup2, SP3, & primary site server01, server02 member of DAG, secondary site: server03 is member of DAG, server 04 is holding only mailbox role but not in DAG.

So far i have chcked FSW is online and Cluster netwrork is showing up and online for server01, server02,server03,server04. and all DBs are in healthy state across the DAG. In this situation, can you please advice what can be done to resolve the issue.
Question by:ipsec600
  • 3
  • 3
LVL 36

Expert Comment

ID: 39702804
Please correct me if my understanding  is wrong..
You mean to say, that server4 is got removed from DAG automatically?
Can you run below shell command to verify
Get-DatabaseAvailabilityGroup -Identity DAGNAME | Format-List Servers

If that's the case I don't think there is any option other than forcefully removing passive copies if any on the server4 and add server again.

Your DAG will still remain online due to voters majority

LVL 19

Expert Comment

by:Adam Farage
ID: 39703921
Verify that all roles are out of maintenance, and if they are or you cannot get them out of maintenance try moving the primary active manager to a different node.

You can run the following to find out which role has the PAM (primary active manager) role, which technically is the cluster owner:

Get-DatabaseAvailabilityGroup -Status | Select Name, PrimaryActiveManager

Open in new window

Once you do that, simply move the PAM role to a different node in your primary datacenter:

Cluster.exe "DAGName" /moveto:newnode

Open in new window

Give that a shot and let us know how you make out.

Author Comment

ID: 39706359
Hello Mahesh, thank you for your post, actually I have been facing issues with my DAG, which is becoming offline, but server04 is ok.

Hello evrydayzawrkday, thank you for your advice, please find the below status now:

While I open failover cluster, I found that DAGBL(DAG name) was in offline state while I opened this post, in the DAG there is two interface, one is showing online, and he other one is showing offline, and while I tried to bring online the sencond interface(offline) receiving the below error:

Error "The operation has failed" An error occured while trying to bring the resource "IPV4 static address 1(Cluster Group) online. While i clicked details found the error code: Ox80071397.
The operation failed because either the specified cluster node is not the owner of the resource or the node is not a possible owner of the resource.

I have found the followings during while I was receiving error:
For DAGBL(DAG name), PAM was server01 which is from primary site.

Then we gracefully rebooted all the exchange servers and found the followings:
DAGBL is offline but after 2 to 3 hrs it automatically become online, but PAM is from secondary site which is server04.

Then I run the Validate this cluster from failvoer cluster console and found the following warning message for a cluster resouce IP address which is basically the interface which is offiline:

This resource is marked with a state of 'Offline'. The functionality that this resource provides is not available while it is in the offline state. The resource may be put in this state by an administrator or program. It may also be a newly created resource which has not been put in the online state or the resource may be dependent on a resource that is not online. Resources can be brought online by choosing the 'Bring this resource online' action in Failover Cluster Manager.

In sumamry, after rebooting all the exchange severs, DAGBL(DAG name) is online but one of the interface is still offline, and receving the above warning message. I am not sure is it by design, because in DAG, out of  two interfaces, one inteface is active & other interface is offline. At this moment, all the Database staus is healty from all DAG members servers.
Also note that while PAM was in server01(primary site) then interface02 was offline from DAG, but interface01 was online. Now while PAM is in server04(secondary site) then interface02 online, but interface01 is offline , meaning that DAG interfaces are automatically switching over, and becoming offline/Online based on PAM. In this situation, What can be done to bring that interface online, please advice.
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

LVL 36

Expert Comment

ID: 39706817
Which network interface are going down ? on replication network or Mapi network?
If you could please share any screen shot, it will be great ?

If you move PAM role across servers, does one interface is going offline in case of all servers ?

You are using two network cards per DAG member, right?
How many network cards do you have per server ?
Are the all DAG members are virtualized ?


Author Comment

ID: 39720758
Hi Mahesh, sorry for replying late due to vacation, Please find the below details:

All the DAG members are virtualized, and using one interface, for that both Mapi and replication network is using only one interface.

I will move PAM role across servers, to observer interface status, and update you. Would you please clarify the below: when I run the failover cluster dependancy report, receivig the below output:

Cluster:       DAGBL
Resource:       Name: DAGBL
Started                     12/15/2013 11:18:08 PM
Completed      12/15/2013 11:18:08 PM
'IP Address:' has no required dependencies.
'IP Address:' has no required dependencies.
'Name: DAGBL' dependencies are 'IP Address:' or 'IP Address:'.
'Cluster Name' required dependencies are IP Address.

represents 'AND' relationship: all child resources must be on-line

represents 'OR' relationship: at least one child resource must be on-line

I have noticed that in the DAG properties "OR" relationship is defined, does it mean that only one interface will remain online between the two interfaces, if it is so then in my case DAG is behaving as usual otherwise, I will have work on it to bring two interfaces online.
LVL 36

Accepted Solution

Mahesh earned 500 total points
ID: 39727408
The report simply states that "DAGBL" is dependent on IP addresses.
IP addresses are primary resource, hence they are independent. As long as all IP addresses are alive, DAGBL name resource should remain online.
You can set "OR" relationship there in dependencies

Not sure why you have used public IP range for DAG, or you are just using it as internal network segment ?
Personally, I recommend that you could use TWO network cards for each DAG node, one for Mapi network and one for DAG replication

If the replication network fails, the DAG will fail back to the MAPI network for replication even if the MAPI network has it’s ReplicationEnabled property set to False
In the event of a failure affecting the MAPI network, a server failover will occur (assuming there are healthy mailbox database copies that can be activated).

Although a single network is supported, we recommend that each DAG have at least two networks: a single MAPI network and a single Replication network. This provides redundancy for the network and the network path, and enables the system to distinguish between a server failure and a network failure. Using a single network adapter prevents the system from distinguishing between these two types of failures.

Lastly you can check below post in same site for more information


Author Comment

ID: 39731902
Hi Mahesh, thanks a lot for your excellent clarification, according to blog " You can set "OR" relationship there in dependencies", my DAG environment is behaving normally, that's what I was looking for. and for the recommendation, sure, I will proceed to implement two network card for DAG considering redundancy.

Featured Post

Does Powershell have you tied up in knots?

Managing Active Directory does not always have to be complicated.  If you are spending more time trying instead of doing, then it's time to look at something else. For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
How to setup exchange federation trust? 4 36
Schedule Outlook Calendar 5 33
Disable email comparison in lync 2010 2 20
Using isinteg or eseutil with Exchange 2010 6 16
Following basic email etiquette rules will help you write a professional email and achieve a good, lasting impression with your contacts.
Scam emails are a huge burden for many businesses. Spotting one is not always easy. Follow our tips to identify if an email you receive is a scam.
In this video we show how to create a Shared Mailbox in Exchange 2013. We show this process by using the Exchange Admin Center. Log into Exchange Admin Center.: First we need to log into the Exchange Admin Center. Navigate to the Recipients >> Sha…
In this video we show how to create an Accepted Domain in Exchange 2013. We show this process by using the Exchange Admin Center. Log into Exchange Admin Center.: First we need to log into the Exchange Admin Center. Navigate to the Mail Flow >> Ac…

820 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question