Exchange 2007 CCR Cluster
We have a CCR Cluster. Our System Attendant Service is not starting up after a fail-over. We receive a 1069 Event ID in EventVwr.
Tonight we did our monthly failover for monthly Windows Patches. Everything fails over and starts except the Exchange System Attendant. We have set the timeout to 150 seconds, no go. We tried to fail back over and still would not start. WE tried failing BACK over again, no go.
Below is the error/Screenshot in the Event Viewer we get.
Event-ID-1069-System-Attendant.docx
Here is the environment:
Exchange 2007, SP1, CCR cluster
Running on Dell PE 2950's, Server 2003 Ent x64, 32 GB RAM
WE have 4 physical servers (Box 1 and 2 = virtual cluster 1) & (Box 3 and 4 = virtual cluster 2)
Attached to 6 Dell MD1100 DAS's (for storage - 3 DAS's for each cluster)
DC's = Server 2003 x64 Ent
*Note-We also failed over cluster 2 tonight and it came up ok, no issues. The issue is with Cluster 1 (above)
The other odd part is we can send emails. Me and the other admin working on this both have our mailboxes on the cluster having issues (cluster 1). We can email each other back and forth and send emails externally to our yahoo and gmail and also get replies. It is going to cause issues by morning when we get in though so this needs to be fixed ASAP.
We have already tried this MS KB but no go:
Change the “Pending timeout period” for this service within “Cluster Administrator” to 150 seconds from 180 seconds.
http://support.microsoft.com/kb/945078
We have also tried this Experts Exchange "fix" but at this point we CANNOT create a new Cluster at this time until it is the LAST resort and the CIO gives the OK.
https://www.experts-exchange.com/questions/24176183/Event-id-1069-cluster-resource-'Network-Name-xMailbox-'-in-Resource-Group-'xMailbox'-failed.html
In this same EE article we did try the command line but this did not work:
You can use the Cluster command-line utility to set the "DNS Registration Must Succeed" private property to "Not Required." To do this, follow these steps:
A) Go to Cluster Administrator.
B). Right-click the corresponding Network Name resource, --> Take offline.
C). Open a command prompt on one of the cluster nodes, and then type the following command:
cluster res " network name resource " /priv requiredns=0
D) In Cluster Administrator, right-click the cluster resource--> click Bring Online.
Inside-Cluster-Admin.docx
Need assistance ASAP.
I am tired and going to bed so I will answer all questions starting at 7am Central Time, USA, when I get to work
Tonight we did our monthly failover for monthly Windows Patches. Everything fails over and starts except the Exchange System Attendant. We have set the timeout to 150 seconds, no go. We tried to fail back over and still would not start. WE tried failing BACK over again, no go.
Below is the error/Screenshot in the Event Viewer we get.
Event-ID-1069-System-Attendant.docx
Here is the environment:
Exchange 2007, SP1, CCR cluster
Running on Dell PE 2950's, Server 2003 Ent x64, 32 GB RAM
WE have 4 physical servers (Box 1 and 2 = virtual cluster 1) & (Box 3 and 4 = virtual cluster 2)
Attached to 6 Dell MD1100 DAS's (for storage - 3 DAS's for each cluster)
DC's = Server 2003 x64 Ent
*Note-We also failed over cluster 2 tonight and it came up ok, no issues. The issue is with Cluster 1 (above)
The other odd part is we can send emails. Me and the other admin working on this both have our mailboxes on the cluster having issues (cluster 1). We can email each other back and forth and send emails externally to our yahoo and gmail and also get replies. It is going to cause issues by morning when we get in though so this needs to be fixed ASAP.
We have already tried this MS KB but no go:
Change the “Pending timeout period” for this service within “Cluster Administrator” to 150 seconds from 180 seconds.
http://support.microsoft.com/kb/945078
We have also tried this Experts Exchange "fix" but at this point we CANNOT create a new Cluster at this time until it is the LAST resort and the CIO gives the OK.
https://www.experts-exchange.com/questions/24176183/Event-id-1069-cluster-resource-'Network-Name-xMailbox-'-in-Resource-Group-'xMailbox'-failed.html
In this same EE article we did try the command line but this did not work:
You can use the Cluster command-line utility to set the "DNS Registration Must Succeed" private property to "Not Required." To do this, follow these steps:
A) Go to Cluster Administrator.
B). Right-click the corresponding Network Name resource, --> Take offline.
C). Open a command prompt on one of the cluster nodes, and then type the following command:
cluster res " network name resource " /priv requiredns=0
D) In Cluster Administrator, right-click the cluster resource--> click Bring Online.
Inside-Cluster-Admin.docx
Need assistance ASAP.
I am tired and going to bed so I will answer all questions starting at 7am Central Time, USA, when I get to work
ASKER
Why would you set it to "manual" vs "automatic"? Is this a MS "Best Practice"?
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Hello ggipson79
Fist of all I'd like to thank you for your appreciation, but based on your last comment your question was misleading because
1- you indicated in the beginning that the problem is with cluster one and cluster two is up with no issues without mentioning any problems with the hub.
2- Services on Cluster nodes is not dependent on services on HUB transport servers besides you informed that you can send emails internally and externally and this is different scenario than the one you explained in your final comments.
Finally thanks again for the appreciation and thanks to make your questions more clear next time to ensure better quality of service
Thanks and Regards,
Karim
Fist of all I'd like to thank you for your appreciation, but based on your last comment your question was misleading because
1- you indicated in the beginning that the problem is with cluster one and cluster two is up with no issues without mentioning any problems with the hub.
2- Services on Cluster nodes is not dependent on services on HUB transport servers besides you informed that you can send emails internally and externally and this is different scenario than the one you explained in your final comments.
Finally thanks again for the appreciation and thanks to make your questions more clear next time to ensure better quality of service
Thanks and Regards,
Karim
ASKER
Karim,
My apologies. We didnt even think of the HT servers. It never crossed our minds since we were able to send email from one of the clusters. We had to start from scratch and work our way backward. It was actually another admin that found the problem.
I do appreciate what you put for me to try. Unfortunately that did not fix it. Had I thought it to be the HT servers then I would have put that in.
So, to make a long story short, I did not fix it and neither did you, it was someone else I work with.
Again, thanks for your help.
Ggipson79
My apologies. We didnt even think of the HT servers. It never crossed our minds since we were able to send email from one of the clusters. We had to start from scratch and work our way backward. It was actually another admin that found the problem.
I do appreciate what you put for me to try. Unfortunately that did not fix it. Had I thought it to be the HT servers then I would have put that in.
So, to make a long story short, I did not fix it and neither did you, it was someone else I work with.
Again, thanks for your help.
Ggipson79
Event ID 1069 is a "generic this resource has failed" event, and really does
not give a whole lot of info to work with.
1) look at events prior to this time in both application as system event
logs, to see if this is giving more clues
2) look at the cluster.log file to see if this gives more clues
Please note: the cluster.log timestamps are in GMT, so you need to calculate
your time-offset in order to line it up with your Event ID 1069 time.
Also try to check the resources dependecies for cluster one and you can cluster two as a reference to compare with, also check and compare the service to ensure that required services are set to manual.
If your configuration is correct and there is no other problems you can try your last resort by stopping the cluster service on the active node to force the other node to take the ownership of the system attendtant.
please don't hesitate to refer back with more info by trying some from the above so i can help more.
Thanks and Regards,
Karim