Failed DC and Can't Clean AD - Getting DsBindW Error in NTDSUtil
I've recently had a DC fail (ServerDC01) for reasons I will not go into here. Needless to say, I learned something and things continue to work for the most part (not sure for how long). The remaining DC (ServerDC03) is naturally having issues because it's partner, and the majority FSMO role-holder, is not responding/playing-ball.
While I could still access ServerDC01 I attempted to demote it during which the demotion was interrupted. The one thing I didn't try was dcpromo /forceremoval (...just didn't think of it at the time. Lesson #2). DC01 is now no longer responding in anyway so I have turned it off. No, even after multiple reboots it is completely inaccessible except to Ping.
Now I am trying to cleanup AD of any reference to DC01 before I rebuild it and seize the FSMO Roles onto DC03. My problem is that I can't do either because I am unable to connect to DC03 using NTDSUtil. Nothing works when I try to 'Connect to Server' using 'localhost', the FQDN, NetBIOS Name or IP (...I know, I know,...but I figured I'd try anyway to be thorough).
The error I get is:
"DsBindW error 0x6d9(There are no more endpoints available from the endpoint mapper.)"
I do NOT want to rebuild this Domain unless I absolutely MUST. Naturally, however, I MUST get this situation corrected before it falls over onto itself one way or another.
I have already read-through Microsoft's RPC KB which I found close to useless as I KNOW the error I'm getting I just don't know how to fix the RPC issue in order to move forward.
Can someone PLEASE tell me how to fix this?!?!? I am just beginning to pour-through the Event Logs in the hopes that I come across something useful but that could take over a week. I figured I would send up this flare as I continue to investigate.
Thanks in Advance.
While I could still access ServerDC01 I attempted to demote it during which the demotion was interrupted. The one thing I didn't try was dcpromo /forceremoval (...just didn't think of it at the time. Lesson #2). DC01 is now no longer responding in anyway so I have turned it off. No, even after multiple reboots it is completely inaccessible except to Ping.
Now I am trying to cleanup AD of any reference to DC01 before I rebuild it and seize the FSMO Roles onto DC03. My problem is that I can't do either because I am unable to connect to DC03 using NTDSUtil. Nothing works when I try to 'Connect to Server' using 'localhost', the FQDN, NetBIOS Name or IP (...I know, I know,...but I figured I'd try anyway to be thorough).
The error I get is:
"DsBindW error 0x6d9(There are no more endpoints available from the endpoint mapper.)"
I do NOT want to rebuild this Domain unless I absolutely MUST. Naturally, however, I MUST get this situation corrected before it falls over onto itself one way or another.
I have already read-through Microsoft's RPC KB which I found close to useless as I KNOW the error I'm getting I just don't know how to fix the RPC issue in order to move forward.
Can someone PLEASE tell me how to fix this?!?!? I am just beginning to pour-through the Event Logs in the hopes that I come across something useful but that could take over a week. I figured I would send up this flare as I continue to investigate.
Thanks in Advance.
See first of all you need to seize the FSMO Roles onto DC03.
Refer to seize roles: http://www.petri.co.il/seizing_fsmo_roles.htm
Then you should try dcpromo /forceremoval on DC01.
Then you need to perform metadacleanup on DC03.
Refer this to perform metadate:http://www.petri.co.il/delete_failed_dcs_from_ad.htm
Note: All the time you need to connect to DC03 domain controller with ntdsutil, not to the rogue machine.
Once everything is in place then you can take DC01 inside the network but you said that it is completely inaccessible except to Ping then I think you need to rebuild and promote it back.
Refer to seize roles: http://www.petri.co.il/seizing_fsmo_roles.htm
Then you should try dcpromo /forceremoval on DC01.
Then you need to perform metadacleanup on DC03.
Refer this to perform metadate:http://www.petri.co.il/delete_failed_dcs_from_ad.htm
Note: All the time you need to connect to DC03 domain controller with ntdsutil, not to the rogue machine.
Once everything is in place then you can take DC01 inside the network but you said that it is completely inaccessible except to Ping then I think you need to rebuild and promote it back.
can you run a dcdiag /v (verbose) on the dc03.
where is the Dc03 pointing for DNS and when was the last successful replication with its partner.
also is the DC03 a Global catalog ?
where is the Dc03 pointing for DNS and when was the last successful replication with its partner.
also is the DC03 a Global catalog ?
ASKER
Ok.
ABHIJ... I Can not sieze the FSMO or clean AD because "My problem is that I can't do either because I am unable to connect to DC03 using NTDSUtil. Nothing works when I try to 'Connect to Server' using 'localhost', the FQDN, NetBIOS Name or IP (...I know, I know,...but I figured I'd try anyway to be thorough)."
NipponSoul - I will try ReplAdmin on DC03 in the morning but I fear that with DC01 down I will not get anything useful. But I am certainly willing to try.
Kprad - ....you know, I didn't think to use the verbose switch when I initially ran DCDiag. I will run this in the morning as well.
...my morning begins at 03:30 EST (-5:00GMT).
ABHIJ... I Can not sieze the FSMO or clean AD because "My problem is that I can't do either because I am unable to connect to DC03 using NTDSUtil. Nothing works when I try to 'Connect to Server' using 'localhost', the FQDN, NetBIOS Name or IP (...I know, I know,...but I figured I'd try anyway to be thorough)."
NipponSoul - I will try ReplAdmin on DC03 in the morning but I fear that with DC01 down I will not get anything useful. But I am certainly willing to try.
Kprad - ....you know, I didn't think to use the verbose switch when I initially ran DCDiag. I will run this in the morning as well.
...my morning begins at 03:30 EST (-5:00GMT).
ASKER
UPDATE:
I was able to get DC01 up and running long enough to attempt another DCPromo which didn't work. I then tried DCPromo /forceremoval which did.
Question: Can I rebuild DC01, add it to the Domain and then re-promote it? Can it be that simple?
In hindsight I realize that I would possibly have been better served by diagnosing any replication issues before trying to get DC01 gracefully demoted...but that ship has sailed.
I will attempt to connect to DC03 again tomorrow morning using ntdsutil. I do not expect a different result as, having done a forced removal of DC01, DC03 will still be expecting to play-ball but wont be able to.
I was able to get DC01 up and running long enough to attempt another DCPromo which didn't work. I then tried DCPromo /forceremoval which did.
Question: Can I rebuild DC01, add it to the Domain and then re-promote it? Can it be that simple?
In hindsight I realize that I would possibly have been better served by diagnosing any replication issues before trying to get DC01 gracefully demoted...but that ship has sailed.
I will attempt to connect to DC03 again tomorrow morning using ntdsutil. I do not expect a different result as, having done a forced removal of DC01, DC03 will still be expecting to play-ball but wont be able to.
You can rebuild the DC01 and promote it as a DC but before that perform metadata cleanup to remove existing entries of DC01.
Use above provided links to perform these steps.
Use above provided links to perform these steps.
ASKER
Following the material above has still not allowed me to connect to the remaining DC in order to clean the metadata.
...one thing I've noticed:
I have a NAS which uses AD authentication. In order to maintain the connection I use a service account. The other day I went to get a few files from the NAS and I was prompted to authenticate which is new behavior. I tried other shares on the NAS which had the same result. Logging into the NAS management console I found that the NAS was getting RPC errors in attempting to establish a connection to the Domain.
This prompted me to RDP to the remaining DC to discover that I had left it in DSRM overnight.
Is RPC disabled normally when in DSRM? I wouldn't think so or else how would you be able to connect to ANY DC using NTDSUtil, Right? Once I rebooted the DC (DC03) back into normal mode the NAS connected just fine.
Is it possible that I have a problem with DSRM on DC03?
...one thing I've noticed:
I have a NAS which uses AD authentication. In order to maintain the connection I use a service account. The other day I went to get a few files from the NAS and I was prompted to authenticate which is new behavior. I tried other shares on the NAS which had the same result. Logging into the NAS management console I found that the NAS was getting RPC errors in attempting to establish a connection to the Domain.
This prompted me to RDP to the remaining DC to discover that I had left it in DSRM overnight.
Is RPC disabled normally when in DSRM? I wouldn't think so or else how would you be able to connect to ANY DC using NTDSUtil, Right? Once I rebooted the DC (DC03) back into normal mode the NAS connected just fine.
Is it possible that I have a problem with DSRM on DC03?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
OK. Yeah, flaked on that. I guess the crux of my statement was "maybe something is wrong with RPC in DSRM which is preventing me from connecting to it in order to use ntdsutil".
I'm really surprised that this isn't an easier issue. I mean, sure, I haven't come up against serious AD issues more than a few times in the last 10+ years but seriously.....
I'm really surprised that this isn't an easier issue. I mean, sure, I haven't come up against serious AD issues more than a few times in the last 10+ years but seriously.....
ASKER
I figured it out.
My problem was in trying to connect to the healthy DC while it was in DSRM. To successfully use NTDSUtil I needed to run it from a different computer and THEN connect to the healthy DC. Once I did that, removing the dead DC was a breeze.
Since I was reinstalling the failed DC, same name & IP, I for-went the additional Steps.
So far things seem to be running smoothly and I will continue to monitor the Event Logs on BOTH DCs.
I will award points in the morning for the Statement "there is no AD in DSRM".
...another question this brings-up: "What's the point of DSRM if you cannot make any changes to AD?"
My problem was in trying to connect to the healthy DC while it was in DSRM. To successfully use NTDSUtil I needed to run it from a different computer and THEN connect to the healthy DC. Once I did that, removing the dead DC was a breeze.
Since I was reinstalling the failed DC, same name & IP, I for-went the additional Steps.
So far things seem to be running smoothly and I will continue to monitor the Event Logs on BOTH DCs.
I will award points in the morning for the Statement "there is no AD in DSRM".
...another question this brings-up: "What's the point of DSRM if you cannot make any changes to AD?"
ASKER
My initial question was why I was unable to connect to the healthy DC while in DSRM (I THINK I pointed that out).
While I figured it out the solution was ultimately VERY simple as I needed to use NTDSUtil from a different computer.
While I figured it out the solution was ultimately VERY simple as I needed to use NTDSUtil from a different computer.
http://forums.techarena.in/active-directory/758034.htm