Corrupted domain controller replication

Hey experts;

I'm running a Win2k3 server as a global catalog DC for our company that also runs Exchange. We had an issue with Exchange/AD corruption a few weeks ago and I had to restore the system state using NTBackup. Somehow, AD didn't fully cooperate with me, and so I then demoted the DC using dcpromo and reinstalled it from another global catalog DC in a remote office.

The problem is that now, the restored DC is giving me continual replication warnings that it cannot replicate with the corrupted, previous version of itself that it thinks is still active. In AD Sites and Services, my DC has two "NTDS Settings"....one for my secondary DC, and one named "NTDS Settings
CNF:1569a4a2-45b2-47cb-8038-06c947043d64", which is the corrupt version of itself.

I've attempted to delete this setting, but it will not go away. I've used ntdsutil.exe to try to clean up my DC's metadata, and when I try to delete the bad version, I get an error: The connected server will not remove its own metadata.

Any thoughts on how to fix this?
lamontcAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

rehanahmedsCommented:
0
rehanahmedsCommented:
dcpromo /forceremoval
0
lamontcAuthor Commented:
I appreciate the advice, but I've already tried that stuff. When I get to the step 14 on http://www.petri.co.il/delete_failed_dcs_from_ad.htm, that's when I get the error: The connected server will not remove its own metadata.  I can't proceed any further once that comes up.

The server was corrupted when it was demoted and then promoted again. Some kind of way it held onto it's own data, and I can't get rid of it now.
0
Has Powershell sent you back into the Stone Age?

If managing Active Directory using Windows Powershell® is making you feel like you stepped back in time, you are not alone.  For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why.

ChiefITCommented:
How many DC's do you have on the domain?
0
lamontcAuthor Commented:
I've got two. Both are listed as global catalogs.
0
ChiefITCommented:
who holds the FSMO roles?
0
lamontcAuthor Commented:
The domain controller that is currently having problems replicating with it's prior self.
0
ChiefITCommented:
LOL:

Have you tried to seize the roles on the other DC, then demote it?
0
lamontcAuthor Commented:
Yes. I've tried seizing the roles with ntdsutil...the problem is that the corrupted DC still shows up in AD. It's really, really weird. Doesn't seem to be causing any issues at present, but I'm concerned about the growth of the domain. If our personnel keeps growing and we've got a crappy AD foundation (with the corrupted DC), it might cause a real headache down the line.
0
ChiefITCommented:
It's probably a good idea to get one DC up and work on the other separtely.

1) OK, you might have to disconnect the problem child DC's NIC and seize the roles with the other. The reason you are not transfering the roles is probably because your DCs see each other. So, you may have to disconnect one and seize with the other.

How to view and transfer roles:
http://support.microsoft.com/kb/324801

There is a difference between seizing and transfering roles:
http://support.microsoft.com/kb/255504

______________________________________________________
2) Then, It sounds like you will have metadata on both DCs. So, we should first remove metadata from your good DC that should currently hold the roles. Remove any instances of that DC that has been disconnected.

The combination of the two "DCdiag and NTDSUTIL" is the method that most everyone uses to remove metadata from a improperly demoted domain controller, or other forms of metadata in AD. There are a couple articles you might want to read when doing this.

1) Phantom, Tombstone, and the AD infrastructure master. (explains the four stages of a Deleted SID).
http://support.microsoft.com/kb/248047

2) How to remove metadata from AD:(Use of the NTDSUTIL)
 http://support.microsoft.com/kb/230306
or preferably,
http://www.petri.co.il/fix_unsuccessful_demotion.htm

I think what I would do in your situation is demote that DC to a stand alone server. That will remove the AD database, that's corrupt. Then, look for and remove all metadata or tombstoned objects after demoting the machine. Once you have removed all metadata and AD, you can then promoted it back into the domain as a domain controller and replicate to this server.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
ChiefITCommented:
rehanahmeds offered some information that I also provided. Sorry about the redundancy. Please credit him for his input if this resolves your problem.
0
lamontcAuthor Commented:
You guys are awesome. I installed a virtual server on the same LAN as the corrupt DC and transferred over all roles. I then demoted the former DC to a member server and cleaned up the AD metadata on the new primary DC. After that, I just reversed the procedure (re-promoted the old DC back to primary) and voila, no more corrupt replication. It took me longer than I wanted to because I was running Exchange 03 on the primary DC and every time I rebooted it would take FOREVER to load, since Exchange couldn't see AD the way it wanted to.

But it's all resolved now. Thanks a million, guys!
0
jaesoulCommented:
I am curious, Why not just demote the server, and re build the demoted server? Why take a risk on anything being unclean on the server?
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Active Directory

From novice to tech pro — start learning today.