AD replication health degraded after removal of last DC in child domain

About 4 weeks ago we demoted the last DC in one of our child domains. Everything went well, no errors occured during the demotion processand we thought everything was taken care of. Recently we created an AD notification job taht watches AD for us. Since then we're randomly started recieving the follow error on all of our remaing domain controllers:

AD replication health degraded: Directory partition DC=***,DC=net
 
Time of Event:        4/4/2011 2:04:00 PM
 
Source Machine Name:  *****-DC02
 
Event Severity:       10
 
KS Name:              6470:AD_ServerHealth
 
Detail Message:       Partition: DC=***,DC=net  
   
Replication Status  
------------------  
Status: There is no replication partner for this directory partition.  
Severity: 10

I've used the NTDSUtil thinking maybe I needed to remove orphaned objects from AD but the it only shows our two remaining domain, the empty root forest and the final remaining child domain. Any ideas as to why our remaining DCs are still looking for the directory partition in a domain that doesn't exist anymore?
ewazochaAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

ewazochaAuthor Commented:
I've checked and the demoted child domain is not existent in the DNS zones.
snusgubbenCommented:
Can you provide a dcdiag?

dcdiag /v /c /f:dcdiag.txt

(open in notepad and use Search and Replace if you want to hide your domain name)
Creating Active Directory Users from a Text File

If your organization has a need to mass-create AD user accounts, watch this video to see how its done without the need for scripting or other unnecessary complexities.

ewazochaAuthor Commented:
Sure, the dcdiag is attached.
dcdiag.txt
snusgubbenCommented:
The search and replace messed up the formatting so it was a little har to read the lines...

Can you run this and post the output?

repadmin /replsum /bysrc /bydest /sort:delta

(it will show you the status of the AD replication forest wide)
ewazochaAuthor Commented:
Source DSA          largest delta    fails/total %%   error
 ******-DC03               55m:15s    0 /  23    0
 ******-DC05               55m:15s    0 /  16    0
 ***NCT-DC02            55m:15s    0 /  43    0
 ***N-DC05                 55m:15s    0 /  14    0
 ***NBR-DC02            54m:40s    0 /  12    0
 ***NBR-DC01            53m:43s    0 /   6    0
 ***NSW-DC05            53m:18s    0 /  11    0
 ***NCT-DC01             52m:46s    0 /  67    0
 ******-DC04                50m:12s    0 /  16    0
 ***NJP-DC03             47m:12s    0 /   6    0
 ***NRI-DC01              46m:30s    0 /   6    0
 ***NJP-DC04             45m:44s    0 /  11    0
 ***NRI-DC03              45m:17s    0 /  11    0
 ***NSW-DC04            44m:29s    0 /   6    0
 ***NAU-DC03             43m:05s    0 /  11    0
 ***NAU-DC02             42m:44s    0 /   6    0
 ***NFR-DC01             42m:23s    0 /   9    0
 ***NFR-DC03             41m:23s    0 /   3    0
 ***NCA-DC02             40m:14s    0 /   6    0
 ***NES-DC01             37m:45s    0 /   5    0
 ***NDE-DC01             37m:40s    0 /   5    0
 ***NUK-DC02             37m:39s    0 /   5    0
 ***NIT-DC01               37m:36s    0 /   5    0
 ***NMA-DC01             30m:03s    0 /   6    0


Destination DSA     largest delta    fails/total %%   error
 ***NCT-DC01             55m:29s    0 /  31    0
 ***NBR-DC01            55m:11s    0 /   6    0
 ***NBR-DC02            54m:17s    0 /  12    0
 ***NSW-DC04           53m:47s    0 /   6    0
 ***N-DC05                 52m:49s    0 /  53    0
 ******-DC03               50m:29s    0 /  24    0
 ******-DC04               50m:15s    0 /  16    0
 ******-DC05               48m:20s    0 /  16    0
 ***NJP-DC04            47m:32s    0 /   6    0
 ***NRI-DC03             47m:06s    0 /   6    0
 ***NJP-DC03            46m:07s    0 /  12    0
 ***NCT-DC02            45m:21s    0 /  31    0
 ***NRI-DC01             45m:20s    0 /  12    0
 ***NSW-DC05           44m:55s    0 /  12    0
 ***NAU-DC02            43m:49s    0 /   6    0
 ***NAU-DC03            43m:23s    0 /  12    0
 ***NFR-DC03            42m:30s    0 /   3    0
 ***NFR-DC01            41m:27s    0 /   9    0
 ***NIT-DC01              41m:09s    0 /   6    0
 ***NES-DC01            37m:24s    0 /   6    0
 ***NCA-DC02            35m:33s    0 /   6    0
 ***NMA-DC01            32m:11s    0 /   6    0
 ***NDE-DC01             29m:05s    0 /   6    0
 ***NUK-DC02             28m:03s    0 /   6    0
snusgubbenCommented:
All NCs are replicated with no errors forest wide. Largest Delta is under 60 minutes, so your replication is ok.

If there was an inbound partner trying to replicate an orphan NC you would have seen it in the output above.

You mentioned a "AD notification job". What sort of job is this?
ewazochaAuthor Commented:
This is a job that is run by NetIQ application, it monitors our AD for errors among other things. I'm beginning to believe that the NetIQ job itself is looking for the missing directory partition and not the domain controllers themselves. I can't really find anything related to the errors I'm seeing from NetIQ in any of the domain controller logs, the only thing that I can find is some information about some objects that are still in AD that should be removed at the next tombstone interval.

I need to have a look at the NetIQ job and see if that's what's going on, once I can confirm this I"ll report back here and assign points for the help.
ewazochaAuthor Commented:
I found the issue.

http://support.microsoft.com/kb/2413670

"This problem occurs because the domain controller in the Active Directory cannot delete the partition for a naming context that is removed from the forest."

So what it looks like is the NetIQ job that runs is still seeing the domain partition that was removed from the forest. It's only our Windows Server 2008 R2 machines that are reporting the problem via NetIQ. The article above describes a hotfix for this issue but the hotfix has been superseded by SP1 for Server 2008. Looks like I'll be scheduling the install of SP1 on our DCs.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
ewazochaAuthor Commented:
Found answer on my own.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Active Directory

From novice to tech pro — start learning today.