GeeMoon
asked on
AD Replications issues
I have a Windows 2003 R2 (SRV1) and 2008 RS (SRV2). They are both Domain controllers. I added 4 new Windows 7 Pro workstations and noticed that some were not able to log into the security domain due to their computer account. I rebooted the troubled workstations and was back in. Obviously it depended on which server the workstations were trying to log in with. I checked the event logs to discover that AD replications was not happening.
I ran Dcdiag and repadmin. I received a number of errors that have me running in circles.
In order to clear the slate and start from the beginning, I went to each server and pushed an AD replication. I received the following error:
Replicate Now - window from SRV1
The Following error occured during the attempt to
synchronize naming context ABC.Local from domain controller
SRV2 to domain controller SRV1: Insufficient attributes
were given to create an object. This object may not exist
because it may have been deleted and already garbage
collected.
I believe AD replication has been down for a while - months. I don't want this to get worse. Can anybody help on this matter? How do I repair/restore replication? More importantly, ensure the health of my AD.
I will follow up with some of the Dcdiag errors
Thank you
I ran Dcdiag and repadmin. I received a number of errors that have me running in circles.
In order to clear the slate and start from the beginning, I went to each server and pushed an AD replication. I received the following error:
Replicate Now - window from SRV1
The Following error occured during the attempt to
synchronize naming context ABC.Local from domain controller
SRV2 to domain controller SRV1: Insufficient attributes
were given to create an object. This object may not exist
because it may have been deleted and already garbage
collected.
I believe AD replication has been down for a while - months. I don't want this to get worse. Can anybody help on this matter? How do I repair/restore replication? More importantly, ensure the health of my AD.
I will follow up with some of the Dcdiag errors
Thank you
Can you post the output of the following commands:
Also check Event Viewer for errors and post those errors. Thanks.
- repadmin /showrepl
- repadmin /replsummary
- dcdiag /q
- netdom query fsmo
Also check Event Viewer for errors and post those errors. Thanks.
ASKER
repadmin /showrepl
Repadmin: running command /showrepl against full DC localhost
Default-First-Site\SRV2
DSA Options: IS_GC
Site Options: (none)
DSA object GUID: 361d267b-fb93-482c-a3ad-9c 9121e5c425
DSA invocationID: 9a9178af-5a1f-4a84-a3ed-24 07546cb633
==== INBOUND NEIGHBORS ========================== ========== ==
DC=ABC,DC=local
Default-First-Site\SRV1 via RPC
DSA object GUID: fa656f49-32b4-4b6a-a83e-d2 490889898a
Last attempt @ 2016-11-16 18:09:55 was successful.
CN=Configuration,DC=ABC,DC =local
Default-First-Site\SRV1 via RPC
DSA object GUID: fa656f49-32b4-4b6a-a83e-d2 490889898a
Last attempt @ 2016-11-16 17:45:02 was successful.
CN=Schema,CN=Configuration ,DC=ABC,DC =local
Default-First-Site\SRV1 via RPC
DSA object GUID: fa656f49-32b4-4b6a-a83e-d2 490889898a
Last attempt @ 2016-11-16 17:45:02 was successful.
DC=DomainDnsZones,DC=ABC,D C=local
Default-First-Site\SRV1 via RPC
DSA object GUID: fa656f49-32b4-4b6a-a83e-d2 490889898a
Last attempt @ 2016-11-16 17:45:02 was successful.
DC=ForestDnsZones,DC=ABC,D C=local
Default-First-Site\SRV1 via RPC
DSA object GUID: fa656f49-32b4-4b6a-a83e-d2 490889898a
Last attempt @ 2016-11-16 17:45:02 was successful.
Repadmin: running command /showrepl against full DC localhost
Default-First-Site\SRV2
DSA Options: IS_GC
Site Options: (none)
DSA object GUID: 361d267b-fb93-482c-a3ad-9c
DSA invocationID: 9a9178af-5a1f-4a84-a3ed-24
==== INBOUND NEIGHBORS ==========================
DC=ABC,DC=local
Default-First-Site\SRV1 via RPC
DSA object GUID: fa656f49-32b4-4b6a-a83e-d2
Last attempt @ 2016-11-16 18:09:55 was successful.
CN=Configuration,DC=ABC,DC
Default-First-Site\SRV1 via RPC
DSA object GUID: fa656f49-32b4-4b6a-a83e-d2
Last attempt @ 2016-11-16 17:45:02 was successful.
CN=Schema,CN=Configuration
Default-First-Site\SRV1 via RPC
DSA object GUID: fa656f49-32b4-4b6a-a83e-d2
Last attempt @ 2016-11-16 17:45:02 was successful.
DC=DomainDnsZones,DC=ABC,D
Default-First-Site\SRV1 via RPC
DSA object GUID: fa656f49-32b4-4b6a-a83e-d2
Last attempt @ 2016-11-16 17:45:02 was successful.
DC=ForestDnsZones,DC=ABC,D
Default-First-Site\SRV1 via RPC
DSA object GUID: fa656f49-32b4-4b6a-a83e-d2
Last attempt @ 2016-11-16 17:45:02 was successful.
ASKER
repadmin /replsummary
Replication Summary Start Time: 2016-11-16 18:16:13
Beginning data collection for replication summary, this may take awhile:
.....
Source DSA largest delta fails/total %% error
SRV2 07d.05h:00m:29s 1 / 5 20 (8606) Insufficient attribut
es were given to create an object. This object may not exist because it may have
been deleted and already garbage collected.
SRV1 31m:11s 0 / 5 0
Destination DSA largest delta fails/total %% error
SRV2 31m:12s 0 / 5 0
SRV1 07d.05h:00m:29s 1 / 5 20 (8606) Insufficient attribut
es were given to create an object. This object may not exist because it may have
been deleted and already garbage collected.
Replication Summary Start Time: 2016-11-16 18:16:13
Beginning data collection for replication summary, this may take awhile:
.....
Source DSA largest delta fails/total %% error
SRV2 07d.05h:00m:29s 1 / 5 20 (8606) Insufficient attribut
es were given to create an object. This object may not exist because it may have
been deleted and already garbage collected.
SRV1 31m:11s 0 / 5 0
Destination DSA largest delta fails/total %% error
SRV2 31m:12s 0 / 5 0
SRV1 07d.05h:00m:29s 1 / 5 20 (8606) Insufficient attribut
es were given to create an object. This object may not exist because it may have
been deleted and already garbage collected.
ASKER
dcdiag /q
Error NT AUTHORITY\ENTERPRISE DOMAIN CONTROLLERS doesn't have
Replicating Directory Changes In Filtered Set
access rights for the naming context:
DC=ForestDnsZones,DC=ABC,D C=local
Error NT AUTHORITY\ENTERPRISE DOMAIN CONTROLLERS doesn't have
Replicating Directory Changes In Filtered Set
access rights for the naming context:
DC=DomainDnsZones,DC=ABC,D C=local
......................... SRV2 failed test NCSecDesc
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 17:24:34
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 17:24:34
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
An error event occurred. EventID: 0xC00A0038
Time Generated: 11/16/2016 17:24:35
Event String:
The Terminal Server security layer detected an error in the protocol
stream and has disconnected the client. Client IP: 80.228.64.131.
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 17:31:12
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 17:31:12
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 17:58:39
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 17:58:39
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 18:02:14
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 18:02:14
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
......................... SRV2 failed test SystemLog
Error NT AUTHORITY\ENTERPRISE DOMAIN CONTROLLERS doesn't have
Replicating Directory Changes In Filtered Set
access rights for the naming context:
DC=ForestDnsZones,DC=ABC,D
Error NT AUTHORITY\ENTERPRISE DOMAIN CONTROLLERS doesn't have
Replicating Directory Changes In Filtered Set
access rights for the naming context:
DC=DomainDnsZones,DC=ABC,D
......................... SRV2 failed test NCSecDesc
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 17:24:34
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 17:24:34
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
An error event occurred. EventID: 0xC00A0038
Time Generated: 11/16/2016 17:24:35
Event String:
The Terminal Server security layer detected an error in the protocol
stream and has disconnected the client. Client IP: 80.228.64.131.
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 17:31:12
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 17:31:12
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 17:58:39
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 17:58:39
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 18:02:14
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
An error event occurred. EventID: 0x00009018
Time Generated: 11/16/2016 18:02:14
Event String:
The following fatal alert was generated: 10. The internal error stat
e is 1203.
......................... SRV2 failed test SystemLog
ASKER
netdom query fsmo
Schema master SRV1.ABC.local
Domain naming master SRV1.ABC.local
PDC SRV1.ABC.local
RID pool manager SRV1.ABC.local
Infrastructure master SRV1.ABC.local
The command completed successfully.
Schema master SRV1.ABC.local
Domain naming master SRV1.ABC.local
PDC SRV1.ABC.local
RID pool manager SRV1.ABC.local
Infrastructure master SRV1.ABC.local
The command completed successfully.
ASKER
Note:
I believe this whole AD Replication issue was caused by a past loss of capacity on the C:\ drive of the PDC (SRV1). A backup job over crowed the server. I resolved the issue, but I think that knocked the replication out of sync. I believe my two servers have a different view of the AD at this point. Would this be enough to prevent the replication from happening? I keep reading about orphan objects and that they need to be cleaned out. I am just trying to give as much info/insight as I can to get to the bottom of this issue.
Thank you for responding. I look forward to your reply
I believe this whole AD Replication issue was caused by a past loss of capacity on the C:\ drive of the PDC (SRV1). A backup job over crowed the server. I resolved the issue, but I think that knocked the replication out of sync. I believe my two servers have a different view of the AD at this point. Would this be enough to prevent the replication from happening? I keep reading about orphan objects and that they need to be cleaned out. I am just trying to give as much info/insight as I can to get to the bottom of this issue.
Thank you for responding. I look forward to your reply
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
From SRV1
I created a test user, OU and GPO. It successfully popped up on SRV2
From SRV2
I performed the above and was able to see the new OU/GPO from SRV1, but not the new user.
From the link, it was suggested that I run the following:
Repadmin /removelingeringobjects DC_Containing_LO, DC_Containing_NO_LO, Partition_Of_LO /ADVISORY_MODE
My interpretation is SRV2 has the lingering Object, to be match against SRV1 (the good AD Database). Do I have this right? Do I run this on SRV1? Should I run this on the other server?
This is what I ran with the actual GUID of the SRV1
Repadmin /removelingeringobjects SRV2.ABC.Local (SRV1 GUID) DC=ABC,DC=local /ADVISORY_MODE
I did the above and found 1 entry in event log, Directory Service ID: 1988
Could this be the only thing causing the replication problem?
Also I read the following:
Partition_Of_LO is the partition which you want to search for lingering objects. For example: dc=Contoso,DC=Com. Do not forget to run this against all the partitions except Schema. Is this what they mean regarding different partitions:
DC=DomainDnsZones,DC=ABC,D C=local
DC=ForestDnsZones,DC=ABC,D C=local
Do I need to rerun the Repadmin /removelingeringobjects
Side note:
It appears that the event log Directory service has stopped since 11/09/2016 on SRV2. SRV1 DS log is up-to-date. No services appear to be stopped on SRV2.
Thanks for all your help
I created a test user, OU and GPO. It successfully popped up on SRV2
From SRV2
I performed the above and was able to see the new OU/GPO from SRV1, but not the new user.
From the link, it was suggested that I run the following:
Repadmin /removelingeringobjects DC_Containing_LO, DC_Containing_NO_LO, Partition_Of_LO /ADVISORY_MODE
My interpretation is SRV2 has the lingering Object, to be match against SRV1 (the good AD Database). Do I have this right? Do I run this on SRV1? Should I run this on the other server?
This is what I ran with the actual GUID of the SRV1
Repadmin /removelingeringobjects SRV2.ABC.Local (SRV1 GUID) DC=ABC,DC=local /ADVISORY_MODE
I did the above and found 1 entry in event log, Directory Service ID: 1988
Could this be the only thing causing the replication problem?
Also I read the following:
Partition_Of_LO is the partition which you want to search for lingering objects. For example: dc=Contoso,DC=Com. Do not forget to run this against all the partitions except Schema. Is this what they mean regarding different partitions:
DC=DomainDnsZones,DC=ABC,D
DC=ForestDnsZones,DC=ABC,D
Do I need to rerun the Repadmin /removelingeringobjects
Side note:
It appears that the event log Directory service has stopped since 11/09/2016 on SRV2. SRV1 DS log is up-to-date. No services appear to be stopped on SRV2.
Thanks for all your help
Since SRV2 seems to be the problem DC as you have pointed out why not demote this then promoto is again. All FSMO rolls appear do be on SRV1 as you posted earlier
ASKER
That was going to be my next question, can I bypass this whole process by pulling SRV2 from the domain. My concern, and the reason why I halted, was I was worried due to the current replication issue that I would end up causing future problems with the AD. Is it possible to successfully pull a DC from a domain suffering a faulty replication issue.
ASKER
I stumbled a bit on assuring the direction (from which server do I run this and how does it sit w/in the command) and identification of the AD server names with in the Repadmin.
I finally ran Repadmin /removelingeringobjects (see the suggested link) with the /advisory_mode from SRV2. I saw lingering objects identified in the event log of SRV2. I pulled away the /advisory_mode from the command, and ran it again. It removed 20 lingering objects from the AD. I created a couple of additional test users and saw it successfully replicated. I pushed replication from both servers - no error messages - SUCCESS!!!!
Thank you for your help.
I finally ran Repadmin /removelingeringobjects (see the suggested link) with the /advisory_mode from SRV2. I saw lingering objects identified in the event log of SRV2. I pulled away the /advisory_mode from the command, and ran it again. It removed 20 lingering objects from the AD. I created a couple of additional test users and saw it successfully replicated. I pushed replication from both servers - no error messages - SUCCESS!!!!
Thank you for your help.
ASKER
SRV2
Starting test: CheckSecurityError
[SRV2] DsReplicaGetInfo(KCC_DS_CO
[SRV2] Unable to query the list of KCC connection failures. Continuing... [SRV2] No security related replication errors were found on this DC! To target the connection to a specific source DC use /ReplSource:<DC>.
----
SRV2 failed test NCSecDesc
Error NT AUTHORITY\ENTERPRISE DOMAIN CONTROLLERS doesn't have Replicating Directory Changes In Filtered Set
access rights for the naming context: DC=ForestDnsZones,DC=ABC,D
* Security Permissions Check for DC=DomainDnsZones,DC=ABC,D
(NDNC,Version 3)
Error NT AUTHORITY\ENTERPRISE DOMAIN CONTROLLERS doesn't have Replicating Directory Changes In Filtered Set
access rights for the naming context: DC=DomainDnsZones,DC=ABC,D
---------
Starting test: Services
Checking Service: NTDS
Could not open NTDS Service on SRV2, error 0x5 "Access is denied."
---------------
Warning: no DNS RPC connectivity (error or non Microsoft DNS server is running)
[Error details: 5 (Type: Win32 - Description: Access is denied.)]