[Webinar] Streamline your web hosting managementRegister Today

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 3630
  • Last Modified:

Howto fix an Active directory replication issue

Hi
I have domain with 2 DCs in it  which has stopped replicating,
I have attached to log files. As anyone seen these errors before ?


Domain Controller Diagnosis

Performing initial setup:
   * Verifying that the local machine riverdc01, is a DC.
   * Connecting to directory service on server riverdc01.
   * Collecting site info.
   * Identifying all servers.
   * Identifying all NC cross-refs.
   * Found 3 DC(s). Testing 1 of them.
   Done gathering initial info.

Doing initial required tests
   
   Testing server: VictoryHouse\RIVERDC01
      Starting test: Connectivity
         * Active Directory LDAP Services Check
         * Active Directory RPC Services Check
         ......................... RIVERDC01 passed test Connectivity

Doing primary tests
   
   Testing server: VictoryHouse\RIVERDC01
      Starting test: Replications
         * Replications Check
         [Replications Check,RIVERDC01] A recent replication attempt failed:
            From RIVERDC02 to RIVERDC01
            Naming Context: DC=ForestDnsZones,DC=riverltd,DC=co,DC=uk
            The replication generated an error (8614):
            Win32 Error 8614
            The failure occurred at 2010-03-09 10:50:25.
            The last success occurred at 2004-01-15 14:13:11.
            459 failures have occurred since the last success.
         [Replications Check,RIVERDC01] A recent replication attempt failed:
            From RIVERDC02 to RIVERDC01
            Naming Context: DC=DomainDnsZones,DC=riverltd,DC=co,DC=uk
            The replication generated an error (8614):
            Win32 Error 8614
            The failure occurred at 2010-03-09 10:50:25.
            The last success occurred at 2004-01-15 14:13:11.
            459 failures have occurred since the last success.
         [Replications Check,RIVERDC01] A recent replication attempt failed:
            From RIVERDC02 to RIVERDC01
            Naming Context: CN=Schema,CN=Configuration,DC=riverltd,DC=co,DC=uk
            The replication generated an error (8614):
            Win32 Error 8614
            The failure occurred at 2010-03-09 10:50:25.
            The last success occurred at 2004-01-15 14:13:11.
            459 failures have occurred since the last success.
         [Replications Check,RIVERDC01] A recent replication attempt failed:
            From RIVERDATA1 to RIVERDC01
            Naming Context: CN=Schema,CN=Configuration,DC=riverltd,DC=co,DC=uk
            The replication generated an error (-2146893022):
            Win32 Error -2146893022
            The failure occurred at 2010-03-09 10:50:25.
            The last success occurred at 2004-01-15 14:13:11.
            459 failures have occurred since the last success.
         [RIVERDATA1] DsBindWithSpnEx() failed with error -2146893022,
         Win32 Error -2146893022.
         [Replications Check,RIVERDC01] A recent replication attempt failed:
            From RIVERDC02 to RIVERDC01
            Naming Context: CN=Configuration,DC=riverltd,DC=co,DC=uk
            The replication generated an error (8614):
            Win32 Error 8614
            The failure occurred at 2010-03-09 10:50:24.
            The last success occurred at 2004-01-15 14:18:44.
            547 failures have occurred since the last success.
         [Replications Check,RIVERDC01] A recent replication attempt failed:
            From RIVERDATA1 to RIVERDC01
            Naming Context: CN=Configuration,DC=riverltd,DC=co,DC=uk
            The replication generated an error (-2146893022):
            Win32 Error -2146893022
            The failure occurred at 2010-03-09 10:50:25.
            The last success occurred at 2004-01-15 14:18:44.
            555 failures have occurred since the last success.
         [Replications Check,RIVERDC01] A recent replication attempt failed:
            From RIVERDATA1 to RIVERDC01
            Naming Context: DC=riverltd,DC=co,DC=uk
            The replication generated an error (-2146893022):
            Win32 Error -2146893022
            The failure occurred at 2010-03-09 10:50:24.
            The last success occurred at 2004-01-15 14:18:50.
            31689 failures have occurred since the last success.
         [Replications Check,RIVERDC01] A recent replication attempt failed:
            From RIVERDC02 to RIVERDC01
            Naming Context: DC=riverltd,DC=co,DC=uk
            The replication generated an error (8614):
            Win32 Error 8614
            The failure occurred at 2010-03-09 10:55:04.
            The last success occurred at 2004-01-15 14:18:50.
            68607 failures have occurred since the last success.
         * Replication Latency Check
         REPLICATION-RECEIVED LATENCY WARNING
         RIVERDC01:  Current time is 2010-03-09 10:55:06.
            DC=ForestDnsZones,DC=riverltd,DC=co,DC=uk
               Last replication recieved from RIVERDC02 at 2004-01-15 14:13:11.
               WARNING:  This latency is over the Tombstone Lifetime of 60 days!
               Latency information for 2 entries in the vector were ignored.
                  2 were retired Invocations.  0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc.  0 had no latency information (Win2K DC).  
            DC=DomainDnsZones,DC=riverltd,DC=co,DC=uk
               Last replication recieved from RIVERDC02 at 2004-01-15 14:13:11.
               WARNING:  This latency is over the Tombstone Lifetime of 60 days!
               Latency information for 2 entries in the vector were ignored.
                  2 were retired Invocations.  0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc.  0 had no latency information (Win2K DC).  
            CN=Schema,CN=Configuration,DC=riverltd,DC=co,DC=uk
               Last replication recieved from RIVERDC02 at 2004-01-15 14:13:11.
               WARNING:  This latency is over the Tombstone Lifetime of 60 days!
               Last replication recieved from RIVERDATA1 at 2004-01-15 14:13:11.
               WARNING:  This latency is over the Tombstone Lifetime of 60 days!
               Latency information for 7 entries in the vector were ignored.
                  7 were retired Invocations.  0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc.  0 had no latency information (Win2K DC).  
            CN=Configuration,DC=riverltd,DC=co,DC=uk
               Last replication recieved from RIVERDC02 at 2004-01-15 14:18:44.
               WARNING:  This latency is over the Tombstone Lifetime of 60 days!
               Last replication recieved from RIVERDATA1 at 2004-01-15 14:18:44.
               WARNING:  This latency is over the Tombstone Lifetime of 60 days!
               Latency information for 7 entries in the vector were ignored.
                  7 were retired Invocations.  0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc.  0 had no latency information (Win2K DC).  
            DC=riverltd,DC=co,DC=uk
               Last replication recieved from RIVERDC02 at 2004-01-15 14:18:50.
               WARNING:  This latency is over the Tombstone Lifetime of 60 days!
               Last replication recieved from RIVERDATA1 at 2004-01-15 14:18:50.
               WARNING:  This latency is over the Tombstone Lifetime of 60 days!
               Latency information for 7 entries in the vector were ignored.
                  7 were retired Invocations.  0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc.  0 had no latency information (Win2K DC).  
         ......................... RIVERDC01 passed test Replications
      Test omitted by user request: Topology
      Test omitted by user request: CutoffServers
      Starting test: NCSecDesc
         * Security Permissions check for all NC's on DC RIVERDC01.
         * Security Permissions Check for
           DC=ForestDnsZones,DC=riverltd,DC=co,DC=uk
            (NDNC,Version 2)
         * Security Permissions Check for
           DC=DomainDnsZones,DC=riverltd,DC=co,DC=uk
            (NDNC,Version 2)
         * Security Permissions Check for
           CN=Schema,CN=Configuration,DC=riverltd,DC=co,DC=uk
            (Schema,Version 2)
         * Security Permissions Check for
           CN=Configuration,DC=riverltd,DC=co,DC=uk
            (Configuration,Version 2)
         * Security Permissions Check for
           DC=riverltd,DC=co,DC=uk
            (Domain,Version 2)
         ......................... RIVERDC01 passed test NCSecDesc
      Starting test: NetLogons
         * Network Logons Privileges Check
         Verified share \\RIVERDC01\netlogon
         Verified share \\RIVERDC01\sysvol
         ......................... RIVERDC01 passed test NetLogons
      Starting test: Advertising
         The DC RIVERDC01 is advertising itself as a DC and having a DS.
         The DC RIVERDC01 is advertising as an LDAP server
         The DC RIVERDC01 is advertising as having a writeable directory
         The DC RIVERDC01 is advertising as a Key Distribution Center
         The DC RIVERDC01 is advertising as a time server
         The DS RIVERDC01 is advertising as a GC.
         ......................... RIVERDC01 passed test Advertising
      Starting test: KnowsOfRoleHolders
         Role Schema Owner = CN=NTDS Settings,CN=RIVERDC01,CN=Servers,CN=VictoryHouse,CN=Sites,CN=Configuration,DC=riverltd,DC=co,DC=uk
         Role Domain Owner = CN=NTDS Settings,CN=RIVERDC01,CN=Servers,CN=VictoryHouse,CN=Sites,CN=Configuration,DC=riverltd,DC=co,DC=uk
         Role PDC Owner = CN=NTDS Settings,CN=RIVERDC02,CN=Servers,CN=VictoryHouse,CN=Sites,CN=Configuration,DC=riverltd,DC=co,DC=uk
         Role Rid Owner = CN=NTDS Settings,CN=RIVERDC02,CN=Servers,CN=VictoryHouse,CN=Sites,CN=Configuration,DC=riverltd,DC=co,DC=uk
         Role Infrastructure Update Owner = CN=NTDS Settings,CN=RIVERDATA1,CN=Servers,CN=VictoryHouse,CN=Sites,CN=Configuration,DC=riverltd,DC=co,DC=uk
         Warning: RIVERDATA1 is the Infrastructure Update Owner, but is not responding to DS RPC Bind.
         [RIVERDATA1] LDAP bind failed with error 8341,
         Win32 Error 8341.
         Warning: RIVERDATA1 is the Infrastructure Update Owner, but is not responding to LDAP Bind.
         ......................... RIVERDC01 failed test KnowsOfRoleHolders
      Starting test: RidManager
         * Available RID Pool for the Domain is 7101 to 1073741823
         * riverdc02.riverltd.co.uk is the RID Master
         * DsBind with RID Master was successful
         * rIDAllocationPool is 4101 to 4600
         * rIDPreviousAllocationPool is 4101 to 4600
         * rIDNextRID: 4341
         ......................... RIVERDC01 passed test RidManager
      Starting test: MachineAccount
         Checking machine account for DC RIVERDC01 on DC RIVERDC01.
         * SPN found :LDAP/riverdc01.riverltd.co.uk/riverltd.co.uk
         * SPN found :LDAP/riverdc01.riverltd.co.uk
         * SPN found :LDAP/RIVERDC01
         * SPN found :LDAP/riverdc01.riverltd.co.uk/RIVERLTD
         * SPN found :LDAP/433ef2e5-0544-4e06-bb98-a6c5f0220eac._msdcs.riverltd.co.uk
         * SPN found :E3514235-4B06-11D1-AB04-00C04FC2DCD2/433ef2e5-0544-4e06-bb98-a6c5f0220eac/riverltd.co.uk
         * SPN found :HOST/riverdc01.riverltd.co.uk/riverltd.co.uk
         * SPN found :HOST/riverdc01.riverltd.co.uk
         * SPN found :HOST/RIVERDC01
         * SPN found :HOST/riverdc01.riverltd.co.uk/RIVERLTD
         * SPN found :GC/riverdc01.riverltd.co.uk/riverltd.co.uk
         ......................... RIVERDC01 passed test MachineAccount
      Starting test: Services
         * Checking Service: Dnscache
         * Checking Service: NtFrs
         * Checking Service: IsmServ
         * Checking Service: kdc
         * Checking Service: SamSs
         * Checking Service: LanmanServer
         * Checking Service: LanmanWorkstation
         * Checking Service: RpcSs
         * Checking Service: w32time
         * Checking Service: NETLOGON
         ......................... RIVERDC01 passed test Services
      Test omitted by user request: OutboundSecureChannels
      Starting test: ObjectsReplicated
         RIVERDC01 is in domain DC=riverltd,DC=co,DC=uk
         Checking for CN=RIVERDC01,OU=Domain Controllers,DC=riverltd,DC=co,DC=uk in domain DC=riverltd,DC=co,DC=uk on 1 servers
            Object is up-to-date on all servers.
         Checking for CN=NTDS Settings,CN=RIVERDC01,CN=Servers,CN=VictoryHouse,CN=Sites,CN=Configuration,DC=riverltd,DC=co,DC=uk in domain CN=Configuration,DC=riverltd,DC=co,DC=uk on 1 servers
            Object is up-to-date on all servers.
         ......................... RIVERDC01 passed test ObjectsReplicated
      Starting test: frssysvol
         * The File Replication Service SYSVOL ready test
         File Replication Service's SYSVOL is ready
         ......................... RIVERDC01 passed test frssysvol
      Starting test: frsevent
         * The File Replication Service Event log test
         There are warning or error events within the last 24 hours after the

         SYSVOL has been shared.  Failing SYSVOL replication problems may cause

         Group Policy problems.
         An Warning Event occured.  EventID: 0x800034C4
            Time Generated: 03/08/2010   13:52:38
            (Event String could not be retrieved)
         ......................... RIVERDC01 failed test frsevent
      Starting test: kccevent
         * The KCC Event log test
         An Error Event occured.  EventID: 0xC00007FA
            Time Generated: 03/09/2010   10:50:24
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0xC00007FA
            Time Generated: 03/09/2010   10:50:25
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0xC00007FA
            Time Generated: 03/09/2010   10:50:25
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0xC00007FA
            Time Generated: 03/09/2010   10:50:25
            (Event String could not be retrieved)
         An Error Event occured.  EventID: 0xC00007FA
            Time Generated: 03/09/2010   10:50:34
            (Event String could not be retrieved)
         ......................... RIVERDC01 failed test kccevent
      Starting test: systemlog
         * The System Event log test
         An Error Event occured.  EventID: 0x40000004
            Time Generated: 03/09/2010   10:00:57
            Event String: The kerberos client received a

KRB_AP_ERR_MODIFIED error from the server

host/riverdata1.riverltd.co.uk.  The target name

used was RIVERLTD\RIVERDATA1$. This indicates

that the password used to encrypt the kerberos

service ticket is different than that on the

target server. Commonly, this is due to

identically named  machine accounts in the target

realm (RIVERLTD.CO.UK), and the client realm.

Please contact your system administrator.
         An Error Event occured.  EventID: 0x40000004
            Time Generated: 03/09/2010   10:16:45
            Event String: The kerberos client received a

KRB_AP_ERR_MODIFIED error from the server

host/riverdata1.riverltd.co.uk.  The target name

used was

LDAP/cf5b5935-4783-4d5c-83fc-0f3d9ce1525c._msdcs.riverltd.co.uk.

 This indicates that the password used to encrypt

the kerberos service ticket is different than

that on the target server. Commonly, this is due

to identically named  machine accounts in the

target realm (RIVERLTD.CO.UK), and the client

realm.   Please contact your system

administrator.
         An Error Event occured.  EventID: 0x40000004
            Time Generated: 03/09/2010   10:16:45
            Event String: The kerberos client received a

KRB_AP_ERR_MODIFIED error from the server

host/riverdata1.riverltd.co.uk.  The target name

used was

ldap/cf5b5935-4783-4d5c-83fc-0f3d9ce1525c._msdcs.riverltd.co.uk.

 This indicates that the password used to encrypt

the kerberos service ticket is different than

that on the target server. Commonly, this is due

to identically named  machine accounts in the

target realm (RIVERLTD.CO.UK), and the client

realm.   Please contact your system

administrator.
         ......................... RIVERDC01 failed test systemlog
      Test omitted by user request: VerifyReplicas
      Starting test: VerifyReferences
         The system object reference (serverReference)

         CN=RIVERDC01,OU=Domain Controllers,DC=riverltd,DC=co,DC=uk and

         backlink on

         CN=RIVERDC01,CN=Servers,CN=VictoryHouse,CN=Sites,CN=Configuration,DC=riverltd,DC=co,DC=uk

         are correct.
         The system object reference (frsComputerReferenceBL)

         CN=RIVERDC01,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=riverltd,DC=co,DC=uk

         and backlink on

         CN=RIVERDC01,OU=Domain Controllers,DC=riverltd,DC=co,DC=uk are

         correct.
         The system object reference (serverReferenceBL)

         CN=RIVERDC01,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=riverltd,DC=co,DC=uk

         and backlink on

         CN=NTDS Settings,CN=RIVERDC01,CN=Servers,CN=VictoryHouse,CN=Sites,CN=Configuration,DC=riverltd,DC=co,DC=uk

         are correct.
         ......................... RIVERDC01 passed test VerifyReferences
      Test omitted by user request: VerifyEnterpriseReferences
      Test omitted by user request: CheckSecurityError
   
   Running partition tests on : ForestDnsZones
      Starting test: CrossRefValidation
         ......................... ForestDnsZones passed test CrossRefValidation
      Starting test: CheckSDRefDom
         ......................... ForestDnsZones passed test CheckSDRefDom
   
   Running partition tests on : DomainDnsZones
      Starting test: CrossRefValidation
         ......................... DomainDnsZones passed test CrossRefValidation
      Starting test: CheckSDRefDom
         ......................... DomainDnsZones passed test CheckSDRefDom
   
   Running partition tests on : Schema
      Starting test: CrossRefValidation
         ......................... Schema passed test CrossRefValidation
      Starting test: CheckSDRefDom
         ......................... Schema passed test CheckSDRefDom
   
   Running partition tests on : Configuration
      Starting test: CrossRefValidation
         ......................... Configuration passed test CrossRefValidation
      Starting test: CheckSDRefDom
         ......................... Configuration passed test CheckSDRefDom
   
   Running partition tests on : riverltd
      Starting test: CrossRefValidation
         ......................... riverltd passed test CrossRefValidation
      Starting test: CheckSDRefDom
         ......................... riverltd passed test CheckSDRefDom
   
   Running enterprise tests on : riverltd.co.uk
      Starting test: Intersite
         Skipping site VictoryHouse, this site is outside the scope provided by

         the command line arguments provided.
         ......................... riverltd.co.uk passed test Intersite
      Starting test: FsmoCheck
         GC Name: \\riverdc01.riverltd.co.uk
         Locator Flags: 0xe00001fc
         PDC Name: \\riverdc02.riverltd.co.uk
         Locator Flags: 0xe00003fd
         Time Server Name: \\riverdc01.riverltd.co.uk
         Locator Flags: 0xe00001fc
         Preferred Time Server Name: \\riverdc02.riverltd.co.uk
         Locator Flags: 0xe00003fd
         KDC Name: \\riverdc01.riverltd.co.uk
         Locator Flags: 0xe00001fc
         ......................... riverltd.co.uk passed test FsmoCheck
      Test omitted by user request: DNS
      Test omitted by user request: DNS

AD-rep-manager.png
0
thombie
Asked:
thombie
3 Solutions
 
Mike ThomasConsultantCommented:
Check DNS and connectivity then check DNS again

Run dcdiag /test:dns and look for errors, this is where you need to start.
0
 
rizla7Commented:
WARNING:  This latency is over the Tombstone Lifetime of 60 days

there is your problem. i think.

here some links to get you going. i cant remember exactly how to do this, but it had to do with updating the timestamps on some objects and resyncing

http://blogs.dirteam.com/blogs/jorge/archive/2005/11/24/153.aspx
http://technet.microsoft.com/en-us/magazine/2007.09.tombstones.aspx?pr=blog
0
 
rizla7Commented:
Also..

Event String: The kerberos client received a
KRB_AP_ERR_MODIFIED error from the server
host/riverdata1.riverltd.co.uk.  The target name
used was RIVERLTD\RIVERDATA1$. This indicates
that the password used to encrypt the kerberos
service ticket is different than that on the
target server. Commonly, this is due to
identically named  machine accounts in the target
realm (RIVERLTD.CO.UK), and the client realm

which could actually be a duplicately named machine account, or it could be the way you added the server to AD or reset its machine key or what not.
0
 
Mike KlineCommented:
Yeah it is the tombstone issue...they haven't replicated in six years.  What you need to do is cleanup that dead DC from active directory

Run a metadata cleanup of the DC(you do this from a good DC)    Some articles on metadata cleanup here:

http://support.microsoft.com/kb/216498

http://www.petri.co.il/delete_failed_dcs_from_ad.htm

http://msmvps.com/blogs/ad/archive/2008/12/17/how-to-remove-a-failed-or-offline-dc.aspx

Thanks

Mike
0
 
thombieAuthor Commented:
Thanks Guys with all your help I hav managed to sort this.  It was the thombstone issue.
0

Featured Post

Learn to develop an Android App

Want to increase your earning potential in 2018? Pad your resume with app building experience. Learn how with this hands-on course.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now