AD Replication

cheto06
cheto06 used Ask the Experts™
on
I have a windows 2003 DC on a windows 2003 AD DFL. I've added two Win2k8 DC to retire the 2k3 DC. However, I am getting replications errors where the new DCs failed to replicate and become GCs.
See dcdiag.exe output below.

C:\Users\administrator.QABIG>dcdiag.exe

Directory Server Diagnosis

Performing initial setup:
   Trying to find home server...
   Home Server = qabigdc2
   * Identified AD Forest.
   Done gathering initial info.

Doing initial required tests

   Testing server: Default-First-Site-Name\QABIGDC2
      Starting test: Connectivity
         ......................... QABIGDC2 passed test Connectivity

Doing primary tests

   Testing server: Default-First-Site-Name\QABIGDC2
      Starting test: Advertising
         Warning: QABIGDC2 has not finished promoting to be a GC.
         Check the event log for domains that cannot be replicated.
         Warning: QABIGDC2 is not advertising as a global catalog.
         Check that server finished GC promotion.
         Check the event log on server that enough source replicas for the GC
         are available.
         ......................... QABIGDC2 failed test Advertising
      Starting test: FrsEvent
         ......................... QABIGDC2 passed test FrsEvent
      Starting test: DFSREvent
         ......................... QABIGDC2 passed test DFSREvent
      Starting test: SysVolCheck
         ......................... QABIGDC2 passed test SysVolCheck
      Starting test: KccEvent
         An Warning Event occurred.  EventID: 0x80000785
            Time Generated: 04/16/2012   09:17:55
            Event String:
            The attempt to establish a replication link for the following writab
le directory partition failed.
         An Warning Event occurred.  EventID: 0x80000786
            Time Generated: 04/16/2012   09:17:55
            Event String:
            The attempt to establish a replication link to a read-only directory
 partition with the following parameters failed.
         ......................... QABIGDC2 passed test KccEvent
      Starting test: KnowsOfRoleHolders
         ......................... QABIGDC2 passed test KnowsOfRoleHolders
      Starting test: MachineAccount
         ......................... QABIGDC2 passed test MachineAccount
      Starting test: NCSecDesc
         Error NT AUTHORITY\ENTERPRISE DOMAIN CONTROLLERS doesn't have
            Replicating Directory Changes In Filtered Set
         access rights for the naming context:
         DC=ForestDnsZones,DC=qabig,DC=com
         Error NT AUTHORITY\ENTERPRISE DOMAIN CONTROLLERS doesn't have
            Replicating Directory Changes In Filtered Set
         access rights for the naming context:
         DC=DomainDnsZones,DC=qabig,DC=com
         Ldap search capabality attribute search failed on server QABIGDC2,
         return value = 81
         ......................... QABIGDC2 failed test NCSecDesc
      Starting test: NetLogons
         ......................... QABIGDC2 passed test NetLogons
      Starting test: ObjectsReplicated
         ......................... QABIGDC2 passed test ObjectsReplicated
      Starting test: Replications
         REPLICATION-RECEIVED LATENCY WARNING
         QABIGDC2:  Current time is 2012-04-16 09:30:36.
            CN=Schema,CN=Configuration,DC=qabig,DC=com
               Last replication received from QAWESTDC1 at
          2009-10-28 07:54:32
               WARNING:  This latency is over the Tombstone Lifetime of 180
         days!
            CN=Configuration,DC=qabig,DC=com
               Last replication received from QAWESTDC1 at
          2009-10-28 07:54:32
               WARNING:  This latency is over the Tombstone Lifetime of 180
         days!
         ......................... QABIGDC2 passed test Replications
      Starting test: RidManager
         ......................... QABIGDC2 passed test RidManager
      Starting test: Services
         ......................... QABIGDC2 passed test Services
      Starting test: SystemLog
         ......................... QABIGDC2 passed test SystemLog
      Starting test: VerifyReferences
         ......................... QABIGDC2 passed test VerifyReferences


   Running partition tests on : ForestDnsZones
      Starting test: CheckSDRefDom
         ......................... ForestDnsZones passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... ForestDnsZones passed test
         CrossRefValidation

   Running partition tests on : DomainDnsZones
      Starting test: CheckSDRefDom
         ......................... DomainDnsZones passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... DomainDnsZones passed test
         CrossRefValidation

   Running partition tests on : Schema
      Starting test: CheckSDRefDom
         ......................... Schema passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... Schema passed test CrossRefValidation

   Running partition tests on : Configuration
      Starting test: CheckSDRefDom
         ......................... Configuration passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... Configuration passed test CrossRefValidation

   Running partition tests on : qabig
      Starting test: CheckSDRefDom
         ......................... qabig passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... qabig passed test CrossRefValidation

   Running enterprise tests on : qabig.com
      Starting test: LocatorCheck
         ......................... qabig.com passed test LocatorCheck
      Starting test: Intersite
         ......................... qabig.com passed test Intersite

C:\Users\administrator.QABIG>
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Top Expert 2013

Commented:
What is the QAWESTDC1 box?  

 WARNING:  This latency is over the Tombstone Lifetime of 180
         days!

That thing hasn't replicated in 2+ years.  Do you know of older boxes that were not demoted properly?  You will need to run a metadata cleanup of those old boxes that have passed the tombstone lifetime period.

Thanks

Mike
AmitIT Architect
Distinguished Expert 2017

Commented:
Is replication completed? Have you rebooted these servers? Secondly, goto sites and services and check if connections are created, if not create it manually and force the replication.
Top Expert 2012

Commented:
Do a metadata cleanup to remove any old failed DCs it seems you have some.

http://www.petri.co.il/delete_failed_dcs_from_ad.htm
Fundamentals of JavaScript

Learn the fundamentals of the popular programming language JavaScript so that you can explore the realm of web development.

Author

Commented:
Yes, it appears that there is dc that was not removed properly.  This is a sub-domain on my forest.  The question is could this be causing my root domains dc's not to sync proprely?

I've tried the metadata cleanup steps and at the very last step I get this, No server found.
However on Sites and Services it is listed and obviously it fails replication when trying to force.

0 - CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=qabig,DC=com
select operation target: select site 0
Site - CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=qabig,DC=com
Domain - DC=qawest,DC=qaus,DC=qabig,DC=com
No current server
No current Naming Context
select operation target: list servers in site
Found 1 server(s)
0 - (null)
select operation target: list sites
Found 1 site(s)
0 - CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=qabig,DC=com
select operation target: selec site 0
Site - CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=qabig,DC=com
Domain - DC=qawest,DC=qaus,DC=qabig,DC=com
No current server
No current Naming Context
select operation target: List Servers in site
Found 1 server(s)
0 - (null)
select operation target:
AmitIT Architect
Distinguished Expert 2017

Commented:
Check DNS, if entries are still present. Remember, DNS clean require advance skills. I would suggeste to Promote server with different name.
Top Expert 2012

Commented:
Go through this cleanup.

http://support.microsoft.com/kb/230306

if the above doesn't work go to through this link.

http://support.microsoft.com/kb/216498

Look at number 19

The problem will cause issues if you have any DC That is down

Author

Commented:
One more output. As you can see The domain shows up but not the DC.

C:\Documents and Settings\user>ntdsutil.exe
ntdsutil.exe: metadata cleanup
metadata cleanup: connections
server connections: connect to server qabigdc1
Binding to qabigdc1 ...
Connected to qabigdc1 using credentials of locally logged on user.
server connections: quit
metadata cleanup: select operation target
select operation target: list domains
Found 7 domain(s)
0 - DC=qabig,DC=com
1 - DC=qaeu,DC=qabig,DC=com
2 - DC=qaru,DC=qaeu,DC=qabig,DC=com
3 - DC=qace,DC=qaeu,DC=qabig,DC=com
4 - DC=qaus,DC=qabig,DC=com
5 - DC=qaeast,DC=qaus,DC=qabig,DC=com
6 - DC=qawest,DC=qaus,DC=qabig,DC=com
select operation target: select domain 6
No current site
Domain - DC=qawest,DC=qaus,DC=qabig,DC=com
No current server
No current Naming Context
select operation target: list sites
Found 1 site(s)
0 - CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=qabig,DC=com
select operation target: select site 0
Site - CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=qabig,DC=com
Domain - DC=qawest,DC=qaus,DC=qabig,DC=com
No current server
No current Naming Context
select operation target: list servers in site
Found 8 server(s)
0 - CN=QABIGDC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,
DC=qabig,DC=com
1 - CN=QAEUDC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,D
C=qabig,DC=com
2 - CN=QARUDC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,D
C=qabig,DC=com
3 - CN=QACEDC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,D
C=qabig,DC=com
4 - CN=QAUSDC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,D
C=qabig,DC=com
5 - CN=QAEASTDC1,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration
,DC=qabig,DC=com
6 - CN=QABIGDC2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,
DC=qabig,DC=com
7 - CN=QABIGDC3,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,
DC=qabig,DC=com
select operation target:
Top Expert 2012

Commented:
Did you go  through the second link? Is this child domain still up?

Author

Commented:
I did but on step 19
Expand DC=Your Domain, DC=COM, PRI, LOCAL, NET.
I only see DC=Your Domain, DC=COM what is PRI, LOCAL,NET?
When I expand this and go to DC=SYSTEM there's no trust domain that refers my domain in question.
Top Expert 2012

Commented:
Some site might have those extensions
Top Expert 2012

Commented:
Give me a screenshot of where you are seeing the DC that is tombstoned

Author

Commented:
Oddly enough, things seem to be better now. yesterday I removed that lost DC from Sites and Services and now I only see the two DCs not able of becoming GCs.
See attached:
DC1, is the the current GC that I want to delete, that passed all dcdiag tests (win2k3)
DC2, has always existed as a DC but was never a GC, now it's failing. (win2k8)
DC3, is the new DC i want to implement to retire DC1 fails GC as well (win2k8)
Additionally I get an event id that still point to QAWESTDC1 (the lost DC)
DC1-NowOK.txt
DC2-Errors.txt
DC3-Errors.txt
EventID.txt
Top Expert 2012

Commented:
dcdiag /test:dns

Author

Commented:
Dc3
C:\Users\administrator.QABIG>dcdiag /test:dns

Directory Server Diagnosis

Performing initial setup:
   Trying to find home server...
   Home Server = qabigdc3
   * Identified AD Forest.
   Done gathering initial info.

Doing initial required tests

   Testing server: Default-First-Site-Name\QABIGDC3
      Starting test: Connectivity
         ......................... QABIGDC3 passed test Connectivity

Doing primary tests

   Testing server: Default-First-Site-Name\QABIGDC3

      Starting test: DNS

         DNS Tests are running and not hung. Please wait a few minutes...
         ......................... QABIGDC3 passed test DNS

   Running partition tests on : ForestDnsZones

   Running partition tests on : DomainDnsZones

   Running partition tests on : Schema

   Running partition tests on : Configuration

   Running partition tests on : qabig

   Running enterprise tests on : qabig.com
      Starting test: DNS
         Test results for domain controllers:

            DC: qabigdc3.qabig.com
            Domain: qabig.com


               TEST: Basic (Basc)
                  Warning: The AAAA record for this DC was not found

               TEST: Dynamic update (Dyn)
                  Warning: Failed to delete the test record _dcdiag_test_record
in zone qabig.com

               TEST: Records registration (RReg)
                  Network Adapter
                  [00000006] Intel(R) PRO/1000 MT Network Connection:
                     Warning:
                     Missing AAAA record at DNS server 10.83.66.20:
                     qabigdc3.qabig.com

                     Warning:
                     Missing AAAA record at DNS server 10.83.66.20:
                     gc._msdcs.qabig.com

                     Warning:
                     Missing AAAA record at DNS server 10.83.66.64:
                     qabigdc3.qabig.com

                     Warning:
                     Missing AAAA record at DNS server 10.83.66.64:
                     gc._msdcs.qabig.com

                     Warning:
                     Missing AAAA record at DNS server 10.83.67.170:
                     qabigdc3.qabig.com

                     Warning:
                     Missing AAAA record at DNS server 10.83.67.170:
                     gc._msdcs.qabig.com

               Warning: Record Registrations not found in some network adapters

               qabigdc3                     PASS WARN PASS PASS WARN WARN n/a
         ......................... qabig.com passed test DNS

C:\Users\administrator.QABIG>

Author

Commented:
DC1

C:\Documents and Settings\juanqa>netdiag /test:dns
.....
    Computer Name: QABIGDC1
    DNS Host Name: qabigdc1.qabig.com
    System info : Microsoft Windows Server 2003 (Build 3790)
    Processor : x86 Family 16 Model 8 Stepping 0, AuthenticAMD
 
Netcard queries test . . . . . . . : Passed

Per interface results:
    Adapter : Local Area Connection

        Netcard queries test . . . : Passed

Global results:

Domain membership test . . . . . . : Passed

NetBT transports test. . . . . . . : Passed
    List of NetBt transports currently configured:
        NetBT_Tcpip_{366D6EC2-1638-4323-B54D-3342426B0EFF}
    1 NetBt transport currently configured.

DNS test . . . . . . . . . . . . . : Passed
    PASS - All the DNS entries for DC are registered on DNS server '10.83.66.20'
 and other DCs also have some of the names registered.
    PASS - All the DNS entries for DC are registered on DNS server '10.83.66.64'
 and other DCs also have some of the names registered.


The command completed successfully

C:\Documents and Settings\juanqa>

Author

Commented:
DC2
C:\Users\administrator.QABIG>dcdiag /test:dns

Directory Server Diagnosis

Performing initial setup:
   Trying to find home server...
   Home Server = qabigdc2
   * Identified AD Forest.
   Done gathering initial info.

Doing initial required tests

   Testing server: Default-First-Site-Name\QABIGDC2
      Starting test: Connectivity
         ......................... QABIGDC2 passed test Connectivity

Doing primary tests

   Testing server: Default-First-Site-Name\QABIGDC2

      Starting test: DNS

         DNS Tests are running and not hung. Please wait a few minutes...
         ......................... QABIGDC2 passed test DNS
   Running partition tests on : ForestDnsZones
   Running partition tests on : DomainDnsZones
   Runing partition tests on : Schema
   Running partition tests on : Configuration
   Running partition tests on : qabig
   Running enterprise tests on : qabig.com
      Starting test: DNS
         Test results for domain controllers:

            DC: qabigdc2.qabig.com
            Domain: qabig.com
               TEST: Basic (Basc)
                  Warning: The AAAA record for this DC was not found
               TEST: Dynamic update (Dyn)
                  Warning: Failed to delete the test record _dcdiag_test_record
in zone qabig.com

               TEST: Records registration (RReg)
                  Network Adapter
                  [00000006] Intel(R) PRO/1000 MT Network Connection:
                     Warning:
                     Missing AAAA record at DNS server 10.83.67.160:
                     qabigdc2.qabig.com

                     Warning:
                     Missing AAAA record at DNS server 10.83.67.160:
                     gc._msdcs.qabig.com

                     Warning:
                     Missing AAAA record at DNS server 10.83.66.64:
                     qabigdc2.qabig.com

                     Warning:
                     Missing AAAA record at DNS server 10.83.66.64:
                     gc._msdcs.qabig.com

               Warning: Record Registrations not found in some network adapters

               qabigdc2                     PASS WARN PASS PASS WARN WARN n/a
         ......................... qabig.com passed test DNS

C:\Users\administrator.QABIG>
Top Expert 2012

Commented:
You are missing DNS records.

Post ipconfig /all

Delete any records for the failed DC

Change binding order make sure IPv4 is listed first

http://theregime.wordpress.com/2008/03/04/how-to-setview-the-nic-bind-order-in-windows/

Author

Commented:
DC1:
C:\Documents and Settings\juanqa>ipconfig /all

Windows IP Configuration

   Host Name . . . . . . . . . . . . : qabigdc1
   Primary Dns Suffix  . . . . . . . : qabig.com
   Node Type . . . . . . . . . . . . : Unknown
   IP Routing Enabled. . . . . . . . : No
   WINS Proxy Enabled. . . . . . . . : No
   DNS Suffix Search List. . . . . . : qabig.com

Ethernet adapter Local Area Connection:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : VMware Accelerated AMD PCNet Adapter
   Physical Address. . . . . . . . . : 00-50-56-A9-17-C4
   DHCP Enabled. . . . . . . . . . . : No
   IP Address. . . . . . . . . . . . : 10.83.66.20
   Subnet Mask . . . . . . . . . . . : 255.255.224.0
   Default Gateway . . . . . . . . . : 10.83.66.2
   DNS Servers . . . . . . . . . . . : 10.83.66.20
                                       10.83.66.64

C:\Documents and Settings\user>

Author

Commented:
DC2:
C:\Users\administrator.QABIG>ipconfig /all

Windows IP Configuration

   Host Name . . . . . . . . . . . . : qabigdc2
   Primary Dns Suffix  . . . . . . . : qabig.com
   Node Type . . . . . . . . . . . . : Hybrid
   IP Routing Enabled. . . . . . . . : No
   WINS Proxy Enabled. . . . . . . . : No
   DNS Suffix Search List. . . . . . : qabig.com

Ethernet adapter Local Area Connection:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Intel(R) PRO/1000 MT Network Connection
   Physical Address. . . . . . . . . : 00-50-56-A2-60-AD
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   IPv4 Address. . . . . . . . . . . : 10.83.67.160(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.252.0
   Default Gateway . . . . . . . . . : 10.83.66.2
   DNS Servers . . . . . . . . . . . : 10.83.67.160
                                       10.83.66.64
   NetBIOS over Tcpip. . . . . . . . : Enabled

Tunnel adapter Local Area Connection* 8:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : isatap.{AB56F644-AF29-4E92-9033-B97F6E081
EEB}
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter Local Area Connection* 9:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Teredo Tunneling Pseudo-Interface
   Physical Address. . . . . . . . . : 02-00-54-55-4E-01
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

C:\Users\administrator.QABIG>

Author

Commented:
DC3:
C:\Users\administrator.QABIG>ipconfig /all

Windows IP Configuration

   Host Name . . . . . . . . . . . . : qabigdc3
   Primary Dns Suffix  . . . . . . . : qabig.com
   Node Type . . . . . . . . . . . . : Hybrid
   IP Routing Enabled. . . . . . . . : No
   WINS Proxy Enabled. . . . . . . . : No
   DNS Suffix Search List. . . . . . : qabig.com
                                     

Ethernet adapter Local Area Connection:

   Connection-specific DNS Suffix  . : qabig.com
   Description . . . . . . . . . . . : Intel(R) PRO/1000 MT Network Connectio
   Physical Address. . . . . . . . . : 00-50-56-A2-63-70
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   IPv4 Address. . . . . . . . . . . : 10.83.67.170(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.224.0
   Default Gateway . . . . . . . . . : 10.83.66.2
   DNS Servers . . . . . . . . . . . : 10.83.66.20
                                       10.83.66.64
                                       127.0.0.1
   NetBIOS over Tcpip. . . . . . . . : Enabled

Tunnel adapter Local Area Connection* 9:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Teredo Tunneling Pseudo-Interface
   Physical Address. . . . . . . . . : 02-00-54-55-4E-01
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter Local Area Connection* 11:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . : qabig.com
   Description . . . . . . . . . . . : Microsoft ISATAP Adapter #2
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

C:\Users\administrator.QABIG>

Author

Commented:
The binding is correct no problems there and no IPv6 enabled either.
Top Expert 2012
Commented:
IPv6 is still enabled. Unchecking this does not disable.

http://support.microsoft.com/kb/929852

Author

Commented:
After removing the qawest tombsotone records replication started to work ok. The AAAA missing records doen't matter now I guess. My DC's were able to promote themselves to GCs

Thanks, for your help

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial