Replication

I’ve been brought in to help with an 2008 R2 Active Directory system.  I had things running pretty well.  Now I have two problems.  The first and most important is the Active Directory DC in a site (BEL- Bellevue, BEL-VM-AD01 and BEL-VM-AD02, both are GC’s) have stopped replicating with the main site, (TUK – Tukwila, DC: TUK-DC-VM-AD01, main FISMO DC for the domain.)  

When I try to replicate manually from Tuk to Bel using;

>REPADMIN /REPLICATE BEL-VM-AD02 TUK-DC-VM-AD01 DC=DomainDNSZones,DC=USCorp,DC=AudSci,DC=COM

I get,

DsReplicaSync() failed with status 1722 (0x6ba):
    The RPC server is unavailable.

When I try from the Bel side

>REPADMIN /REPLICATE TUK-DC-VM-AD01 BEL-VM-AD01 DC=DomainDNSZones,DC=USCorp,DC=AudSci,DC=COM

I get,

Error: An LDAP lookup operation failed with the following error:

    LDAP Error 81(0x51): Server Down
    Server Win32 Error 0(0x0):
    Extended Information:

Logs from Bel show Event ID; 1311  The Knowledge Consistency Checker (KCC) has detected a problem …..

The DNS settings on the nic’s of the servers are correct.
LDAP SSL ports 636 are not blocked
RPC port of 135 is not blocked.
The registry keys are correct.
Repadmin /syncall shows no errors.

Repadmin /showrepl;

Source: TukDC\TUK-DC-VM-AD001
******* 1971 CONSECUTIVE FAILURES since 2015-08-29 17:34:52
Last error: 1722 (0x6ba):
            The RPC server is unavailable.

And

Source: TukDC\TUK-DC-VM-AD001
******* 1971 CONSECUTIVE FAILURES since 2015-08-29 17:34:52
Last error: 1722 (0x6ba):
            The RPC server is unavailable.


I could really use some help.
suppergAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Mal OsborneAlpha GeekCommented:
My first guess here is a DNS problem. I am assuming that the two Bellevue DCs are DNS servers, and point to themselves, while Tukwila has its own DNS servers?  (probably the DC).

I would do the following:
1. From a Belleview DC, check you can ping TUK-DC-VM-AD01.
2. Check and configure firewalls to allow DNS (UDP port 53) between the Belleview CD and TUK-DC-VM-AD01.
3. Set one Belleview DC to use TUK-DC-VM-AD01 as its DNS server.
4. Reboot the Belleview DC.
5. Try replicating again.

I am basing this on the theory that DNS in Belleview is somehow out of date, and cannot be used to properly locate the DC in Tukwila. Once replication has occurred, hopefully DNS will update, and you can change the setting back.
Dirk MareSystems Engineer (Acting IT Manager)Commented:
Can you confirm connectivity between DC.

Can all DC's ping ip and DNS name of all other DC's?

DirkMare
suppergAuthor Commented:
Hello Dirk Mare, yes all DNS AD DC's can and do ping.
Determine the Perfect Price for Your IT Services

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden with our free interactive tool and use it to determine the right price for your IT services. Download your free eBook now!

suppergAuthor Commented:
Hi Malmensa,  Those were good suggestions.  Unfortunately none of them work.   Something interesting has transpired overnight.  Now BEL-VM-AD01 and TUK-DC-VM-AD01 are replicating !
BEL-VM-AD02 and TUK-DC-VM-AD01 are still failing to replicate.  But BEL-VM-AD01 and BEL-VM-AD02 are replicating with each other.  So I don't have to worry about these things getting too far out of date and tomb stoning on me.  Other things I have tried this morning;

1. In NTDS Setting for the site I have deleted the connection between BEL-VM-AD02 and Tuk-DC-VM-Ad01, rebooted BEL-VM-Ad02 and added the connection back.  Still fails.  

2. I added the IP address of TUK-DC-VM-AD01 in the DNS setting for Conditional Forwards (on BEL-VM-AD02) and rebooted BEL-VM-AD02.  Tried replication and it still fails.  But I noticed that IP was already showing up in the Conditional Forwards on TUK-DC-VM-Ad01 so even though replication is reporting as failed, some communication is taking place.  And it didn't come from BEL-VM-AD01 because that IP Address had not appeared there yet.  This is all very strange.
Dirk MareSystems Engineer (Acting IT Manager)Commented:
Have you tried disabling IPv6 on all DC?

Are all the server in the correct "Sites and Services" all sites setup correctly with the correct ip ranges and subnets?

DirkMare
Mal OsborneAlpha GeekCommented:
Hang on.

Assuming that the Bellevue servers are in their own site, one should be selected by the KCC as the bridgehead, and only it will replicate to Tukwila.

The two Bellevue servers will replicate with each other, but only one will talk to the rest of the DCs.

Below is a pretty good explanation:

https://technet.microsoft.com/en-us/library/cc755994(v=ws.10).aspx

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
JaihuntCommented:
What is the Primary and secondary DNS servers in the BEL-VM-AD02 and BEL-VM-AD01. Configure it pointing to TUK-DC-VM-RAD02. and do ipconfig /registerDNS.

check both side you are able to nslookup DSA object GUID(you can see this in repadmin /showreps)
suppergAuthor Commented:
Malmensa, you absolutely hit the nail on the head.  THANK YOU.  I really appreciate the link you provided and I'm reading up on Bridgehead servers.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Active Directory

From novice to tech pro — start learning today.