2003 + 2008R2 DC FSMO Transfer and Replication Issues

Hello,

We seem to be experiencing issues with AD replication after adding two Server 2008 R2 servers to a 2003 domain with existing Server 2003 DCs.  I'm not sure where to continue troubleshooting at this point.

We have 3 existing Server 2003 DCs, and are adding 2 new Server 2008R2 DCs.  So far, what I've done is moved the Schema Master, Domain Naming Master, and PDC roles over to one of the 2008 R2 servers.  This doesn't seem to cause any issues, and I can see from several machines that the roles have transferred (through 'netdom query fsmo', or through the GUI).  However, when I attempt to transfer the remaining roles over to the second new DC, the role owner simply shows up as 'ERROR' in the GUI, and doesn't display any owner for those two remaining roles in the command line.

I am noticing some warnings in the Event log, which seem to have something to do with replication.

Event ID 1925 - The attempt to establish a replication link for the following writable directory partition failed.  
(The Source directory service is the 2nd new DC.

Right now, both new DCs are pointed to the same DNS server.  I have tested remote EventViewer between the DCs to ensure that RPC is working.  I am attaching two DCDiags.  DC1 is the original Server 2003 DC which held all of the FSMO roles initially.  DC4 is the new DC, which I have attempted to transfer Infrastructure and RID roles to unsuccessfully.

All suggestions are greatly appreciated.
DC1-Diag.txt
DC3-diag.txt
LVL 1
ClearBlueTechnologiesAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Aaron TomoskyDirector of Solutions ConsultingCommented:
What functional level is the domain? Did you get the adprep folder from your 08 install and run these commands?
http://blogs.technet.com/b/omers/archive/2010/06/30/step-by-step-guide-for-upgrading-active-directory-from-microsoft-windows-2003-to-microsoft-windows-server-2008.aspx
ClearBlueTechnologiesAuthor Commented:
Aaron, the functional level is 2003.  Yes, I went through the adprep using adprep from the Server 2008R2 installation CD.  At first, I only did adprep /forestprep and adprep /domainprep /gpprep.  I did adprep /rodcprep after I noticed we were having some issues.
Aaron TomoskyDirector of Solutions ConsultingCommented:
1. did you run those commands on the schema master? If not, run them again on the 2008 server now that it's the schema master
2. is sites and services showing your DCs in the right locations?
3. does adrepstatus show errors? http://www.microsoft.com/en-us/download/details.aspx?id=30005
4. also check your dns settings, what I have done in the past is set all dcs to the new dc with only one dns entry, I also force replication through sites and services between each change in roles.

Even though it's for 2012, this site has some good commands to check the health of the domain and the schema level
http://blogs.msmvps.com/mweber/2012/07/27/upgrading-an-active-directory-domain-from-windows-server-2008-or-windows-server-2008-r2-to-windows-server-2012/

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

MaheshArchitectCommented:
There must be some DNS and replication issues, as a fact DCs are unable to find server holding FSMO roles correctly
On all domain controllers run netdom query fsmo and ensure that all output is showing same results
Check DNS records [CNAME, NS and Host(A)] are correct on all domain controllers
Check under _msdcs.domain.com zone if there are CNAME records are available for each DC and it resolves to correct domain controllers IP and host name
If any CNAME record is not resolved, just navigate to AD sites \ sitename \ server \servername \ntds seting properties and from general tab copy CNAME and make new record
once your DNS issues get resolved and replication gets restored correctly you will get correct picture of FSMO roles

Also ensure that no lingering objects exists on any DCs
http://technet.microsoft.com/en-us/library/cc949124(v=ws.10).aspx
http://technet.microsoft.com/en-us/library/cc949134(v=ws.10).aspx
Aaron TomoskyDirector of Solutions ConsultingCommented:
was this an upgrade of a 2000 ad? if so, you may be missing the top level _msdcs.domain.com folder in dns.
http://support2.microsoft.com/kb/817470/en-us
ClearBlueTechnologiesAuthor Commented:
It turned out to be an RPC issue caused by our firewalls on the new servers.  I wasn't aware of the ADRepstatus tool suggested by Aaron Tomosky, which revealed the issue pretty clearly.  I used ADRepstatus in combination with Portqry.exe to verify that the necessary ports were not open, which was causing all kinds of replication issues.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Active Directory

From novice to tech pro — start learning today.