Adding a new DC to existing domain

Before you do any major changes to active directory, you need to test AD's health and fix any issues.  DCDIAG /C /E /V and REPADMIN /SHOWREPL should be run and the information provided should be analyzed with any unexplained errors corrected.  If you have a DC that is not replicating anymore it may be tombstoned. If it is you have no choice but to delete it as a failed DC.  Demoting it on the physical (or virtual) server will NOT properly remove it from AD since it can't communicate properly with AD anymore.

Its not showing tombstoned in the tools results below is some additional information. I will run those commands and see what I can find out.

This is the debug file that the server created. Our current DC's are dc01, dc02 and fs02.. There is no mention of fs02 in the output and DC01 was the server that showed it wasn't replicating when I ran the tool


[2020/01/07:14:51:52.212]
Adprep created the log file 'C:\Windows\debug\adprep\logs\20200107145152-test\ADPrep.log'
[2020/01/07:14:51:52.212]
Adprep successfully initialized global variables.

[Status/Consequence]

Adprep is continuing.
[2020/01/07:14:51:52.219]
Adprep was about to call the following LDAP API. ldap_search_s(). The base entry to start the search is CN=Schema,CN=Configuration,DC=domain,DC=com.
[2020/01/07:14:51:52.220]
LDAP API ldap_search_s() finished, return code is 0x0
[2020/01/07:14:51:52.220]
Adprep was about to call the following LDAP API. ldap_search_s(). The base entry to start the search is CN=DC01,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=domain,DC=com.
[2020/01/07:14:51:52.221]
LDAP API ldap_search_s() finished, return code is 0x0
[2020/01/07:14:51:52.224]
Adprep was about to call the following LDAP API. ldap_search_s(). The base entry to start the search is CN=Schema,CN=Configuration,DC=domain,DC=com.
[2020/01/07:14:51:52.225]
LDAP API ldap_search_s() finished, return code is 0x0
[2020/01/07:14:51:52.228]
Adprep was about to call the following LDAP API. ldap_search_s(). The base entry to start the search is CN=Schema,CN=Configuration,DC=domain,DC=com.
[2020/01/07:14:51:52.229]
LDAP API ldap_search_s() finished, return code is 0x0
[2020/01/07:14:51:52.230]
Adprep was about to call the following LDAP API. ldap_search_s(). The base entry to start the search is CN=Infrastructure,DC=domain,DC=com.
[2020/01/07:14:51:52.231]
LDAP API ldap_search_s() finished, return code is 0x0
[2020/01/07:14:51:52.231]
Adprep was about to call the following LDAP API. ldap_search_s(). The base entry to start the search is CN=DC02,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=domain,DC=com.
[2020/01/07:14:51:52.231]
LDAP API ldap_search_s() finished, return code is 0x0
[2020/01/07:14:51:52.236]
Adprep discovered the schema FSMO: DC01.domain.com.
[2020/01/07:14:51:52.238]
Adprep connected to the schema FSMO: DC01.domain.com.
[2020/01/07:14:51:52.238]
Adprep was about to call the following LDAP API. ldap_search_s(). The base entry to start the search is (null).
[2020/01/07:14:51:52.239]
LDAP API ldap_search_s() finished, return code is 0x0
[2020/01/07:14:51:52.239]
Adprep successfully retrieved information from the Active Directory Domain Services.
[2020/01/07:14:51:52.239]
Adprep was about to call the following LDAP API. ldap_search_s(). The base entry to start the search is DC=domain,DC=com.
[2020/01/07:14:51:52.239]
LDAP API ldap_search_s finished, return code is 0x0
[2020/01/07:14:51:52.239]
Adprep was about to call the following LDAP API. ldap_search_s(). The base entry to start the search is (null).
[2020/01/07:14:51:52.239]
LDAP API ldap_search_ext_s finished, return code is 0x0
[2020/01/07:14:51:52.239]
Adprep was about to call the following LDAP API. ldap_search_s(). The base entry to start the search is (null).
[2020/01/07:14:51:52.240]
LDAP API ldap_search_s finished, return code is 0x0
[2020/01/07:14:51:52.252]
Adprep discovered the Infrastructure FSMO: dc02.domain.com.
[2020/01/07:14:51:52.255]
Adprep connected to the Infrastructure FSMO: dc02.domain.com.
[2020/01/07:14:51:52.255]
Adprep was about to call the following LDAP API. ldap_search_s(). The base entry to start the search is (null).
[2020/01/07:14:51:52.255]
LDAP API ldap_search_s() finished, return code is 0x0
[2020/01/07:14:51:52.255]
Adprep successfully retrieved information from the Active Directory Domain Services.
[2020/01/07:14:51:52.255]
Adprep was about to call the following LDAP API. ldap_search_s(). The base entry to start the search is DC=domain,DC=com.
[2020/01/07:14:51:52.256]
LDAP API ldap_search_s finished, return code is 0x0
[2020/01/07:14:51:52.256]
Adprep was about to call the following LDAP API. ldap_search_s(). The base entry to start the search is (null).
[2020/01/07:14:51:52.257]
LDAP API ldap_search_ext_s finished, return code is 0x0
[2020/01/07:14:51:52.257]
Adprep was about to call the following LDAP API. ldap_search_s(). The base entry to start the search is (null).
[2020/01/07:14:51:52.257]
LDAP API ldap_search_s finished, return code is 0x0
[2020/01/07:14:51:52.265]
Adprep failed to verify whether schema master has completed a replication cycle after last reboot.

[Status/Consequence]

The schema is not upgraded.

[User Action]

Check the log file ADPrep.log in the C:\Windows\debug\adprep\logs\20200107145152-test directory for possible cause of failure.
[2020/01/07:14:51:52.265]
Adprep encountered an LDAP error.

Error code: 0xc. Server extended error code: 0x20ae, Server error message: 000020AE: SvcErr: DSID-032103B3, problem 5010 (UNAVAIL_EXTENSION), data 8610



DSID Info:
DSID: 0x180f0975
ldap error = 0xc
NT BUILD: 14393
NT BUILD: 2969

The replication problem may be something simple such as a time sync problem - especially is one of the DCs is a VM.

DCDIAG and REPADMIN are your friends here, but don’t make it worse - your AD is already breaking - don’t break it further by demoting the errant DC. Fix the problem, let replication take place fully, then make the changes you need to make.

The results of the dcdiag are attached, In the output its stating to run
 "repadmin  /options DC01_inbound_REPL"  
 "repadmin /options DC01 -disable_outbound_repl"

Do you think that is correct?
dcdiag.txt

NETLOGON Service is paused on [DC01]

check the netlogon service
might be USN issue (check for event id 2095 in directory service log)

A Windows Server domain controller logs Directory Services event 2095 when it encounters a USN rollback
https://support.microsoft.com/en-us/help/875495/how-to-detect-and-recover-from-a-usn-rollback-in-windows-server-dc

Seth,
The service is in a paused state I did have to revert to a clone about a week or so ago.

I do not see a 2095 event id in the Directory Service logs

Here is an error that points to the USN issue
dc01error.docx

The server in question is being replaced so my question is can I just demote the server that is having replication issues and not worry about fixing the issue because its being replaced anyways.

Nope, as it is having replication issues.
You will probably need to perform a metadata cleanup

The article offers 3 options, only the first two (Gui tools, ntdsutil) are described though.
After the metadata cleanup, no replica errors should show up in AD logs.

Ok, the server in question does hold 2 FSMO roles Schema Master and Domain Naming master.. I have tried to transfer but because of the issues its not allowing me to. Do you think that option 3 is the way to go with that being the case also?

I attempted to seize them but it errored

fsmo maintenance: seize schema master
Attempting safe transfer of schema FSMO before seizure.
ldap_modify_sW error 0x34(52 (Unavailable).
Ldap extended error message is 000020AF: SvcErr: DSID-0321041F, problem 5002 (UN
AVAILABLE), data 8456

Win32 error returned is 0x20af(The requested FSMO operation failed. The current
FSMO holder could not be contacted.)
)
Depending on the error code this may indicate a connection,
ldap, or role transfer error.
Transfer of schema FSMO failed, proceeding with seizure ...
Server "dc02" knows about 5 roles
Schema - CN=NTDS Settings,CN=DC02,CN=Servers,CN=Default-First-Site-Name,CN=Sites
,CN=Configuration,DC=domain,DC=com
Naming Master - CN=NTDS Settings,CN=DC01,CN=Servers,CN=Default-First-Site-Name,C
N=Sites,CN=Configuration,DC=domain,DC=com
PDC - CN=NTDS Settings,CN=DC02,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN
=Configuration,DC=domain,DC=com
RID - CN=NTDS Settings,CN=DC02,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN
=Configuration,DC=domain,DC=com
Infrastructure - CN=NTDS Settings,CN=DC02,CN=Servers,CN=Default-First-Site-Name,
CN=Sites,CN=Configuration,DC=domain,DC=com
fsmo maintenance:

That output indicates that the "friendly" transfer (which it attempts before seizing) failed, but the seizure succeeded:

Transfer of schema FSMO failed, proceeding with seizure ...
Server "dc02" knows about 5 roles
Schema - CN=NTDS Settings,CN=DC02,CN=Servers,CN=Default-First-Site-Name,CN=Sites
,CN=Configuration,DC=domain,DC=com

DC02 is now the owner of the Schema Master role.

You are correct I had ran netdom /query FSMO to verify and I needed to wait a min or so at least that's what I had to do for the naming master I just changed also.

So do you think now I can remove meta data and remove the roles from the server?

After moving those FSMO roles to another server I am no longer erroring out on the AD configuration wizard on the new DC.

Id did fail later in the promotion. I ran dcdiag again and the errors pointed me a MS article that pointed me to the verification that the issue happens because of the clone I put in place. That pointed me to these steps for fixing. I will attempt it tonight after hours if you have additional thoughts please share.

Recovering from a USN rollback

There are three approaches to recover from a USN rollback.


Remove the Domain Controller from the domain

To do this, follow these steps:
1.Remove Active Directory from the domain controller to force it to be a standalone server.

 For more information, see the following article in the Microsoft Knowledge Base:  
332199  Domain controllers do not demote gracefully when you use the Active Directory Installation Wizard to force demotion in Windows Server 2003 and in Windows 2000 Server

2.Shut down the demoted server.
3.On a healthy domain controller, clean up the metadata of the demoted domain controller.

For more information, see the following article in the Microsoft Knowledge Base:  
216498  How to remove data in Active Directory after an unsuccessful domain controller demotion

4.If the incorrectly restored domain controller hosts operations master roles, transfer these roles to a healthy domain controller.

For more information, see the following article in the Microsoft Knowledge Base:  
255504  Using Ntdsutil.exe to transfer or seize FSMO roles to a domain controller

5.Restart the demoted server.
6.If you are required to, install Active Directory on the stand-alone server again.
7.If the domain controller was previously a global catalog, configure the domain controller to be a global catalog.

For more information, see the following article in the Microsoft Knowledge Base:  
313994  How to create or move a global catalog in Windows 2000

8.If the domain controller previously hosted operations master roles, transfer the operations master roles back to the domain controller.

Yes, those are the correct steps. (That's part of the article that Seth Simmons linked above.)

Yep I have to get a couple of things worked out today then change all my dhcp servers with the new DNS entires I already did a few and no issues so far.

I finally got a chance to try the removal process and I had an issue. I was able to remove the DC roles from server manager. When I went to remove the meta data it wouldn't allow me to remove it,  I tried it thru AD Users and Computers, AD Sites and Service and I tried using the ntdsutil and all failed to remove/change.. The next step is to dump the server out of AD but I hesitate to do that since the first step in the instructions for removing the meta data failed.
I was using these instructions posted by a couple of you
https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/deploy/ad-ds-metadata-cleanup

Any suggestions?

Can you show us what errors you received when trying to remove it?

I was able to get this completed. Thanks for all the input