Re-promoting a Windows Server 2019 Domain Controller DC3 in a set of 3.
I have a system with 3 Domain Controllers (Server 2019). One of them got out of sync (my best description) and wasn't replicating - while the other 2 seem to be working fine.
The problem seemed to be RPC errors but all 3 DCs were running together in a test lab before being deployed.
This one was deployed a month or so after the first two, having been powered down during the interim.
The failure would cause me to suspect the real network they are now in somehow - but "how?" is elusive.
I have wireshark captures taken during replication attempts but I don't see much that jumps out.
Eventually, I was advised to take the "bad" DC off the domain and re-promote it.
I've proceeded with thatprocess and have tried a lot of suggested things but so far no luck in promoting this DC.
Fortunately, this DC wasn't in the mainstream of operations and, without replication, had less configuration data on it than the others.
So, taking a step back is of little concern.
I think I did a pretty good job of bringing the DC down but I'm unsure of its status. There's a first time for everything...
I still see remnants of its name on the others for example.
So, I think the first step is to confirm that stuff is removed as required in preparation for promotion.
I've run a bunch of tools but don't know when to be happy or sad re: the results. Just no experience.
So, I'm wasting calendar time by running down blind alleys.
I *did* try to promote this DC but the process won't complete.
The process ends with what appears to be a need for further DNS configuration.
I could use some real help.
The problem seemed to be RPC errors but all 3 DCs were running together in a test lab before being deployed.
This one was deployed a month or so after the first two, having been powered down during the interim.
The failure would cause me to suspect the real network they are now in somehow - but "how?" is elusive.
I have wireshark captures taken during replication attempts but I don't see much that jumps out.
Eventually, I was advised to take the "bad" DC off the domain and re-promote it.
I've proceeded with thatprocess and have tried a lot of suggested things but so far no luck in promoting this DC.
Fortunately, this DC wasn't in the mainstream of operations and, without replication, had less configuration data on it than the others.
So, taking a step back is of little concern.
I think I did a pretty good job of bringing the DC down but I'm unsure of its status. There's a first time for everything...
I still see remnants of its name on the others for example.
So, I think the first step is to confirm that stuff is removed as required in preparation for promotion.
I've run a bunch of tools but don't know when to be happy or sad re: the results. Just no experience.
So, I'm wasting calendar time by running down blind alleys.
I *did* try to promote this DC but the process won't complete.
The process ends with what appears to be a need for further DNS configuration.
I could use some real help.
Use dcdiag to make sure the AD is clean and has no reference to the DC #3
ASKER
arnold: Thank you! What dcdiag test would you recommend exactly for this purpose?
arnold: I did all those things - even though I feel unsure about step 3 being 100% effective.
When I try to promote the DC, here is what I get:
arnold: I did all those things - even though I feel unsure about step 3 being 100% effective.
When I try to promote the DC, here is what I get:
Check which name servers are configured on this system? It could still have configuration remnants I.e. Points to itself for name resolutions.
ipconfig /all | find /i "name server"
dcdiag shoukd be run on the DCs #1 and #2.
#3 is no longer a DC, sounds missed that in step 2 prior to disjoining from the domain to demote the DC.
often when disjoin, rejoin, it says that it located an existing object and whether you want to use it..
ipconfig /all | find /i "name server"
dcdiag shoukd be run on the DCs #1 and #2.
#3 is no longer a DC, sounds missed that in step 2 prior to disjoining from the domain to demote the DC.
often when disjoin, rejoin, it says that it located an existing object and whether you want to use it..
ASKER
dcdiag shoukd be run on the DCs #1 and #2.Yes, but which of the many commands do you recommend?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Why three DCs? Except in RARE circumstances, you should never need more than 2 per site. Do you have multiple sites?
It's unclear how you removed the DC from the network. If it got "out of sync" by more than 60 days, then you needed to treat it as a failed DC and remove it as such. For the most part, deleting the "failed" DC from the Domain Controllers OU should be largely sufficient, but you would still need to clean up some items in DNS and likely AD Sites and Services.
If it got corrupted and AD was no longer replicating, there may have been other ways to resolve the issues, but if you
As for DCDIAG Before promoting a second (or subsequent DC), I always run DCDIAG /C /E /V from all DCs (in a small environment). Then I resolve any unexplained issues before attempting to promote another system to be a DC.
It's unclear how you removed the DC from the network. If it got "out of sync" by more than 60 days, then you needed to treat it as a failed DC and remove it as such. For the most part, deleting the "failed" DC from the Domain Controllers OU should be largely sufficient, but you would still need to clean up some items in DNS and likely AD Sites and Services.
If it got corrupted and AD was no longer replicating, there may have been other ways to resolve the issues, but if you
As for DCDIAG Before promoting a second (or subsequent DC), I always run DCDIAG /C /E /V from all DCs (in a small environment). Then I resolve any unexplained issues before attempting to promote another system to be a DC.
ASKER
Lee W: Just FYI
There are 3 sites that are all fairly tightly coupled with fiber. There are disaster recovery plans in place that the 3 DCs support.
We originally thought of having 2 DCs (one per site again). So, except for the objectives, the system is operationally fine less one DC.
We have rarely had the fiber system go down but it did happen once - and recently.
The network was brought up entirely off site.
Then, our integration process left the last one out too long.
And, just now, we learned that we needed to treat it as a failed DC so have removed it.
And here we are....
arnold and Lee W: Thank you both for the guidance re: dcdiag! That saves a lot of time sorting through the tools. I don't mind figuring it out myself but in this case there are a lot of words being used that aren't immediately familiar re: meaning. That's a big deal in a practical sense. So thanks for being translators!
There are 3 sites that are all fairly tightly coupled with fiber. There are disaster recovery plans in place that the 3 DCs support.
We originally thought of having 2 DCs (one per site again). So, except for the objectives, the system is operationally fine less one DC.
We have rarely had the fiber system go down but it did happen once - and recently.
The network was brought up entirely off site.
Then, our integration process left the last one out too long.
And, just now, we learned that we needed to treat it as a failed DC so have removed it.
And here we are....
arnold and Lee W: Thank you both for the guidance re: dcdiag! That saves a lot of time sorting through the tools. I don't mind figuring it out myself but in this case there are a lot of words being used that aren't immediately familiar re: meaning. That's a big deal in a practical sense. So thanks for being translators!
ASKER
OK, so those have been run and I'll analyze them in the morning.
As I mentioned, I originally had all 3 working together and then moved #1 and #2 out into the real world.
However, #3 was in the original configurations and was destined to be reunited with the others.
So now I still see #3 in the configurations of #1 and #2 and need to remove it.
As I mentioned, I originally had all 3 working together and then moved #1 and #2 out into the real world.
However, #3 was in the original configurations and was destined to be reunited with the others.
So now I still see #3 in the configurations of #1 and #2 and need to remove it.
ASKER
As I look through the DCs and the logs, I notice this redundancy:
I imagine it should be removed but I need to decide which one?
If I were to guess, I'd say the second, longer entry.
It seems like I did this before but it's still there (if I did).
I imagine it should be removed but I need to decide which one?If I were to guess, I'd say the second, longer entry.
It seems like I did this before but it's still there (if I did).
Does each DC have multiple IPs?
i.e. DC2 how many IPs does it have one ipv4 and one ipv6?
You can check each connection's properties to see what it is.
i.e. DC2 how many IPs does it have one ipv4 and one ipv6?
You can check each connection's properties to see what it is.
ASKER
Does each DC have multiple IPs?
Not ipv4. Only one.
i.e. DC2 how many IPs does it have one ipv4 and one ipv6?
Yes. One each.
You can check each connection's properties to see what it is.
If you mean in NIC properties, right?
I have come to just let ipv6 exist with no changes.
I did read that turning off ipv6 solved a DC re-promoting problem.... but I couldn't tell how old the solution was.
Is that still recommended - or recommended at times?
The SAME THING is true of #1 but the display doesn't show the redundancy...
Not ipv4. Only one.
i.e. DC2 how many IPs does it have one ipv4 and one ipv6?
Yes. One each.
You can check each connection's properties to see what it is.
If you mean in NIC properties, right?
I have come to just let ipv6 exist with no changes.
I did read that turning off ipv6 solved a DC re-promoting problem.... but I couldn't tell how old the solution was.
Is that still recommended - or recommended at times?
The SAME THING is true of #1 but the display doesn't show the redundancy...
ASKER
#3 (DC1-RAY) is NOT any longer listed in the NIC DNS entries on #1 nor #2. Yet:
On #1 I get:
On #2 I get the same thing.
I *think* I've done everything to get rid of these references, yet they are still there....
On #1 I get:
DNS server: x.x.1.2 (DC1-RAY)
1 test failure on this DNS server
PTR record query for the 1.0.0.127.in-addr.arpa. failed on the DNS server x.x.1.2 [Error details: 1460 (Type: Win32 - Description: This operation returned because the timeout period expired.)]
Name resolution is not functional. _ldap._tcp.net.rayfed.com. failed on the DNS server x.x.1.2
[Error details: 1460 (Type: Win32 - Description: This operation returned because the timeout period expired.)]On #2 I get the same thing.
I *think* I've done everything to get rid of these references, yet they are still there....
The error is not significant it merely means that the localhost 127.0.0.1 could not be looked up by ip.
The fix to avoid the warning is to open DNS management add a reverse zone 127.0.0 and to add a PTR record 1 that points to localhost.
It might be using the name server records that point to the localhost versus LAN IPs.
The fix to avoid the warning is to open DNS management add a reverse zone 127.0.0 and to add a PTR record 1 that points to localhost.
It might be using the name server records that point to the localhost versus LAN IPs.
In ntds you can select each entry from the right pane of the image and get properties it will tell you the type of relationship, ..
ASKER
I can go so far as to figure out that you mean: Active Directory Sites and Services / Sites / [site] / Servers / DCxyz / NTDS Settings. Right?
But truthfully, shorthand is tough for me right now because of my limited language in this community.
Now "type of relationship" is completely lost on me. What type am I looking for and where?
But truthfully, shorthand is tough for me right now because of my limited language in this community.
Now "type of relationship" is completely lost on me. What type am I looking for and where?
In the image where you have two connections from dc2 reflected. Select one and right-click select properties. Within it will show you info. You can then do the same for the other vonnection and it will indicate the distinction between the two.
On newer servers, I think since 2012 the IPv4/IPv6 should not be disabled. Those suggestions for IPv6 disabling dealt. With issues when newer 2008, 2012 were being added to existing 2003 AD ....
On newer servers, I think since 2012 the IPv4/IPv6 should not be disabled. Those suggestions for IPv6 disabling dealt. With issues when newer 2008, 2012 were being added to existing 2003 AD ....
ASKER
arnold: Thank you again for the quick reply!
So the ipv4/ipv6 question is settled (again) :-)
I asked "What am I looking for and where?"
I can look through the 4 tabs that are presented and they look pretty much the same.
I can open both of them up side-by-side in Attribute Editor and scroll down through the long list of Attributes and, again, I see nothing much useful. The difference is their names which we can see in the NTDS display on the right. If there's another difference of note, I've not found it.
Anyway, I followed the route of adding a PTR record to 127.0.0.1. We'll see how that affects things.
Thanks again!
So the ipv4/ipv6 question is settled (again) :-)
"it will indicate the distinction between the two"Well, that may be obvious to you but it sure isn't obvious to me.
I asked "What am I looking for and where?"
I can look through the 4 tabs that are presented and they look pretty much the same.
I can open both of them up side-by-side in Attribute Editor and scroll down through the long list of Attributes and, again, I see nothing much useful. The difference is their names which we can see in the NTDS display on the right. If there's another difference of note, I've not found it.
Anyway, I followed the route of adding a PTR record to 127.0.0.1. We'll see how that affects things.
Thanks again!
Sorry, initially I had the impression that dc#3 us the issue. In your most recent comments including the images, dc3 but you reference with issues in dc1.
I am uncertain what state/stage you are in.
Metadata cleanup to remove the DC you unjointed.
The old object deleted from ad?
Recheck the network configuration on the demoted ad to make sure name server records point to existing DCs.
The link to the ..... That the name server/DC can be reached.
nslookup -q=SRV _ldap._tcp.dc._msdcs.youra
Since you have multiple sites the above query instead of DC which reflects all DCs in the environment would be narrowed to DCs at the soecific site if ...
Try to rejoin this system as another DC.
I am uncertain what state/stage you are in.
Metadata cleanup to remove the DC you unjointed.
The old object deleted from ad?
Recheck the network configuration on the demoted ad to make sure name server records point to existing DCs.
The link to the ..... That the name server/DC can be reached.
nslookup -q=SRV _ldap._tcp.dc._msdcs.youra
Since you have multiple sites the above query instead of DC which reflects all DCs in the environment would be narrowed to DCs at the soecific site if ...
Try to rejoin this system as another DC.
ASKER
I'm sorry about the numbering. There's a difference between the question here #1, #2, #3 and the actual names DC1=#3, DC2=#1, DC3=#2. I was rather hoping to not have to confuse with that.
I'm nearing the end of the cleanup I believe which includes the old object deleted from AD.
And, I have the impression that a pristine system of 2 operating DCs is rather important before embarking on rejoining #3.
Thus the DCDIAG reports / discussion on #1 and #2.
I *may* be ready to begin the rejoin process.
I'm nearing the end of the cleanup I believe which includes the old object deleted from AD.
And, I have the impression that a pristine system of 2 operating DCs is rather important before embarking on rejoining #3.
Thus the DCDIAG reports / discussion on #1 and #2.
I *may* be ready to begin the rejoin process.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
arnold: OK. Thanks!
ASKER
Thanks all!
This one was deployed a month or so after the first two, having been powered down during the interim."
The object has been tombstoned.
You need to do it in the following stages.
1) make sure you have a local account on this DC.
2) disjoin it from the domain
3) do metadata cleanup to remove all references to this DC including deleting its object from the ad computers...
4) addict it as a new member/DC.