Upgrading from AD 2003 to 2012

I have two 2012 domain controllers (Prometheus and Chronos), and both the Domain and Functional Level are at Windows Server 2003.  We've migrated from Windows 2000 to 2003 domain many years ago.  Everything has been working fine, and have not seen any errors with AD or replication, and we have not experiences any issues DNS or communications issues with our on-prem Exchange 2010 server.

All of our computers are Windows 10 Pro Computers.

So, I decided to view Active Directory Sites and Services, and discovered an DC (Nemesis) is still listed under server, but should not be there.  I believe it was a former DC and was properly demoted many years back.

Nemesis - Sites and Services
So, I am not sure whether I should simply delete it.  I do not want to start having AD issues.  

So, I also decided to check DNS.  As you can see below within the forward lookup zones (.msdcs.ch13.local and ch13.local) Nemesis appears a name server.  The ch13.local\_msdcs only showing this server (Nemesis), and not the other servers (Prometheus or Chronos) does not appear to be correct.

DNS View
So, I don't know whether I need to delete "Nemesis" Server within Sites and Services and also the entries within DNS.  Not sure doing so will actually cause damage and not help.  Remember, AD and DNS have been working fine with no issues/errors for many years.

Below I added the output for Repadmin /Showrepl and DCDiag with details including DNs.

Repadmin-showrepl-Prometheus.txt

Repadmin-showrepl-Prometheus.txt
cmp119IT ManagerAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

footechCommented:
Yes, delete it under Sites and Services.  Then I would also go through the process of a metadata cleanup to verify there are no remnants of it left behind (though it wouldn't surprise me if there weren't).

For DNS you need to manually update your nameservers for each zone (and also the _msdcs delegation).  Only existing nameservers should be listed.  If you ran dcdiag /test:dns the delegation as it stands right now would get flagged.  Things can often run fine with old entries, but it can cause confusion and there's no reason to keep them around.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
MASEE Solution Guide - Technical Dept HeadCommented:
Cleanup ADmetadata, delete all entries from DNS servers as well.
https://www.petri.com/delete_failed_dcs_from_ad
FOXActive Directory/Exchange EngineerCommented:
It seems like they didn't do the metadata clean up

1. Verify if DHCP has that DNS server pushing to the workstations- if so remove it.
ref link:  https://www.petri.com/forums/forum/microsoft-networking-services/dhcp/56204-change-dns-in-dhcp-server

I'm sure it can't be holding any fsmo roles or else you would get errors as it is non existent
run this at command line  
Netdom query fsmo

Once you are sure nothing is pointing to it.  Find it in Active Directory> Domain Controllers and Delete it
ref link:  https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/manage/ad-forest-recovery-cleaning-metadata

MetaDataCleanup- if you have to force
ref link: https://www.petri.com/delete_failed_dcs_from_ad
Active Protection takes the fight to cryptojacking

While there were several headline-grabbing ransomware attacks during in 2017, another big threat started appearing at the same time that didn’t get the same coverage – illicit cryptomining.

Lee W, MVPTechnology and Business Process AdvisorCommented:
When you demote a DC or otherwise remove it from AD, it doesn't always remove information from Sites and Services and DNS.  I make a point of checking over these whenever I remove a DC and then manually deleting it.  I would ALWAYS make backups first, but you should be safe deleting those items manually, provided you have run DCDIAG and there are no issues especially regarding the "nemesis" machine.
DrDave242Senior Support EngineerCommented:
I suspect a metadata cleanup of Nemesis has already been performed sometime in the past. In fact, I'd bet money that it was removed improperly and then had a metadata cleanup performed, rather than being demoted properly.

Notice in the AD Sites and Services screenshot that there's no "expand" arrow next to Nemesis like there is next to the other two DCs. This indicates that Nemesis doesn't have an NTDS Settings object associated with it; it's just an empty object in that console. The repadmin output shows that neither of the other DCs is attempting to replicate from it, so it's not really doing anything at all.

It's quite common, if not universal, for a metadata cleanup to leave behind an empty server object in Sites and Services, as well as some DNS records. I'm pretty sure I've had to manually remove that stuff every time I've ever performed a metadata cleanup.
cmp119IT ManagerAuthor Commented:
The following displays the result of dcdiag test dns.  You can see TEST: Delegations failed with Error:  DNS server: nemesis.ch13.local.

I will look at the link MAS provided.

dcdiag-testdns.txt

Referencing Fox' s comments:  

DHCP does not have Nemesis server listed.  It has both Prometheus and Chronos IPs listed.

The server Promotheus holds all FSMO roles.

Within Active Directory Users and Computers\Domain Controllers, only Prometheus and Chronos are listed.  

Referencing the link to find it in AD>DC and delete it link, it indicates using RSAT.  Since I am doing all of this directly from the domain contoller, I do not believe RSAT is necessary.  

Referencing the MetaDataCleanup link, I believe it pertains to failed DC promo issues and the need to reinstall DC with the same name: Nemesis.  Dcpromo on this server did not fail, and I do not plan on installing future DCs with this name.
Lee W, MVPTechnology and Business Process AdvisorCommented:
You can't have remnants of a DC in a healthy active directory.  If your AD (as per Metadata Cleanup) still sees the Nemesis DC, then you need to clean it up not just ignore it.

As for the DCDIAG tests, make sure you don't still have an entry for it in DNS as a Name Server for the domain or forwarder or something. Check the properties of the DNS servers AND the zones in DNS management.
DrDave242Senior Support EngineerCommented:
The following displays the result of dcdiag test dns.  You can see TEST: Delegations failed with Error:  DNS server: nemesis.ch13.local.

This test failed because the only server listed in the delegation is nonexistent. To fix this, right-click on the delegation record (the gray _msdcs folder in the ch13.local zone) and select Properties. Remove Nemesis from the list of name servers for the delegation and add the other two DCs.
MASEE Solution Guide - Technical Dept HeadCommented:
Please be extremely careful while working on ADSI. Wrong deletion may end up in full forest reconstruction.
cmp119IT ManagerAuthor Commented:
DrDave242 -

Okay I am looking at this and ch13.local zone name servers already has the two DCS listed, but _msdcs on lists the missing DC: Nemesis.  So, I am concerned that if I remove Nemesis, and the other two DCS are listed as Name Server that maybe no name servers will be listed at all.

Nemesis Name Server
DrDave242Senior Support EngineerCommented:
The gray _msdcs record is a delegation record, which indicates that any queries for records inside the _msdcs.ch13.local namespace should be sent to a specified list of servers - the name servers listed in the delegation record. This is separate from the list of name servers for the ch13.local zone, which is what you're looking at in that screenshot (although Nemesis should be removed from that list as well, since it's no longer a valid name server for that zone).

In your case, it doesn't technically matter that the delegation record has an incorrect list of name servers, because all of the servers that host the ch13.local zone also host the _msdcs.ch13.local zone, so your two DCs are effectively delegating the zone to themselves. In other words, any query for a record in the _msdcs.ch13.local zone is going to be sent to one of those two servers anyway, so the delegation isn't really serving any purpose; it's just there by default. Correcting its list of name servers will "clean up" DNS and should cause that dcdiag test to pass, but it won't have any real-world effect on DNS resolution in your domain, which is already functional.
cmp119IT ManagerAuthor Commented:
Okay, on a closed test network (no internet either) I spun both DCs, and removed Nemesis server NS server for both ch13.local and _msdcs.ch13.local zones.  

Note, ch13.local\_msdcs still displays Nemesis as a NS.  Not sure that matters or if I should simply remove and no name servers will be listed.

I ran dcdiag /test:dns, and it still states Test: Delegation DNS ServerL nemesis unavailable missing glue a record.

New Dcdiag on test network.
DrDave242Senior Support EngineerCommented:
Note, ch13.local\_msdcs still displays Nemesis as a NS.  Not sure that matters or if I should simply remove and no name servers will be listed.

Remove Nemesis and then add the other two DCs, since they're the correct name servers for that zone.
cmp119IT ManagerAuthor Commented:
I think that's the problem, they are already listed.  I simply removed Nemesis and left the two servers as NS.

Promtheus and Chronos not listed NS
cmp119IT ManagerAuthor Commented:
I messed up.  I did not change the NS for _msdcs, and have done so now.  Sorry or mixup.
cmp119IT ManagerAuthor Commented:
Okay, I updated the NS servers so that Nemesis is now removed from DNS, and also from Sites and Services.  I am speaking of the closed test network.

I ran dcdiag /test:dns and nemesis is no longer coming up with an error.  However, ch13.local failed the test DNS.

Closed Network dns test failed
footechCommented:
This isn't related to cleaning up the invalid records.  I'd say it's due to being on a closed network.  You can't query the root hints or forwarders while on it.
cmp119IT ManagerAuthor Commented:
Okay over the weekend, I updated DNS by removing "Nemesis" as a name server.  Immediately after updating  results appear clean now.  See attached results.

However, after carefully reviewing DNS - _msdcs.ch13.local, I can see under domains the container seems a bit odd or out of the norm.

DNS After Changedcdiag-21JUL2018.txt
DrDave242Senior Support EngineerCommented:
I don't see anything wrong in that screenshot. It matches my own two-DC test domain:

DNS Manager screenshot
If you're concerned about the name of the folder under domains, that's normal. The name of that folder is the GUID of the domain.
cmp119IT ManagerAuthor Commented:
Thanks for letting me know about the GUID.  

Okay, dcdiag results look good and DNS looks good as well.

Referencing Fox's comments:

1. Verify if DHCP has that DNS server pushing to the workstations- if so remove it.  DHCP has no entries relating the defunct server (Nemesis).

I'm sure it can't be holding any fsmo roles or else you would get errors as it is non existent:  DC (Prometheus) holds all FSMO roles


Once you are sure nothing is pointing to it.  Find it in Active Directory> Domain Controllers and Delete it
ref link:  https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/manage/ad-forest-recovery-cleaning-metadata

MetaDataCleanup- if you have to force
ref link: https://www.petri.com/delete_failed_dcs_from_ad

I followed the MetaDataCleanup steps on DC Prometheus by spinning it up on a Closed Test Network, and the part pertaining to:

"Select operation target: list servers in site" reveals the existing DCs (Prometheus & Chronos).  To me that indicates Metadata Cleanup is not necessary on the live domain/DCs.

Metadata Cleanup Result
So, I presume I am good to go on the live domain/DCS by removing server:  "Nemesis" Sites and Services.

If so, I would think we should be good to go as far as raising the domain and forest functional levels to Windows Server 2012.

Before I actually commit to raising the domain functional levels, what exactly is supposed to happen?  I mean will new features immediately pop up in AD what weren't there before, or will I need to download updates, or will a DC reboot be necessary for everything to update, etc.

I am just wondering what sort of changes to expect after changing the functional levels. I also want to ensure Exchange 2010 will not have any problems with AD or DNS after doing so.
DrDave242Senior Support EngineerCommented:
Raising the functional levels is a very quick procedure, and you don't have to reboot or install updates afterward for new features to show up. You won't notice anything different, though, unless you go looking for it. This article contains the list of features available at every forest and domain functional level:

https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/active-directory-functional-levels

Since you're going from 2003 to 2012, the most important feature in terms of day-to-day operation is likely to be the AD Recycle Bin, which is available beginning at the 2008 R2 forest functional level. I recommend enabling it ASAP after you're done raising the functional levels, because it makes restoring deleted AD objects a lot simpler. Be advised, though, that any AD object that was deleted before enabling the recycle bin will be unrecoverable once it's enabled. Also, it can't be disabled once it's enabled, but why would you want to?

Oh, the availability of DFS-R for SYSVOL replication is pretty important too. DFS-R is much more robust than FRS, and you'll want to consider migrating to it at some point in the not-too-distant future.
cmp119IT ManagerAuthor Commented:
Final question, can I raise the functional level now, during production hours with users are busy working, or do I need to wait after work hours to do so.
DrDave242Senior Support EngineerCommented:
I hesitate a little bit to say that you can do it now, just because I can't positively guarantee that nothing will go wrong, but I've personally never experienced a problem when raising the functional levels. It's an almost instantaneous operation that won't interfere with users at all.
cmp119IT ManagerAuthor Commented:
Just to be on the safe side, I will raise the levels either tonight remotely or when I get into the office in the morning.  I will respond and update you on the results.  Thanks.
cmp119IT ManagerAuthor Commented:
One more thing, I will raise the domain functional level first, and then the forest level.  Do I need to wait a period of time between raising each level, or can I simply do it one right after another.  I figure if raising the domain functional level is successful, then I would think I can go ahead and immediately update the forest level.
DrDave242Senior Support EngineerCommented:
You are correct; there's no need to wait if the domain operation is successful.
footechCommented:
I seem to recall needing to restart the Kerberos service (if the DC wasn't restarted) after raising the DFL from 2003.
https://blogs.technet.microsoft.com/exchange/2015/02/13/considering-updating-your-domain-functional-level-from-windows-2003-read-this/
cmp119IT ManagerAuthor Commented:
Okay, last night I successfully raised the domain and forest levels to Windows Server 2012 without issue.  I did not see any Event ID 10 or 14 errors in the system event viewer on both DCs, but decided to restart all servers on the domain several hours after raising the levels. All servers and DNS appear to be functioning without issue.  Exchange is also working as expected.  I also activated/enabled AD recycle bin as well.  

Thank you all for your assistance on this matter.
DrDave242Senior Support EngineerCommented:
Glad to hear it!
cmp119IT ManagerAuthor Commented:
I thought I closed this question assigned the best and assisted solutions.  Can you please confirm this case is closed now.  Thanks.
MASEE Solution Guide - Technical Dept HeadCommented:
Hi cmp119,
Please check this article to close the question
How do I a close a question

MAS
EE Solution Guide
cmp119IT ManagerAuthor Commented:
That's the problem, there is not option I have my answer, and the sidebar only displays close question.  I already applied the best and assisted solutions, so there is nothing I can do on my end.
DrDave242Senior Support EngineerCommented:
Several experts made meaningful contributions here. If anyone feels that points should be allocated differently, don't hesitate to object.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows OS

From novice to tech pro — start learning today.