Upgrading from AD 2003 to 2012

cmp119
cmp119 used Ask the Experts™
on
I have two 2012 domain controllers (Prometheus and Chronos), and both the Domain and Functional Level are at Windows Server 2003.  We've migrated from Windows 2000 to 2003 domain many years ago.  Everything has been working fine, and have not seen any errors with AD or replication, and we have not experiences any issues DNS or communications issues with our on-prem Exchange 2010 server.

All of our computers are Windows 10 Pro Computers.

So, I decided to view Active Directory Sites and Services, and discovered an DC (Nemesis) is still listed under server, but should not be there.  I believe it was a former DC and was properly demoted many years back.

Nemesis - Sites and Services
So, I am not sure whether I should simply delete it.  I do not want to start having AD issues.  

So, I also decided to check DNS.  As you can see below within the forward lookup zones (.msdcs.ch13.local and ch13.local) Nemesis appears a name server.  The ch13.local\_msdcs only showing this server (Nemesis), and not the other servers (Prometheus or Chronos) does not appear to be correct.

DNS View
So, I don't know whether I need to delete "Nemesis" Server within Sites and Services and also the entries within DNS.  Not sure doing so will actually cause damage and not help.  Remember, AD and DNS have been working fine with no issues/errors for many years.

Below I added the output for Repadmin /Showrepl and DCDiag with details including DNs.

Repadmin-showrepl-Prometheus.txt

Repadmin-showrepl-Prometheus.txt
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Top Expert 2014
Commented:
Yes, delete it under Sites and Services.  Then I would also go through the process of a metadata cleanup to verify there are no remnants of it left behind (though it wouldn't surprise me if there weren't).

For DNS you need to manually update your nameservers for each zone (and also the _msdcs delegation).  Only existing nameservers should be listed.  If you ran dcdiag /test:dns the delegation as it stands right now would get flagged.  Things can often run fine with old entries, but it can cause confusion and there's no reason to keep them around.
MASEE Solution Guide - Technical Dept Head
Most Valuable Expert 2017

Commented:
Cleanup ADmetadata, delete all entries from DNS servers as well.
https://www.petri.com/delete_failed_dcs_from_ad
FOXActive Directory/Exchange Engineer
Top Expert 2015

Commented:
It seems like they didn't do the metadata clean up

1. Verify if DHCP has that DNS server pushing to the workstations- if so remove it.
ref link:  https://www.petri.com/forums/forum/microsoft-networking-services/dhcp/56204-change-dns-in-dhcp-server

I'm sure it can't be holding any fsmo roles or else you would get errors as it is non existent
run this at command line  
Netdom query fsmo

Once you are sure nothing is pointing to it.  Find it in Active Directory> Domain Controllers and Delete it
ref link:  https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/manage/ad-forest-recovery-cleaning-metadata

MetaDataCleanup- if you have to force
ref link: https://www.petri.com/delete_failed_dcs_from_ad
Ensure you’re charging the right price for your IT

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

Lee W, MVPTechnology and Business Process Advisor
Most Valuable Expert 2013
Commented:
When you demote a DC or otherwise remove it from AD, it doesn't always remove information from Sites and Services and DNS.  I make a point of checking over these whenever I remove a DC and then manually deleting it.  I would ALWAYS make backups first, but you should be safe deleting those items manually, provided you have run DCDIAG and there are no issues especially regarding the "nemesis" machine.
DrDave242Principal Support Engineer
Commented:
I suspect a metadata cleanup of Nemesis has already been performed sometime in the past. In fact, I'd bet money that it was removed improperly and then had a metadata cleanup performed, rather than being demoted properly.

Notice in the AD Sites and Services screenshot that there's no "expand" arrow next to Nemesis like there is next to the other two DCs. This indicates that Nemesis doesn't have an NTDS Settings object associated with it; it's just an empty object in that console. The repadmin output shows that neither of the other DCs is attempting to replicate from it, so it's not really doing anything at all.

It's quite common, if not universal, for a metadata cleanup to leave behind an empty server object in Sites and Services, as well as some DNS records. I'm pretty sure I've had to manually remove that stuff every time I've ever performed a metadata cleanup.
cmp119IT Manager

Author

Commented:
The following displays the result of dcdiag test dns.  You can see TEST: Delegations failed with Error:  DNS server: nemesis.ch13.local.

I will look at the link MAS provided.

dcdiag-testdns.txt

Referencing Fox' s comments:  

DHCP does not have Nemesis server listed.  It has both Prometheus and Chronos IPs listed.

The server Promotheus holds all FSMO roles.

Within Active Directory Users and Computers\Domain Controllers, only Prometheus and Chronos are listed.  

Referencing the link to find it in AD>DC and delete it link, it indicates using RSAT.  Since I am doing all of this directly from the domain contoller, I do not believe RSAT is necessary.  

Referencing the MetaDataCleanup link, I believe it pertains to failed DC promo issues and the need to reinstall DC with the same name: Nemesis.  Dcpromo on this server did not fail, and I do not plan on installing future DCs with this name.
Lee W, MVPTechnology and Business Process Advisor
Most Valuable Expert 2013
Commented:
You can't have remnants of a DC in a healthy active directory.  If your AD (as per Metadata Cleanup) still sees the Nemesis DC, then you need to clean it up not just ignore it.

As for the DCDIAG tests, make sure you don't still have an entry for it in DNS as a Name Server for the domain or forwarder or something. Check the properties of the DNS servers AND the zones in DNS management.
DrDave242Principal Support Engineer
Commented:
The following displays the result of dcdiag test dns.  You can see TEST: Delegations failed with Error:  DNS server: nemesis.ch13.local.

This test failed because the only server listed in the delegation is nonexistent. To fix this, right-click on the delegation record (the gray _msdcs folder in the ch13.local zone) and select Properties. Remove Nemesis from the list of name servers for the delegation and add the other two DCs.
MASEE Solution Guide - Technical Dept Head
Most Valuable Expert 2017

Commented:
Please be extremely careful while working on ADSI. Wrong deletion may end up in full forest reconstruction.
cmp119IT Manager

Author

Commented:
DrDave242 -

Okay I am looking at this and ch13.local zone name servers already has the two DCS listed, but _msdcs on lists the missing DC: Nemesis.  So, I am concerned that if I remove Nemesis, and the other two DCS are listed as Name Server that maybe no name servers will be listed at all.

Nemesis Name Server
DrDave242Principal Support Engineer
Commented:
The gray _msdcs record is a delegation record, which indicates that any queries for records inside the _msdcs.ch13.local namespace should be sent to a specified list of servers - the name servers listed in the delegation record. This is separate from the list of name servers for the ch13.local zone, which is what you're looking at in that screenshot (although Nemesis should be removed from that list as well, since it's no longer a valid name server for that zone).

In your case, it doesn't technically matter that the delegation record has an incorrect list of name servers, because all of the servers that host the ch13.local zone also host the _msdcs.ch13.local zone, so your two DCs are effectively delegating the zone to themselves. In other words, any query for a record in the _msdcs.ch13.local zone is going to be sent to one of those two servers anyway, so the delegation isn't really serving any purpose; it's just there by default. Correcting its list of name servers will "clean up" DNS and should cause that dcdiag test to pass, but it won't have any real-world effect on DNS resolution in your domain, which is already functional.
cmp119IT Manager

Author

Commented:
Okay, on a closed test network (no internet either) I spun both DCs, and removed Nemesis server NS server for both ch13.local and _msdcs.ch13.local zones.  

Note, ch13.local\_msdcs still displays Nemesis as a NS.  Not sure that matters or if I should simply remove and no name servers will be listed.

I ran dcdiag /test:dns, and it still states Test: Delegation DNS ServerL nemesis unavailable missing glue a record.

New Dcdiag on test network.
DrDave242Principal Support Engineer

Commented:
Note, ch13.local\_msdcs still displays Nemesis as a NS.  Not sure that matters or if I should simply remove and no name servers will be listed.

Remove Nemesis and then add the other two DCs, since they're the correct name servers for that zone.
cmp119IT Manager

Author

Commented:
I think that's the problem, they are already listed.  I simply removed Nemesis and left the two servers as NS.

Promtheus and Chronos not listed NS
cmp119IT Manager

Author

Commented:
I messed up.  I did not change the NS for _msdcs, and have done so now.  Sorry or mixup.
cmp119IT Manager

Author

Commented:
Okay, I updated the NS servers so that Nemesis is now removed from DNS, and also from Sites and Services.  I am speaking of the closed test network.

I ran dcdiag /test:dns and nemesis is no longer coming up with an error.  However, ch13.local failed the test DNS.

Closed Network dns test failed
Top Expert 2014
Commented:
This isn't related to cleaning up the invalid records.  I'd say it's due to being on a closed network.  You can't query the root hints or forwarders while on it.
cmp119IT Manager

Author

Commented:
Okay over the weekend, I updated DNS by removing "Nemesis" as a name server.  Immediately after updating  results appear clean now.  See attached results.

However, after carefully reviewing DNS - _msdcs.ch13.local, I can see under domains the container seems a bit odd or out of the norm.

DNS After Changedcdiag-21JUL2018.txt
DrDave242Principal Support Engineer

Commented:
I don't see anything wrong in that screenshot. It matches my own two-DC test domain:

DNS Manager screenshot
If you're concerned about the name of the folder under domains, that's normal. The name of that folder is the GUID of the domain.
cmp119IT Manager

Author

Commented:
Thanks for letting me know about the GUID.  

Okay, dcdiag results look good and DNS looks good as well.

Referencing Fox's comments:

1. Verify if DHCP has that DNS server pushing to the workstations- if so remove it.  DHCP has no entries relating the defunct server (Nemesis).

I'm sure it can't be holding any fsmo roles or else you would get errors as it is non existent:  DC (Prometheus) holds all FSMO roles


Once you are sure nothing is pointing to it.  Find it in Active Directory> Domain Controllers and Delete it
ref link:  https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/manage/ad-forest-recovery-cleaning-metadata

MetaDataCleanup- if you have to force
ref link: https://www.petri.com/delete_failed_dcs_from_ad

I followed the MetaDataCleanup steps on DC Prometheus by spinning it up on a Closed Test Network, and the part pertaining to:

"Select operation target: list servers in site" reveals the existing DCs (Prometheus & Chronos).  To me that indicates Metadata Cleanup is not necessary on the live domain/DCs.

Metadata Cleanup Result
So, I presume I am good to go on the live domain/DCS by removing server:  "Nemesis" Sites and Services.

If so, I would think we should be good to go as far as raising the domain and forest functional levels to Windows Server 2012.

Before I actually commit to raising the domain functional levels, what exactly is supposed to happen?  I mean will new features immediately pop up in AD what weren't there before, or will I need to download updates, or will a DC reboot be necessary for everything to update, etc.

I am just wondering what sort of changes to expect after changing the functional levels. I also want to ensure Exchange 2010 will not have any problems with AD or DNS after doing so.
DrDave242Principal Support Engineer
Commented:
Raising the functional levels is a very quick procedure, and you don't have to reboot or install updates afterward for new features to show up. You won't notice anything different, though, unless you go looking for it. This article contains the list of features available at every forest and domain functional level:

https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/active-directory-functional-levels

Since you're going from 2003 to 2012, the most important feature in terms of day-to-day operation is likely to be the AD Recycle Bin, which is available beginning at the 2008 R2 forest functional level. I recommend enabling it ASAP after you're done raising the functional levels, because it makes restoring deleted AD objects a lot simpler. Be advised, though, that any AD object that was deleted before enabling the recycle bin will be unrecoverable once it's enabled. Also, it can't be disabled once it's enabled, but why would you want to?

Oh, the availability of DFS-R for SYSVOL replication is pretty important too. DFS-R is much more robust than FRS, and you'll want to consider migrating to it at some point in the not-too-distant future.
cmp119IT Manager

Author

Commented:
Final question, can I raise the functional level now, during production hours with users are busy working, or do I need to wait after work hours to do so.
DrDave242Principal Support Engineer

Commented:
I hesitate a little bit to say that you can do it now, just because I can't positively guarantee that nothing will go wrong, but I've personally never experienced a problem when raising the functional levels. It's an almost instantaneous operation that won't interfere with users at all.
cmp119IT Manager

Author

Commented:
Just to be on the safe side, I will raise the levels either tonight remotely or when I get into the office in the morning.  I will respond and update you on the results.  Thanks.
cmp119IT Manager

Author

Commented:
One more thing, I will raise the domain functional level first, and then the forest level.  Do I need to wait a period of time between raising each level, or can I simply do it one right after another.  I figure if raising the domain functional level is successful, then I would think I can go ahead and immediately update the forest level.
DrDave242Principal Support Engineer

Commented:
You are correct; there's no need to wait if the domain operation is successful.
Top Expert 2014

Commented:
I seem to recall needing to restart the Kerberos service (if the DC wasn't restarted) after raising the DFL from 2003.
https://blogs.technet.microsoft.com/exchange/2015/02/13/considering-updating-your-domain-functional-level-from-windows-2003-read-this/
cmp119IT Manager

Author

Commented:
Okay, last night I successfully raised the domain and forest levels to Windows Server 2012 without issue.  I did not see any Event ID 10 or 14 errors in the system event viewer on both DCs, but decided to restart all servers on the domain several hours after raising the levels. All servers and DNS appear to be functioning without issue.  Exchange is also working as expected.  I also activated/enabled AD recycle bin as well.  

Thank you all for your assistance on this matter.
DrDave242Principal Support Engineer

Commented:
Glad to hear it!
cmp119IT Manager

Author

Commented:
I thought I closed this question assigned the best and assisted solutions.  Can you please confirm this case is closed now.  Thanks.
MASEE Solution Guide - Technical Dept Head
Most Valuable Expert 2017

Commented:
Hi cmp119,
Please check this article to close the question
How do I a close a question

MAS
EE Solution Guide
cmp119IT Manager

Author

Commented:
That's the problem, there is not option I have my answer, and the sidebar only displays close question.  I already applied the best and assisted solutions, so there is nothing I can do on my end.
DrDave242Principal Support Engineer

Commented:
Several experts made meaningful contributions here. If anyone feels that points should be allocated differently, don't hesitate to object.
cmp119IT Manager

Author

Commented:
Thank you!

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial