Looking for guidance removing old MS Exchange 2003 installation and replacing MS Server 2003 Domain Controllers.

I am inexperienced in this, but it has fallen to me, and I'm getting the sense that there are a number of issues that need to be worked out first.

Some relevant information regarding the environment:
   7 Domain Controllers with a network of around 30 servers (mixed virtual and physical) and around 100 physical Windows 7 workstations.  6 of the 7 DCs are virtualized, all of them are currently running Windows Server 2003.  One exists at each of 6 branch locations except in the main office where there are two, the PDC and a secondary (which is the physical)...  For purpose of discussion, we'll call the old physical Domain/Exchange server "OldDCEx",

   OldDCEx should have been retired a decade ago, but at one time it was the everything server:  It was the PDC, Microsoft Exchange 2003, BES, Active Directory, DNS, DHCP, Time, Internal websites, SQL database(s) Access Databases, and File shares... some of this is still used yet today.

At some point a new separate Exchange 2003 server was built and added to the exchange group.  All user mailboxes were migrated.  The older exchange installation was not fully removed.

OldDCEx was the PDC, but that role was migrated to a virtual server (still Windows server 2003)

A new virtual MS Exchange server was built running MS Exchange 2010 on top of Windows server 2008, and added to the mix.  Most users were then migrated to the new machine, though still not all.

Fast Forward a few years to 2013, and prep to upgrade off Windows Server 2003 and Exchange 2003 began.  Unfortunately, the person working that project ended up leaving the company before it was finished.  I am picking up the pieces, but I'm a little daunted, and it seems there are a few possible issues with our domain that may need fixing before proceeding.


steps that I've taken:

On OldDCEx, I've gone through the public, schedulefreebusy, and offlineaddressbook folders and discovered that many were still homed on this server.  I've replicated them on the other exchange servers.  (I know I should have just replicated them to the Exchange 2010 server, but that wasn't an obvious choice where the 2003 servers appeared by name.  I can look into that more thouroghly when I get to removing the newer Exchange 2003 server I suppose.)

Disabled exchange services on OldDCEx for a few days to see if there were any issues that would arise as a result.  So far, nothing that I would connect back to this.

Built new Windows 2012 virtual machines for the replacement DCs including a physical for the PDC.


Things I've been noticing recently, though not necessarily after the step above:

Recently we had a number of user workstations produce a black screen event.  The first time, they affected multiple machines successively in the middle of their users logged in session.  The occurance repeated itself for a few weeks at random times until we traced it back to a memory error that was happening on a router in our network.  legit traffic was having a hard time reaching the DCs because the device flooded the switch and firewall that it was connected to.  Powering off the device by itself emediately resolved the issue, and after the switch got an iOS upgrade the issue seemed to go away for good.

On the other hand, every once and a while now, we have a few users complain of similar issues after a computer reboot or unlock. One time it lasted exactly until we rebooted the PDC and OldDCEx.


Our exchange 2010 server is throwing errors like this now and then referencing both OldDCEx and the PDC:
-----------------------------
Log Name:      System
Source:        NETLOGON
Date:          7/1/2015 12:21:47 AM
Event ID:      5783
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      [exchange 2010 server]
Description:
The session setup to the Windows NT or Windows 2000 Domain Controller \\[OldDCEx] for the domain [OurDomain] is not responsive.  The current RPC call from Netlogon on \\[EXCHANGE 2010 server] to \\[OldDCEx] has been cancelled.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="NETLOGON" />
    <EventID Qualifiers="0">5783</EventID>
    <Level>2</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2015-07-01T05:21:47.000000000Z" />
    <EventRecordID>165477</EventRecordID>
    <Channel>System</Channel>
    <Computer>[exchange 2010 server]</Computer>
    <Security />
  </System>
  <EventData>
    <Data>\\[OldDCEx]</Data>
    <Data>[domain name]</Data>
    <Data>[EXCHANGE 2010 server]</Data>
  </EventData>
</Event>
-----------------------------


Our PDC is throwing some errors like these a bit more often than I am comfortable with:
-----------------------------
Event Type:      Error
Event Source:      DNS
Event Category:      None
Event ID:      4016
Date:            6/25/2015
Time:            4:46:01 PM
User:            N/A
Computer:      [our PDC]
Description:
The DNS server timed out attempting an Active Directory service operation on ---.  Check Active Directory to see that it is functioning properly. The event data contains the error.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 55 00 00 00               U...    
-----------------------------

and this:
-----------------------------
Event Type:      Error
Event Source:      DNS
Event Category:      None
Event ID:      4016
Date:            7/1/2015
Time:            12:13:40 AM
User:            N/A
Computer:      [our PDC]
Description:
The DNS server timed out attempting an Active Directory service operation on DC=[xxx],DC=[xxx.xxx.xxx].in-addr.arpa,cn=MicrosoftDNS,DC=DomainDnsZones,DC=[ourdomain],DC=local.  Check Active Directory to see that it is functioning properly. The event data contains the error.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 55 00 00 00               U...    

-----------------------------



Where should I begin, how should I proceed?  Any suggestions?
Intelli-SeekerAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Simon Butler (Sembee)ConsultantCommented:
When Exchange is installed on a domain controller, all Exchange servers will only use that domain controller.
Therefore the first thing you need to do is remove Exchange from that old domain controller.
To ensure everything is ok, remove the public folder database first - if Exchange doesn't let you, then you need to check replicas and that instances are blank.
Then drop the mailbox database.
The uninstall of Exchange should go clean.

Once you have done that, remove the GC role from the server and restart the other Exchange servers.
Finally DCPROMO that server out, then drop from the domain and retire.

You can then think about introducing the later versions of Windows as domain controllers. Exchange 2010 has no problem with the later version.
I wouldn't do anything until you have removed Exchange 2003 from the DC though.

Simon.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Tony GiangrecoCommented:
I agree with Simon Butler (Sembee), but you should also replace the Windows 2000 server mentioned above. You can bring in a 2008 or 2012 to replace it but replacing all Windows 2000 & XP area systems in the network would be a high priority.

If you use Windows 2012, you need to replace all XP & 2000 servers anyway and raise your forest and domain functional level to at least 2003.  Also make sure your FSMO roles are not on any Windows 2000 server.

It looks like you have a lot of work to do. i suggest documenting each server and organize the project, Meet with your team and assign different tasks (if you have other team members) so you can focus and are not responsible for everything.

Hope this helps!
Intelli-SeekerAuthor Commented:
Thanks Simon,
Just to confirm - Exchange will not uninstall if there is something hasn't been properly replicated out yet?  So I don't need to worry about having missed something stupid? :-)


Thanks TG-TIS,

I should have included that FSMO roles are all on the PDC (not OldDCEx), and all of our DCs are all Win 2003, and the function level is 2003 as well.  They are all slated to get replaced as part of this project, but as you said... it's a big project, which is why I launched this thread.  I was getting lost in the direction to proceed and steps to take...  I do have someone that will be helping, but we're a small shop, and neither experienced in this.  

These comments are very helpful, thanks!
Determine the Perfect Price for Your IT Services

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden with our free interactive tool and use it to determine the right price for your IT services. Download your free eBook now!

Simon Butler (Sembee)ConsultantCommented:
If you drop the databases before attempting the uninstall, then that will confirm you have everything out. Otherwise you are correct - it should throw some errors.
The annoying thing about Exchange 2003 though is the uninstaller will throw errors, crash out and leave Exchange in place -  BUT remove the entry from add/remove programs.
That then means you have to reinstall Exchange to get it back in so you can remove it correctly. Complete headache.
There is a guide for removing the first Exchange 2003 server here:
http://support.microsoft.com/default.aspx?kbid=822931
Follow those steps and you should be fine.

Simon.
Intelli-SeekerAuthor Commented:
Exchange uninstalled without a hitch, thank you!

Do I need to remove the other exchange 2003 server before I can address the domain controller upgrades?  I'd like to get started on that asap.

Thanks again.
Simon Butler (Sembee)ConsultantCommented:
Exchange 2003 isn't supported with Windows 2012 or higher domain controllers. Therefore get rid of the Exchange 2003 server before you start to introduce higher version DCs.

Simon.
Intelli-SeekerAuthor Commented:
Thanks again Simon.  Your comments have been most helpful

I probably won't be able to remove the other exchange server in the next few days though, so I may not comment on this thread for a few days again.

(Also, I was thinking that it would be more appropriate to close this discussion when I finish the MS Exchange stuff, and start up a new one when I start working on the Domain controllers rather than rolling the discussion in that direction.)
Intelli-SeekerAuthor Commented:
well, I can't remember if this was the case with the first Exchange that I removed earlier, but I ran into a public folder called "System Configuration", which is set to replicate to both the 2010 and 2003 public folder stores.  the folder replication tab says the 2003 is "In Sync" while the 2010 says "Local Modified".

As I'm trying to decommission exchange 2003 altogether, I figured I should be able to change either the replication message priority, or do something to coax it forward.  Trying to change the Replication Message priority brings up a prompt for user credentials, but... it won't let me make changes.  My credentials are domain/exchange enterprise admin...  unsure what the issue is here.

It also gives the following message when trying to delete the public folder store, though I think this is normal:

publicfolderstore.JPG
Simon Butler (Sembee)ConsultantCommented:
You need to clear the error messages.
They are pretty easy to clear - simply delete the OAB from the Exchange 2003 server (you probably created a new one on the new installation of Exchange). Once you have cleared that, when you try to remove the public folder database it should prompt you for another database - failing that, simply change the default public folder database on the properties of the mailbox database.

Sync status should be ignored. The only thing that matters is what is in public folder instances. If that is empty then you have replicated all of the information off.

Simon.
Intelli-SeekerAuthor Commented:
I was able to get rid one of the error message, but still have the "it is used by one or more Offline Address Lists to maintain system folders" message.

I'll be working on this tonight or tomorrow.  the public folder instances are empty, and when I go to delete the public folder store, it does prompt for another public store to take over the role... I'm still thrown off by that system configuration folder - it's the only thing that is still homed on this Exchange server - but I don't think it matters, if I read what you said above correctly.  Thanks again!
Simon Butler (Sembee)ConsultantCommented:
You need to look at the OAB configuration.
On the later version of Exchange, ensure that you have either
a. Deselected the option to use public folders for distribution.
b. Delete the OAB on Exchange 2003 and then create a new one on the Exchange 2010 server - with web distribution only enabled.

Simon.
Intelli-SeekerAuthor Commented:
Hi Simon,

I did option (a.) on the 2010 server as you suggested above.  I've also powered off the old 2003 sever to verify that the OAB and ScheduleFreeBusy stuff is still working, and it seems to be, so I feel like I'm almost there, but when I use:

get-offlineAddressBook | fl

the 2010 server tells me:

PublicFolderDatabase is [myexchange2003server]\First storage Group\Public Folder Store <[myexchange2003server]>

when I expected it to point to [myexchange2010server] instead.  (I did confirm that the 2010 server is definitely the "originating" server for the OAB.)

This is still confusing to me so forgive me if I seem to be dragging my feet... Is this to be expected?  How would I go about changing it if it isn't?  I've read suggestions about re-pointing it to the new server using ADSI edit, by modifying the siteFolderServer here:  http://blog.ronnypot.nl/?p=212  but even if I wanted to try something like that, the option is grayed out...

You had mentioned deleting the OAB folder from the 2003 server in an earlier post.  Is that the solution?  Just delete the OAB "system" folder from the 2003 server?

Thanks again!
Simon Butler (Sembee)ConsultantCommented:
What I meant by deleting the OAB on the Exchange 2003 server is to go in to the Offline Address Book configuration in ESM and delete the OAB that is listed. No need to go in to ADSIEDIT.

However if you are seeing just the one OAB when you run get-offlineaddressbook, the first thing I would do is create a new OAB in Exchange 2010, then change the databases to use that new OAB. Wait at least 24 hours before doing anything else so that all clients are picking up the change.

Simon.
Intelli-SeekerAuthor Commented:
I created a new OAB, linked the mailbox databases and set it as default.  I had an issue downloading it on my outlook client originally, but I then manually triggered an update and restarted a few services and was able to download the address book successfully.  That being said, I'll wait till tomorrow to remove the old address book as you suggested.  

Assuming that goes well, I'll (finally) have this exchange 2003 server uninstalled tomorrow.  Yay!

Thanks again!
Intelli-SeekerAuthor Commented:
ok - so uninstalling exchange 2003 could not commence until the removal of one or more bridgehead connector and  RUS objects.  Then the uninstall process started fine but throws the following error message:

Setup failed while installing sub-component NNTP Service with error code 0xC0070424 (Please consult the installation logs for a detailed description). You may cancel the installation or try the failed step again.

PS - the setup log is a mile long, so I'll only post it if you want it.

Thanks again for all your help!
Intelli-SeekerAuthor Commented:
Completed the uninstall using  https://support.microsoft.com/en-us/kb/833396
removed the exchange server from the administrative group.  Apparently removing the administrative group is not recommended?  I guess that that sums it up then for my exchange server removal.  

As I noted earlier, I'll open a new thread if/when I need assistance with the DC upgrade/replacement.

Thanks so much for your assistance!
Intelli-SeekerAuthor Commented:
Thanks for sticking around and helping me through it all Simon.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Active Directory

From novice to tech pro — start learning today.