Major domain controller active directory problems

I have 3 domain controllers, 1. was the main 2008 R2, 2. 2012 R2 current main, 3. 2012 R2 as second. The 2. dc was promoted, etc to become the main, and is verified as the main, and the 1. old main was down graded. When I shut down the old main (1.) every computer loses connectivity to the domain. The old main (1) has now went bad and is disrupting AD services in the environment. I can't bring it back to life. How can I force computers to use the 2. and 3. domain controllers? How can dc 2 see itself as the main?
Zakee AbdurrasheedSystems AdministratorAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Alex GreenProject Systems EngineerCommented:
You need to run a DCdiag first.

When you demoted the domain controller 1, did you verify that active directory had fully syncronised to domain controller 2?

When you get the DCdiag, you may find that your FSMO roles are on DC1.

You now need to seize those roles over to DC2.

Untill you have run DCDiag and tried to find where the issue is, we cannot give you specific advice for this issue.

Regards
Alex
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
I did verify that ad had fully syncronised to the dc 2.

I ran DCdiag on dc 2 just now.

 Zakee: Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

C:\Users\administrator.GECUSVI>netdom query dc
The specified domain either does not exist or could not be contacted.

The command failed to complete successfully.


C:\Users\administrator.GECUSVI>repadmin /syncall /AdeP
Syncing all NC's held on GEC-dc1.
Syncing partition: DC=ForestDnsZones,DC=gecusvi,DC=com
CALLBACK MESSAGE: Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Ser
vers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (net
work error): 1722 (0x6ba):
    The RPC server is unavailable.
CALLBACK MESSAGE: The following replication is in progress:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: SyncAll Finished.

SyncAll reported the following errors:
Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Servers,CN=Default-Fi
rst-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (network error): 1722
(0x6ba):
    The RPC server is unavailable.

Syncing partition: DC=DomainDnsZones,DC=gecusvi,DC=com
CALLBACK MESSAGE: Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Ser
vers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (net
work error): 1722 (0x6ba):
    The RPC server is unavailable.
CALLBACK MESSAGE: The following replication is in progress:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: SyncAll Finished.

SyncAll reported the following errors:
Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Servers,CN=Default-Fi
rst-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (network error): 1722
(0x6ba):
    The RPC server is unavailable.

Syncing partition: CN=Schema,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Ser
vers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (net
work error): 1722 (0x6ba):
    The RPC server is unavailable.
CALLBACK MESSAGE: The following replication is in progress:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: SyncAll Finished.

SyncAll reported the following errors:
Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Servers,CN=Default-Fi
rst-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (network error): 1722
(0x6ba):
    The RPC server is unavailable.

Syncing partition: CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Ser
vers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (net
work error): 1722 (0x6ba):
    The RPC server is unavailable.
CALLBACK MESSAGE: The following replication is in progress:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: SyncAll Finished.

SyncAll reported the following errors:
Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Servers,CN=Default-Fi
rst-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (network error): 1722
(0x6ba):
    The RPC server is unavailable.

Syncing partition: DC=gecusvi,DC=com
CALLBACK MESSAGE: Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Ser
vers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (net
work error): 1722 (0x6ba):
    The RPC server is unavailable.
CALLBACK MESSAGE: The following replication is in progress:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: SyncAll Finished.

SyncAll reported the following errors:
Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Servers,CN=Default-Fi
rst-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (network error): 1722
(0x6ba):
    The RPC server is unavailable.


C:\Users\administrator.GECUSVI>
05:29 PM Zakee: Doing primary tests

   Testing server: Default-First-Site-Name\GEC-DC1
      Starting test: Advertising
         Fatal Error:DsGetDcName (GEC-DC1) call failed, error 1355
         The Locator could not find the server.
         ......................... GEC-DC1 failed test Advertising
      Starting test: FrsEvent
         There are warning or error events within the last 24 hours after the
         SYSVOL has been shared.  Failing SYSVOL replication problems may cause
         Group Policy problems.
         ......................... GEC-DC1 passed test FrsEvent
      Starting test: DFSREvent
         ......................... GEC-DC1 passed test DFSREvent
      Starting test: SysVolCheck
         ......................... GEC-DC1 passed test SysVolCheck
      Starting test: KccEvent
         A warning event occurred.  EventID: 0x80000B46
            Time Generated: 12/12/2018   07:46:21
            Event String:
            The security of this directory server can be significantly enhanced
by configuring the server to reject SASL (Negotiate, Kerberos, NTLM, or Digest)
LDAP binds that do not request signing (integrity verification) and LDAP simple
binds that are performed on a clear text (non-SSL/TLS-encrypted) connection.  Ev
en if no clients are using such binds, configuring the server to reject them wil
l improve the security of this server.
         A warning event occurred.  EventID: 0x8000082C
            Time Generated: 12/12/2018   07:47:21
            Event String:
         ......................... GEC-DC1 passed test KccEvent
      Starting test: KnowsOfRoleHolders
         ......................... GEC-DC1 passed test KnowsOfRoleHolders
      Starting test: MachineAccount
         ......................... GEC-DC1 passed test MachineAccount
      Starting test: NCSecDesc
         ......................... GEC-DC1 passed test NCSecDesc
      Starting test: NetLogons
         Unable to connect to the NETLOGON share! (\\GEC-DC1\netlogon)
         [GEC-DC1] An net use or LsaPolicy operation failed with error 67,
         The network name cannot be found..
         ......................... GEC-DC1 failed test NetLogons
      Starting test: ObjectsReplicated
         ......................... GEC-DC1 passed test ObjectsReplicated
      Starting test: Replications
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-STX-DC1 to GEC-DC1
            Naming Context: DC=ForestDnsZones,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 05:57:39.
            2 failures have occurred since the last success.
            The guid-based DNS name
            801d01e9-beec-4dfd-94fc-ef9d0cea3957._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [GEC-STX-DC1] DsBindWithSpnEx() failed with error 1722,
         The RPC server is unavailable..
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-STX-DC1 to GEC-DC1
            Naming Context: DC=DomainDnsZones,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 05:57:39.
            2 failures have occurred since the last success.
            The guid-based DNS name
            801d01e9-beec-4dfd-94fc-ef9d0cea3957._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-STX-DC1 to GEC-DC1
            Naming Context: CN=Schema,CN=Configuration,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 05:57:39.
            2 failures have occurred since the last success.
            The guid-based DNS name
            801d01e9-beec-4dfd-94fc-ef9d0cea3957._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-DC2 to GEC-DC1
            Naming Context: CN=Schema,CN=Configuration,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 07:00:04.
            1 failures have occurred since the last success.
            The guid-based DNS name
            e94b54e1-600e-4fbd-a5e6-c78050d093f1._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-STX-DC1 to GEC-DC1
            Naming Context: CN=Configuration,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 05:57:39.
            2 failures have occurred since the last success.
            The guid-based DNS name
            801d01e9-beec-4dfd-94fc-ef9d0cea3957._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-DC2 to GEC-DC1
            Naming Context: CN=Configuration,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 06:59:22.
            1 failures have occurred since the last success.
            The guid-based DNS name
            e94b54e1-600e-4fbd-a5e6-c78050d093f1._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-STX-DC1 to GEC-DC1
            Naming Context: DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 06:47:56.
            2 failures have occurred since the last success.
            The guid-based DNS name
            801d01e9-beec-4dfd-94fc-ef9d0cea3957._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         ......................... GEC-DC1 failed test Replications
      Starting test: RidManager
         ......................... GEC-DC1 passed test RidManager
      Starting test: Services
         ......................... GEC-DC1 passed test Services
      Starting test: SystemLog
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:03:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:08:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:13:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:18:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:23:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:24:18
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:28:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:33:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:38:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:43:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         A warning event occurred.  EventID: 0x000003F6
            Time Generated: 12/12/2018   07:46:11
            Event String:
            Name resolution for the name _ldap._tcp.dc._msdcs.gecusvi.com. timed
 out after none of the configured DNS servers responded.
         A warning event occurred.  EventID: 0x00002724
            Time Generated: 12/12/2018   07:46:47
            Event String:
            This computer has at least one dynamically assigned IPv6 address.For
 reliable DHCPv6 server operation, you should use only static IPv6 addresses.
         A warning event occurred.  EventID: 0x000727AA
            Time Generated: 12/12/2018   07:46:58
            Event String:
            The WinRM service failed to create the following SPNs: WSMAN/GEC-dc1
.gecusvi.com; WSMAN/GEC-dc1.
         A warning event occurred.  EventID: 0x00001796
            Time Generated: 12/12/2018   07:47:45
            Event String:
            Microsoft Windows Server has detected that NTLM authentication is pr
esently being used between clients and this server. This event occurs once per b
oot of the server on the first time a client uses NTLM with this server.
         A warning event occurred.  EventID: 0xC000042B
            Time Generated: 12/12/2018   07:49:37
            Event String:
            The RD Session Host server cannot register 'TERMSRV' Service Princip
al Name to be used for server authentication. The following error occured: The s
pecified domain either does not exist or could not be contacted.
         An error event occurred.  EventID: 0x00000469
            Time Generated: 12/12/2018   07:50:57
            Event String:
            The processing of Group Policy failed because of lack of network con
nectivity to a domain controller. This may be a transient condition. A success m
essage would be generated once the machine gets connected to the domain controll
er and Group Policy has successfully processed. If you do not see a success mess
age for several hours, then contact your administrator.
         An error event occurred.  EventID: 0x00000469
            Time Generated: 12/12/2018   07:50:57
            Event String:
            The processing of Group Policy failed because of lack of network con
nectivity to a domain controller. This may be a transient condition. A success m
essage would be generated once the machine gets connected to the domain controll
er and Group Policy has successfully processed. If you do not see a success mess
age for several hours, then contact your administrator.
         ......................... GEC-DC1 failed test SystemLog
      Starting test: VerifyReferences
         ......................... GEC-DC1 passed test VerifyReferences


   Running partition tests on : ForestDnsZones
      Starting test: CheckSDRefDom
         ......................... ForestDnsZones passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... ForestDnsZones passed test
         CrossRefValidation

   Running partition tests on : DomainDnsZones
      Starting test: CheckSDRefDom
         ......................... DomainDnsZones passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... DomainDnsZones passed test
         CrossRefValidation

   Running partition tests on : Schema
      Starting test: CheckSDRefDom
         ......................... Schema passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... Schema passed test CrossRefValidation

   Running partition tests on : Configuration
      Starting test: CheckSDRefDom
         ......................... Configuration passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... Configuration passed test CrossRefValidation

   Running partition tests on : gecusvi
      Starting test: CheckSDRefDom
         ......................... gecusvi passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... gecusvi passed test CrossRefValidation

   Running enterprise tests on : gecusvi.com
      Starting test: LocatorCheck
         Warning: DcGetDcName(GC_SERVER_REQUIRED) call failed, error 1355
         A Global Catalog Server could not be located - All GC's are down.
         Warning: DcGetDcName(TIME_SERVER) call failed, error 1355
         A Time Server could not be located.
         The server holding the PDC role is down.
         Warning: DcGetDcName(GOOD_TIME_SERVER_PREFERRED) call failed, error
         1355
         A Good Time Server could not be located.
         Warning: DcGetDcName(KDC_REQUIRED) call failed, error 1355
         A KDC could not be located - All the KDCs are down.
         ......................... gecusvi.com failed test LocatorCheck
      Starting test: Intersite
         ......................... gecusvi.com passed test Intersite

C:\Users\administrator.GECUSVI>


FYI: gec-dc1 = dc2.    gec-stx-dc1 = dc1 (hard drive failure)
Alex GreenProject Systems EngineerCommented:
Are you sure that you demoted the domain controller correctly?

What process did you follow? Because by the looks of it, you've either had a catastrophic failure of the demotion or done it completely wrong.

Regards

Alex
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

Zakee AbdurrasheedSystems AdministratorAuthor Commented:
I did a while back following instructions on the internet. Now that dc1 has died, and other domain controllers are sitting there. Is there a way to remove the failed dc1??
Alex GreenProject Systems EngineerCommented:
Be more specific, what instructions, you must have a link for it, beacuse according that, there is no way at all that the DCPROMO has worked anywhere near correctly, you have DC1 references in your sites and services, DNS and DFSR. Which is for all intents and purposes, everywhere.

Did you just delete the server from AD and shut it down? Honesty will help here, we're going to have to do a full metadata cleanup of this DC now.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
here's the link I followed: https://winsvr.wordpress.com/2012/12/17/transferring-fsmo-roles-from-ws-2008r2-dc-to-ws-2012-dc/

I didn't delete the dc1 (failed server) from AD, it stayed on until this morning when it died. So now I'm finding out that the other ADs are not doing job of DC.
Alex GreenProject Systems EngineerCommented:
That's by far the worst guide i've ever seen.... without a doubt... Sorry but this could be interesting.

Ok look,

First up, lets seize the roles from the failed demoted DC.

https://blogs.technet.microsoft.com/canitpro/2015/10/14/step-by-step-seizing-the-operation-master-roles-in-windows-server-2012-r2/


Follow that, then shutdown DC1, see if you continue to have the login issues. This will basically take all the roles available.

Next up, clean up the old domain controller, follow the link below.

https://www.petri.com/delete_failed_dcs_from_ad

Next up,

Run DCDiag again.

Attach the results as a text file, dropping that much info into the comment isn't very user friendly.
Alex GreenProject Systems EngineerCommented:
Do all the above from your DC2 by the way.
Alex GreenProject Systems EngineerCommented:
You don't need to do any of what they said since there is a direct upgrade path from 2008r2 to 2012r2 btw.

netdom /query FSMO

Run that as well, you could also boot up your 2008r2 box, see if it's still working and then transfer the roles

Run that first though, see how it goes.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
thanks, working on it now!
Alex GreenProject Systems EngineerCommented:
Run netdom /FSMO first fella, we need to see where all your roles are, if we can migrate them to your 2012 box we will try to do that nicely before we strip out the old DC

If we can migrate the roles, we can consider using DCPromo to take it out of action, which is much nicer than having to force it out using those links :-)

Regards
Alex
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
DC1 is dead in the water... no way to access it. hard drive bit the dust.

Once I complete the seizing of the rolls, I reboot dc2? or just move on to clean up?
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
when I try to run netdom /FSMO on dc2 after seizing, it says The specified domain either does not exist or could not babe contacted. The command failed to complete successfully.
Alex GreenProject Systems EngineerCommented:
Urgh ok.....


If your AD hasn't replicated you've lost your domain, do you have backups?
Alex GreenProject Systems EngineerCommented:
Are you sure you ran DCDiag prior to carrying out this work and you saw it replicating to DC2?

Could you seize the roles to DC2?
Alex GreenProject Systems EngineerCommented:
It looks as though it should be on DC2

    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:


Right, can you launch AD users and computers and can you see the infrastructure?
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
I did, and it showed all FSMO on dc2. but now that dc1 is dead, having issues.

I ran the seizing just now on DC2. Should I use DC3 to seize the roles?
Alex GreenProject Systems EngineerCommented:
No,

Launch AD users and computers and connect to DC2 if it doesn't find it. Tell me your accounts are there please.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
when I try to launch AD users and computers it says Naming information cannot be located because: the specified domain either does not exist or could not be contacted...
Alex GreenProject Systems EngineerCommented:
Run DC diag again, this time please attach it as a file.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
attached dcdiag results
dcdiag.txt
Alex GreenProject Systems EngineerCommented:
Sorry

DCDIAG  /v/c/d/e

That'll test all servers.

Also

repadmin

Is there Zero way to get DC1 back up?
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
Alex GreenProject Systems EngineerCommented:
Right,

We need to check NTDSUTIL to see if DC2 is even in there

Command prompt

NTDSUTIL

Metadata Cleanup

Connections

Connect to server DC2

Then q and enter should take you back to metadata cleanup

Select operation target

List domains

See what that has in there for me
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
Alex GreenProject Systems EngineerCommented:
Restart the netlogon service on DC2

Also, if you log directly onto DC2, and launch ADUC, does that work?

Lastly, launch Sites and services, see if that comes back with your DC lists.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
after netlogon service restart on both good DCs, still unable to connect to ADUC or ADSS.
Alex GreenProject Systems EngineerCommented:
OK have you logged onto the server and checked it from there?
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
yes, I logged onto both good DCs and got same errors.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
I'm able to get the failed DC (gec-stx-dc1) to boot into directory services restore mode, but whenever I try to boot normally or into something that provides network connectivity, I get the blue screen of death.
Alex GreenProject Systems EngineerCommented:
What's the BSOD error?
Alex GreenProject Systems EngineerCommented:
Also,

I want you to navigate to \\dc2\sysvol

Go in there, tell me if you have folders in there
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
i'm at \\dc2\sysvol what's next? I see the domain folder in that folder.
Alex GreenProject Systems EngineerCommented:
Right,

That's replicated apparently.

What I want you to do, is DCDIAG /c /v

I need this to be done on DC2, so please log on, run it from there.

Next up

netdom query FSMO

Check the results on there and tell me what DC each points to.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
results of netdomo query FSMO: The specified domain either does not exist or could not be contacted.
Alex GreenProject Systems EngineerCommented:
OK,

On DC2

Get-ADDomainController | ft Name,IsGlobalCatalog
Shaun VermaakTechnical SpecialistCommented:
Do you have all DCs configured as DNS servers on all the clients DNS settings?
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
I have only the good DCs (gec-dc1 and gec-dc2) as DNS servres on all clients DNS settings.

'Get-ADDomainController ' is not  recognized as an internal or external command, operable program or batch file'
Alex GreenProject Systems EngineerCommented:
Shaun,

He can't even access active directory from within a domain controller (DC2), Settings have been modified directly within ADSIEdit on DC1.

https://winsvr.wordpress.com/2012/12/17/transferring-fsmo-roles-from-ws-2008r2-dc-to-ws-2012-dc/

He's followed that, and since that the DC won't boot, it throws back a BSOD.

DCDIAG shows everything still being on DC1

He can't seize roles.

There isn't a global catalog server according to the last DCDiag.

Netdom query FSMO from DC2, whilst he's logged onto the DC, it can't find the domain.
Alex GreenProject Systems EngineerCommented:
Import-module ACtiveDirectory on DC2 and then run that command please.
Alex GreenProject Systems EngineerCommented:
Also,

on DC2, when you launch the server manager, it must have active directory installed as a role correct?

Thanks
Alex
Alex GreenProject Systems EngineerCommented:
Also,


One last effort from me on this one

https://blogs.technet.microsoft.com/canitpro/2015/10/14/step-by-step-seizing-the-operation-master-roles-in-windows-server-2012-r2/

Follow that, you will need to put in your FQDN of your DC2 domain controller.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
Yes it has AD installed.

'Import-module' is not a recognized command
MASEE Solution Guide - Technical Dept HeadCommented:
Can you find a backup of the old DC? That is the easy solution for now.
Then we will provide you some guidance how to demote.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
yes I have a windows backup of the old DC
Alex GreenProject Systems EngineerCommented:
The old DC can boot into "Directory services restore mode"

It won't however boot into windows, I suspect because of the Schema DLL issue.  

The only other thing I can think of is either try an authorative restore from DSRM, or a back up at this point as you said.
MASEE Solution Guide - Technical Dept HeadCommented:
Then restore it for now.
STop replication on all servers. once restore is complete start the replication.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
dc1 just booted up!! I need to push it over to dc2 STAT!
Alex GreenProject Systems EngineerCommented:
NO!

Hold on
Alex GreenProject Systems EngineerCommented:
Before you change anything lets take our time with this so it's not a tragic situation.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
kk
Alex GreenProject Systems EngineerCommented:
Right first of all,

Your DC has booted up, please log onto DC2 and try to launch Active Directory
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
I get the same initial error when launching AD, but I can change domain controller over to dc1 and it works!
Alex GreenProject Systems EngineerCommented:
Right,

Next up, lets check your Global Catalog servers

From that active directory

Follow this

https://www.dtonias.com/determine-global-catalog-server/

You want to make sure that DC2 is a GC
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
all 3 are Global Catalog
Alex GreenProject Systems EngineerCommented:
Right,

repadmin /syncall

That will syncronise your domain, in theory all domain controllers will syncronise with each other with the newest info.

Check that then check DCDIAG /c /v to make sure they replicated.
Alex GreenProject Systems EngineerCommented:
When you've done that, from the 2008 box, run "Netdom Query FSMO"

That will tell you where your FSMO roles are

I assume they will all be on DC1

https://www.dtonias.com/transfer-fsmo-roles-domain-controller/

Follow that, use the GUI on DC1 to transfer the roles safely. I'm going to be on the road for a bit, I'll be back in a few hours. Best of luck though, I'll check back in a few hours to see where you got to.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
ran netdom on the dc1, and it says all the roles are on dc2. completed successfully.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
dcdiag /v /c is too long to all be in command prompt, the beginning is being truncated by the time it's finishing.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
I just did netdom query FSMO on dc2 and it says that itself (dc2) has all the rolls... what's up with that. If it has all the rolls, then why is it depending on dc1?? doesn't the domain come over to dc2 as well? it seems like when dc1 is down, the entire domain dies.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
now under ADSS on dc2, it's showing dc1 as a server, but it doesn't have NTDS under it. So i'm guessing it's no longer a GC from DC2 and DC3's perspective?
Alex GreenProject Systems EngineerCommented:
And I'm back,

Right, if it were me I'd go through

https://winsvr.wordpress.com/2012/12/17/transferring-fsmo-roles-from-ws-2008r2-dc-to-ws-2012-dc/

and set it back to how it was.

When you installed active directory, did you do a DCPromo or did you use server manager?

THanks
Alex
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
I used server manager
Alex GreenProject Systems EngineerCommented:
ok,

Is this being virtualised off, if so I'd do the following, build a NEW 2012 box, do not modify your other boxes or your 2003 box.

https://blog.thesysadmins.co.uk/server-2012-add-domain-controller.html

Then wait for replication, ensure that you can view AD users and computer from 2012r2 before anything else.

Regards
Alex
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
what to do about the dc1? it's still acting as though it's active, and if it goes down, the domain goes down.

dc1 is not virtualized, the others are VMs
Alex GreenProject Systems EngineerCommented:
Leave DC1 well alone,

At this point, is your domain up and running again?

Thanks
Alex
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
yes domain is up and running... I just fear for if it goes down again... it's on its last legs!
Alex GreenProject Systems EngineerCommented:
Yeah, in which case leave it as it, promote a new DC following the guide I've listed, ensure it links with DC1.
Zakee AbdurrasheedSystems AdministratorAuthor Commented:
The old “dead” DC came back online and then everything started back working. Special thanks to Alex Green for helping me a hella lot!!

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Active Directory

From novice to tech pro — start learning today.