Major domain controller active directory problems

Zakee Abdurrasheed
Zakee Abdurrasheed used Ask the Experts™
on
I have 3 domain controllers, 1. was the main 2008 R2, 2. 2012 R2 current main, 3. 2012 R2 as second. The 2. dc was promoted, etc to become the main, and is verified as the main, and the 1. old main was down graded. When I shut down the old main (1.) every computer loses connectivity to the domain. The old main (1) has now went bad and is disrupting AD services in the environment. I can't bring it back to life. How can I force computers to use the 2. and 3. domain controllers? How can dc 2 see itself as the main?
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
AlexSenior Infrastructure Analyst

Commented:
You need to run a DCdiag first.

When you demoted the domain controller 1, did you verify that active directory had fully syncronised to domain controller 2?

When you get the DCdiag, you may find that your FSMO roles are on DC1.

You now need to seize those roles over to DC2.

Untill you have run DCDiag and tried to find where the issue is, we cannot give you specific advice for this issue.

Regards
Alex
Zakee AbdurrasheedSystems Administrator

Author

Commented:
I did verify that ad had fully syncronised to the dc 2.

I ran DCdiag on dc 2 just now.

 Zakee: Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

C:\Users\administrator.GECUSVI>netdom query dc
The specified domain either does not exist or could not be contacted.

The command failed to complete successfully.


C:\Users\administrator.GECUSVI>repadmin /syncall /AdeP
Syncing all NC's held on GEC-dc1.
Syncing partition: DC=ForestDnsZones,DC=gecusvi,DC=com
CALLBACK MESSAGE: Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Ser
vers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (net
work error): 1722 (0x6ba):
    The RPC server is unavailable.
CALLBACK MESSAGE: The following replication is in progress:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: SyncAll Finished.

SyncAll reported the following errors:
Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Servers,CN=Default-Fi
rst-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (network error): 1722
(0x6ba):
    The RPC server is unavailable.

Syncing partition: DC=DomainDnsZones,DC=gecusvi,DC=com
CALLBACK MESSAGE: Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Ser
vers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (net
work error): 1722 (0x6ba):
    The RPC server is unavailable.
CALLBACK MESSAGE: The following replication is in progress:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: SyncAll Finished.

SyncAll reported the following errors:
Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Servers,CN=Default-Fi
rst-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (network error): 1722
(0x6ba):
    The RPC server is unavailable.

Syncing partition: CN=Schema,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Ser
vers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (net
work error): 1722 (0x6ba):
    The RPC server is unavailable.
CALLBACK MESSAGE: The following replication is in progress:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: SyncAll Finished.

SyncAll reported the following errors:
Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Servers,CN=Default-Fi
rst-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (network error): 1722
(0x6ba):
    The RPC server is unavailable.

Syncing partition: CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Ser
vers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (net
work error): 1722 (0x6ba):
    The RPC server is unavailable.
CALLBACK MESSAGE: The following replication is in progress:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: SyncAll Finished.

SyncAll reported the following errors:
Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Servers,CN=Default-Fi
rst-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (network error): 1722
(0x6ba):
    The RPC server is unavailable.

Syncing partition: DC=gecusvi,DC=com
CALLBACK MESSAGE: Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Ser
vers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (net
work error): 1722 (0x6ba):
    The RPC server is unavailable.
CALLBACK MESSAGE: The following replication is in progress:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: SyncAll Finished.

SyncAll reported the following errors:
Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Servers,CN=Default-Fi
rst-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (network error): 1722
(0x6ba):
    The RPC server is unavailable.


C:\Users\administrator.GECUSVI>
05:29 PM Zakee: Doing primary tests

   Testing server: Default-First-Site-Name\GEC-DC1
      Starting test: Advertising
         Fatal Error:DsGetDcName (GEC-DC1) call failed, error 1355
         The Locator could not find the server.
         ......................... GEC-DC1 failed test Advertising
      Starting test: FrsEvent
         There are warning or error events within the last 24 hours after the
         SYSVOL has been shared.  Failing SYSVOL replication problems may cause
         Group Policy problems.
         ......................... GEC-DC1 passed test FrsEvent
      Starting test: DFSREvent
         ......................... GEC-DC1 passed test DFSREvent
      Starting test: SysVolCheck
         ......................... GEC-DC1 passed test SysVolCheck
      Starting test: KccEvent
         A warning event occurred.  EventID: 0x80000B46
            Time Generated: 12/12/2018   07:46:21
            Event String:
            The security of this directory server can be significantly enhanced
by configuring the server to reject SASL (Negotiate, Kerberos, NTLM, or Digest)
LDAP binds that do not request signing (integrity verification) and LDAP simple
binds that are performed on a clear text (non-SSL/TLS-encrypted) connection.  Ev
en if no clients are using such binds, configuring the server to reject them wil
l improve the security of this server.
         A warning event occurred.  EventID: 0x8000082C
            Time Generated: 12/12/2018   07:47:21
            Event String:
         ......................... GEC-DC1 passed test KccEvent
      Starting test: KnowsOfRoleHolders
         ......................... GEC-DC1 passed test KnowsOfRoleHolders
      Starting test: MachineAccount
         ......................... GEC-DC1 passed test MachineAccount
      Starting test: NCSecDesc
         ......................... GEC-DC1 passed test NCSecDesc
      Starting test: NetLogons
         Unable to connect to the NETLOGON share! (\\GEC-DC1\netlogon)
         [GEC-DC1] An net use or LsaPolicy operation failed with error 67,
         The network name cannot be found..
         ......................... GEC-DC1 failed test NetLogons
      Starting test: ObjectsReplicated
         ......................... GEC-DC1 passed test ObjectsReplicated
      Starting test: Replications
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-STX-DC1 to GEC-DC1
            Naming Context: DC=ForestDnsZones,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 05:57:39.
            2 failures have occurred since the last success.
            The guid-based DNS name
            801d01e9-beec-4dfd-94fc-ef9d0cea3957._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [GEC-STX-DC1] DsBindWithSpnEx() failed with error 1722,
         The RPC server is unavailable..
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-STX-DC1 to GEC-DC1
            Naming Context: DC=DomainDnsZones,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 05:57:39.
            2 failures have occurred since the last success.
            The guid-based DNS name
            801d01e9-beec-4dfd-94fc-ef9d0cea3957._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-STX-DC1 to GEC-DC1
            Naming Context: CN=Schema,CN=Configuration,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 05:57:39.
            2 failures have occurred since the last success.
            The guid-based DNS name
            801d01e9-beec-4dfd-94fc-ef9d0cea3957._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-DC2 to GEC-DC1
            Naming Context: CN=Schema,CN=Configuration,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 07:00:04.
            1 failures have occurred since the last success.
            The guid-based DNS name
            e94b54e1-600e-4fbd-a5e6-c78050d093f1._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-STX-DC1 to GEC-DC1
            Naming Context: CN=Configuration,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 05:57:39.
            2 failures have occurred since the last success.
            The guid-based DNS name
            801d01e9-beec-4dfd-94fc-ef9d0cea3957._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-DC2 to GEC-DC1
            Naming Context: CN=Configuration,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 06:59:22.
            1 failures have occurred since the last success.
            The guid-based DNS name
            e94b54e1-600e-4fbd-a5e6-c78050d093f1._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-STX-DC1 to GEC-DC1
            Naming Context: DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 06:47:56.
            2 failures have occurred since the last success.
            The guid-based DNS name
            801d01e9-beec-4dfd-94fc-ef9d0cea3957._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         ......................... GEC-DC1 failed test Replications
      Starting test: RidManager
         ......................... GEC-DC1 passed test RidManager
      Starting test: Services
         ......................... GEC-DC1 passed test Services
      Starting test: SystemLog
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:03:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:08:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:13:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:18:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:23:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:24:18
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:28:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:33:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:38:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:43:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         A warning event occurred.  EventID: 0x000003F6
            Time Generated: 12/12/2018   07:46:11
            Event String:
            Name resolution for the name _ldap._tcp.dc._msdcs.gecusvi.com. timed
 out after none of the configured DNS servers responded.
         A warning event occurred.  EventID: 0x00002724
            Time Generated: 12/12/2018   07:46:47
            Event String:
            This computer has at least one dynamically assigned IPv6 address.For
 reliable DHCPv6 server operation, you should use only static IPv6 addresses.
         A warning event occurred.  EventID: 0x000727AA
            Time Generated: 12/12/2018   07:46:58
            Event String:
            The WinRM service failed to create the following SPNs: WSMAN/GEC-dc1
.gecusvi.com; WSMAN/GEC-dc1.
         A warning event occurred.  EventID: 0x00001796
            Time Generated: 12/12/2018   07:47:45
            Event String:
            Microsoft Windows Server has detected that NTLM authentication is pr
esently being used between clients and this server. This event occurs once per b
oot of the server on the first time a client uses NTLM with this server.
         A warning event occurred.  EventID: 0xC000042B
            Time Generated: 12/12/2018   07:49:37
            Event String:
            The RD Session Host server cannot register 'TERMSRV' Service Princip
al Name to be used for server authentication. The following error occured: The s
pecified domain either does not exist or could not be contacted.
         An error event occurred.  EventID: 0x00000469
            Time Generated: 12/12/2018   07:50:57
            Event String:
            The processing of Group Policy failed because of lack of network con
nectivity to a domain controller. This may be a transient condition. A success m
essage would be generated once the machine gets connected to the domain controll
er and Group Policy has successfully processed. If you do not see a success mess
age for several hours, then contact your administrator.
         An error event occurred.  EventID: 0x00000469
            Time Generated: 12/12/2018   07:50:57
            Event String:
            The processing of Group Policy failed because of lack of network con
nectivity to a domain controller. This may be a transient condition. A success m
essage would be generated once the machine gets connected to the domain controll
er and Group Policy has successfully processed. If you do not see a success mess
age for several hours, then contact your administrator.
         ......................... GEC-DC1 failed test SystemLog
      Starting test: VerifyReferences
         ......................... GEC-DC1 passed test VerifyReferences


   Running partition tests on : ForestDnsZones
      Starting test: CheckSDRefDom
         ......................... ForestDnsZones passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... ForestDnsZones passed test
         CrossRefValidation

   Running partition tests on : DomainDnsZones
      Starting test: CheckSDRefDom
         ......................... DomainDnsZones passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... DomainDnsZones passed test
         CrossRefValidation

   Running partition tests on : Schema
      Starting test: CheckSDRefDom
         ......................... Schema passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... Schema passed test CrossRefValidation

   Running partition tests on : Configuration
      Starting test: CheckSDRefDom
         ......................... Configuration passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... Configuration passed test CrossRefValidation

   Running partition tests on : gecusvi
      Starting test: CheckSDRefDom
         ......................... gecusvi passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... gecusvi passed test CrossRefValidation

   Running enterprise tests on : gecusvi.com
      Starting test: LocatorCheck
         Warning: DcGetDcName(GC_SERVER_REQUIRED) call failed, error 1355
         A Global Catalog Server could not be located - All GC's are down.
         Warning: DcGetDcName(TIME_SERVER) call failed, error 1355
         A Time Server could not be located.
         The server holding the PDC role is down.
         Warning: DcGetDcName(GOOD_TIME_SERVER_PREFERRED) call failed, error
         1355
         A Good Time Server could not be located.
         Warning: DcGetDcName(KDC_REQUIRED) call failed, error 1355
         A KDC could not be located - All the KDCs are down.
         ......................... gecusvi.com failed test LocatorCheck
      Starting test: Intersite
         ......................... gecusvi.com passed test Intersite

C:\Users\administrator.GECUSVI>


FYI: gec-dc1 = dc2.    gec-stx-dc1 = dc1 (hard drive failure)
AlexSenior Infrastructure Analyst

Commented:
Are you sure that you demoted the domain controller correctly?

What process did you follow? Because by the looks of it, you've either had a catastrophic failure of the demotion or done it completely wrong.

Regards

Alex
Ensure you’re charging the right price for your IT

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

Zakee AbdurrasheedSystems Administrator

Author

Commented:
I did a while back following instructions on the internet. Now that dc1 has died, and other domain controllers are sitting there. Is there a way to remove the failed dc1??
AlexSenior Infrastructure Analyst

Commented:
Be more specific, what instructions, you must have a link for it, beacuse according that, there is no way at all that the DCPROMO has worked anywhere near correctly, you have DC1 references in your sites and services, DNS and DFSR. Which is for all intents and purposes, everywhere.

Did you just delete the server from AD and shut it down? Honesty will help here, we're going to have to do a full metadata cleanup of this DC now.
Zakee AbdurrasheedSystems Administrator

Author

Commented:
here's the link I followed: https://winsvr.wordpress.com/2012/12/17/transferring-fsmo-roles-from-ws-2008r2-dc-to-ws-2012-dc/

I didn't delete the dc1 (failed server) from AD, it stayed on until this morning when it died. So now I'm finding out that the other ADs are not doing job of DC.
AlexSenior Infrastructure Analyst

Commented:
That's by far the worst guide i've ever seen.... without a doubt... Sorry but this could be interesting.

Ok look,

First up, lets seize the roles from the failed demoted DC.

https://blogs.technet.microsoft.com/canitpro/2015/10/14/step-by-step-seizing-the-operation-master-roles-in-windows-server-2012-r2/


Follow that, then shutdown DC1, see if you continue to have the login issues. This will basically take all the roles available.

Next up, clean up the old domain controller, follow the link below.

https://www.petri.com/delete_failed_dcs_from_ad

Next up,

Run DCDiag again.

Attach the results as a text file, dropping that much info into the comment isn't very user friendly.
AlexSenior Infrastructure Analyst

Commented:
Do all the above from your DC2 by the way.
AlexSenior Infrastructure Analyst

Commented:
You don't need to do any of what they said since there is a direct upgrade path from 2008r2 to 2012r2 btw.

netdom /query FSMO

Run that as well, you could also boot up your 2008r2 box, see if it's still working and then transfer the roles

Run that first though, see how it goes.
Zakee AbdurrasheedSystems Administrator

Author

Commented:
thanks, working on it now!
AlexSenior Infrastructure Analyst

Commented:
Run netdom /FSMO first fella, we need to see where all your roles are, if we can migrate them to your 2012 box we will try to do that nicely before we strip out the old DC

If we can migrate the roles, we can consider using DCPromo to take it out of action, which is much nicer than having to force it out using those links :-)

Regards
Alex
Zakee AbdurrasheedSystems Administrator

Author

Commented:
DC1 is dead in the water... no way to access it. hard drive bit the dust.

Once I complete the seizing of the rolls, I reboot dc2? or just move on to clean up?
Zakee AbdurrasheedSystems Administrator

Author

Commented:
when I try to run netdom /FSMO on dc2 after seizing, it says The specified domain either does not exist or could not babe contacted. The command failed to complete successfully.
AlexSenior Infrastructure Analyst

Commented:
Urgh ok.....


If your AD hasn't replicated you've lost your domain, do you have backups?
AlexSenior Infrastructure Analyst

Commented:
Are you sure you ran DCDiag prior to carrying out this work and you saw it replicating to DC2?

Could you seize the roles to DC2?
AlexSenior Infrastructure Analyst

Commented:
It looks as though it should be on DC2

    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:


Right, can you launch AD users and computers and can you see the infrastructure?
Zakee AbdurrasheedSystems Administrator

Author

Commented:
I did, and it showed all FSMO on dc2. but now that dc1 is dead, having issues.

I ran the seizing just now on DC2. Should I use DC3 to seize the roles?
AlexSenior Infrastructure Analyst

Commented:
No,

Launch AD users and computers and connect to DC2 if it doesn't find it. Tell me your accounts are there please.
Zakee AbdurrasheedSystems Administrator

Author

Commented:
when I try to launch AD users and computers it says Naming information cannot be located because: the specified domain either does not exist or could not be contacted...
AlexSenior Infrastructure Analyst

Commented:
Run DC diag again, this time please attach it as a file.
Zakee AbdurrasheedSystems Administrator

Author

Commented:
attached dcdiag results
dcdiag.txt
AlexSenior Infrastructure Analyst

Commented:
Sorry

DCDIAG  /v/c/d/e

That'll test all servers.

Also

repadmin

Is there Zero way to get DC1 back up?
Zakee AbdurrasheedSystems Administrator

Author

Commented:
AlexSenior Infrastructure Analyst

Commented:
Right,

We need to check NTDSUTIL to see if DC2 is even in there

Command prompt

NTDSUTIL

Metadata Cleanup

Connections

Connect to server DC2

Then q and enter should take you back to metadata cleanup

Select operation target

List domains

See what that has in there for me
Zakee AbdurrasheedSystems Administrator

Author

Commented:
AlexSenior Infrastructure Analyst

Commented:
Restart the netlogon service on DC2

Also, if you log directly onto DC2, and launch ADUC, does that work?

Lastly, launch Sites and services, see if that comes back with your DC lists.
Zakee AbdurrasheedSystems Administrator

Author

Commented:
after netlogon service restart on both good DCs, still unable to connect to ADUC or ADSS.
AlexSenior Infrastructure Analyst

Commented:
OK have you logged onto the server and checked it from there?
Zakee AbdurrasheedSystems Administrator

Author

Commented:
yes, I logged onto both good DCs and got same errors.
Zakee AbdurrasheedSystems Administrator

Author

Commented:
I'm able to get the failed DC (gec-stx-dc1) to boot into directory services restore mode, but whenever I try to boot normally or into something that provides network connectivity, I get the blue screen of death.
AlexSenior Infrastructure Analyst

Commented:
What's the BSOD error?
AlexSenior Infrastructure Analyst

Commented:
Also,

I want you to navigate to \\dc2\sysvol

Go in there, tell me if you have folders in there
Zakee AbdurrasheedSystems Administrator

Author

Commented:
i'm at \\dc2\sysvol what's next? I see the domain folder in that folder.
AlexSenior Infrastructure Analyst

Commented:
Right,

That's replicated apparently.

What I want you to do, is DCDIAG /c /v

I need this to be done on DC2, so please log on, run it from there.

Next up

netdom query FSMO

Check the results on there and tell me what DC each points to.
Zakee AbdurrasheedSystems Administrator

Author

Commented:
Zakee AbdurrasheedSystems Administrator

Author

Commented:
results of netdomo query FSMO: The specified domain either does not exist or could not be contacted.
AlexSenior Infrastructure Analyst

Commented:
OK,

On DC2

Get-ADDomainController | ft Name,IsGlobalCatalog
Shaun VermaakTechnical Specialist
Awarded 2017
Distinguished Expert 2018

Commented:
Do you have all DCs configured as DNS servers on all the clients DNS settings?
Zakee AbdurrasheedSystems Administrator

Author

Commented:
I have only the good DCs (gec-dc1 and gec-dc2) as DNS servres on all clients DNS settings.

'Get-ADDomainController ' is not  recognized as an internal or external command, operable program or batch file'
AlexSenior Infrastructure Analyst

Commented:
Shaun,

He can't even access active directory from within a domain controller (DC2), Settings have been modified directly within ADSIEdit on DC1.

https://winsvr.wordpress.com/2012/12/17/transferring-fsmo-roles-from-ws-2008r2-dc-to-ws-2012-dc/

He's followed that, and since that the DC won't boot, it throws back a BSOD.

DCDIAG shows everything still being on DC1

He can't seize roles.

There isn't a global catalog server according to the last DCDiag.

Netdom query FSMO from DC2, whilst he's logged onto the DC, it can't find the domain.
AlexSenior Infrastructure Analyst

Commented:
Import-module ACtiveDirectory on DC2 and then run that command please.
AlexSenior Infrastructure Analyst

Commented:
Also,

on DC2, when you launch the server manager, it must have active directory installed as a role correct?

Thanks
Alex
AlexSenior Infrastructure Analyst

Commented:
Also,


One last effort from me on this one

https://blogs.technet.microsoft.com/canitpro/2015/10/14/step-by-step-seizing-the-operation-master-roles-in-windows-server-2012-r2/

Follow that, you will need to put in your FQDN of your DC2 domain controller.
Zakee AbdurrasheedSystems Administrator

Author

Commented:
Yes it has AD installed.

'Import-module' is not a recognized command
MASEE Solution Guide - Technical Dept Head
Most Valuable Expert 2017

Commented:
Can you find a backup of the old DC? That is the easy solution for now.
Then we will provide you some guidance how to demote.
Zakee AbdurrasheedSystems Administrator

Author

Commented:
yes I have a windows backup of the old DC
AlexSenior Infrastructure Analyst

Commented:
The old DC can boot into "Directory services restore mode"

It won't however boot into windows, I suspect because of the Schema DLL issue.  

The only other thing I can think of is either try an authorative restore from DSRM, or a back up at this point as you said.
MASEE Solution Guide - Technical Dept Head
Most Valuable Expert 2017

Commented:
Then restore it for now.
STop replication on all servers. once restore is complete start the replication.
Zakee AbdurrasheedSystems Administrator

Author

Commented:
dc1 just booted up!! I need to push it over to dc2 STAT!
AlexSenior Infrastructure Analyst

Commented:
NO!

Hold on
AlexSenior Infrastructure Analyst

Commented:
Before you change anything lets take our time with this so it's not a tragic situation.
Zakee AbdurrasheedSystems Administrator

Author

Commented:
kk
AlexSenior Infrastructure Analyst

Commented:
Right first of all,

Your DC has booted up, please log onto DC2 and try to launch Active Directory
Zakee AbdurrasheedSystems Administrator

Author

Commented:
I get the same initial error when launching AD, but I can change domain controller over to dc1 and it works!
AlexSenior Infrastructure Analyst

Commented:
Right,

Next up, lets check your Global Catalog servers

From that active directory

Follow this

https://www.dtonias.com/determine-global-catalog-server/

You want to make sure that DC2 is a GC
Zakee AbdurrasheedSystems Administrator

Author

Commented:
all 3 are Global Catalog
AlexSenior Infrastructure Analyst

Commented:
Right,

repadmin /syncall

That will syncronise your domain, in theory all domain controllers will syncronise with each other with the newest info.

Check that then check DCDIAG /c /v to make sure they replicated.
AlexSenior Infrastructure Analyst

Commented:
When you've done that, from the 2008 box, run "Netdom Query FSMO"

That will tell you where your FSMO roles are

I assume they will all be on DC1

https://www.dtonias.com/transfer-fsmo-roles-domain-controller/

Follow that, use the GUI on DC1 to transfer the roles safely. I'm going to be on the road for a bit, I'll be back in a few hours. Best of luck though, I'll check back in a few hours to see where you got to.
Zakee AbdurrasheedSystems Administrator

Author

Commented:
ran netdom on the dc1, and it says all the roles are on dc2. completed successfully.
Zakee AbdurrasheedSystems Administrator

Author

Commented:
dcdiag /v /c is too long to all be in command prompt, the beginning is being truncated by the time it's finishing.
Zakee AbdurrasheedSystems Administrator

Author

Commented:
I just did netdom query FSMO on dc2 and it says that itself (dc2) has all the rolls... what's up with that. If it has all the rolls, then why is it depending on dc1?? doesn't the domain come over to dc2 as well? it seems like when dc1 is down, the entire domain dies.
Zakee AbdurrasheedSystems Administrator

Author

Commented:
now under ADSS on dc2, it's showing dc1 as a server, but it doesn't have NTDS under it. So i'm guessing it's no longer a GC from DC2 and DC3's perspective?
AlexSenior Infrastructure Analyst

Commented:
And I'm back,

Right, if it were me I'd go through

https://winsvr.wordpress.com/2012/12/17/transferring-fsmo-roles-from-ws-2008r2-dc-to-ws-2012-dc/

and set it back to how it was.

When you installed active directory, did you do a DCPromo or did you use server manager?

THanks
Alex
Zakee AbdurrasheedSystems Administrator

Author

Commented:
I used server manager
AlexSenior Infrastructure Analyst

Commented:
ok,

Is this being virtualised off, if so I'd do the following, build a NEW 2012 box, do not modify your other boxes or your 2003 box.

https://blog.thesysadmins.co.uk/server-2012-add-domain-controller.html

Then wait for replication, ensure that you can view AD users and computer from 2012r2 before anything else.

Regards
Alex
Zakee AbdurrasheedSystems Administrator

Author

Commented:
what to do about the dc1? it's still acting as though it's active, and if it goes down, the domain goes down.

dc1 is not virtualized, the others are VMs
AlexSenior Infrastructure Analyst

Commented:
Leave DC1 well alone,

At this point, is your domain up and running again?

Thanks
Alex
Zakee AbdurrasheedSystems Administrator

Author

Commented:
yes domain is up and running... I just fear for if it goes down again... it's on its last legs!
AlexSenior Infrastructure Analyst

Commented:
Yeah, in which case leave it as it, promote a new DC following the guide I've listed, ensure it links with DC1.
Systems Administrator
Commented:
The old “dead” DC came back online and then everything started back working. Special thanks to Alex Green for helping me a hella lot!!

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial