Major domain controller active directory problems

You need to run a DCdiag first.

When you demoted the domain controller 1, did you verify that active directory had fully syncronised to domain controller 2?

When you get the DCdiag, you may find that your FSMO roles are on DC1.

You now need to seize those roles over to DC2.

Untill you have run DCDiag and tried to find where the issue is, we cannot give you specific advice for this issue.

Regards
Alex

I did verify that ad had fully syncronised to the dc 2.

I ran DCdiag on dc 2 just now.

 Zakee: Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

C:\Users\administrator.GECUSVI>netdom query dc
The specified domain either does not exist or could not be contacted.

The command failed to complete successfully.


C:\Users\administrator.GECUSVI>repadmin /syncall /AdeP
Syncing all NC's held on GEC-dc1.
Syncing partition: DC=ForestDnsZones,DC=gecusvi,DC=com
CALLBACK MESSAGE: Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Ser
vers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (net
work error): 1722 (0x6ba):
    The RPC server is unavailable.
CALLBACK MESSAGE: The following replication is in progress:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: SyncAll Finished.

SyncAll reported the following errors:
Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Servers,CN=Default-Fi
rst-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (network error): 1722
(0x6ba):
    The RPC server is unavailable.

Syncing partition: DC=DomainDnsZones,DC=gecusvi,DC=com
CALLBACK MESSAGE: Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Ser
vers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (net
work error): 1722 (0x6ba):
    The RPC server is unavailable.
CALLBACK MESSAGE: The following replication is in progress:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: SyncAll Finished.

SyncAll reported the following errors:
Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Servers,CN=Default-Fi
rst-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (network error): 1722
(0x6ba):
    The RPC server is unavailable.

Syncing partition: CN=Schema,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Ser
vers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (net
work error): 1722 (0x6ba):
    The RPC server is unavailable.
CALLBACK MESSAGE: The following replication is in progress:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: SyncAll Finished.

SyncAll reported the following errors:
Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Servers,CN=Default-Fi
rst-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (network error): 1722
(0x6ba):
    The RPC server is unavailable.

Syncing partition: CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Ser
vers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (net
work error): 1722 (0x6ba):
    The RPC server is unavailable.
CALLBACK MESSAGE: The following replication is in progress:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: SyncAll Finished.

SyncAll reported the following errors:
Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Servers,CN=Default-Fi
rst-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (network error): 1722
(0x6ba):
    The RPC server is unavailable.

Syncing partition: DC=gecusvi,DC=com
CALLBACK MESSAGE: Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Ser
vers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (net
work error): 1722 (0x6ba):
    The RPC server is unavailable.
CALLBACK MESSAGE: The following replication is in progress:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:
    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: SyncAll Finished.

SyncAll reported the following errors:
Error contacting server CN=NTDS Settings,CN=GEC-STX-DC1,CN=Servers,CN=Default-Fi
rst-Site-Name,CN=Sites,CN=Configuration,DC=gecusvi,DC=com (network error): 1722
(0x6ba):
    The RPC server is unavailable.


C:\Users\administrator.GECUSVI>
05:29 PM Zakee: Doing primary tests

   Testing server: Default-First-Site-Name\GEC-DC1
      Starting test: Advertising
         Fatal Error:DsGetDcName (GEC-DC1) call failed, error 1355
         The Locator could not find the server.
         ......................... GEC-DC1 failed test Advertising
      Starting test: FrsEvent
         There are warning or error events within the last 24 hours after the
         SYSVOL has been shared.  Failing SYSVOL replication problems may cause
         Group Policy problems.
         ......................... GEC-DC1 passed test FrsEvent
      Starting test: DFSREvent
         ......................... GEC-DC1 passed test DFSREvent
      Starting test: SysVolCheck
         ......................... GEC-DC1 passed test SysVolCheck
      Starting test: KccEvent
         A warning event occurred.  EventID: 0x80000B46
            Time Generated: 12/12/2018   07:46:21
            Event String:
            The security of this directory server can be significantly enhanced
by configuring the server to reject SASL (Negotiate, Kerberos, NTLM, or Digest)
LDAP binds that do not request signing (integrity verification) and LDAP simple
binds that are performed on a clear text (non-SSL/TLS-encrypted) connection.  Ev
en if no clients are using such binds, configuring the server to reject them wil
l improve the security of this server.
         A warning event occurred.  EventID: 0x8000082C
            Time Generated: 12/12/2018   07:47:21
            Event String:
         ......................... GEC-DC1 passed test KccEvent
      Starting test: KnowsOfRoleHolders
         ......................... GEC-DC1 passed test KnowsOfRoleHolders
      Starting test: MachineAccount
         ......................... GEC-DC1 passed test MachineAccount
      Starting test: NCSecDesc
         ......................... GEC-DC1 passed test NCSecDesc
      Starting test: NetLogons
         Unable to connect to the NETLOGON share! (\\GEC-DC1\netlogon)
         [GEC-DC1] An net use or LsaPolicy operation failed with error 67,
         The network name cannot be found..
         ......................... GEC-DC1 failed test NetLogons
      Starting test: ObjectsReplicated
         ......................... GEC-DC1 passed test ObjectsReplicated
      Starting test: Replications
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-STX-DC1 to GEC-DC1
            Naming Context: DC=ForestDnsZones,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 05:57:39.
            2 failures have occurred since the last success.
            The guid-based DNS name
            801d01e9-beec-4dfd-94fc-ef9d0cea3957._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [GEC-STX-DC1] DsBindWithSpnEx() failed with error 1722,
         The RPC server is unavailable..
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-STX-DC1 to GEC-DC1
            Naming Context: DC=DomainDnsZones,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 05:57:39.
            2 failures have occurred since the last success.
            The guid-based DNS name
            801d01e9-beec-4dfd-94fc-ef9d0cea3957._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-STX-DC1 to GEC-DC1
            Naming Context: CN=Schema,CN=Configuration,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 05:57:39.
            2 failures have occurred since the last success.
            The guid-based DNS name
            801d01e9-beec-4dfd-94fc-ef9d0cea3957._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-DC2 to GEC-DC1
            Naming Context: CN=Schema,CN=Configuration,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 07:00:04.
            1 failures have occurred since the last success.
            The guid-based DNS name
            e94b54e1-600e-4fbd-a5e6-c78050d093f1._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-STX-DC1 to GEC-DC1
            Naming Context: CN=Configuration,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 05:57:39.
            2 failures have occurred since the last success.
            The guid-based DNS name
            801d01e9-beec-4dfd-94fc-ef9d0cea3957._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-DC2 to GEC-DC1
            Naming Context: CN=Configuration,DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 06:59:22.
            1 failures have occurred since the last success.
            The guid-based DNS name
            e94b54e1-600e-4fbd-a5e6-c78050d093f1._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         [Replications Check,GEC-DC1] A recent replication attempt failed:
            From GEC-STX-DC1 to GEC-DC1
            Naming Context: DC=gecusvi,DC=com
            The replication generated an error (8524):
            The DSA operation is unable to proceed because of a DNS lookup failu
re.

            The failure occurred at 2018-12-12 07:46:51.
            The last success occurred at 2018-12-12 06:47:56.
            2 failures have occurred since the last success.
            The guid-based DNS name
            801d01e9-beec-4dfd-94fc-ef9d0cea3957._msdcs.gecusvi.com
            is not registered on one or more DNS servers.
         ......................... GEC-DC1 failed test Replications
      Starting test: RidManager
         ......................... GEC-DC1 passed test RidManager
      Starting test: Services
         ......................... GEC-DC1 passed test Services
      Starting test: SystemLog
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:03:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:08:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:13:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:18:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:23:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:24:18
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:28:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:33:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:38:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         An error event occurred.  EventID: 0x0000041E
            Time Generated: 12/12/2018   07:43:14
            Event String:
            The processing of Group Policy failed. Windows could not obtain the
name of a domain controller. This could be caused by a name resolution failure.
Verify your Domain Name System (DNS) is configured and working correctly.
         A warning event occurred.  EventID: 0x000003F6
            Time Generated: 12/12/2018   07:46:11
            Event String:
            Name resolution for the name _ldap._tcp.dc._msdcs.gecusvi.com. timed
 out after none of the configured DNS servers responded.
         A warning event occurred.  EventID: 0x00002724
            Time Generated: 12/12/2018   07:46:47
            Event String:
            This computer has at least one dynamically assigned IPv6 address.For
 reliable DHCPv6 server operation, you should use only static IPv6 addresses.
         A warning event occurred.  EventID: 0x000727AA
            Time Generated: 12/12/2018   07:46:58
            Event String:
            The WinRM service failed to create the following SPNs: WSMAN/GEC-dc1
.gecusvi.com; WSMAN/GEC-dc1.
         A warning event occurred.  EventID: 0x00001796
            Time Generated: 12/12/2018   07:47:45
            Event String:
            Microsoft Windows Server has detected that NTLM authentication is pr
esently being used between clients and this server. This event occurs once per b
oot of the server on the first time a client uses NTLM with this server.
         A warning event occurred.  EventID: 0xC000042B
            Time Generated: 12/12/2018   07:49:37
            Event String:
            The RD Session Host server cannot register 'TERMSRV' Service Princip
al Name to be used for server authentication. The following error occured: The s
pecified domain either does not exist or could not be contacted.
         An error event occurred.  EventID: 0x00000469
            Time Generated: 12/12/2018   07:50:57
            Event String:
            The processing of Group Policy failed because of lack of network con
nectivity to a domain controller. This may be a transient condition. A success m
essage would be generated once the machine gets connected to the domain controll
er and Group Policy has successfully processed. If you do not see a success mess
age for several hours, then contact your administrator.
         An error event occurred.  EventID: 0x00000469
            Time Generated: 12/12/2018   07:50:57
            Event String:
            The processing of Group Policy failed because of lack of network con
nectivity to a domain controller. This may be a transient condition. A success m
essage would be generated once the machine gets connected to the domain controll
er and Group Policy has successfully processed. If you do not see a success mess
age for several hours, then contact your administrator.
         ......................... GEC-DC1 failed test SystemLog
      Starting test: VerifyReferences
         ......................... GEC-DC1 passed test VerifyReferences


   Running partition tests on : ForestDnsZones
      Starting test: CheckSDRefDom
         ......................... ForestDnsZones passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... ForestDnsZones passed test
         CrossRefValidation

   Running partition tests on : DomainDnsZones
      Starting test: CheckSDRefDom
         ......................... DomainDnsZones passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... DomainDnsZones passed test
         CrossRefValidation

   Running partition tests on : Schema
      Starting test: CheckSDRefDom
         ......................... Schema passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... Schema passed test CrossRefValidation

   Running partition tests on : Configuration
      Starting test: CheckSDRefDom
         ......................... Configuration passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... Configuration passed test CrossRefValidation

   Running partition tests on : gecusvi
      Starting test: CheckSDRefDom
         ......................... gecusvi passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... gecusvi passed test CrossRefValidation

   Running enterprise tests on : gecusvi.com
      Starting test: LocatorCheck
         Warning: DcGetDcName(GC_SERVER_REQUIRED) call failed, error 1355
         A Global Catalog Server could not be located - All GC's are down.
         Warning: DcGetDcName(TIME_SERVER) call failed, error 1355
         A Time Server could not be located.
         The server holding the PDC role is down.
         Warning: DcGetDcName(GOOD_TIME_SERVER_PREFERRED) call failed, error
         1355
         A Good Time Server could not be located.
         Warning: DcGetDcName(KDC_REQUIRED) call failed, error 1355
         A KDC could not be located - All the KDCs are down.
         ......................... gecusvi.com failed test LocatorCheck
      Starting test: Intersite
         ......................... gecusvi.com passed test Intersite

C:\Users\administrator.GECUSVI>


FYI: gec-dc1 = dc2.    gec-stx-dc1 = dc1 (hard drive failure)

Are you sure that you demoted the domain controller correctly?

What process did you follow? Because by the looks of it, you've either had a catastrophic failure of the demotion or done it completely wrong.

Regards

Alex

I did a while back following instructions on the internet. Now that dc1 has died, and other domain controllers are sitting there. Is there a way to remove the failed dc1??

Be more specific, what instructions, you must have a link for it, beacuse according that, there is no way at all that the DCPROMO has worked anywhere near correctly, you have DC1 references in your sites and services, DNS and DFSR. Which is for all intents and purposes, everywhere.

Did you just delete the server from AD and shut it down? Honesty will help here, we're going to have to do a full metadata cleanup of this DC now.

here's the link I followed: https://winsvr.wordpress.com/2012/12/17/transferring-fsmo-roles-from-ws-2008r2-dc-to-ws-2012-dc/

I didn't delete the dc1 (failed server) from AD, it stayed on until this morning when it died. So now I'm finding out that the other ADs are not doing job of DC.

That's by far the worst guide i've ever seen.... without a doubt... Sorry but this could be interesting.

Ok look,

First up, lets seize the roles from the failed demoted DC.

https://blogs.technet.microsoft.com/canitpro/2015/10/14/step-by-step-seizing-the-operation-master-roles-in-windows-server-2012-r2/


Follow that, then shutdown DC1, see if you continue to have the login issues. This will basically take all the roles available.

Next up, clean up the old domain controller, follow the link below.

https://www.petri.com/delete_failed_dcs_from_ad

Next up,

Run DCDiag again.

Attach the results as a text file, dropping that much info into the comment isn't very user friendly.

Do all the above from your DC2 by the way.

You don't need to do any of what they said since there is a direct upgrade path from 2008r2 to 2012r2 btw.

netdom /query FSMO

Run that as well, you could also boot up your 2008r2 box, see if it's still working and then transfer the roles

Run that first though, see how it goes.

thanks, working on it now!

Run netdom /FSMO first fella, we need to see where all your roles are, if we can migrate them to your 2012 box we will try to do that nicely before we strip out the old DC

If we can migrate the roles, we can consider using DCPromo to take it out of action, which is much nicer than having to force it out using those links :-)

Regards
Alex

DC1 is dead in the water... no way to access it. hard drive bit the dust.

Once I complete the seizing of the rolls, I reboot dc2? or just move on to clean up?

when I try to run netdom /FSMO on dc2 after seizing, it says The specified domain either does not exist or could not babe contacted. The command failed to complete successfully.

Urgh ok.....


If your AD hasn't replicated you've lost your domain, do you have backups?

Are you sure you ran DCDiag prior to carrying out this work and you saw it replicating to DC2?

Could you seize the roles to DC2?

It looks as though it should be on DC2

    From: CN=NTDS Settings,CN=GEC-DC1,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
    To  : CN=NTDS Settings,CN=GEC-DC2,CN=Servers,CN=Default-First-Site-Name,CN=S
ites,CN=Configuration,DC=gecusvi,DC=com
CALLBACK MESSAGE: The following replication completed successfully:


Right, can you launch AD users and computers and can you see the infrastructure?

I did, and it showed all FSMO on dc2. but now that dc1 is dead, having issues.

I ran the seizing just now on DC2. Should I use DC3 to seize the roles?

No,

Launch AD users and computers and connect to DC2 if it doesn't find it. Tell me your accounts are there please.

when I try to launch AD users and computers it says Naming information cannot be located because: the specified domain either does not exist or could not be contacted...

Run DC diag again, this time please attach it as a file.

attached dcdiag results
dcdiag.txt

Sorry

DCDIAG  /v/c/d/e

That'll test all servers.

Also

repadmin

Is there Zero way to get DC1 back up?

dcdiag-full.txt

Right,

We need to check NTDSUTIL to see if DC2 is even in there

Command prompt

NTDSUTIL

Metadata Cleanup

Connections

Connect to server DC2

Then q and enter should take you back to metadata cleanup

Select operation target

List domains

See what that has in there for me

ntdsutil.txt

Restart the netlogon service on DC2

Also, if you log directly onto DC2, and launch ADUC, does that work?

Lastly, launch Sites and services, see if that comes back with your DC lists.

after netlogon service restart on both good DCs, still unable to connect to ADUC or ADSS.

OK have you logged onto the server and checked it from there?

yes, I logged onto both good DCs and got same errors.

I'm able to get the failed DC (gec-stx-dc1) to boot into directory services restore mode, but whenever I try to boot normally or into something that provides network connectivity, I get the blue screen of death.

What's the BSOD error?

Also,

I want you to navigate to \\dc2\sysvol

Go in there, tell me if you have folders in there

i'm at \\dc2\sysvol what's next? I see the domain folder in that folder.

Right,

That's replicated apparently.

What I want you to do, is DCDIAG /c /v

I need this to be done on DC2, so please log on, run it from there.

Next up

netdom query FSMO

Check the results on there and tell me what DC each points to.

dcdiag-gec-dc1.txt

results of netdomo query FSMO: The specified domain either does not exist or could not be contacted.

OK,

On DC2

Get-ADDomainController | ft Name,IsGlobalCatalog

Do you have all DCs configured as DNS servers on all the clients DNS settings?

I have only the good DCs (gec-dc1 and gec-dc2) as DNS servres on all clients DNS settings.

'Get-ADDomainController ' is not  recognized as an internal or external command, operable program or batch file'

Shaun,

He can't even access active directory from within a domain controller (DC2), Settings have been modified directly within ADSIEdit on DC1.

https://winsvr.wordpress.com/2012/12/17/transferring-fsmo-roles-from-ws-2008r2-dc-to-ws-2012-dc/

He's followed that, and since that the DC won't boot, it throws back a BSOD.

DCDIAG shows everything still being on DC1

He can't seize roles.

There isn't a global catalog server according to the last DCDiag.

Netdom query FSMO from DC2, whilst he's logged onto the DC, it can't find the domain.

Import-module ACtiveDirectory on DC2 and then run that command please.

Also,

on DC2, when you launch the server manager, it must have active directory installed as a role correct?

Thanks
Alex

Also,


One last effort from me on this one

https://blogs.technet.microsoft.com/canitpro/2015/10/14/step-by-step-seizing-the-operation-master-roles-in-windows-server-2012-r2/

Follow that, you will need to put in your FQDN of your DC2 domain controller.

Yes it has AD installed.

'Import-module' is not a recognized command

Can you find a backup of the old DC? That is the easy solution for now.
Then we will provide you some guidance how to demote.

yes I have a windows backup of the old DC

The old DC can boot into "Directory services restore mode"

It won't however boot into windows, I suspect because of the Schema DLL issue.  

The only other thing I can think of is either try an authorative restore from DSRM, or a back up at this point as you said.

Then restore it for now.
STop replication on all servers. once restore is complete start the replication.

dc1 just booted up!! I need to push it over to dc2 STAT!

NO!

Hold on

Before you change anything lets take our time with this so it's not a tragic situation.

kk

Right first of all,

Your DC has booted up, please log onto DC2 and try to launch Active Directory

I get the same initial error when launching AD, but I can change domain controller over to dc1 and it works!

Right,

Next up, lets check your Global Catalog servers

From that active directory

Follow this

https://www.dtonias.com/determine-global-catalog-server/

You want to make sure that DC2 is a GC

all 3 are Global Catalog

Right,

repadmin /syncall

That will syncronise your domain, in theory all domain controllers will syncronise with each other with the newest info.

Check that then check DCDIAG /c /v to make sure they replicated.

When you've done that, from the 2008 box, run "Netdom Query FSMO"

That will tell you where your FSMO roles are

I assume they will all be on DC1

https://www.dtonias.com/transfer-fsmo-roles-domain-controller/

Follow that, use the GUI on DC1 to transfer the roles safely. I'm going to be on the road for a bit, I'll be back in a few hours. Best of luck though, I'll check back in a few hours to see where you got to.

ran netdom on the dc1, and it says all the roles are on dc2. completed successfully.

dcdiag /v /c is too long to all be in command prompt, the beginning is being truncated by the time it's finishing.

I just did netdom query FSMO on dc2 and it says that itself (dc2) has all the rolls... what's up with that. If it has all the rolls, then why is it depending on dc1?? doesn't the domain come over to dc2 as well? it seems like when dc1 is down, the entire domain dies.

now under ADSS on dc2, it's showing dc1 as a server, but it doesn't have NTDS under it. So i'm guessing it's no longer a GC from DC2 and DC3's perspective?

And I'm back,

Right, if it were me I'd go through

https://winsvr.wordpress.com/2012/12/17/transferring-fsmo-roles-from-ws-2008r2-dc-to-ws-2012-dc/

and set it back to how it was.

When you installed active directory, did you do a DCPromo or did you use server manager?

THanks
Alex

I used server manager

ok,

Is this being virtualised off, if so I'd do the following, build a NEW 2012 box, do not modify your other boxes or your 2003 box.

https://blog.thesysadmins.co.uk/server-2012-add-domain-controller.html

Then wait for replication, ensure that you can view AD users and computer from 2012r2 before anything else.

Regards
Alex

what to do about the dc1? it's still acting as though it's active, and if it goes down, the domain goes down.

dc1 is not virtualized, the others are VMs

Leave DC1 well alone,

At this point, is your domain up and running again?

Thanks
Alex

yes domain is up and running... I just fear for if it goes down again... it's on its last legs!

Yeah, in which case leave it as it, promote a new DC following the guide I've listed, ensure it links with DC1.