Link to home
Start Free TrialLog in
Avatar of birchy66
birchy66

asked on

Active Directory cannot connect to domain and no Global Catalog server in dcdiag

This is the dcdiag on PDC, GC,
 

C:\Users...>dcdiag

Directory Server Diagnosis

Performing initial setup:
   Trying to find home server...
   Home Server = SMCDC01
   * Identified AD Forest.
   Done gathering initial info.

Doing initial required tests

   Testing server: Default-First-Site\SMCDC01
      Starting test: Connectivity
         ......................... SMCDC01 passed test Connectivity

Doing primary tests

   Testing server: Default-First-Site\SMCDC01
      Starting test: Advertising
         Fatal Error:DsGetDcName (SMCDC01) call failed, error 1355
         The Locator could not find the server.
         ......................... SMCDC01 failed test Advertising
      Starting test: FrsEvent
         There are warning or error events within the last 24 hours after the
         SYSVOL has been shared.  Failing SYSVOL replication problems may cause
         Group Policy problems.
         ......................... SMCDC01 failed test FrsEvent
      Starting test: DFSREvent
         There are warning or error events within the last 24 hours after the
         SYSVOL has been shared.  Failing SYSVOL replication problems may cause
         Group Policy problems.
         ......................... SMCDC01 failed test DFSREvent
      Starting test: SysVolCheck
         ......................... SMCDC01 passed test SysVolCheck
      Starting test: KccEvent
         ......................... SMCDC01 passed test KccEvent
      Starting test: KnowsOfRoleHolders
         ......................... SMCDC01 passed test KnowsOfRoleHolders
      Starting test: MachineAccount
         ......................... SMCDC01 passed test MachineAccount
      Starting test: NCSecDesc
         ......................... SMCDC01 passed test NCSecDesc
      Starting test: NetLogons
         Unable to connect to the NETLOGON share! (\\SMCDC01\netlogon)
         [SMCDC01] An net use or LsaPolicy operation failed with error 67,
         Win32 Error 67.
         ......................... SMCDC01 failed test NetLogons
      Starting test: ObjectsReplicated
         ......................... SMCDC01 passed test ObjectsReplicated
      Starting test: Replications
         [Replications Check,SMCDC01] A recent replication attempt failed:
            From SMCDC03 to SMCDC01
            Naming Context: DC=ForestDnsZones,DC=StMarys,DC=local
            The replication generated an error (1908):
            Win32 Error 1908
            The failure occurred at 2009-09-10 09:50:26.
            The last success occurred at 2009-09-10 09:46:55.
            1 failures have occurred since the last success.
            Kerberos Error.
            A KDC was not found to authenticate the call.
            Check that sufficient domain controllers are available.
         [Replications Check,SMCDC01] A recent replication attempt failed:
            From SMCDC03 to SMCDC01
            Naming Context: CN=Schema,CN=Configuration,DC=StMarys,DC=local
            The replication generated an error (1908):
            Win32 Error 1908
            The failure occurred at 2009-09-10 09:49:29.
            The last success occurred at 2009-09-10 09:46:55.
            1 failures have occurred since the last success.
            Kerberos Error.
            A KDC was not found to authenticate the call.
            Check that sufficient domain controllers are available.
         ......................... SMCDC01 failed test Replications
      Starting test: RidManager
         ......................... SMCDC01 passed test RidManager
      Starting test: Services
         ......................... SMCDC01 passed test Services
      Starting test: SystemLog
         An Warning Event occurred.  EventID: 0x800007DC
            Time Generated: 09/10/2009   10:06:21
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x800007DC
            Time Generated: 09/10/2009   10:06:21
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x800007DC
            Time Generated: 09/10/2009   10:06:21
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x800007DC
            Time Generated: 09/10/2009   10:06:21
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x800007DC
            Time Generated: 09/10/2009   10:06:21
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x800007DC
            Time Generated: 09/10/2009   10:06:21
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x800007DC
            Time Generated: 09/10/2009   10:06:21
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x800007DC
            Time Generated: 09/10/2009   10:06:21
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x800007DC
            Time Generated: 09/10/2009   10:06:21
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x800007DC
            Time Generated: 09/10/2009   10:06:23
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x8004001A
            Time Generated: 09/10/2009   10:06:58
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x8000001D
            Time Generated: 09/10/2009   10:07:43
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x8000A000
            Time Generated: 09/10/2009   10:08:13
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x8000A000
            Time Generated: 09/10/2009   09:46:22
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x8000A000
            Time Generated: 09/10/2009   09:46:25
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x8000A000
            Time Generated: 09/10/2009   09:46:25
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000469
            Time Generated: 09/10/2009   09:46:27
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x825A000C
            Time Generated: 09/10/2009   09:46:32
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000469
            Time Generated: 09/10/2009   09:47:41
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x00001695
            Time Generated: 09/10/2009   09:51:27
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x00001695
            Time Generated: 09/10/2009   09:51:27
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Warning Event occurred.  EventID: 0x00001695
            Time Generated: 09/10/2009   09:51:27
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000406
            Time Generated: 09/10/2009   09:56:30
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000406
            Time Generated: 09/10/2009   10:01:35
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000406
            Time Generated: 09/10/2009   10:06:38
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000469
            Time Generated: 09/10/2009   10:11:38
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000457
            Time Generated: 09/10/2009   10:21:46
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000457
            Time Generated: 09/10/2009   10:21:46
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000457
            Time Generated: 09/10/2009   10:21:50
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000457
            Time Generated: 09/10/2009   10:21:51
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000457
            Time Generated: 09/10/2009   10:21:51
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000457
            Time Generated: 09/10/2009   10:21:52
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         ......................... SMCDC01 failed test SystemLog
      Starting test: VerifyReferences
         ......................... SMCDC01 passed test VerifyReferences


   Running partition tests on : DomainDnsZones
      Starting test: CheckSDRefDom
         ......................... DomainDnsZones passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... DomainDnsZones passed test
         CrossRefValidation

   Running partition tests on : ForestDnsZones
      Starting test: CheckSDRefDom
         ......................... ForestDnsZones passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... ForestDnsZones passed test
         CrossRefValidation

   Running partition tests on : Schema
      Starting test: CheckSDRefDom
         ......................... Schema passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... Schema passed test CrossRefValidation

   Running partition tests on : Configuration
      Starting test: CheckSDRefDom
         ......................... Configuration passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... Configuration passed test CrossRefValidation

   Running partition tests on : StMarys
      Starting test: CheckSDRefDom
         ......................... StMarys passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... StMarys passed test CrossRefValidation

   Running enterprise tests on : StMarys.local
      Starting test: LocatorCheck
         Warning: DcGetDcName(GC_SERVER_REQUIRED) call failed, error 1355
         A Global Catalog Server could not be located - All GC's are down.

Urgent help required  - no one can logon.  Any help would be useful.
Avatar of Chris Dent
Chris Dent
Flag of United Kingdom of Great Britain and Northern Ireland image


Please would you show us:

ipconfig /all
netdiag

Or can you confirm the DNS settings for your servers / clients.

Can you confirm that you do have a Global Catalog available (AD Sites and Services)?

And would you provide details of the error messages from the Event Logs.

Cheers,

Chris
Avatar of birchy66
birchy66

ASKER

DNS server settings are the two DC's and these are displayed in ipconfig results
The server that I ran the dcdiag on is listed as a global catalog server in AD sites and services but I cannot get into AD Domains and Trusts to verify the OM role.

Event error info to follow
Here are the most pertinent event logs as requested:

DNS Event log warning

Log Name:      DNS Server
Source:        Microsoft-Windows-DNS-Server-Service
Date:          10/09/2009 09:50:22
Event ID:      4013
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      SMCDC01.StMarys.local
Description:
The DNS server is waiting for Active Directory Domain Services (AD DS) to signal that the initial synchronization of the directory has been completed. The DNS server service cannot start until the initial synchronization is complete because critical DNS data might not yet be replicated onto this domain controller. If events in the AD DS event log indicate that there is a problem with DNS name resolution, consider adding the IP address of another DNS server for this domain to the DNS server list in the Internet Protocol properties of this computer. This event will be logged every two minutes until AD DS has signaled that the initial synchronization has successfully completed.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-DNS-Server-Service" Guid="{71A551F5-C893-4849-886B-B5EC8502641E}" EventSourceName="DNS" />
    <EventID Qualifiers="32768">4013</EventID>
    <Version>0</Version>
    <Level>3</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2009-09-10T08:50:22.000Z" />
    <EventRecordID>65</EventRecordID>
    <Correlation />
    <Execution ProcessID="0" ThreadID="0" />
    <Channel>DNS Server</Channel>
    <Computer>SMCDC01.StMarys.local</Computer>
    <Security />
  </System>
  <EventData Name="DNS_EVENT_DS_OPEN_WAIT">
  </EventData>
</Event>

Directory Services Error

Log Name:      Directory Service
Source:        Microsoft-Windows-ActiveDirectory_DomainService
Date:          10/09/2009 11:00:49
Event ID:      1126
Task Category: Global Catalog
Level:         Error
Keywords:      Classic
User:          ANONYMOUS LOGON
Computer:      SMCDC01.StMarys.local
Description:
Active Directory Domain Services was unable to establish a connection with the global catalog.
 
Additional Data
Error value:
1355 The specified domain either does not exist or could not be contacted.
Internal ID:
3200d50
 
User Action:
Make sure a global catalog is available in the forest, and is reachable from this domain controller. You may use the nltest utility to diagnose this problem.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-ActiveDirectory_DomainService" Guid="{0e8478c5-3605-4e8c-8497-1e730c959516}" EventSourceName="NTDS General" />
    <EventID Qualifiers="49152">1126</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>18</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8080000000000000</Keywords>
    <TimeCreated SystemTime="2009-09-10T10:00:49.785Z" />
    <EventRecordID>6687</EventRecordID>
    <Correlation />
    <Execution ProcessID="588" ThreadID="784" />
    <Channel>Directory Service</Channel>
    <Computer>SMCDC01.StMarys.local</Computer>
    <Security UserID="S-1-5-7" />
  </System>
  <EventData>
    <Data>3200d50</Data>
    <Data>1355</Data>
    <Data>The specified domain either does not exist or could not be contacted.</Data>
  </EventData>
</Event>

File Replication Service Warning

Log Name:      File Replication Service
Source:        NtFrs
Date:          10/09/2009 09:56:35
Event ID:      13508
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      SMCDC01.StMarys.local
Description:
The File Replication Service is having trouble enabling replication from SMCDC03 to SMCDC01 for c:\windows\sysvol\domain using the DNS name smcdc03.StMarys.local. FRS will keep retrying.
 Following are some of the reasons you would see this warning.
 
 [1] FRS can not correctly resolve the DNS name smcdc03.StMarys.local from this computer.
 [2] FRS is not running on smcdc03.StMarys.local.
 [3] The topology information in the Active Directory Domain Services for this replica has not yet replicated to all the Domain Controllers.
 
 This event log message will appear once per connection, After the problem is fixed you will see another event log message indicating that the connection has been established.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="NtFrs" />
    <EventID Qualifiers="32768">13508</EventID>
    <Level>3</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2009-09-10T08:56:35.000Z" />
    <EventRecordID>684</EventRecordID>
    <Channel>File Replication Service</Channel>
    <Computer>SMCDC01.StMarys.local</Computer>
    <Security />
  </System>
  <EventData>
    <Data>SMCDC03</Data>
    <Data>SMCDC01</Data>
    <Data>c:\windows\sysvol\domain</Data>
    <Data>smcdc03.StMarys.local</Data>
    <Binary>00000000</Binary>
  </EventData>
</Event>


Can you run:

nslookup -q=srv _gc._tcp.StMarys.local

And confirm that it returns some of your domain controllers?

Is Event ID 4013 repeating? Or just a single instance? If it's only one, does that coincide with a reboot of the server? If so, we won't worry about it.

Chris

Thanks for the reply.

Here is the nslookup result:

C:\Users\administrator.STMARYS>nslookup -q=srv _gc._tcp.StMarys.local
Server:  smcdc01.stmarys.local
Address:  10.11.55.253

_gc._tcp.StMarys.local  SRV service location:
          priority       = 0
          weight         = 100
          port           = 3268
          svr hostname   = smcdc01.stmarys.local
_gc._tcp.StMarys.local  SRV service location:
          priority       = 0
          weight         = 100
          port           = 3268
          svr hostname   = smcdc03.stmarys.local
smcdc01.stmarys.local   internet address = 10.11.55.253
smcdc03.stmarys.local   internet address = 10.11.55.239

Event 4013 does coincide with a reboot yes.

Would you run this please?

nltest /DsGetDc:stmarys.local

From the above I would expect it to return one of your domain controllers and for that to have the GC flag. Note that this calls the same function DcDiag where it returned a fatal error previously.

There are a couple more records in DNS we may look at if that fails. However, you might also consider running:

netdiag /fix

Which should force it to re-register the records we'd be looking for.

Can you confirm that SysVol and NetLogon are shared on both of your domain controllers?

Chris

Sorry, one more.

Did you also run DCDiag on the second DC? The two will share the load, so it's not really important which has the PDC Emulator role. And I assume there's no smcdc02?

Chris
DCdiag from the other DC is as follows and no there isn't an smcdc02:

C:\Users\administrator.STMARYS>dcdiag

Directory Server Diagnosis

Performing initial setup:
   Trying to find home server...
   Home Server = smcdc03
   * Identified AD Forest.
   Done gathering initial info.

Doing initial required tests

   Testing server: Default-First-Site\SMCDC03
      Starting test: Connectivity
         ......................... SMCDC03 passed test Connectivity

Doing primary tests

   Testing server: Default-First-Site\SMCDC03
      Starting test: Advertising
         Fatal Error:DsGetDcName (SMCDC03) call failed, error 1355
         The Locator could not find the server.
         ......................... SMCDC03 failed test Advertising
      Starting test: FrsEvent
         There are warning or error events within the last 24 hours after the
         SYSVOL has been shared.  Failing SYSVOL replication problems may cause
         Group Policy problems.
         ......................... SMCDC03 passed test FrsEvent
      Starting test: DFSREvent
         ......................... SMCDC03 passed test DFSREvent
      Starting test: SysVolCheck
         ......................... SMCDC03 passed test SysVolCheck
      Starting test: KccEvent
         ......................... SMCDC03 passed test KccEvent
      Starting test: KnowsOfRoleHolders
         ......................... SMCDC03 passed test KnowsOfRoleHolders
      Starting test: MachineAccount
         ......................... SMCDC03 passed test MachineAccount
      Starting test: NCSecDesc
         ......................... SMCDC03 passed test NCSecDesc
      Starting test: NetLogons
         Unable to connect to the NETLOGON share! (\\SMCDC03\netlogon)
         [SMCDC03] An net use or LsaPolicy operation failed with error 67,
         Win32 Error 67.
         ......................... SMCDC03 failed test NetLogons
      Starting test: ObjectsReplicated
         ......................... SMCDC03 passed test ObjectsReplicated
      Starting test: Replications
         ......................... SMCDC03 passed test Replications
      Starting test: RidManager
         ......................... SMCDC03 passed test RidManager
      Starting test: Services
         ......................... SMCDC03 passed test Services
      Starting test: SystemLog
         An Error Event occurred.  EventID: 0x00000406
            Time Generated: 09/10/2009   10:44:31
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000406
            Time Generated: 09/10/2009   10:49:34
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000406
            Time Generated: 09/10/2009   10:54:36
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000406
            Time Generated: 09/10/2009   10:59:39
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000406
            Time Generated: 09/10/2009   11:04:42
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000406
            Time Generated: 09/10/2009   11:09:45
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000406
            Time Generated: 09/10/2009   11:14:47
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000406
            Time Generated: 09/10/2009   11:19:50
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000406
            Time Generated: 09/10/2009   11:24:53
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000406
            Time Generated: 09/10/2009   11:29:56
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000406
            Time Generated: 09/10/2009   11:34:58
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         An Error Event occurred.  EventID: 0x00000406
            Time Generated: 09/10/2009   11:39:59
            EvtFormatMessage failed, error 15100 Win32 Error 15100.
            (Event String (event log = System) could not be retrieved, error
            0x3afc)
         ......................... SMCDC03 failed test SystemLog
      Starting test: VerifyReferences
         ......................... SMCDC03 passed test VerifyReferences


   Running partition tests on : ForestDnsZones
      Starting test: CheckSDRefDom
         ......................... ForestDnsZones passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... ForestDnsZones passed test
         CrossRefValidation

   Running partition tests on : DomainDnsZones
      Starting test: CheckSDRefDom
         ......................... DomainDnsZones passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... DomainDnsZones passed test
         CrossRefValidation

   Running partition tests on : Schema
      Starting test: CheckSDRefDom
         ......................... Schema passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... Schema passed test CrossRefValidation

   Running partition tests on : Configuration
      Starting test: CheckSDRefDom
         ......................... Configuration passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... Configuration passed test CrossRefValidation

Thanks

If the nltest command above fails, would you be able to give us this from both DCs:

netdiag /v /l

This will create a netdiag.log file, the file will be pretty long so adding it as an attachment would be preferable if you don't mind.

Chris


The nltest result is as follows:

C:\Users\administrator.STMARYS>nltest /DsGetDc:stmarys.local
Getting DC name failed: Status = 1355 0x54b ERROR_NO_SUCH_DOMAIN

Also the sysvol shares are up but no netlogon share.  I checked the default directory (%SystemRoot%\sysvol\sysvol\<DomainName>\scripts) and it is empty.

Servers are 2008 so unable to do a netdiag. Please find a dcdiag attached. Hopefully this provides the same information.

Sorry Chris,
Here is the attachment
dcdiag.txt

Ack, always forget about that with 2008. We'll have to run some more manual checks in DNS to see what's missing.

Can you run...

nslookup -q=srv _gc._tcp.Default-First-Site._sites.stmarys.local
nslookup -q=srv _ldap._tcp.gc._msdcs.stmarys.local
nslookup -q=srv _ldap._tcp.Default-First-Site._sites.gc._msdcs.stmarys.local

That'll do for a start, if they all work I'll throw together a quick script to check the lot.

If the last two fail, can you verify that the _msdcs.stmarys.local zone exists (or that you have a folder called _msdcs in stmarys.local)?

Chris
The last one failed. Results attached. However in DNS there is a folder called _msdcs in stmarys.local.

Andy
nslookup.txt

See this is where you really need NetDiag.

Ah well... try restarting the NetLogon service please? Then run the nslookup queries again?

And lets have DCDiag tell us a bit more with these:

DCDiag /c /v /f:dcdiag.log

That runs all tests with verbose output and logs it to the dcdiag.log file.

Do you have PowerShell installed on either of the servers? If not, would you add it? You'll find it under Features. We can run a few quick tests from there to emulate some of stuff netdiag used to tell us.

Chris
Thanks Chris

Installing powershell now. The nslookups all worked after restarting netlogon.  Please find the dcdiag log file and nslookup results attached.
nslookup2.txt
and the dcdiag.log
dcdiag.log

We're still failing on this one?

nltest /DsGetDc:stmarys.local

Could you open up Services and verify that the "Kerberos Key Distribution Center" service is running? That's the KDC it mentions. If it is, but is still failing connections (as mentioned in DCDiag) is there any chance we can go for a reboot?

It's inability to find the netlogon share is a bit troubling. Do you have any errors relating to that in either the NTFRS or DFS logs?

Chris
The KDC service is running and I have done a reboot. I cannot access AD users and Computers or site and services. DCdiag looks the same. Also I notice that sysvol is no longer shared.

I can't see any particular errors relating to the netlogon share.

Andy

We can force the server to rebuild SysVol (and NetLogon) but you will potentially lose any group policies you have configured which is far from ideal.

Do you have Exchange as well? Or just AD on its own?

Do any of the errors in the FRS log mention Journal Wrap?

Chris
Is there a way of exporting group policies and importing them after Sysvol and Netlogon are rebuilt?

We do have just AD and there are no errors in the FRS log relating to journal wrap.  If the worst comes to the worst I would be happy to get the server up and running again and rebuild the GPO's afterwards.

Andy

Yes if it will let you into the Group Policy Management Console, which isn't looking too promising if it won't let you into the rest.

Do both DCs exhibit the same symptoms?

Chris
Yes both exhibit the same symptoms. One thing I hadn't mentioned as it didn't seem relevant was that this is a virtualised server and I have a snapshot of it from a couple of weeks back. This is not much good for restoring but I could revert to this snapshot in isolation and export the GPO's as not much has changed with them.

Andy

Lets rebuild the DNS part since that's pretty straight-forward. It would be nice to rule that out as being broken.

Are you able to open the DNS Console at the moment?

If so:

1. Nominate a server that's going to be the DNS server while we do this. Doesn't matter which you choose.
2. On both servers (and potentially all clients) change the Preferred DNS server to the nominated server (TCP/IP configuration). Do not configure an Alternate DNS server at this stage.
3. On the nominated server:
  a. Open the DNS Console
  b. Expand Forward Lookup Zones
  c. Select stmarys.local
  d. Right click and open the zone properties
  e. Next to Type, select Change. Untick store in Active Directory
  f. Under Dynamic Updates, change it to Secure and Non-Secure
  g. Repeat for _msdcs.stmarys.local
4. On both Domain Controllers run:

  ipconfig /registerdns
  net stop netlogon && net start netlogon

That takes DNS out of AD, we want to ensure that is available throughout.

Next, if you can reboot DC01 again please do so. See how well it started up, we'll hope that DNS is fine at this stage. If it's still failing to load SysVol and NetLogon we can move onto having it rebuild that.

Chris

Can't find this:

  g. Repeat for _msdcs.stmarys.local

There is an _msdcs folder under stmarys.local but there is not a properties option when I right click on it. Is this what you meant?

I have completed the other steps and restarting now.

Andy

Okay, don't worry about _msdcs as a zone. That appears if the domain is built as 2008, but if it was built a long time ago and upgraded it wouldn't be present.

Either way it doesn't matter, as long as you have the folder somewhere :)

Chris
Okay, server has rebooted. still no sysvol or netlogon on net share

Andy
ASKER CERTIFIED SOLUTION
Avatar of Chris Dent
Chris Dent
Flag of United Kingdom of Great Britain and Northern Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Chris

I took a snapshot of both and completed the steps as directed.  What should I see now / what is next?

Thanks for your help

Andy

Any sign of NetLogon / SysVol?

Chris
Yes!  And I can access AD on both servers. What else do I need to do?

Does DNS need re associating with AD?

DCDiag again? See how happy it is before switching DNS back to AD Integrated.

Chris
It's better than it was but still with errors. failing on Dfsrevent and systemlog.  Please see attached

Andy
dcdiag2.log

Check the date stamps on the event log errors it's flagging. They may well be from earlier, if you can't see any since the last reboot then I'd be quite happy :)

Chris
Hi Chris

Yes you are correct the time stamps are before the sysvol rebuild

Can you advise of the next step.

Thanks

Switch DNS back to AD Integrated (DNS Console, zone properties, Change Type, and tick the box), and set updates to Secure Only.

Then it's just a case of keep an eye on it, make sure it doesn't become upset again.

Chris
Have switched DNS back and rebooting smcdc01 for good measure.  Will check that users / systems can logon once it is rebuilt. I haven't checked GPO's, am I likely to need to reinstate these?

Andy

BTW you will have more than earned your points if this works!

Potentially. We can recreate the default policies easily, but anything which isn't default is likely to have been killed off. Of course, having an operational domain is more important at this stage :)

Chris
Well I have tested computer and user accounts for logging on and all seems well. Group policies also seem to be working fine for drive mappings, folder redirection  and general security also!  You have definitely saved my bacon. Now I just have to work out why it happened and ensure it does not recurr.

Thanks

Andy

Yeah, that one will be the harder one, a good long look at the event logs is in order :)

Do you have Aging configured in DNS? I did wonder if some of the records we were missing were removed by Scavenging.

Chris