Solved

Crashed domain controller questions

Posted on 2011-02-27
28
1,671 Views
Last Modified: 2012-05-11
I know that these terms (PDC,BDC)  are out of date, but i dont know how to say this quickly without using them.  

My PDC is down.   The BDC was shtudown by someone else because, when it was running the PDC had issues that were fixed when the BDC was stopped.

I've restarted the BDC (running in a VM) and attempted to replicate.  No Joy, since its past the 60 day period.  I attempted to replicate anyway with the help of the registry hack,

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NTDS\Parameters
Allow Replication With Divergent and Corrupt Partner

Did not help.

I ran dcdiag /fix and netdiag /fix

I get errors like this in the event log of the pdc

Active Directory Replication encountered the existence of objects in the following partition that have been deleted from the local domain controllers (DCs) Active Directory database.  Not all direct or transitive replication partners replicated in the deletion before the tombstone lifetime number of days passed.  Objects that have been deleted and garbage collected from an Active Directory partition but still exist in the writable partitions of other DCs in the same domain, or read-only partitions of global catalog servers in other domains in the forest are known as "lingering objects". 

Open in new window



Now the when I run dcdiag /e /c /v  I see these errors.


Performing initial setup:
   * Verifying that the local machine PDC, is a DC. 
   * Connecting to directory service on server PDC.
   * Collecting site info.
   * Identifying all servers.
   * Identifying all NC cross-refs.
   * Found 2 DC(s). Testing 2 of them.
   Done gathering initial info.

Doing initial required tests
   
   Testing server: Default-First-Site-Name\PDC
      Starting test: Connectivity
         * Active Directory LDAP Services Check
         * Active Directory RPC Services Check
         ......................... PDC passed test Connectivity
   
   Testing server: Default-First-Site-Name\BDC
      Starting test: Connectivity
         * Active Directory LDAP Services Check
         * Active Directory RPC Services Check
         ......................... BDC passed test Connectivity

 The replication generated an error (8606):
  Insufficient attributes were given to create an object.  This object may not exist because it may    have been deleted and already garbage collected.

   Testing server: Default-First-Site-Name\PDC
      Starting test: Replications
         * Replications Check
         [Replications Check,PDC] A recent replication attempt failed:
            From BDC to PDC
            Naming Context: DC=DomainDnsZones,DC=myco,DC=net
            The replication generated an error (8606):
            Insufficient attributes were given to create an object.  This object may not exist because it may have been deleted and already garbage collected.
            The failure occurred at 2011-02-27 21:57:00.
            The last success occurred at 2010-02-01 18:53:03.
            9424 failures have occurred since the last success.
         [Replications Check,PDC ] A recent replication attempt failed:
            From BDC to PDC 
            Naming Context: DC=myco,DC=net

      Starting test: Advertising
         Warning: DsGetDcName returned information for \\bdc.myco.net, when we were trying to reach PDC.
         Server is not responding or is not considered suitable.
         The DC PDCis advertising itself as a DC and having a DS.
         The DC PDCis advertising as an LDAP server
         The DC PDCis advertising as having a writeable directory
         The DC PDCis advertising as a Key Distribution Center
         The DC PDCis advertising as a time server
         The DS PDCis advertising as a GC.
         ......................... PDC failed test Advertising

      Starting test: RidManager
         * Available RID Pool for the Domain is 2100 to 1073741823
         * PDC.myco.net is the RID Master
         * DsBind with RID Master was successful
         * rIDAllocationPool is 1100 to 1599
         * rIDPreviousAllocationPool is 1100 to 1599
         * rIDNextRID: 1599
         * Warning :Next rid pool not allocated
         * Warning :There is less than 0% available RIDs in the current pool
         ......................... PDC passed test RidManager


  
   Testing server: Default-First-Site-Name\BDC
      Starting test: Replications
         * Replications Check
         [Replications Check,BDC] A recent replication attempt failed:
            From PRDto BDC
            Naming Context: DC=ForestDnsZones,DC=myco,DC=net
            The replication generated an error (-2146893022):
            The target principal name is incorrect.
            The failure occurred at 2011-02-27 22:39:25.
            The last success occurred at 2010-02-01 18:46:46.
            123 failures have occurred since the last success.
         [Replications Check,BDC] A recent replication attempt failed:
            From PDC to BDC
            Naming Context: DC=DomainDnsZones,DC=myco,DC=net
            The replication generated an error (-2146893022):
            The target principal name is incorrect.
            The failure occurred at 2011-02-27 22:39:25.
            The last success occurred at 2010-02-01 18:46:46.
            150 failures have occurred since the last success.
         [Replications Check,BDC] A recent replication attempt failed:
            From PDC to BDC
            Naming Context: CN=Schema,CN=Configuration,DC=myco,DC=net
            The replication generated an error (-2146893022):
            The target principal name is incorrect.
            The failure occurred at 2011-02-27 22:39:25.
            The last success occurred at 2010-02-01 18:46:46.
            116 failures have occurred since the last success.
         [Replications Check,BDC] A recent replication attempt failed:
            From PDC to BDC
            Naming Context: CN=Configuration,DC=myco,DC=net
            The replication generated an error (-2146893022):
            The target principal name is incorrect.
            The failure occurred at 2011-02-27 22:39:25.
            The last success occurred at 2010-02-01 18:46:46.
            130 failures have occurred since the last success.
         [Replications Check,BDC] A recent replication attempt failed:
            From PDC  to BDC
            Naming Context: DC=myco,DC=net
            The replication generated an error (-2146893022):
            The target principal name is incorrect.
            The failure occurred at 2011-02-27 22:39:25.
            The last success occurred at 2010-02-01 19:02:59.
            4467 failures have occurred since the last success.
         * Replication Latency Check
         REPLICATION-RECEIVED LATENCY WARNING
         BDC:  Current time is 2011-02-27 22:39:25.
            DC=ForestDnsZones,DC=myco,DC=net
               Last replication recieved from PDC  at 2010-02-01 18:46:46.
               WARNING:  This latency is over the Tombstone Lifetime of 60 days!
            DC=DomainDnsZones,DC=myco,DC=net
               Last replication recieved from PDC  at 2010-02-01 18:46:46.
               WARNING:  This latency is over the Tombstone Lifetime of 60 days!
            CN=Schema,CN=Configuration,DC=myco,DC=net
               Last replication recieved from PDC  at 2010-02-01 18:46:46.
               WARNING:  This latency is over the Tombstone Lifetime of 60 days!
            CN=Configuration,DC=myco,DC=net
               Last replication recieved from PDC  at 2010-02-01 18:46:46.
               WARNING:  This latency is over the Tombstone Lifetime of 60 days!
            DC=myco,DC=net
               Last replication recieved from PDC  at 2010-02-01 19:02:59.
               WARNING:  This latency is over the Tombstone Lifetime of 60 days!


            The replication generated an error (8606):
            Insufficient attributes were given to create an object.  This object may not exist because it may have been deleted and already garbage collected.
            The failure occurred at 2011-02-27 22:28:31.
            The last success occurred at 2010-02-01 18:58:30.
            9502 failures have occurred since the last success.
         * Replication Latency Check
         REPLICATION-RECEIVED LATENCY WARNING
         PDC :  Current time is 2011-02-27 22:39:25.
            DC=DomainDnsZones,DC=myco,DC=net
               Last replication recieved from BDC at 2010-02-01 18:53:03.
               WARNING:  This latency is over the Tombstone Lifetime of 60 days!
            DC=myco,DC=net
               Last replication recieved from BDC at 2010-02-01 18:58:30.
               WARNING:  This latency is over the Tombstone Lifetime of 60 days!
         ......................... PDC  passed test Replications
      Starting test: Topology



      Starting test: FsmoCheck
         GC Name: \\bdc.myco.net
         Locator Flags: 0xe00001fc
         Warning: DcGetDcName(PDC_REQUIRED) call failed, error 1355
         A Primary Domain Controller could not be located.
         The server holding the PDC role is down.

 Starting test: Advertising
         Warning: DsGetDcName returned information for \\bdc.myco.net, when we were trying to reach pdc.
         Server is not responding or is not considered suitable.


  Starting test: RidManager
         * Available RID Pool for the Domain is 2100 to 1073741823
         * pdc.myco.net is the RID Master
         * DsBind with RID Master was successful
         * rIDAllocationPool is 1100 to 1599
         * rIDPreviousAllocationPool is 1100 to 1599
         * rIDNextRID: 1599
         * Warning :Next rid pool not allocated
         * Warning :There is less than 0% available RIDs in the current pool
         ......................... pdc passed test RidManager


      Starting test: Advertising
         The DC bdc is advertising itself as a DC and having a DS.
         The DC bdc is advertising as an LDAP server
         The DC bdc is advertising as having a writeable directory
         The DC bdc is advertising as a Key Distribution Center
         The DC bdc is advertising as a time server
         The DS bdc is advertising as a GC.
         ......................... bdc passed test Advertising


      Starting test: RidManager
         * Available RID Pool for the Domain is 2100 to 1073741823
         * bdc.myco.net is the RID Master
         * DsBind with RID Master was successful
         * rIDAllocationPool is 1600 to 2099
         * rIDPreviousAllocationPool is 1600 to 2099
         * rIDNextRID: 1633



      Starting test: systemlog
         * The System Event log test
         An Error Event occured.  EventID: 0x40000004
            Time Generated: 02/27/2011   21:46:45
            Event String: The kerberos client received a
KRB_AP_ERR_MODIFIED error from the server
host/pdc.myco.net.  The target name
used was . This indicates that the password used
to encrypt the kerberos service ticket is
different than that on the target server.
Commonly, this is due to identically named
machine accounts in the target realm
(myco.NET), and the client realm.   Please
contact your system administrator. 
When i run 


    Starting test: FsmoCheck
         GC Name: \\bdc.myco.net
         Locator Flags: 0xe00001fc
         Warning: DcGetDcName(PDC_REQUIRED) call failed, error 1355
         A Primary Domain Controller could not be located.
         The server holding the PDC role is down.
         Time Server Name: \\bdc.myco.net
         Locator Flags: 0xe00001fc
         Preferred Time Server Name: \\bdc.myco.net
         Locator Flags: 0xe00003e5
         KDC Name: \\bdc.myco.net
         Locator Flags: 0xe00001fc
         ......................... myco.net failed test FsmoCheck

Open in new window

So, my questions are:

1) Is the BDC in good shape.  Meaning, if i seize the FSMO roles on the BDC, will that fix my problems?  I hesitate to do this since the BDC has not been active for more than 60 days.  I'm willing to do that and rebuild the PDC but only if its not going to make more problems.

2) I think the Domain Admin password was changed between when the BDC was shutoff and the PDC crashed. Does that matter.

3) If both PDC and BDC are bad, and i need to rebuild from scratch, what should I do?  Goto each machine on the network and remove them from the domain?  That would be ugly, but i'm just wondering what my options are.

Looking forward to your response.








0
Comment
Question by:cdesk458
  • 13
  • 11
  • 2
  • +2
28 Comments
 
LVL 4

Expert Comment

by:szichen
Comment Utility
How can you try to replicate if the PDC is down?
0
 
LVL 70

Expert Comment

by:KCTS
Comment Utility
The whole purpose of having a seconf DC is so that it can be used in the event of a DC failure - by switching off the second DC tahter than confruntion the issues that were causing the problem you have compounded the issue. As the only copy of AD you have is beyond the tombstone its not going to be easy to recover it - try http://www.techrepublic.com/article/rescuing-a-failed-domain-controller-disaster-recovery-in-action/6082663
0
 
LVL 10

Expert Comment

by:Muzafar Momin
Comment Utility
removed BDC out of network, seize the role, connect one system to BDC with cross cable and check the status if all is fine then you can simple remove PDC from your network and add BDC to it and then go ahead and create new ADC

Follow the same steps for PDC and the you will be in better possion to take the decission
0
 
LVL 10

Expert Comment

by:Muzafar Momin
Comment Utility
followed to above steps you will need to do metadata cleanup for faulty DC
0
 

Author Comment

by:cdesk458
Comment Utility
Im new to this entire process so I just want to check that I have all of the facts correct.

When I run dcdiag /test:fsmocheck  I get

Starting test: FsmoCheck
         Warning: DcGetDcName(PDC_REQUIRED) call failed, error 1355
         A Primary Domain Controller could not be located.
         The server holding the PDC role is down.

Open in new window


When I run  dcdiag /test:ridmanager

      Starting test: RidManager
         * Available RID Pool for the Domain is 2100 to 1073741823
         * pdc.myco.net is the RID Master
         * DsBind with RID Master was successful
         * rIDAllocationPool is 1100 to 1599
         * rIDPreviousAllocationPool is 1100 to 1599
         * rIDNextRID: 1599
         * Warning :Next rid pool not allocated
         * Warning :There is less than 0% available RIDs in the current pool

Open in new window


From what I understand its a catch-22 situation:

PDC went down since its ran our of RID’s
Only way PDC can issue more RID’s is to replicate
BUT BDC is past its tombstone date and therefore PDC will not replicate with it.

Question: Are there any other things to check, or am i close to having the diagnosis part complete?




0
 
LVL 11

Accepted Solution

by:
Renato Montenegro Rustice earned 500 total points
Comment Utility
Let me see if I can help here.

Your "PDC" is experiencing those issues (like depletion of the RIDs) because it's not the holder of the fsmo roles and it can't find the current owner. If you have an "BDC" so old as 60 days, I dont think you should try to use this living-dead. Use it only if it's your only choice, but I think you can try other ways.

So, just turn off your "BDC", turn on your "PDC", seize the roles and let nature take place. And, and it's very important, keep your "BDC" turned off. Having two DCs competing for the roles can led your AD to serious data corruption. When you get your DC running fine again, build another brand new domain controller. Let your "BDC" rest in peace.

And, please, take a regular system state backup from both DCs. Monitor the backup routine regularly. Keep an eye at your Event Viewer. It's the best place to check out if AD is happy.

Follow this document in order to seize the roles:

Using Ntdsutil.exe to transfer or seize FSMO roles to a domain controller
http://support.microsoft.com/kb/255504

Look at the section "Seize FSMO roles".
0
 

Author Comment

by:cdesk458
Comment Utility
rmrustice:

Thanks for the assistance. I did what you suggested

1) turned off BDC
2) seized RID manager
3) seized PDC

Then ran this test

c:\>dcdiag /test:ridmanager

Domain Controller Diagnosis

Performing initial setup:
   Done gathering initial info.

Doing initial required tests

   Testing server: Default-First-Site-Name\PDC
      Starting test: Connectivity
         ......................... PDC passed test Connectivity

Doing primary tests

   Testing server: Default-First-Site-Name\PDC
      Starting test: RidManager
         ......................... PDC passed test RidManager

   Running partition tests on : ForestDnsZones

   Running partition tests on : DomainDnsZones

   Running partition tests on : Schema

   Running partition tests on : Configuration

   Running partition tests on : myco

   Running enterprise tests on : myco.net

C:\>dcdiag /test:fsmocheck

Domain Controller Diagnosis

Performing initial setup:
   Done gathering initial info.

Doing initial required tests

   Testing server: Default-First-Site-Name\PDC
      Starting test: Connectivity
         ......................... PDC passed test Connectivity

Doing primary tests

   Testing server: Default-First-Site-Name\PDC

   Running partition tests on : ForestDnsZones

   Running partition tests on : DomainDnsZones

   Running partition tests on : Schema

   Running partition tests on : Configuration

   Running partition tests on : myco

   Running enterprise tests on : myco.net
      Starting test: FsmoCheck
         Warning: DcGetDcName(GC_SERVER_REQUIRED) call failed, error 1355
         A Global Catalog Server could not be located - All GC's are down.
         Warning: DcGetDcName(PDC_REQUIRED) call failed, error 1355
         A Primary Domain Controller could not be located.
         The server holding the PDC role is down.
         Warning: DcGetDcName(TIME_SERVER) call failed, error 1355
         A Time Server could not be located.
         The server holding the PDC role is down.
         Warning: DcGetDcName(KDC_REQUIRED) call failed, error 1355
         A KDC could not be located - All the KDCs are down.
         ......................... myco.net failed test FsmoCheck

Open in new window


But now the other roles are down???  
0
 
LVL 11

Expert Comment

by:Renato Montenegro Rustice
Comment Utility
It seems like you are running without a Global Catalog (the GC). The GC is not one of the FSMO roles, but it plays an important part in the Active Directory. Possibly, the "BDC" was a GC too.

Follow these instructions to enable it on the surviving domain controller:

Enable or disable a global catalog
http://technet.microsoft.com/en-us/library/cc758330(WS.10).aspx

Restart the server to speed up the process. When you create your additional domain controller (the new one), set it as a Global Catalog too.

Make sure all FSMO roles are in place in the surviving domain controller. Run the following command and send the results back to us:

netdom query fsmo

If you notice any other role being hosted by the old server, seize it too. Then, send us the resulting netdom output. Send the updated dcdiag /v output too.
0
 
LVL 11

Expert Comment

by:Renato Montenegro Rustice
Comment Utility
Just to help you. You moved two roles. There are five of them. 3 to go:

Infrastructure Master
Schema Master
Domain Naming Master

Have a nice seizure :)
0
 

Author Comment

by:cdesk458
Comment Utility
Did not work.  Should i reboot????

C:\Documents and Settings\Administrator.PDC>ntdsutil
ntdsutil: roles
fsmo maintenance: connections
server connections: connect to server PDC
Binding to PDC ...
Connected to PDC using credentials of locally logged on user.
server connections: q
fsmo maintenance: ?

 ?                             - Show this help information
 Connections                   - Connect to a specific domain controller
 Help                          - Show this help information
 Quit                          - Return to the prior menu
 Seize domain naming master    - Overwrite domain role on connected server
 Seize infrastructure master   - Overwrite infrastructure role on connected serv
er
 Seize PDC                     - Overwrite PDC role on connected server
 Seize RID master              - Overwrite RID role on connected server
 Seize schema master           - Overwrite schema role on connected server
 Select operation target       - Select sites, servers, domains, roles and
                                 naming contexts
 Transfer domain naming master - Make connected server the domain naming master
 Transfer infrastructure master - Make connected server the infrastructure maste
r
 Transfer PDC                  - Make connected server the PDC
 Transfer RID master           - Make connected server the RID master
 Transfer schema master        - Make connected server the schema master


fsmo maintenance: seize domain naming master
Attempting safe transfer of domain naming FSMO before seizure.
ldap_modify_sW error 0x34(52 (Unavailable).
Ldap extended error message is 000020AF: SvcErr: DSID-03210333, problem 5002 (UN
AVAILABLE), data 8438

Win32 error returned is 0x20af(The requested FSMO operation failed. The current
FSMO holder could not be contacted.)
)
Depending on the error code this may indicate a connection,
ldap, or role transfer error.
Transfer of domain naming FSMO failed, proceeding with seizure ...
Server "PDC" knows about 5 roles
Schema - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN
=Sites,CN=Configuration,DC=myco,DC=net
Domain - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN
=Sites,CN=Configuration,DC=myco,DC=net
PDC - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN=Si
tes,CN=Configuration,DC=myco,DC=net
RID - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN=Si
tes,CN=Configuration,DC=myco,DC=net
Infrastructure - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site
-Name,CN=Sites,CN=Configuration,DC=myco,DC=net
fsmo maintenance: seize infrastructure master
Attempting safe transfer of infrastructure FSMO before seizure.
ldap_modify_sW error 0x34(52 (Unavailable).
Ldap extended error message is 000020AF: SvcErr: DSID-03210333, problem 5002 (UN
AVAILABLE), data 8438

Win32 error returned is 0x20af(The requested FSMO operation failed. The current
FSMO holder could not be contacted.)
)
Depending on the error code this may indicate a connection,
ldap, or role transfer error.
Transfer of infrastructure FSMO failed, proceeding with seizure ...
Server "PDC" knows about 5 roles
Schema - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN
=Sites,CN=Configuration,DC=myco,DC=net
Domain - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN
=Sites,CN=Configuration,DC=myco,DC=net
PDC - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN=Si
tes,CN=Configuration,DC=myco,DC=net
RID - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN=Si
tes,CN=Configuration,DC=myco,DC=net
Infrastructure - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site
-Name,CN=Sites,CN=Configuration,DC=myco,DC=net
fsmo maintenance: ?

 ?                             - Show this help information
 Connections                   - Connect to a specific domain controller
 Help                          - Show this help information
 Quit                          - Return to the prior menu
 Seize domain naming master    - Overwrite domain role on connected server
 Seize infrastructure master   - Overwrite infrastructure role on connected serv
er
 Seize PDC                     - Overwrite PDC role on connected server
 Seize RID master              - Overwrite RID role on connected server
 Seize schema master           - Overwrite schema role on connected server
 Select operation target       - Select sites, servers, domains, roles and
                                 naming contexts
 Transfer domain naming master - Make connected server the domain naming master
 Transfer infrastructure master - Make connected server the infrastructure maste
r
 Transfer PDC                  - Make connected server the PDC
 Transfer RID master           - Make connected server the RID master
 Transfer schema master        - Make connected server the schema master

fsmo maintenance: seize pdc
Attempting safe transfer of PDC FSMO before seizure.
FSMO transferred successfully - seizure not required.
Server "PDC" knows about 5 roles
Schema - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN
=Sites,CN=Configuration,DC=myco,DC=net
Domain - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN
=Sites,CN=Configuration,DC=myco,DC=net
PDC - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN=Si
tes,CN=Configuration,DC=myco,DC=net
RID - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN=Si
tes,CN=Configuration,DC=myco,DC=net
Infrastructure - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site
-Name,CN=Sites,CN=Configuration,DC=myco,DC=net
fsmo maintenance: seize rid master
The Selected Server is already the RID role owner
fsmo maintenance: seize schema master
Attempting safe transfer of schema FSMO before seizure.
ldap_modify_sW error 0x34(52 (Unavailable).
Ldap extended error message is 000020AF: SvcErr: DSID-03210333, problem 5002 (UN
AVAILABLE), data 8438

Win32 error returned is 0x20af(The requested FSMO operation failed. The current
FSMO holder could not be contacted.)
)
Depending on the error code this may indicate a connection,
ldap, or role transfer error.
Transfer of schema FSMO failed, proceeding with seizure ...
Server "PDC" knows about 5 roles
Schema - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN
=Sites,CN=Configuration,DC=myco,DC=net
Domain - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN
=Sites,CN=Configuration,DC=myco,DC=net
PDC - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN=Si
tes,CN=Configuration,DC=myco,DC=net
RID - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site-Name,CN=Si
tes,CN=Configuration,DC=myco,DC=net
Infrastructure - CN=NTDS Settings,CN=PDC,CN=Servers,CN=Default-First-Site
-Name,CN=Sites,CN=Configuration,DC=myco,DC=net
fsmo maintenance: q
ntdsutil: q
Disconnecting from PDC...

C:\Documents and Settings\Administrator.PDC>dcdiag

Domain Controller Diagnosis

Performing initial setup:
   Done gathering initial info.

Doing initial required tests

   Testing server: Default-First-Site-Name\PDC
      Starting test: Connectivity
         ......................... PDC passed test Connectivity

Doing primary tests

   Testing server: Default-First-Site-Name\PDC
      Starting test: Replications
         [Replications Check,PDC] A recent replication attempt failed:
            From BDC to PDC
            Naming Context: DC=ForestDnsZones,DC=myco,DC=net
            The replication generated an error (1256):
            The remote system is not available. For information about network tr
oubleshooting, see Windows Help.
            The failure occurred at 2011-03-01 10:59:59.
            The last success occurred at 2011-02-28 21:57:00.
            13 failures have occurred since the last success.
         [BDC] DsBindWithSpnEx() failed with error 1722,
         The RPC server is unavailable..
         [Replications Check,PDC] A recent replication attempt failed:
            From BDC to PDC
            Naming Context: DC=DomainDnsZones,DC=myco,DC=net
            The replication generated an error (1256):
            The remote system is not available. For information about network tr
oubleshooting, see Windows Help.
            The failure occurred at 2011-03-01 10:59:59.
            The last success occurred at 2010-02-01 18:53:03.
            9467 failures have occurred since the last success.
         [Replications Check,PDC] A recent replication attempt failed:
            From BDC to PDC
            Naming Context: CN=Schema,CN=Configuration,DC=myco,DC=net
            The replication generated an error (1722):
            The RPC server is unavailable.
            The failure occurred at 2011-03-01 11:00:41.
            The last success occurred at 2011-02-28 21:57:00.
            13 failures have occurred since the last success.
            The source remains down. Please check the machine.
         [Replications Check,PDC] A recent replication attempt failed:
            From BDC to PDC
            Naming Context: CN=Configuration,DC=myco,DC=net
            The replication generated an error (1722):
            The RPC server is unavailable.
            The failure occurred at 2011-03-01 11:00:20.
            The last success occurred at 2011-02-28 21:57:00.
            13 failures have occurred since the last success.
            The source remains down. Please check the machine.
         [Replications Check,PDC] A recent replication attempt failed:
            From BDC to PDC
            Naming Context: DC=myco,DC=net
            The replication generated an error (1722):
            The RPC server is unavailable.
            The failure occurred at 2011-03-01 10:59:59.
            The last success occurred at 2010-02-01 18:58:30.
            9545 failures have occurred since the last success.
            The source remains down. Please check the machine.
         REPLICATION-RECEIVED LATENCY WARNING
         PDC:  Current time is 2011-03-01 11:10:59.
            DC=ForestDnsZones,DC=myco,DC=net
               Last replication recieved from BDC at 2011-02-28 21:57:00.

            DC=DomainDnsZones,DC=myco,DC=net
               Last replication recieved from BDC at 2010-02-01 18:53:03.

               WARNING:  This latency is over the Tombstone Lifetime of 60 days!

            CN=Schema,CN=Configuration,DC=myco,DC=net
               Last replication recieved from BDC at 2011-02-28 21:57:00.

            CN=Configuration,DC=myco,DC=net
               Last replication recieved from BDC at 2011-02-28 21:57:00.

            DC=myco,DC=net
               Last replication recieved from BDC at 2010-02-01 18:58:30.

               WARNING:  This latency is over the Tombstone Lifetime of 60 days!

         ......................... PDC passed test Replications
      Starting test: NCSecDesc
         ......................... PDC passed test NCSecDesc
      Starting test: NetLogons
         ......................... PDC passed test NetLogons
      Starting test: Advertising
         Fatal Error:DsGetDcName (PDC) call failed, error 1355
         The Locator could not find the server.
         ......................... PDC failed test Advertising
      Starting test: KnowsOfRoleHolders
         ......................... PDC passed test KnowsOfRoleHolders
      Starting test: RidManager
         ......................... PDC passed test RidManager
      Starting test: MachineAccount
         ......................... PDC passed test MachineAccount
      Starting test: Services
         ......................... PDC passed test Services
      Starting test: ObjectsReplicated
         ......................... PDC passed test ObjectsReplicated
      Starting test: frssysvol
         ......................... PDC passed test frssysvol
      Starting test: frsevent
         There are warning or error events within the last 24 hours after the
         SYSVOL has been shared.  Failing SYSVOL replication problems may cause
         Group Policy problems.
         ......................... PDC failed test frsevent
      Starting test: kccevent
         ......................... PDC passed test kccevent
      Starting test: systemlog
         ......................... PDC passed test systemlog
      Starting test: VerifyReferences
         ......................... PDC passed test VerifyReferences

   Running partition tests on : ForestDnsZones
      Starting test: CrossRefValidation
         ......................... ForestDnsZones passed test CrossRefValidation

      Starting test: CheckSDRefDom
         ......................... ForestDnsZones passed test CheckSDRefDom

   Running partition tests on : DomainDnsZones
      Starting test: CrossRefValidation
         ......................... DomainDnsZones passed test CrossRefValidation

      Starting test: CheckSDRefDom
         ......................... DomainDnsZones passed test CheckSDRefDom

   Running partition tests on : Schema
      Starting test: CrossRefValidation
         ......................... Schema passed test CrossRefValidation
      Starting test: CheckSDRefDom
         ......................... Schema passed test CheckSDRefDom

   Running partition tests on : Configuration
      Starting test: CrossRefValidation
         ......................... Configuration passed test CrossRefValidation
      Starting test: CheckSDRefDom
         ......................... Configuration passed test CheckSDRefDom

   Running partition tests on : myco
      Starting test: CrossRefValidation
         ......................... myco passed test CrossRefValidation
      Starting test: CheckSDRefDom
         ......................... myco passed test CheckSDRefDom

   Running enterprise tests on : myco.net
      Starting test: Intersite
         ......................... myco.net passed test Intersite
      Starting test: FsmoCheck
         Warning: DcGetDcName(GC_SERVER_REQUIRED) call failed, error 1355
         A Global Catalog Server could not be located - All GC's are down.
         Warning: DcGetDcName(PDC_REQUIRED) call failed, error 1355
         A Primary Domain Controller could not be located.
         The server holding the PDC role is down.
         Warning: DcGetDcName(TIME_SERVER) call failed, error 1355
         A Time Server could not be located.
         The server holding the PDC role is down.
         Warning: DcGetDcName(KDC_REQUIRED) call failed, error 1355
         A KDC could not be located - All the KDCs are down.
         ......................... myco.net failed test FsmoCheck

Open in new window

C:\Documents and Settings\Administrator.PDC>
0
 
LVL 11

Expert Comment

by:Renato Montenegro Rustice
Comment Utility
It seems like the roles were seized successfuly. It first tries to transfer and, when it cant be done, it seizes.

Just the get a summary, run the netdom:

netdom query fsmo

I think there is still an error in your Global Catalog. Make sure you have enable it in Sites and Services. Then, restart the machine and run dcdiag again.

Enable or disable a global catalog
http://technet.microsoft.com/en-us/library/cc758330(WS.10).aspx
0
 
LVL 11

Expert Comment

by:Renato Montenegro Rustice
Comment Utility
Dont forget to paste the netdom output here.
0
 

Author Comment

by:cdesk458
Comment Utility
Global Cache was enabled, but description was blank so i added "global catalog"

I rebooted.

I ran Netdom query fsmo - reports no errors

There are errors in dcdiag

C:\Documents and Settings\Administrator.PDC>dcdiag

Domain Controller Diagnosis

Performing initial setup:
   Done gathering initial info.

Doing initial required tests

   Testing server: Default-First-Site-Name\PDC
      Starting test: Connectivity
         ......................... PDC passed test Connectivity

Doing primary tests

   Testing server: Default-First-Site-Name\PDC
      Starting test: Replications
         REPLICATION-RECEIVED LATENCY WARNING
         PDC:  Current time is 2011-03-01 12:44:24.
            DC=ForestDnsZones,DC=myco,DC=net
               Last replication recieved from BDC at 2011-02-28 21:57:00.

            DC=DomainDnsZones,DC=myco,DC=net
               Last replication recieved from BDC at 2010-02-01 18:53:03.

               WARNING:  This latency is over the Tombstone Lifetime of 60 days!

            CN=Schema,CN=Configuration,DC=myco,DC=net
               Last replication recieved from BDC at 2011-02-28 21:57:00.

            CN=Configuration,DC=myco,DC=net
               Last replication recieved from BDC at 2011-02-28 21:57:00.

            DC=myco,DC=net
               Last replication recieved from BDC at 2010-02-01 18:58:30.

               WARNING:  This latency is over the Tombstone Lifetime of 60 days!

         ......................... PDC passed test Replications
      Starting test: NCSecDesc
         ......................... PDC passed test NCSecDesc
      Starting test: NetLogons
         ......................... PDC passed test NetLogons
      Starting test: Advertising
         ......................... PDC passed test Advertising
      Starting test: KnowsOfRoleHolders
         ......................... PDC passed test KnowsOfRoleHolders
      Starting test: RidManager
         ......................... PDC passed test RidManager
      Starting test: MachineAccount
         ......................... PDC passed test MachineAccount
      Starting test: Services
         ......................... PDC passed test Services
      Starting test: ObjectsReplicated
         ......................... PDC passed test ObjectsReplicated
      Starting test: frssysvol
         ......................... PDC passed test frssysvol
      Starting test: frsevent
         ......................... PDC passed test frsevent
      Starting test: kccevent
         An Error Event occured.  EventID: 0xC0250827
            Time Generated: 03/01/2011   12:31:34
            (Event String could not be retrieved)
         An Warning Event occured.  EventID: 0x8000051C
            Time Generated: 03/01/2011   12:35:02
            Event String: The Knowledge Consistency Checker (KCC) has
         ......................... PDC failed test kccevent
      Starting test: systemlog
         An Error Event occured.  EventID: 0x00000423
            Time Generated: 03/01/2011   12:31:47
            Event String: The DHCP service failed to see a directory server
         An Error Event occured.  EventID: 0x00000423
            Time Generated: 03/01/2011   12:32:16
            Event String: The DHCP service failed to see a directory server
         An Error Event occured.  EventID: 0x00004105
            Time Generated: 03/01/2011   12:41:57
            Event String: The maximum account identifier allocated to this
         ......................... PDC failed test systemlog
      Starting test: VerifyReferences
         ......................... PDC passed test VerifyReferences

   Running partition tests on : ForestDnsZones
      Starting test: CrossRefValidation
         ......................... ForestDnsZones passed test CrossRefValidation

      Starting test: CheckSDRefDom
         ......................... ForestDnsZones passed test CheckSDRefDom

   Running partition tests on : DomainDnsZones
      Starting test: CrossRefValidation
         ......................... DomainDnsZones passed test CrossRefValidation

      Starting test: CheckSDRefDom
         ......................... DomainDnsZones passed test CheckSDRefDom

   Running partition tests on : Schema
      Starting test: CrossRefValidation
         ......................... Schema passed test CrossRefValidation
      Starting test: CheckSDRefDom
         ......................... Schema passed test CheckSDRefDom

   Running partition tests on : Configuration
      Starting test: CrossRefValidation
         ......................... Configuration passed test CrossRefValidation
      Starting test: CheckSDRefDom
         ......................... Configuration passed test CheckSDRefDom

   Running partition tests on : myco
      Starting test: CrossRefValidation
         ......................... myco passed test CrossRefValidation
      Starting test: CheckSDRefDom
         ......................... myco passed test CheckSDRefDom

   Running enterprise tests on : myco.net
      Starting test: Intersite
         ......................... myco.net passed test Intersite
      Starting test: FsmoCheck
         ......................... myco.net passed test FsmoCheck

Open in new window



Also, the bad BDC is still showing up in sites and services even thought its offline.
0
 

Author Comment

by:cdesk458
Comment Utility
I attempted to add a user one time, and got message about problem with resources, then i ran dcdiag /test:ridmanager and got no errors.  So i attempted to add the user again and it worked!!

0
Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 11

Expert Comment

by:Renato Montenegro Rustice
Comment Utility
I think is running fine now. The remaining errors are related to existing logs in Event Viewer. No big deal.

The best for the last. Lets now send BDC back to the dead.

Please, run a System State at this point.

Here is the document:

How to remove data in Active Directory after an unsuccessful domain controller demotion
http://support.microsoft.com/kb/216498/en-us

Follow the document carefully. Your goal is to wipe out every reference about the "BDC" from Active Directory. The main part is the cleaning of the data using ntdsutil, adsiedit and the DNS.

Feel free to ask us when you find any dificulty during the process.
0
 

Author Comment

by:cdesk458
Comment Utility
You are the MAN!! Saved our A@#$..  
0
 

Author Comment

by:cdesk458
Comment Utility
System State?   is this the system state from NTBackup.exe ??
0
 
LVL 11

Expert Comment

by:Renato Montenegro Rustice
Comment Utility
Oh thank you. It's really a pleasure to help. And, as a matter of fact, I love working with Active Directory and have seen many cases like that. I love those. I always learn something new.

I hope you manage to solve the problem. That's really important to me.
0
 
LVL 11

Expert Comment

by:Renato Montenegro Rustice
Comment Utility
Yes, just run ntbackup and then pick the system state.
0
 
LVL 11

Expert Comment

by:Renato Montenegro Rustice
Comment Utility
Is it a Windows 2003 box, right? If it's a Windows 2008, the procedure is quite different.
0
 

Author Comment

by:cdesk458
Comment Utility
Just for future readers of this:  One thing that is confusing about the process is that after you seize the roles you should wait several minutes before testing if its accomplished its goals.  

Examples: I seized the ridmanager role and then attempted to add a user, it failed.  Ran the ridmanager test again, it showed no errors. Then attempted to add a user again it worked.  So, i think i just needed to wait for few minutes for the system to do its magic.

Ok, system is backed-up now im going to read the Failed DC article.  Will keep you posted.
0
 
LVL 11

Expert Comment

by:Renato Montenegro Rustice
Comment Utility
I use and recommend the Coffee method when doing some stuff in AD. Some tasks really take time to complete. Besides replication, there are some hidden threads doing things all the time. So, go for a coffee and give it some 15 minutes.
0
 

Author Comment

by:cdesk458
Comment Utility
Metadata cleanup is complete.

Installed the new BDC, with a different name. Its visible in AD sites and services..

I did a dcdiag /fix to add all of the correct DNS for the new BDC

But, I still see the old  BDC listed in Sites and Services, I used the COFFEE method, but  it did not go away.  In fact it got replicated to the new BDC!!!  So, i deleted it in Sites and Service on PDC and New BDC. Is that enough???

I think i followed all of the steps in the article you sent.

I did not follow the adsledit steps since this was not the last DC in a child domain.. I only have one domain, so i ASSUME its not a child domain.

0
 
LVL 11

Expert Comment

by:Renato Montenegro Rustice
Comment Utility
No problem. Just removing it from there will suffice. Make sure it was removed from the Computers folder too.

The adsiedit part I was talking about was about the File Replication Service. Make sure there is no reference for the old DC. I think the NTDSUTIL will remove it properly from there.

I think your are done now!

Dont forget about the system state backup routine.
0
 
LVL 11

Expert Comment

by:Renato Montenegro Rustice
Comment Utility
Also, check the Domain Controllers OU. If the BDC is there, remove it.
0
 

Author Comment

by:cdesk458
Comment Utility
We're using Backup Exec for backup.. There is a Backup system state option for Domain Controlers.  Is this sufficient?  Are there any nuances with Backup Exec and AD??

I guess we should backup the system state every day, and backup the domain controller machine or VM image every week.

0
 

Author Closing Comment

by:cdesk458
Comment Utility
He did a great job..  A guru without a doubt!!
0
 
LVL 11

Expert Comment

by:Renato Montenegro Rustice
Comment Utility
I dont how the Backup Exec System State works. If you have a virtual environment, use it to test a restore isolated from the production network.

It was a pleasure to work with you. Glad to know you enjoy it too.

0

Featured Post

What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

Join & Write a Comment

Installing a printer using group policy preferences is not that hard let’s take a look at it. First lets open up your group policy console and edit the policy you want to add it to. I recommend creating a new policy for each printer makes it a l…
In this article, we will see the basic design consideration while designing a Multi-tenant web application in a simple manner. Though, many frameworks are available in the market to develop a multi - tenant application, but do they provide data, cod…
This tutorial will walk an individual through the steps necessary to join and promote the first Windows Server 2012 domain controller into an Active Directory environment running on Windows Server 2008. Determine the location of the FSMO roles by lo…
This tutorial will walk an individual through the process of configuring their Windows Server 2012 domain controller to synchronize its time with a trusted, external resource. Use Google, Bing, or other preferred search engine to locate trusted NTP …

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now