Unable To Validate Domain Trust

Well, the intraforest domain migration was an epic fail.  I showed up on site, RDP'd into the target DC, and ran the ADMT utility, selected the group migration wizard, selected the source domain, <any domain controller>, selected the target domain, and selected HDQ-DC, but received the following error message:

"The specified domain either does not exist or could not be contacted."

So, I confirmed that I could ping the source DC, BRDC1.brick.sca.local, and I also confirmed that I could RDP into it.  I checked that DC locally, and confirmed ADDS and DNS services were running.  So I thought, let me check the trusts.

From the HDQ-DC DC, I opened AD Domains and Trusts and right clicked on the parent domain and noticed the trust.  Remove was greyed out.  I selected properties and confirmed the trust type is set to Parent-Child.  I clicked on Validate, and recv'd the following error:

"Windows cannot find an AD Domain Controller for the Brick. sca.local domain.  Verify that an AD Domain Controller is available and try again."

From the Brick DC, I open AD Domains and Trusts, and right clicked on the parent domain and received the following error in the properties box:

"The AD Domain Services object could not be displayed.  Information for this object is not currently available possibly due to a network or AD Domain Controller failure."

I then, right clicked on the child domain and selected Validate.  I entered my Enterprise Admin credentials and recv'd the following error:

"The outgoing trust was successfully validated.  The Secure Channel (SC) reset on AD Domain Controller \\DC4.sca.local of domain sca.local to domain Brick.sca.local failed with error - There are currently no logon servers available to service the logon request."

I selected Yes, to reset the trust passwords and it errored out with "The trust cannot be repaired because there are currently no logon servers available to service the logon request."

Sorry for posting such a long thread, but I wanted to provide experts with the steps that I had taken and the errors that occurred at the exact points in which they occurred.  Please, in simplest of terms, provide me with steps on what I need to do to resolve this.  

What do I have to do in DNS?  What do I have to do in Domains and Trusts?  What do I have to do in Sites and Services?

Thanks.
--
Dan
SCAITAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Carol ChisholmCommented:
I would suggest putting fixed entries in the LMHOSTS files for the DCs on each side. This hard codes the IP addresses.
This is written for older versions but it still works.

http://support.microsoft.com/kb/314108
SCAITAuthor Commented:
Thanks.  I'll make an attempt on that tomorrow.  In the mantime, does anyone else have any other suggestions for me to consider?

--
Dan
SandeshdubeySenior Server EngineerCommented:
It seems that you have parent-child domain arhitecture.Can you confirm the same.By default parent-child domain has two way transitive trust.It seems to me that secure channel between parent-child domain is broken.

Verify the health of dc by dcdiag /q and repadmin /replsum.

First ensure correct dns setting on DCs as this:http://awinish.wordpress.com/2011/04/09/configuring-dns-in-child-domain/
http://msmvps.com/blogs/acefekay/archive/2010/10/01/dns-parent-child-dns-delegation-how-to-create-a-dns-delegation.aspx

Ensure that required port are open for AD communication:http://msmvps.com/blogs/acefekay/archive/2011/11/01/active-directory-firewall-ports-let-s-try-to-make-this-simple.aspx
 
To reset the secure channel see this:
Child domain no longer trust can't validate the trust
http://social.technet.microsoft.com/Forums/windowsserver/en-US/78621a02-4dd8-4d7b-b543-b27f6e4c4c57/child-domain-no-longer-trust-cant-validate-the-trust

In case if it is different forest domain then you can recreate the trust.create seconadry zone for dns resolution.

Checklist: Creating a forest trust
http://technet.microsoft.com/en-us/library/cc756852%28WS.10%29.aspx
Simplify Active Directory Administration

Administration of Active Directory does not have to be hard.  Too often what should be a simple task is made more difficult than it needs to be.The solution?  Hyena from SystemTools Software.  With ease-of-use as well as powerful importing and bulk updating capabilities.

SCAITAuthor Commented:
Ok thanks, I'll work on that today.  And you are correct, we do in fact have a Parent -Child domain architecture, whereas SCA.local is the parent and Brick.sca.local is the child.  I see the two way transitive trust, and as you mentioned the secure channel may be broken as I receive an error specifically related to the secure channel.
SCAITAuthor Commented:
Results from dcdiag /q:

An Warning Event occurred.  EventID: 0x80000785
            Time Generated: 08/06/2013   10:12:48
            Event String:
            The attempt to establish a replication link for the following writab
le directory partition failed.
         ......................... BRDC1 failed test KccEvent
         [HDQ-DC] DsBindWithSpnEx() failed with error 1722,
         The RPC server is unavailable..
         Warning: HDQ-DC is the Schema Owner, but is not responding to DS RPC
         Bind.
         Ldap search capabality attribute search failed on server HDQ-DC,
         return value = 81
         Warning: HDQ-DC is the Schema Owner, but is not responding to LDAP
         Bind.
         Warning: HDQ-DC is the Domain Owner, but is not responding to DS RPC
         Bind.
         Warning: HDQ-DC is the Domain Owner, but is not responding to LDAP
         Bind.
         ......................... BRDC1 failed test KnowsOfRoleHolders
         

[Replications Check,BRDC1] A recent replication attempt failed:
            From NYSYODC01 to BRDC1
            Naming Context: CN=Schema,CN=Configuration,DC=sca,DC=local
            The replication generated an error (1722):
            The RPC server is unavailable.
            The failure occurred at 2013-08-06 08:48:37.
            The last success occurred at 2011-10-15 05:52:20.
            5236 failures have occurred since the last success.
            [NYSYODC01] DsBindWithSpnEx() failed with error 1722,
            The RPC server is unavailable..
            The source remains down. Please check the machine.

[Replications Check,BRDC1] A recent replication attempt failed:
            From NJNEWDC01 to BRDC1
            Naming Context: CN=Schema,CN=Configuration,DC=sca,DC=local
            The replication generated an error (1722):
            The RPC server is unavailable.
            The failure occurred at 2013-08-06 10:18:19.
            The last success occurred at 2012-10-29 20:27:37.
            26297 failures have occurred since the last success.
            [NJNEWDC01] DsBindWithSpnEx() failed with error 1722,
            The RPC server is unavailable..
            The source remains down. Please check the machine.

[Replications Check,BRDC1] A recent replication attempt failed:
            From PANTODC01 to BRDC1
            Naming Context: CN=Schema,CN=Configuration,DC=sca,DC=local
            The replication generated an error (1723):
            The RPC server is too busy to complete this operation.
            The failure occurred at 2013-08-06 10:18:21.
            The last success occurred at 2013-07-25 14:23:52.
            1129 failures have occurred since the last success.

[Replications Check,BRDC1] A recent replication attempt failed:
            From NJCLIDC001 to BRDC1
            Naming Context: CN=Schema,CN=Configuration,DC=sca,DC=local
            The replication generated an error (1722):
            The RPC server is unavailable.
            The failure occurred at 2013-08-06 10:18:42.
            The last success occurred at 2012-03-13 19:24:41.
            48362 failures have occurred since the last success.
            [NJCLIDC001] DsBindWithSpnEx() failed with error 1722,
            The RPC server is unavailable..
            The source remains down. Please check the machine.

[Replications Check,BRDC1] A recent replication attempt failed:
            From NJEGGDC01 to BRDC1
            Naming Context: CN=Schema,CN=Configuration,DC=sca,DC=local
            The replication generated an error (1722):
            The RPC server is unavailable.
            The failure occurred at 2013-08-06 10:19:03.
            The last success occurred at 2012-10-29 20:27:37.
            26298 failures have occurred since the last success.
            [NJEGGDC01] DsBindWithSpnEx() failed with error 1722,
            The RPC server is unavailable..
            The source remains down. Please check the machine.

[Replications Check,BRDC1] A recent replication attempt failed:
            From DC4 to BRDC1
            Naming Context: CN=Schema,CN=Configuration,DC=sca,DC=local
            The replication generated an error (5):
            Access is denied.
            The failure occurred at 2013-08-06 10:19:04.
            The last success occurred at 2013-07-26 10:23:53.
            1050 failures have occurred since the last success.
            ......................... BRDC1 failed test SystemLog

C:\>
SCAITAuthor Commented:
The DNS test failed with several errors stating

"The AAAA record for this DC was not found"

"Error: Forwarders list has invalid forwarder x.x.x.x"

"Missing SRV record at DNS server x.x.x.x"
Carol ChisholmCommented:
Check for forwarders and conditional forwarders in your DNS servers on both domains. If you are asking an external DNS server for information about one of your domain it will not work.

are the x.x.x.x both the same server?
SCAITAuthor Commented:
The errors are indicating IP addresses of two of our servers at headquarters, the first is our PDC (HDQ-DC), and the second is another DC (DC4).  

There are Name Server entries for both of these servers on both ends under Forward Lookup Zones.  I'm unable to add a Conditional Forwarder becasue it already exists in the Forward Lookup Zones.
Carol ChisholmCommented:
And can you ping AND nslookup all the DCs by name and by IP address from both ends?
Just checking that the basic networking is working.



Did something change back in March?

The last success occurred at 2012-03-13 19:24:41.
48362 failures have occurred since the last success.
SCAITAuthor Commented:
Wow, I'm not sure, I wasn't here in March, but my guess would be the VPN tunnel had broken then.  When I started here the VPN tunnel between these two sites was already down/failed, and I had to recreate the tunnel, and now that the tunnel is online, I'm trying to get this DNS issue resolved so that I can migrate the child domain.

Pings and nslookup succeeds on both ends.

I used the following syntax:

nslookup
set type=all
_ldap._tcp.dc._msdcs.sca.local
SCAITAuthor Commented:
Sorry, but many of these links, contain additional links to other posts which are either specific for Windows 2000, or are just too cumbersome and complex with involved procedures.  Some of you have provided simple steps for manually removing the trust objects; thank you.  Can I please also receive simple steps for re-creating the trust?  

PDC in HDQ is running Windows Server 2008 Ent R2 and the DC at Brick is running Windows Server 2008 Std.

Thanks.
Carol ChisholmCommented:
Well it looks as if the trust failed in March from your log.
However as you say that is past history.

The trust is set in Active Directory Domains and Trusts, but until you can resolve the DNS / NETBIOS problem  you won't be able to recreate the trust. The technology for trusts has not changed much in ages, you need a really basic NETBIOS type connection.

You have to be able to resolve by both ping and nslookup the COMPUTERNAME of the DC (not the FQDN or any other record), Your DC also has to know that this machine is a DC for the domain you want to contact. Any locally stored AD information will be out of date if replication has failed since March.

You then have to have a route to and from that IP address.

You will have to brute force the connection:
That is why I suggested the LMHOSTS file, it is a brute force method using a text file. It will override any errors in your DNS and AD and will ensure that each DC can find the DC of the other domain.

"Use the following steps to create a correctly formatted LMHOSTS file: 1.Using a text editor, for example, Notepad.exe or Edit.com, create a file called LMHOSTS, and then save it in the following folder:

%SystemRoot%\System32\Drivers\Etc

Note that the file name is LMHOSTS, with no extension. If you are using Notepad.exe, Notepad.exe may automatically append .txt. If Notepad.exe does this, rename the file, using no extension, at a command prompt.

Add the following entries to the LMHOSTS file: It is important to get the exact case and number of spaces.

x.x.x.x   HQ-PDC #PRE #DOM:HQDOMAIN
x.x.x.x   "HQDOMAIN_NAME    \0x1b"   #PRE
Note also that DOMAIN_NAME in this entry is case-sensitive. Make sure to use all capital letters. Replace 10.0.0.1 with the IP address of your primary domain controller (PDC), replace PDCName with the NetBIOS name of your PDC, and replace DOMAIN_NAME with the name of your Windows NT-based domain.

Then when you resolve computer NetBIOS names to IPs you can think about going into AD Domains and Trusts to delete then recreate...

Historically trusts existed before domain controllers were always DNS servers, so to fix them you always have to go back in time.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
SCAITAuthor Commented:
Ok thanks, I'm working on the LMHOSTS file now, but wanted to post the results of the nslookup from each side...the results are interesting...

The Brick DC is still resolving 10.x.0.x as the old, failed DC (DC2) from HQ, while it's still resolving the new DC.  So I have to figure a way to stop BRDC1 from resolving dc2, and only resolving HDQ-DC.

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\Administrator.SCA>nslookup 10.40.0.5
Server:  hdq-dc.sca.local
Address:  10.x.0.x

Name:    brdc1.brick.sca.local
Address:  10.40.0.5


C:\Users\Administrator.SCA>nslookup brdc1.brick.sca.local
Server:  hdq-dc.sca.local
Address:  10.x.0.x

Name:    brdc1.brick.sca.local
Address:  10.40.0.5


C:\Users\Administrator.SCA>


===================================


From BRDC1

Microsoft Windows [Version 6.0.6002]
Copyright (c) 2006 Microsoft Corporation.  All rights reserved.

C:\Users\Administrator>nslookup 10.x.0.x
Server:  brdc1.brick.sca.local
Address:  10.40.0.5

Name:    dc2.sca.local
Address:  10.x.0.x


C:\Users\Administrator>nslookup hdq-dc.sca.local
Server:  brdc1.brick.sca.local
Address:  10.40.0.5

Name:    hdq-dc.sca.local
Address:  10.x.0.x


C:\Users\Administrator>nslookup dc2.sca.local
Server:  brdc1.brick.sca.local
Address:  10.40.0.5

Name:    dc2.sca.local
Address:  10.x.0.x


C:\Users\Administrator>
Carol ChisholmCommented:
I thought you had some basic DNS issues. Glad you have found them. Now you can start to fix them.

If you have a failed DC it is quite a pain to clear up. You have to trawl through the DNS entries, and possible clear the metadata. You will have to do that at some stage.

http://technet.microsoft.com/en-us/library/cc816907(v=ws.10).aspx

http://support.microsoft.com/kb/216498
SCAITAuthor Commented:
LMHOSTS file created and entries made.

Also, metadata cleanup was done already (DC2 removed):

ntdsutil: metadata cleanup
metadata cleanup: connections
Connected to localhost using credentials of locally logged on user.
server connections: connect to server dc2
Disconnecting from localhost...
Binding to dc2 ...
DsBindWithSpnExW error 0x6ba(The RPC server is unavailable.)
server connections: quit
metadata cleanup: select operation target
select operation target: list domains
Not connected to a server - use "Connections"
select operation target: quit
metadata cleanup: connections
server connections: connect to server hdq-dc
Binding to hdq-dc ...
Connected to hdq-dc using credentials of locally logged on user.
server connections: quit
metadata cleanup: select operation target
select operation target: list domains
Found 3 domain(s)
0 - DC=sca,DC=local
1 - DC=philly,DC=sca,DC=local
2 - DC=brick,DC=sca,DC=local
select operation target: select domain 0
No current site
Domain - DC=sca,DC=local
No current server
No current Naming Context
select operation target: list sites
Found 9 site(s)
0 - CN=Berlin,CN=Sites,CN=Configuration,DC=sca,DC=local
1 - CN=Philadelphia,CN=Sites,CN=Configuration,DC=sca,DC=local
2 - CN=Brick,CN=Sites,CN=Configuration,DC=sca,DC=local
3 - CN=Norristown,CN=Sites,CN=Configuration,DC=sca,DC=local
4 - CN=Newark,CN=Sites,CN=Configuration,DC=sca,DC=local
5 - CN=EggHarbor,CN=Sites,CN=Configuration,DC=sca,DC=local
6 - CN=CALI,CN=Sites,CN=Configuration,DC=sca,DC=local
7 - CN=Clifton,CN=Sites,CN=Configuration,DC=sca,DC=local
8 - CN=CANY,CN=Sites,CN=Configuration,DC=sca,DC=local
select operation target: select site 0
Site - CN=Berlin,CN=Sites,CN=Configuration,DC=sca,DC=local
Domain - DC=sca,DC=local
No current server
No current Naming Context
select operation target: list servers in site
Found 3 server(s)
0 - CN=DC001,CN=Servers,CN=Berlin,CN=Sites,CN=Configuration,DC=sca,DC=local
1 - CN=DC4,CN=Servers,CN=Berlin,CN=Sites,CN=Configuration,DC=sca,DC=local
2 - CN=HDQ-DC,CN=Servers,CN=Berlin,CN=Sites,CN=Configuration,DC=sca,DC=local
select operation target:
Carol ChisholmCommented:
so are ping and NSLOOKUP better?

Then you want to tracert to the other DC.
Carol ChisholmCommented:
And when that is all working you can go into AD domains and trusts and remove or add trusts...
SCAITAuthor Commented:
When I ping 10.x.0.x I get the successful replies.

When I ping hdq-dc I get the successful replies.

When I ping the old DC domain name, I still get the succesful replies.  I just deleted all traces of DC2 from the forward and reverse lookup zones.  Why is it still associating the old domain name with 10.x.10.x and how can I remove that?
SCAITAuthor Commented:
I'm now going through every DNS/forward/reverse lookup entry in the child domain.
Carol ChisholmCommented:
Somewhere you have an old entry:

Look in LMHOSTS, HOSTS, DNS, AD sites and services, AD itself.
DNS forwarders (conditional or otherwise)
DHCP scope? (really way out here)
FSMO role?

sysvol/netlogon shares ?

logon scripts?

Group policy objects?

Sorry to be so expansive but you really have a complicated situation.
SCAITAuthor Commented:
Good points.  I've checked all of DNS on both DC's.  I've already migrated the FSMO roles.  I checked AD Sites and Services, Domains and Trusts, and Users and Computers.

What's wierd is that in LDAP.exe, that has seemed to have been replicated as I do not see the DN of the old DC anywhere.  

Check out the ldap entries:

-- CN=Servers,CN=Berlin,CN=Sites,CN=Configuration,DC=sca,DC=local
   -- CN=HDQ-DC,CN=Servers,CN=Berlin,CN=Sites,CN=Configuration,DC=sca,DC=local


I'm going to continue to dig into this.  I'll check Group Policy next.
SCAITAuthor Commented:
FYI...I tried resetting the two way trust via NETDOM and this is what NETDOM output:

C:\Users\Administrator.SCA>netdom trust sca.local /Domain:brick.sca.local /UserD
:Brick.sca.local\administrator /PasswordD:* /UserO:sca.local\administrator /Pass
wordO:* /Reset /TwoWay
Type the password associated with the domain user:

Type the password associated with the object user:

The specified domain either does not exist or could not be contacted.

The command failed to complete successfully.


C:\Users\Administrator.SCA>
Carol ChisholmCommented:
I would still try without the FQDNs:

http://technet.microsoft.com/en-us/library/cc835085.aspx
says:

Also look at the example passwords:

netdom trust /d:marketing.contoso.com engineering.contoso.com /add /twoway /Uo:admin@engineering.contoso.com /Ud:admin@marketing.contoso.com

Just out of curiosity can you join a workstation to the remote domain?
SCAITAuthor Commented:
Today I ran dsgetdc from both ends, this is what I got:

From Brick.sca.local
================
C:\Users\Administrator>nltest /dsgetdc:brick.sca.local /force
           DC: \\BRDC1.brick.sca.local
      Address: \\10.40.0.5
     Dom Guid: 39dedad1-c2ae-446a-9b7c-8f0d3844c53f
     Dom Name: brick.sca.local
  Forest Name: sca.local
 Dc Site Name: BRICK
Our Site Name: BRICK
        Flags: PDC GC DS LDAP KDC TIMESERV WRITABLE DNS_DC DNS_DOMAIN DNS_FOREST
 CLOSE_SITE FULL_SECRET
The command completed successfully

C:\Users\Administrator>nltest /dsgetdc:sca.local /force
           DC: \\DC4.sca.local
      Address: \\10.10.0.4
     Dom Guid: a39ff148-43a8-4a2b-b842-7ae1078a6029
     Dom Name: sca.local
  Forest Name: sca.local
 Dc Site Name: Berlin
Our Site Name: Brick
        Flags: GC DS LDAP KDC TIMESERV WRITABLE DNS_DC DNS_DOMAIN DNS_FOREST CLO
SE_SITE FULL_SECRET
The command completed successfully

C:\Users\Administrator>



From HDQ
================
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\Administrator.SCA>nltest /dsgetdc:brick.sca.local /force
Getting DC name failed: Status = 1355 0x54b ERROR_NO_SUCH_DOMAIN

C:\Users\Administrator.SCA>nltest /dsgetdc:sca.local /force
           DC: \\HDQ-DC.sca.local
      Address: \\10.10.0.6
     Dom Guid: a39ff148-43a8-4a2b-b842-7ae1078a6029
     Dom Name: sca.local
  Forest Name: sca.local
 Dc Site Name: Berlin
Our Site Name: Berlin
        Flags: PDC GC DS LDAP KDC TIMESERV GTIMESERV WRITABLE DNS_DC DNS_DOMAIN
DNS_FOREST CLOSE_SITE FULL_SECRET WS
The command completed successfully

C:\Users\Administrator.SCA>

====================

So, the Brick domain sees DC4 as the PDC, not HDQ-DC.  How can I set it so it sees HDQ-DC as the PDC?

--
Dan
SCAITAuthor Commented:
Today, I created new ldap and kerberos SRV (service locator) records in DNS on BRDC1 pointing to HDQ-DC.  I will register DNS cache and will attemp to replicate and will check the resutls.
Carol ChisholmCommented:
Well done and bonne courage as they say. Looks as if you are getting closer. You have a much  better view of the detail than I do.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Server 2008

From novice to tech pro — start learning today.