Authentication problems : NO_CLIENT_SITE when looking in the c:\windows\debug\netlogon.log

This is very common when the subnet that the clients are connecting from is not allocated to one of your sites in Sites and Services.

With an Enterprise Admin account, go into AD Sites and Services and add the subnet and subnet mask, and then assign that subnet to the same site where that client's domain controllers are located.

This helps to ensure that users can first be directed to domain controllers at the same site, instead of perhaps logging in to domain controllers across slower WAN links.

If you want to check all your netlogon files then you can use a few scripts that are out there from Microsoft

http://blogs.technet.com/b/askpfeplat/archive/2011/12/26/in-search-of-roaming-active-directory-clients-how-to-scriptomatically-identify-missing-active-directory-subnet-definitions.aspx

http://gallery.technet.microsoft.com/scriptcenter/Collect-Active-Directory-fa4dff2e

Thanks

Mike

Thanks. By the looks of the logs, it's going all the way back to May, so we had no issues then.

The site to site entries are in there and were from the beginning.

I'm also seeing the error attached. People still can't get authenticated! This is all after we had a network issue today and now this!
AD.jpg

SCHANNEL errors could be due to a failure to reach an authoritative domain controller or because of a failure of the workstation to authenticate to the domain.

Is this happening for everyone, or just a few users?

Are you seeing those errors on the domain controllers or on the client workstations?

This is happening across the board, including myself.

I'm seeing these errors on the DC's. We have multi-domain site.

I am on the new domain (uk.fc.local) and when I type the UNC path to the fileserver (Matches.com domain), I am prompted to enter user/password credentials. And it will only allow me access if I enter the domain admin credentials.

The erros I am seeing are on the UK.FC.LOCAL AD's (All on Windows 2008 R2). The Matches.com is the old AD (All on Windows 2000) and isn't showing any errors at all.

I can try restarting ALL AD servers across the estate? I'm at a loss, as I don't know what to do.

Are you seeing errors on your DCs (event logs, dcdiag, repdamin)

Is there a trust between the two domains?

Thanks

Mike

OK, so you have two domains... and they can talk to each other... and you have a trust established between the two domains... right?

You ruled out any network problems... and can ping between domain controllers for each domain?

What about firewall ports between them? Anything closed that shouldn't be?

I'm only seeing these errors on the DC's event viewer logs. There are trusts between the two domains, yes and there always have been.

The only thing that had happened in the day, was that one of our DC's sat on a datastore which was running out of space. We moved datastore content over to new datastore and that's all. Restarted that DC (which is a VM) and it was up and running fine.

Firewall ports are all open and it's an MPLS network. I can ping, I can log onto ALL of the DCs.

And now this:

Name resolution for the name _ldap._tcp.UK-CAM._sites.dc._msdcs.fchk.matches.com timed out after none of the configured DNS servers responded.

Can you post more of the event that shows the SCHANNEL source?
These events may be irrelevant, and may not be part of the problem.

Also, from one of your domain controllers on a site where your clients are having problems, run a DCDIAG and see what errors show up there.

OK, well if DNS servers are not responding then that certainly raises some concerns. Clients depend on DNS to find the SRV records for the domain controllers. If the clients can't find the DCs then they can't authenticate.

Go to a client workstation and run NSLOOKUP and see what server it connects to. Then do a simple query to that server to resolve one of your DCs and make sure it comes back with an answer.

This is the DCDiag from one of our AD servers on the UK.FC.loCAL site, which is where I'm based.


Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\Administrator>dcdiag

Directory Server Diagnosis

Performing initial setup:
   Trying to find home server...
   Home Server = FCCAMAD01
   * Identified AD Forest.
   Done gathering initial info.

Doing initial required tests

   Testing server: UK-Camden\FCCAMAD01
      Starting test: Connectivity
         ......................... FCCAMAD01 passed test Connectivity

Doing primary tests

   Testing server: UK-Camden\FCCAMAD01
      Starting test: Advertising
         ......................... FCCAMAD01 passed test Advertising
      Starting test: FrsEvent
         ......................... FCCAMAD01 passed test FrsEvent
      Starting test: DFSREvent
         ......................... FCCAMAD01 passed test DFSREvent
      Starting test: SysVolCheck
         ......................... FCCAMAD01 passed test SysVolCheck
      Starting test: KccEvent
         ......................... FCCAMAD01 passed test KccEvent
      Starting test: KnowsOfRoleHolders
         ......................... FCCAMAD01 passed test KnowsOfRoleHolders
      Starting test: MachineAccount
         * The current DC is not in the domain controller's OU
         ......................... FCCAMAD01 failed test MachineAccount
      Starting test: NCSecDesc
         ......................... FCCAMAD01 passed test NCSecDesc
      Starting test: NetLogons
         ......................... FCCAMAD01 passed test NetLogons
      Starting test: ObjectsReplicated
         ......................... FCCAMAD01 passed test ObjectsReplicated
      Starting test: Replications
         [Replications Check,FCCAMAD01] A recent replication attempt failed:
            From FCROOTAD02 to FCCAMAD01
            Naming Context: DC=ForestDnsZones,DC=fc,DC=local
            The replication generated an error (1256):
            The remote system is not available. For information about network tr
oubleshooting, see Windows Help.

            The failure occurred at 2013-07-09 15:59:32.
            The last success occurred at 2013-07-09 14:59:13.
            1 failures have occurred since the last success.
         [Replications Check,FCCAMAD01] A recent replication attempt failed:
            From FCROOTAD02 to FCCAMAD01
            Naming Context: CN=Schema,CN=Configuration,DC=fc,DC=local
            The replication generated an error (1722):
            The RPC server is unavailable.
            The failure occurred at 2013-07-09 15:59:33.
            The last success occurred at 2013-07-09 14:59:13.
            1 failures have occurred since the last success.
            The source FCROOTAD02 is responding now.
         [Replications Check,FCCAMAD01] A recent replication attempt failed:
            From FCROOTAD02 to FCCAMAD01
            Naming Context: CN=Configuration,DC=fc,DC=local
            The replication generated an error (1726):
            The remote procedure call failed.
            The failure occurred at 2013-07-09 15:59:32.
            The last success occurred at 2013-07-09 14:59:13.
            1 failures have occurred since the last success.
            The replication RPC call executed for too long at the server and
            was cancelled.
            Check load and resource usage on FCROOTAD02.
         [Replications Check,FCCAMAD01] A recent replication attempt failed:
            From FCROOTAD02 to FCCAMAD01
            Naming Context: DC=fc,DC=local
            The replication generated an error (1256):
            The remote system is not available. For information about network tr
oubleshooting, see Windows Help.

            The failure occurred at 2013-07-09 15:59:32.
            The last success occurred at 2013-07-09 15:47:39.
            1 failures have occurred since the last success.
         [Replications Check,FCCAMAD01] A recent replication attempt failed:
            From FCROOTAD02 to FCCAMAD01
            Naming Context: DC=us,DC=fc,DC=local
            The replication generated an error (1256):
            The remote system is not available. For information about network tr
oubleshooting, see Windows Help.

            The failure occurred at 2013-07-09 15:59:32.
            The last success occurred at 2013-07-09 14:59:13.
            1 failures have occurred since the last success.
         ......................... FCCAMAD01 failed test Replications
      Starting test: RidManager
         ......................... FCCAMAD01 passed test RidManager
      Starting test: Services
         ......................... FCCAMAD01 passed test Services
      Starting test: SystemLog
         A warning event occurred.  EventID: 0x00009016
            Time Generated: 07/09/2013   16:08:33
            Event String:
            No suitable default server credential exists on this system. This wi
ll prevent server applications that expect to make use of the system default cre
dentials from accepting SSL connections. An example of such an application is th
e directory server. Applications that manage their own credentials, such as the
internet information server, are not affected by this.
         A warning event occurred.  EventID: 0x00009016
            Time Generated: 07/09/2013   16:09:31
            Event String:
            No suitable default server credential exists on this system. This wi
ll prevent server applications that expect to make use of the system default cre
dentials from accepting SSL connections. An example of such an application is th
e directory server. Applications that manage their own credentials, such as the
internet information server, are not affected by this.
         A warning event occurred.  EventID: 0x00009016
            Time Generated: 07/09/2013   16:09:50
            Event String:
            No suitable default server credential exists on this system. This wi
ll prevent server applications that expect to make use of the system default cre
dentials from accepting SSL connections. An example of such an application is th
e directory server. Applications that manage their own credentials, such as the
internet information server, are not affected by this.
         A warning event occurred.  EventID: 0x00009016
            Time Generated: 07/09/2013   16:09:59
            Event String:
            No suitable default server credential exists on this system. This wi
ll prevent server applications that expect to make use of the system default cre
dentials from accepting SSL connections. An example of such an application is th
e directory server. Applications that manage their own credentials, such as the
internet information server, are not affected by this.
         A warning event occurred.  EventID: 0x00009016
            Time Generated: 07/09/2013   16:10:00
            Event String:
            No suitable default server credential exists on this system. This wi
ll prevent server applications that expect to make use of the system default cre
dentials from accepting SSL connections. An example of such an application is th
e directory server. Applications that manage their own credentials, such as the
internet information server, are not affected by this.
         A warning event occurred.  EventID: 0x00009016
            Time Generated: 07/09/2013   16:10:06
            Event String:
            No suitable default server credential exists on this system. This wi
ll prevent server applications that expect to make use of the system default cre
dentials from accepting SSL connections. An example of such an application is th
e directory server. Applications that manage their own credentials, such as the
internet information server, are not affected by this.
         A warning event occurred.  EventID: 0x00009016
            Time Generated: 07/09/2013   16:10:09
            Event String:
            No suitable default server credential exists on this system. This wi
ll prevent server applications that expect to make use of the system default cre
dentials from accepting SSL connections. An example of such an application is th
e directory server. Applications that manage their own credentials, such as the
internet information server, are not affected by this.
         A warning event occurred.  EventID: 0x00009016
            Time Generated: 07/09/2013   16:10:39
            Event String:
            No suitable default server credential exists on this system. This wi
ll prevent server applications that expect to make use of the system default cre
dentials from accepting SSL connections. An example of such an application is th
e directory server. Applications that manage their own credentials, such as the
internet information server, are not affected by this.
         A warning event occurred.  EventID: 0x00009016
            Time Generated: 07/09/2013   16:11:58
            Event String:
            No suitable default server credential exists on this system. This wi
ll prevent server applications that expect to make use of the system default cre
dentials from accepting SSL connections. An example of such an application is th
e directory server. Applications that manage their own credentials, such as the
internet information server, are not affected by this.
         A warning event occurred.  EventID: 0x00009016
            Time Generated: 07/09/2013   16:12:06
            Event String:
            No suitable default server credential exists on this system. This wi
ll prevent server applications that expect to make use of the system default cre
dentials from accepting SSL connections. An example of such an application is th
e directory server. Applications that manage their own credentials, such as the
internet information server, are not affected by this.
         A warning event occurred.  EventID: 0x00009016
            Time Generated: 07/09/2013   16:12:07
            Event String:
            No suitable default server credential exists on this system. This wi
ll prevent server applications that expect to make use of the system default cre
dentials from accepting SSL connections. An example of such an application is th
e directory server. Applications that manage their own credentials, such as the
internet information server, are not affected by this.
         A warning event occurred.  EventID: 0x00009016
            Time Generated: 07/09/2013   16:12:09
            Event String:
            No suitable default server credential exists on this system. This wi
ll prevent server applications that expect to make use of the system default cre
dentials from accepting SSL connections. An example of such an application is th
e directory server. Applications that manage their own credentials, such as the
internet information server, are not affected by this.
         A warning event occurred.  EventID: 0x00009016
            Time Generated: 07/09/2013   16:12:28
            Event String:
            No suitable default server credential exists on this system. This wi
ll prevent server applications that expect to make use of the system default cre
dentials from accepting SSL connections. An example of such an application is th
e directory server. Applications that manage their own credentials, such as the
internet information server, are not affected by this.
         A warning event occurred.  EventID: 0x00001695
            Time Generated: 07/09/2013   16:32:44
            Event String:
            Dynamic registration or deletion of one or more DNS records associat
ed with DNS domain 'uk.fc.local.' failed.  These records are used by other compu
ters to locate this server as a domain controller (if the specified domain is an
 Active Directory domain) or as an LDAP server (if the specified domain is an ap
plication partition).
         A warning event occurred.  EventID: 0x00001695
            Time Generated: 07/09/2013   16:32:44
            Event String:
            Dynamic registration or deletion of one or more DNS records associat
ed with DNS domain 'fc.local.' failed.  These records are used by other computer
s to locate this server as a domain controller (if the specified domain is an Ac
tive Directory domain) or as an LDAP server (if the specified domain is an appli
cation partition).
         A warning event occurred.  EventID: 0x00001695
            Time Generated: 07/09/2013   16:32:44
            Event String:
            Dynamic registration or deletion of one or more DNS records associat
ed with DNS domain 'DomainDnsZones.uk.fc.local.' failed.  These records are used
 by other computers to locate this server as a domain controller (if the specifi
ed domain is an Active Directory domain) or as an LDAP server (if the specified
domain is an application partition).
         A warning event occurred.  EventID: 0x000003F6
            Time Generated: 07/09/2013   16:34:07
            Event String:
            Name resolution for the name _ldap._tcp.UK-Camden._sites.dc._msdcs.f
chk.matches.com timed out after none of the configured DNS servers resp
onded.
         ......................... FCCAMAD01 passed test SystemLog
      Starting test: VerifyReferences
         Some objects relating to the DC FCCAMAD01 have problems:
            [1] Problem: Missing Expected Value
             Base Object:
            CN=FCCAMAD01,OU=Domain Controllers,OU=Servers,DC=uk,DC=fc,DC=local
             Base Object Description: "DC Account Object"
             Value Object Attribute Name: frsComputerReferenceBL
             Value Object Description: "SYSVOL FRS Member Object"
             Recommended Action: See Knowledge Base Article: Q312862

         ......................... FCCAMAD01 failed test VerifyReferences


   Running partition tests on : DomainDnsZones
      Starting test: CheckSDRefDom
         ......................... DomainDnsZones passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... DomainDnsZones passed test
         CrossRefValidation

   Running partition tests on : uk
      Starting test: CheckSDRefDom
         ......................... uk passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... uk passed test CrossRefValidation

   Running partition tests on : ForestDnsZones
      Starting test: CheckSDRefDom
         ......................... ForestDnsZones passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... ForestDnsZones passed test
         CrossRefValidation

   Running partition tests on : Schema
      Starting test: CheckSDRefDom
         ......................... Schema passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... Schema passed test CrossRefValidation

   Running partition tests on : Configuration
      Starting test: CheckSDRefDom
         ......................... Configuration passed test CheckSDRefDom
      Starting test: CrossRefValidation
         ......................... Configuration passed test CrossRefValidation

   Running enterprise tests on : fc.local
      Starting test: LocatorCheck
         ......................... fc.local passed test LocatorCheck
      Starting test: Intersite
         ......................... fc.local passed test Intersite

C:\Users\Administrator>

Well, you have some failed tests in there.

I'm assuming that FCCAMAD01 is the DC you ran the test on.
See the error "The current DC is not in the domain controller's OU"
Why?

Then you have other messages indicating failures to communicate between DCs:
From FCROOTAD02 to FCCAMAD01
Naming Context: CN=Schema,CN=Configuration,DC=fc,DC=local
The replication generated an error (1722):
The RPC server is unavailable.
Why is it unavailable?

You might want to reboot FCROOTAD02 as well.

Also check a DCDIAG from another DC and see what similar failures might come up.

Hang on... I thought you said your domain was UK.FC.Local?

Then what is US.FC.Local?

Put all your efforts into the DNS/Network issue.  I guess once that is cleared up the other items will start work.

Any changes made on the network in the last few days?

Thanks

Mike

My domain is UK.FC.LOCAL yes. The US.FC.LOCAL was created a while back to prepare the US to get migrated from their old systems onto the new which hasn't occured yet.

I'm only on the Uk.FC.LOCAL.

No, no changes to our network at all.

Are you guys seeing any errors that shouldn't exist?

Look at the DCDIAG results you sent. There is a reference there to a machine that is looking for US.FC.LOCAL. Should that be there?

Why is your FCCAMAD01 domain controller not in the Domain Controllers OU?
That is another message that showed up in the DCDIAG.

Also, see my earlier note about nslookup tests. It is most certainly looking like network connectivity issues, either because DNS is failing to resolve the queries, or because the RPC server is not responding for the destination DC when attempting replication.

So when one machine (either a DC or another client) is attempting to connect to another DC, it will query DNS for it first. If it finds the record in DNS, it makes a network connection to port 135 to request an RPC session. Then the destination replies back with a port number to connect on. Then the source machine will use that port number to connect to the machine to continue it's transaction.

There are loads of errors in your  DCDIAG that show RPC failures.

dhoffman_98 - didn't see the earlier post. Looking into this now.

Okay, well first things first. I restarted FCROOTAD02. When I try to log on, it won't log me on at all. It sits at the logon screen for around 15-20mins when you enter credentials.

The fact that it is showing RPC failures would seem to indicate that it is already past the DNS portion, and that it has found an answer in DNS. If it never found an answer in DNS, then it would not be even trying to make the RPC Call.

So FCROOTAD02 is a domain controller, and it's not even letting you log in with your domain admin credentials? Something is just not right about that. I can understand a different machine causing issues if it can't authenticate to a DC, but a DC to fail to authenticate?

Are any of your FSMO roles on that machine?

Right, so I found that we have another OU which has all of the DC's in it. Should I change that and put them all in the default 'Domain Controllers'?

Secondly, when I try to log on using remote desktop it takes forever and it just 'hangs'. Instead, I went onto my Vcenter and accessed the VM directly and even then it takes forever to just log in as an Admin, but nevertheless it logs on.

The thing is the FCROOTAD02 is the controller for the FC.LOCAL. So it's the main domain. The UK.FC.LOCAL would be the subdomain of that.

OK, well either way... it should not be taking that long to process a logon.

I don't suppose you have any available incidents for opening a case with Microsoft Premier Support?

I wish I could help, but I'm at a loss without being there and seeing things up close. (Not to mention that I actually have other things I'm supposed to be focusing on for work this morning).

I'll still try to help out as I can, but it is starting to sound like you are having some major connectivity issue between machines.

Are you sure your networking people didn't mess something up? They didn't start closing ports or anything?

After attempting to log onto it many times, I managed to select the 'Last Known Configuration'. I logged on and it said:

Windows has encountered a critical problem and will restart automatically in one minute. Please save your work Now!

I looked this up and it was a virus according to all of the forums.

I've tried using Malwarebytes and Microsoft removal tool but nothing. So I'm trying to see if there's something hidden in there.

OK, well if you truly have a virus, then of course do whatever it takes to detect and clean it.

Viruses and WORMs are nasty enough on workstations... get them on a domain controller... bad news.

Hopefully if you have other domain controllers that are replicated, and to which the virus did not affect your Active Directory, you might end up blowing away that machine and rebuilding it, then allowing replication to restore the AD data.

I've been told to just take this particular one off the network. And to rebuild one.

So the child domains 'UK.FC.LOCAL' have been okay. It's the 'FC.LOCAL' which has been the major problem and I'm assuming the UK.FC.LOCAL DC's are talking to the FC.LOCAL.

It should be as simple as creating a new VM and then just DCPROMO right? And creating a new AD controller in the FC.LOCAL domain?

Yes... as long as you have another DC in the FC.LOCAL domain, then you shoudln't have a problem.

Build a new machine and make it a member of FC.LOCAL. Make sure you can log into it properly (validating that it is resolving DNS and can find another DC from FC.LOCAL).

Then add your ADDS role, and then DCPROMO the machine and make it an additional DC.

Awesome. Thanks man.

And then what about in the Sites and Services? Will I have to add that in there? As the current one has been added in sites and services, seeing as there's another FC.LOCAL AD machine in our datacentre and they are supposed to talk/replicate to one another?

When they replicate, and assuming it's on the same subnet that is assigned to the existing site, it will automatically populate under the NTDS settings for that site after replication is completed.

But then if I don't want to demote the current FC.LOCAL AD as I want to diagnose the fault later and I just disable its network card, then if it's in the Sites and Services, will it still not cause a problem even if I create a new one? If it's in there, I'd assume that servers will still throw a fit as they can't 'replicate' to it even though they can to the new one and still cause problems? Or do you think that as long as they can talk to an FC.LOCAL AD server, that things will work?

When you create the new one, you will give it a new name. So instead of FCROOTAD02, you use a different name.

They will both be in there.

However... I would advise you against bringing the bad one back on line and connected to the network again. If anything, I'd suggest completely removing it from AD and just wiping it.

The ISTG will automatically build the site-to-site topology based on which machines are online. So if the new machine is the only one online, then the ISTG will build the new replication partnership based on that machine. Your old machine will no longer be a replication partner if it can't be contacted.

Okay, that's fine, I'm willing to do the wiping but not tonight. If I build a new one and just power off the old one, that's still alright?

Of course... you can wipe it whenever.

Do you think this FC.LOCAL would be causing the authentication issues? I mean, it is the child domain UK.FC.LOCAL which is where all of the user AD accounts are etc right?

Absoutely, i want to build a new one. I just wouldnt have thought with that server having replication issues, would have caused users authentication issues on Matches.com domain.

P.s thank you for all your responses. Seriously appreciate it

Glad to help. Follow up if you have more questions.