DNS resolution really slow on Win 2012 R2 with forwarders

Tim Phillips
Tim Phillips used Ask the Experts™
on
This is strange to me, but my DNS server on my network is periodically resolving public DNS really slow and sometimes timing out.  However, if I put Google's DNS servers as my primary DNS on my laptop, my laptop it is lightening fast.  What gives?  I tried clearing DNS cache on the server and rebooting it over the weekend, but it has not helped.

Extra info: my DNS server is on my Active Directory server and is Active Directory integrated.
Capture.PNG
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Distinguished Expert 2018

Commented:
To ensure an apples to apples comparison, you are doing your tests on the same network, using the same gateway device, and your DNS server(s) are also using the same Google servers as their only forwarders as your test laptop?
Have you tried removing your forwarder entries and just use the root hint servers. I have seen problems with forwarders and slow response in the past.

Scott Gorcester
Tim PhillipsWindows Systems Administrator

Author

Commented:
@Cliff I am on the same network using the same router with 8.8.8.8 as my DNS (which is being used as the forwarder on the DNS server)

@Scott I didn't realize that would work.  I thought you had to specify DNS servers to forward to.  So, if I check the "use root hints if forwarders not available" box and remove the forwarders then DNS for public sites will still work?  It may even fix the speed issue?
JavaScript Best Practices

Save hours in development time and avoid common mistakes by learning the best practices to use for JavaScript.

You do not require a forwarder as root hints can provide all of the necessary information. I would also look for bad forwarder addresses as this can slow things down considerably. I also look to see if my root hints are properly listed and resolving.

Scott
Tim PhillipsWindows Systems Administrator

Author

Commented:
I removed the forwarders and checked that I can still resolve names.  So far so good.  It was pretty fast too.  However, the problem is intermittent so I'll keep this open and see how tomorrow goes.  If we don't have DNS resolution problems tomorrow then I think we've found the culprit.
Tim PhillipsWindows Systems Administrator

Author

Commented:
Bummer, that didn't fix it.  I'm still getting slow DNS responses.  A page will timeout and then about 5 seconds later load just fine.  I was able to get a screen cap of the error before the page successfully loads.  See attached.
DNS-Slow.PNG
Lets try putting a forwarder back in, use 4.2.2.1 for now, also have a look at this post.

https://support.opendns.com/entries/21684834-getting-slow-response-to-my-dns-lookups-

Scott
Tim PhillipsWindows Systems Administrator

Author

Commented:
Since we are using AT&T Uverse, I updated our forwarders to be AT&T's DNS servers.  Maybe that'll help since they are on their network?

DNS1: 68.94.156.1
DNS2: 68.94.157.1

I noticed from the link you sent that some people fixed issues using ISP DNS instead of public.
Not a bad idea although I have not had this issue with using 8.8.8.8 or 4.2.2.2 in the past. The worst lookup issues I have found typically relate to an incorrect forwarder address. The incorrect forwarder would take forever to fail and then the lookups would succeed.

Scott
Tim PhillipsWindows Systems Administrator

Author

Commented:
Hmm, I still have the issue.  In addition to these DNS issues I also get slow bandwidth for even LAN connectivity.  I'm thinking that may be related.  I didn't think it was since DNS is so lean, but then again if there is latency it could be affecting the DNS packets as well (causing timeouts).
Check for IP address conflicts, does the server have a single nic with a single IP address? What happens if you run an internet speed test?

Scott
Tim PhillipsWindows Systems Administrator

Author

Commented:
The server is a VM and the host has several NICs setup so I don't think that's it.  I haven't seen any IP conflicts.  

However, I am suspect of two older switches installed here.  One is an old Cisco 1900 switch.  Quite possibly dying.  The other is a Zyxel switch... I just don't like Zyxel so that makes me a little biased there.
Tim PhillipsWindows Systems Administrator

Author

Commented:
Oh yeah, and speed test came back fine on the DNS server.
Older switches might be the culprit. Only concerned if the Domain controller has multiple nics or IP's.

Scott
Tim PhillipsWindows Systems Administrator

Author

Commented:
Not sure if this helps, but running this test gives intermittent fails.  Haven't tried replacing the switches, but I'm doubtful that is really the issue.
DNS-Test-Fails.PNG
Tim PhillipsWindows Systems Administrator

Author

Commented:
Found this in the DNS log.  Dunno if it means anything pertinent.
DNS-Event-Error.PNG
Tim PhillipsWindows Systems Administrator

Author

Commented:
Hmmm, I found old DNS server IPs on the network adapter on our DNS server.  What IPs should I have in there?  I have the loopback (127.0.0.1) and another server that we have an AD trust with.  Should I have others in there like 8.8.8.8?  What order should they be in?
The recursive test may fail if you are currently not using forwarders. Have you tried rebooting your router? Also is the firewall turned on on the DNS server?

Scott
YES OLD DNS server entries could absolutely cause this, NO do NOT put 8.8.8.8 on the DNS server IPV4 configuration. JUST PUT the DNS server itself there. I don't typically use 127.0.0.1 (loopback)

I think you found the issue

Scott
How many DNS servers do you have on the network

Scott
Tim PhillipsWindows Systems Administrator

Author

Commented:
Ok, I put in only two entries on the adapter: actual IP of the DNS server itself and below that the IP of the other AD server with which we have a trust (it made building the trust possible, don't know if it is still needed though).

This server is the only DNS server we have on the network.  (the AD server we have a trust with is on another network we connect to over site-to-site VPN)

This server does have its Windows Firewall enabled.

Anything else?
Okay that sounds fine, you can put the address of each domain controller as primary and the other domain controller as secondary, also make sure that both servers are replicating properly and maybe confirm that your records are updating properly on both Domain controllers.

This should resolve your issues

Scott
Tim PhillipsWindows Systems Administrator

Author

Commented:
Hmmm.... I removed all but the local IP of the domain controller in the DNS list on the only network adapter on the DC.  We are STILL getting DNS errors...

However, a pattern has emerged, it happens a lot more frequently at lunch and in the late afternoon...  Now, we have a 300Mbps symetrical pipe (Uverse) with QoS giving streaming data (ie YouTube) only 20Mbps.  So... what gives?  Why is DNS failing when we have a lot of users surfing?  Should I QoS DNS with a high priority (port 53)?  Do you think it is getting choked out?

We have about 50 users here.
Can you post the DNS errors you are seeing?

Scott
Tim PhillipsWindows Systems Administrator

Author

Commented:
I have posted the errors we are getting earlier in this thread, but I've attached a new one.
DNS-Event-Error2.PNG
Tim PhillipsWindows Systems Administrator

Author

Commented:
I found this in our logs when I do a debug log with our DNS server.  That IP is for a DNS server I removed (I thought gracefully).  When I look in the SOA on our actual DNS server the other one is NOT listed... Where is this getting referenced?
Tim PhillipsWindows Systems Administrator

Author

Commented:
Forgot to upload it
DNS-SOA-Error.PNG
You may have to look through your DNS server entries and manually clean up the stale records.

Scott
Tim PhillipsWindows Systems Administrator

Author

Commented:
I'm not sure what I'm looking for.  Any recommendations?  I've gone through DNS a million times looking for settings referencing old DNS servers.  What gets me is that the new DNS server should failover to other DNS servers if it gets a response, but it isn't doing that.
Tim PhillipsWindows Systems Administrator

Author

Commented:
Here are my "advanced" settings in DNS.  Is something miss-configured?
DNS-Advanced-Properties.PNG
Tim PhillipsWindows Systems Administrator

Author

Commented:
If the problem is that DNS fails to resolve and then a "few" seconds later resolves, could reducing the timeout time help?  See attached.
DNS-Forwarders.PNG
I don't see any problems here
Joseph HornseyPresident and Janitor

Commented:
I didn't see this mentioned yet (but I may have missed it), but is there anything on your network blocking UDP packets or UDP 53 in particular?

The DNS service attempts to resolve via UDP 53.  When that times out, it resolves using TCP 53.  That would account for the delay.

I had a very similar situation happen to one of my customers yesterday.  In this case, though, the network was being flooded with BOOTPC packets from a Windows 2012 R2 DHCP Server whose rogue detection was out of control.  Stopping that service and letting the network settle down solved the problem.

Go into task manager, expand "More details", go to the "Performance" tab and then click on "Open Resource Meter" at the bottom.  Go to the "Network" tab and look to see if any services are acting wonky.
Tim PhillipsWindows Systems Administrator

Author

Commented:
I worked with MS support and found out about an issue with MS DNS that makes the packets rather large.  By slimming down the DNS packet size we significantly decreased the errors we were seeing.  This leads me to think it has something to do with network traffic.  I also implemented QoS to prioritize DNS requests to help the issue.

Article: https://support.microsoft.com/en-us/kb/832223
Tim PhillipsWindows Systems Administrator

Author

Commented:
I did notice a lot of LSASS.exe traffic on our DC from a dev box.  I'll try to replace that dev box.  Also, I know its bad practice, but just how "bad" is it that our DC is also our print server?
Joseph HornseyPresident and Janitor

Commented:
Yeah... I think MS uses EDNS which is extended DNS.  I know that when we've had Cisco firewalls, we've had to disable the inspection on DNS because they don't like the larger packet size.

What firewall are you using?
Tim PhillipsWindows Systems Administrator

Author

Commented:
Palo Alto 500
Joseph HornseyPresident and Janitor

Commented:
Maybe disable:
- Passive DNS monitoring.
- DNS sinkholing

If, they're even enabled.
Windows Systems Administrator
Commented:
After a lot of troubleshooting and working with several vendors to diagnose the issue... it turns out our modem has a session limit on it (Max TCPIP sessions).  That's the issue.  We were maxing out on sessions and not able to make new connections to DNS servers for DNS requests.  ...I hate ISP's.  ...Uverse...
Tim PhillipsWindows Systems Administrator

Author

Commented:
I found out that it was the modem.  No one suggested that the modem was the issue.  Not really their fault, very strange issue.
Joseph HornseyPresident and Janitor

Commented:
Wow... a session limit on an ISP's modem?  I mean, I've seen that many times on firewalls that require per-user licenses... but an ISP?

Glad you figured it out!
Tim PhillipsWindows Systems Administrator

Author

Commented:
Yeah, watch out for AT&T Uverse (Business Account).  300mbps/300mbps but with a session limit...
Joseph HornseyPresident and Janitor

Commented:
Did they modify that?  I mean, that's not reasonable.
Tim PhillipsWindows Systems Administrator

Author

Commented:
They said it is a limitation of the modem and that it is the only modem they offer.  I'm not pleased with this answer and working with the sales rep to find a viable solution.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial