Link to home
Start Free TrialLog in
Avatar of mabutterfield
mabutterfieldFlag for United States of America

asked on

DNS listening on too many ports

Running windows 2003 server, active directory controller and DNS server.  While troubleshooting another problem, I noticed the server was listening on ALOT of ports.  Upon further diagonistics using TCPView and netstat, I determined that the Microsoft DNS Server is listening on about 2500 random UDP ports.  I've run a virus scan, nothing came up.  The server is patched and up to date.

I've attached the output of netstat -an - p udp with the dns server running, stopped, then started again.
dnsservicerunning.txt
dnsservice-stopped.txt
dnsservic-restarted.txt
Avatar of Chris Dent
Chris Dent
Flag of United Kingdom of Great Britain and Northern Ireland image


It could potentially be using those to listen for responses to issued queries. Is the DNS Server a busy one?

I'd put a packet sniffer on there to see what all of those are actually doing. WireShark is good for that (and free):

http://www.wireshark.org/

Chris
Avatar of mabutterfield

ASKER

I'll load it up this afternoon and see what it does and get back to you.
Why is it listening on 0.0.0.0 and the 127 loopback address?

Can you provide an IPconfig?

Is this a multihomed server?
Chris,  I'm installing wireshark now.  I couldn't do it yesterday b/c it was in production.  I'm installing it now and will post shortly

ChiefIT,

here's the output of ipconfig /all.  I also noticed and error and a warning in the DNS Server error log.  I'm uploading them also.

The server has multiple NICs in it, but only is being used.


dns-log.txt
dns-log-warn.txt
ipconfig.txt
i've done a wireshark dump and exported the packet summaries, and sanitized.  

it looks to me like there's alot of return traffic coming from other IP:53 to localIP: which would explain a connection, if the server was issuing the query.  However, the server should be doing the replies, and since UDP is stateless, it shouldn't be listening on all those ports.

If you want me to export more packet detail, or a longer dump let me know.  (I filtered the capture for UDP only, which is why you don't see any TCP traffic)



dump-summary.txt
Shift primary/secondary DNS to use the local DNS server as primary server and the other server as secondary server.
Change the DNS-zones to be stored in AD to get better replication.
Only 1 forward and 1 reverse lookup zone loaded.  both are already stored in active directory.  

DNS points first to 10.x.x.48, then to this machine 10.x.x.6.  Both are AD machines, and run DNS.  I cannot terminal into .48 to see if it's doing the same thing.  

You have unnecessary DNS-queries from 10.x.x.6 to 10.x.x.48 because of using the remote address as primary server.
On 10.x.x.6, use itself as its primary DNS server and 10.x.x.48 as its secondary DNS server.
As henjoh stated, "change the servers around" 10.x.x.6 should be the primary DNS server. Otherwise, you are going to 10.x.x.48 for this server's DNS queries.
______________________________________________________________
Event ID:      4015

4015 errors is usually a result of this server not having its host A record registered in DNS. Since you are going to 10.x.x.48 for DNS as the PREFERED DNS server, this means the Host A record for your 10.X.X.6 server may not exist on the 10.x.x.48 server.

After changing the preferred DNS server to be itself, you might want to register the Host A record.
To register the Host A on both servers, go to the command prompt and type:
IPconfig /flushDNS
IPconfig /registerDNS
Net stop netlogon
Net start netlogon

Then force replicate from that server to the other server:

http://www.windowsitpro.com/article/articleid/13396/how-do-i-force-replication-between-two-domain-controllers-in-a-site.html

_____________________________________________________________________________

Now that we worked those two out, there is still a problem with it trying to do a DNS query on 0.0.0.0 IP address and also the 127 loopback address.

In my opinion, this might be your DHCP server not knowing who your DNS servers are. Even if DHCP is the same server, you might want to configure your DHCP server options to define the default gateway and DNS servers. Doing so would define the path to those servers.

To do this, go into MMC consle, or the DHCP snapin, and expand the snapin to where you see the OPTIONS folder. Enter the options folder and configure your DNS server. Much like prefered DNS servers in the NIC, you also want your server to be the first server for DHCP. Example

DHCP options for DNS servers:
If your server's IP is 10.48.4.6 and the other server is 10.48.4.48, you want these two servers in this order of precedence for DNS.

10.48.4.6
10.48.4.48

I have seen DHCP tell the client to query DNS to see if the IP is in use. This is a part of the DHCPACK package.
DHCP is using ARP/ping for conflict detection, not DNS. The dynamic registration of DNS-records is done through DHCP client service.

Even if configuring the DNS to use a single IP (dnsmgmt.msc->server properties->Interfaces-tab), it will when checking netdiag bind port 53 to all IPs including 127.0.0.1
The amount of reserved ports when starting DNS service looks like "as design" to handle performance. The benchmark described at the link below had 1300 dynamic updates/second.
If checking other ports with netstat, you will see that also they are assigned to 0.0.0.0 when the port is in LISTENING state.

http://technet2.microsoft.com/windowsserver/en/library/5e81fbe2-764a-47c4-bc7a-0da6f447897b1033.mspx?mfr=true

> However, the server should be doing the replies, and since UDP is stateless, it shouldn't be
> listening on all those ports.

You mean your server doesn't query public name servers?

If your server makes a query against a remote server (as a response to a client requesting recursion) it will open up a UDP Port to listen on matching the source port of the original request.

As you point out, UDP is stateless, therefore opening a port to listen is the only way it's going to see and match up the response. I've yet to find documentation on how long it keeps them open, or how many it just arbitrarily opens, however the behaviour is not abnormal.

Chris
ChiefIT,

I changed the primary lookup server to be the local ip address 10.17.68.6.  I also did nslookup, pointing to both servers, and attempted to resolve the name.  It resolved fine on the first try.  I went ahead and did the flushdns/registerdns and stop/start netlogon anyway.  

the DHCP server is 10.17.68.48 (the other dns/ad server)  it gives out the dns of .48 as primary, and .6 as secondary, and sets up the local domain name correctly.  the dhcp server is set to 'enable dynamic updates' and ' always update dns A and ptr records'

henjoh09 and Chris-Dent,

I agree about the fact that it would have to open a port to listen for a reply when it queries another server, because UDP is stateless.  However, I would think (what's your opinion) that it would listen on an IP Address, not 0.0.0.0 if it was waiting for a reply.  

Also, this is a pretty small network, about 50 users.  upon starting the DNS server, about 2000-2500 ports are opened up within the time it takes me to type 'netstat -an -p udp'.  I don't think it's making that many queries, and why would it open ports to listen for replies, if it didn't make a request?  To prove this, I stopped dns server, started wireshark to listen for udp packets, then started dns server again.  it didn't make a bunch of queries.  (i'll post the results of that wireshark momentarily)

I've checked several other dns servers that I have in similar situations (windows 2003 AD/dns).  none of them are listening on anywhere near that many ports.

additionally, if you're talking about inbound ports, which is what the DNS server is made to do, it only needs to listen on port 53, and will reply from port 53 to the source port of what was requested.  The need for more performance would not require additional ports, it would just open new connections(less) on that same port.


here is the wireshark dump while i was starting dns, as promised.
dump-starting-dns.txt
In the event log, if I click for more information, this is what I get as a result.

I'm going to see what I can troubleshoot with ldap to see if there's any problems.

will also run netdiag /debug
error-log-ms-response.txt

> However, I would think (what's your opinion) that it would listen on an IP Address,
> not 0.0.0.0 if it was waiting for a reply

I haven't seen a case where it listens on anything but 0.0.0.0 for those high-numbered UDP Ports. This holds true even if you restrict the interfaces DNS listens on under the server properties.

But you're right, it's too many unless it's an exceptionally high-load system.

Out of interest, one of my DNS servers also suffers from the problem, sitting on a large number of high-numbered UDP Ports.

This only exhibits on version 5.2.3790.4318 of dns.exe. I have another server running 5.2.3790.4171 which doesn't display the issue. Patching it up now to see if it starts ;)

Chris
ASKER CERTIFIED SOLUTION
Avatar of Chris Dent
Chris Dent
Flag of United Kingdom of Great Britain and Northern Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
The version of dns.exe on this machine is 5.2.3790.3161 (Win2003 SP1)

I'm checking other machines that I can get my hands on.

2 machines at:
5.2.3790.4171  (Win 2003 R2 SP2) seems to be working fine.

1 machine at
5.2.3790.3161 (Win 2003 SP1) has same issue

Mine are all 2003 R2 SP2, the only difference is the version of dns.exe. The latest version showing the port usage.

Unfortunately, I don't have any version earlier than 4171. I suspect it's not actually a problem, but it is a bit of an oddity. I suspect a fair bit more reading is in order to fully understand why it does it.

Chris
I have  had the issue of the new DNS functionality blocking legitimate programs from working.  Port 5160, for example, is used for a licensing server in my network.  My research uncovered this page:

http://support.microsoft.com/kb/812873

That page describes how to reserve some ports from being used by process that open up random ports.
Avatar of Kryztoval
Kryztoval

I found the following link really insightful.

http://support.microsoft.com/default.aspx/kb/956188

It does not only say what, why but it also tells you how to limit the number of ports.