On my workstations, nslookup against 208.67.222.222 is perfectly fine.

Just the OpenDNS VM forwarding out queries is reported to be having problems.
How can I trace if my firewall is dropping any outgoing DNS traffic?

You have DNS inspection set up in the ASA?

A sanitized config would come in handy :)

You mentioned TCP.  TCP is used for zone transfers, not for normal lookups.  Lookups use UDP.

What's going on with TCP?

According to OpenDNS support, this was logged in their VM appliance, which resides inside my LAN.

Wed Aug 29 15:47:48 2012     TCP connection to 208.67.222.222 fails: Timeout
Wed Aug 29 15:48:48 2012     TCP connection to 208.67.220.220 fails: Timeout
Wed Aug 29 15:48:48 2012     TCP connection to 208.67.222.222 fails: Timeout
Wed Aug 29 15:49:48 2012     TCP connection to 208.67.222.222 fails: Timeout

According to them, not all connections are timing out, there are a few successes.
They are suspecting some form of throttling or IPS.



I have DNS inspection in ASA

policy-map type inspect dns preset_dns_map
 parameters
   message-length maximum 512
class inspection_default
   inspect dns preset_dns_map

class-map my-ip-class
  match access-list IPS

class my-ips-class
  ips promiscuous fail-open
  user-statistics accounting

access-list IPS extended permit ip any any
 match access-list IPS

If there my access list is blocking, then it should theoretically drop all that traffic right? But some seems to be getting through...

I check  the logs on my ASA and filtered via the IP of the appliance. All I see are the DNS traffic mentioned in my original question. I don't see any other dropped/denied logs.

It is already set to debug...

I did already try packet tracer and it didn't show any problems. I've lodge a support call with OpenDNS and they are going to do a tcpdump on the VM. Will probably have an answer in a couple of days. Let's see how it goes before I close and award the points.

We'll be here :)

According to OpenDNS support, this was logged in their VM appliance, which resides inside my LAN.

Wed Aug 29 15:47:48 2012     TCP connection to 208.67.222.222 fails: Timeout
Wed Aug 29 15:48:48 2012     TCP connection to 208.67.220.220 fails: Timeout
Wed Aug 29 15:48:48 2012     TCP connection to 208.67.222.222 fails: Timeout
Wed Aug 29 15:49:48 2012     TCP connection to 208.67.222.222 fails: Timeout

192.168.2.4    40067      208.67.222.222     53                     Built outbound UDP connection for outside:208.67.222.222/53 to inside:192.168.2.4/40067(202.127.70.160/40067)
208.67.222.222   53     192.168.2.4           40067                  Teardown UDP connection for outside:208.67.222.222/53 to inside:192.168.2.4/40067 duration 0:00:00 bytes 66

It's entirely possible that the ASA is blocking TCP connections, but allowing UDP connections.

I think you need to find out from OpenDNS what the device is trying to do via TCP.

If they're trying to perform a zone transfer, then you may need to modify the ASA configuration to allow it.

A DNS forward request, or a recursive lookup, would use UDP/53.


I suggest using the packet tracer again, using TCP/53 as the destination port.

It's entirely possible that the ASA is blocking TCP connections, but allowing UDP connections.

I think you need to find out from OpenDNS what the device is trying to do via TCP.

If they're trying to perform a zone transfer, then you may need to modify the ASA configuration to allow it.

A DNS forward request, or a recursive lookup, would use UDP/53.

I suggest using the packet tracer again, using TCP/53 as the destination port.

I have checked both. Outgoing TCP and UDP/53 are all allowed.
In fact, all outgoing traffic initiated from the LAN is not blocked.

But TCP/53 might be inspected and discarded.

So how do I check that? I need something outside my firewall capturing traffic to verify?

I'm still suggesting that you run the packet tracer again, using TCP/53.

I'm still suggesting that you run the packet tracer again, using TCP/53.

I wasn't clear in my answer but I have ran both TCP/53 and UDP/53 on the packet tracer

Thu Sep  6 15:21:41 2012 OpenDNS Servers: All DNS fail => All DNS ok
Thu Sep  6 15:21:41 2012     UDP traffic to 208.67.220.220 succeeds: 67.215.65.132
Thu Sep  6 15:21:41 2012     TCP connection to 208.67.220.220 succeeds
Thu Sep  6 15:21:41 2012     UDP traffic to 208.67.222.222 succeeds: 67.215.65.132
Thu Sep  6 15:21:41 2012     TCP connection to 208.67.222.222 succeeds
Thu Sep  6 15:22:42 2012 OpenDNS Servers: All DNS ok => Not all DNS ok
Thu Sep  6 15:22:42 2012     UDP traffic to 208.67.220.220 succeeds: 67.215.65.132
Thu Sep  6 15:22:42 2012     TCP connection to 208.67.220.220 fails: Timeout
Thu Sep  6 15:22:42 2012     UDP traffic to 208.67.222.222 succeeds: 67.215.65.132
Thu Sep  6 15:22:42 2012     TCP connection to 208.67.222.222 succeeds
Thu Sep  6 15:23:41 2012 OpenDNS Servers: Not all DNS ok => All DNS ok
Thu Sep  6 15:23:41 2012     UDP traffic to 208.67.220.220 succeeds: 67.215.65.132
Thu Sep  6 15:23:41 2012     TCP connection to 208.67.220.220 succeeds
Thu Sep  6 15:23:41 2012     UDP traffic to 208.67.222.222 succeeds: 67.215.65.132
Thu Sep  6 15:23:41 2012     TCP connection to 208.67.222.222 succeeds
Thu Sep  6 15:24:42 2012 OpenDNS Servers: All DNS ok => All DNS fail
Thu Sep  6 15:24:42 2012     UDP traffic to 208.67.220.220 succeeds: 67.215.65.132
Thu Sep  6 15:24:42 2012     TCP connection to 208.67.220.220 fails: Timeout
Thu Sep  6 15:24:42 2012     UDP traffic to 208.67.222.222 succeeds: 67.215.65.132
Thu Sep  6 15:24:42 2012     TCP connection to 208.67.222.222 fails: Timeout

Select allOpen in new window

I was provided their tcpdumps...it is weird that TCP sometimes timeout.
But UDP always seems to succeed.
But TCP does goes through at time.

It turns out that to be a problem with the OpenDNS virtual appliance.
After I created another instance with double the memory and processor, the TCP timeout stopped.