Remote sites, DNS lookups question

I have an issue I've been working with the last few days, tearing my hair out. I am guessing it's a matter of setting split-dns, maybe but at this point I'm not sure.

I have a main site, using ASA5505 as a VPN headend. Remote sites also use an ASA5505 with EasyVPN because they are all dynamic IPs. Networking works, I have split tunneling enabled and working. No real functionality issue other than me not wanting all the DNS queries to come back to corporate. I monitor the traffic using iftop, on a spanned interface from the main ASA, with a port filter of 53. On a remote site, I can do a nslookup, and it will query the DNS server of choice as assigned by the local DHCP server. (In this case, google 8.8.8.8) The lookup happens, but on my iftop I see the remote client has actually queried the corporate DNS servers. Miniscule traffic, about 700b or so per request.

I never even noticed this before, until a few weeks ago a server here died, happened to be the primary DNS server. All the remote sites stopped being able to do any DNS resolution until I changed the ASA at corporate to use a public DNS server under group policy.

That being said, what do I need to do configuration-wise to make all requests that aren't "mydomain.com" only ask the DNS server as configured at each remote site, and requests to the mydomain.com actually traverse the tunnel and query my corporate DNS server?

Most of the remote sites are not in the domain, they are just homes that will be using the VPN function to use Cisco IP phones while at home, and possibly connect to some shares. If they need to access shares, even then I'm not really worried about DNS, I can map by IP in those cases.
J-RodderAsked:
Who is Participating?
 
J-RodderConnect With a Mentor Author Commented:
I have attached the dnslint from corporate and remote.

This might be part of the problem here:

https://supportforums.cisco.com/thread/2071302

I did find the split-dns option under the split tunneling options for group policy, can't believe I missed that before. I guess part of it was knowing exactly where to drill down looking. But, as that poster says, it doesn't work for the hardware to hardware connections.

Split-dns should only works via vpn client on PC not hareware client like ASA5505 cause it is PC to initiate DNS query. A PC behind hareware VPN client don't know anything about this split-dns setup.

Can you change your DHCP setup on your 5505 to assign DNS server IP to the client in the following order

<corporate_DNS_IP> <public_DNS_IP>

In this way, PC behind ASA5505 should try corporate DNS server first and then your ISP DNS server.

Sure that works, but doesn't resolve the issue of me wanting to keep the DNS split unless it's specifically for domain traffic. Ah well, maybe it's just not possible with this setup?

I have played with it a bit more after configuring the split-dns option, and now at least it looks like there's attempted communication where before there was none on the wire. If I query a hostname, I'll get a "server failed" on the remote end, and can see the request packets showing up on the 10.10 server.
dnslint-corporate.htm
dnslint-remote.htm
0
 
arnoldCommented:
how are the remote Local DNS setup?  Do they link into the domain?
Do they replicate the mydomain.com so they have a local copy?
the issue you might have is that the settings on the mydomain.com are too short for the Expiry i.e. they replicate the zone, but the default expiry on the mydomain.com zone is 1day such that once the DNS server went down, the mydomain.com copies expired after a day.

You could configure the remote DNS servers to only query your corporate DNS when requests are for mydomain.com

Not sure what types of requests you were seeing.
0
 
J-RodderAuthor Commented:
Well I managed to get the DNS lookups unlinked from coming in to corporate, however now I can't do lookups to the FQDN of mydomain.com sites. Less than ideal, but better than before.

Basically the remote sites will use whomever their ISP is (or google DNS) to do lookups from that site. They really have no need of doing lookups to the corporate DNS, but I wanted it to work for sake of completeness.

I don't have control of the remote DNS servers, as they will be public for internet lookups. I tried configuring my corporate DNS as a secondary, for FQDN lookups but that didn't work. I just looked at nslookup from a remote site, and changed to the corporate DNS server, but there's something being filtered or denied. Maybe DNS inspection or something? Instead of returning the FQDN of the corporate DNS server in the "Name:" field as it does when doing a nslookup directly from corporate subnet, it returns the same IP.

I can post my main ASA config here if that would help any. Basically in a nutshell my issue right now is, NSLOOKUP from corporate works completely fine. NSLOOKUP to same DNS server over the VPN does not. At least however, I have stopped the dependence on the corporate link for DNS lookups.

group-policy INTERVPN internal
group-policy INTERVPN attributes
 dns-server value 8.8.8.8 192.168.10.10
 vpn-tunnel-protocol ikev1 
 split-tunnel-policy tunnelspecified
 split-tunnel-network-list value INTERVPN_splitTunnelAcl
 default-domain value interspacetech.net
 ip-phone-bypass enable
 nem enable

Open in new window


0
KuppingerCole Reviews AlgoSec in Executive Report

Leading analyst firm, KuppingerCole reviews AlgoSec's Security Policy Management Solution, and the security challenges faced by companies today in their Executive View report.

 
arnoldCommented:
If there is no Local DNS at the remote sites, you wide up in the scenario you are in.
Either all DNS requests go through to the internal DNS server or no internal traffic is available.

Do the remote offices rely on the AD or are they isolated from corporate?
Having a local DNS servers would let you do what you want. i.e. forward all non mydomain.com requests to the ISP's DNS server while requests for mydomain.com will flow through the VPN to the corporate DNS servers if copies of the DNS zone are not available on the local DNS server.
0
 
J-RodderAuthor Commented:
The remote sites are not in the corporate domain, no. So far for my Plug and Play deployment, I am pairing a DD-WRT enabled router, which is turned into basically a wireless switch and DHCP server. The DHCP server on the DDWRT hands out the IPs, and the DG of the ASA. It has DNS functionality there, I guess it's a matter of configuring that as a DNS server to do the forwarding I need? Probably overkill for my situation, but I hate to leave things unresolved in case I come up on them again in the future. :)
0
 
J-RodderAuthor Commented:
I guess what is bothering me is, I *should* be able to force a query via nslookup to the DNS server of my choice and get a response, no?
0
 
arnoldCommented:
nslookup www.yourdomain.com dns_server_ip

You can not force anything.  If the VPN is down and you use the remote LAN IP the attempt might trigger the VPN connection attempt, but it might not complete before the lookup timeout.
0
 
J-RodderAuthor Commented:
VPN is up. What I meant by "force" was forcing the issue of what DNS to query via nslookup. I can ping the internal IP of the corporate DNS server from the remote site. So from a remote host on the subnet of 192.168.17.0, I issue an nslookup. First server is loads is what I have configured, in this case 8.8.8.8. Fine, so remote hosts use google for now to do all lookups by default, not traversing the VPN link. I issue a "server 192.168.10.10" from nslookup. supposedly changes to that server. No requests can be made, but I question if it actually even made a connection to the server, as the requests aren't showing up on the wire via my iftop sniffing.

I guess I am wondering where the problem actually lies, since I can't "access" the corporate DNS server over the VPN. I mean it's there and ailve, but looks like not usuable for DNS functions without some changes *somewhere*.
0
 
arnoldCommented:
within nslookup lserver is the correct approach to set the dns server.
nslookup www.yourdomain.com IP_op_your_remote_LAN_DNS_SERVER
should give you the same information.
what is the local IP range?
You may have an IP overlap and while you think you are pining the corporate dns on 192.168.10.10 you are actually pinging a local system.

0
 
J-RodderAuthor Commented:
I don't have an overlap. "nslookup www.yourdomain.com IP_op_your_remote_LAN_DNS_SERVER" is the same thing as

nslookup
server ip_of_new_dns_server
domain_to_query

subnets are simple. Corporate is 192.168.10.0/24, remote sites are 192.168.x.0/24. I am quite sure I connectable to the DNS server. I even opened up an RDP session to it over the VPN just for fun. My gut feeling is that the traffic is being either dropped or filtered via the VPN link. nslookup to the corporate DNS server from main site, by IP resolves to the FQDN inside of the nslookup prompt.

From 192.168.10.0/24 (corporate)
C:\Users\jclark>nslookup
Default Server:  hercules.interspacetech.net
Address:  192.168.10.11

> lserver 192.168.10.10
Default Server:  artemis.interspacetech.net
Address:  192.168.10.10

Open in new window


those are correct. From the remote site:

C:\Users\jclark>nslookup
Default Server: google-public-dns-a.google.com
Address: 8.8.8.8

>lserver 192.168.10.10
Default Server: [192.168.10.10]
Address: 192.168.10.10

Open in new window


nslookup to the same DNS server IP from the remote site will not respond with the FQDN anymore, just the IP.

I can still resolve to sites that aren't in my corporate domain from the remote site, when pointed to the 10.10 server, but anything that *should* be resolving internally, as in hosts within my corporate domain, are not. Does this make any sense?
0
 
arnoldCommented:
The domain you use is public and you're remote systems use ISP DNS servers which resolve the external hostname versus the internal one.

i.e. nslookup hostname.yourdomain.net hits the ISP dns and then provides an external IP rather than an internal one. The access to the external IP is not by way of the VPN and likely runs into the external firewall that does not allow port 53 access.

Can you check whether VPN access to port 53 is both TCP and UDP?

run the following to see what happens.
nslookup -debug hostname.mydomain.net 192.168.10.10
0
 
J-RodderAuthor Commented:
I understand how external and internal DNS works. From my understanding, nslookup allows me to directly query the server of my choice, and get the resolution from that server... Here's the debug output.

C:\Users\Interspace>nslookup -debug service02.interspacetech.net 192.168.10.10
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 1, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 0,  additional = 0

    QUESTIONS:
        10.10.168.192.in-addr.arpa, type = PTR, class = IN

------------
Server:  UnKnown
Address:  192.168.10.10

------------
Got answer:
    HEADER:
        opcode = QUERY, id = 2, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net.interspacetech.net, type = A, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1799 (29 mins 59 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 3, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net.interspacetech.net, type = AAAA, class = IN

    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1799 (29 mins 59 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 4, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net, type = A, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1800 (30 mins)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 5, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net, type = AAAA, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1799 (29 mins 59 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
*** UnKnown can't find service02.interspacetech.net: Non-existent domain

Open in new window

0
 
arnoldCommented:
The issue is you have a search domain defined for your domain which is appended to the query host if not terminated
service02.interspacetech.net.interspacetech.net
instead of
service02.interspacetech.net

run nslookup service02.interspacetech.net. 192.168.10.10 and you should get the answer you are expecting.
0
 
J-RodderAuthor Commented:
Before when I have seen the "UnKnown" via nslookup, was on domains that didn't have RDNS configured properly, or at all. I appreciate your help in attacking this, as I am close to pulling my hair out. :)
0
 
J-RodderAuthor Commented:
I did notice the appending before I posted, so for fun I had run a query with just the name, letting it append. I also ran it your way, this was the output:

C:\Users\Interspace>nslookup -debug service02 192.168.10.10
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 1, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 0,  additional = 0

    QUESTIONS:
        10.10.168.192.in-addr.arpa, type = PTR, class = IN

------------
Server:  UnKnown
Address:  192.168.10.10

------------
Got answer:
    HEADER:
        opcode = QUERY, id = 2, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net, type = A, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1237 (20 mins 37 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 3, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net, type = AAAA, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1237 (20 mins 37 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
*** UnKnown can't find service02: Non-existent domain

C:\Users\Interspace>nslookup -debug service02.interspacetech.net. 192.168.10.10
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 1, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 0,  additional = 0

    QUESTIONS:
        10.10.168.192.in-addr.arpa, type = PTR, class = IN

------------
Server:  UnKnown
Address:  192.168.10.10

------------
Got answer:
    HEADER:
        opcode = QUERY, id = 2, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net, type = A, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1228 (20 mins 28 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 3, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net, type = AAAA, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1227 (20 mins 27 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
*** UnKnown can't find service02.interspacetech.net.: Non-existent domain

C:\Users\Interspace>

Open in new window

0
 
arnoldCommented:
Did not pay attention to the query, but your DNS server 192.168.10.10 is the cause for the issue, it does not seem to see the request as coming from an "authorized source" and is referring the request to query the worldnic.net servers for the domain.
this is either the DNS on 192.168.10.10 is not authoritative for the domain or something else is going on

Why are you using ipv6 (type=AAAA) for the lookup try
nslookup -debug -q=A server02 192.168.10.10
0
 
J-RodderAuthor Commented:
Well that's something I didn't consider. I am pretty sure it's all configured properly on this end. (my reverse DNS isn't updating properly right now from DHCP but I don't think that's related to this issue, right?) Ran that debug, and same *non* result. UnKnown server response, telling me it's a non-existent domain. The worldnic DNS is Network Solutions iirc, and is where interspacetech.net is hosted. Maybe the nslookup never really connected to 192.168.10.10 and instead the requests are being handed off to the interspacetech.net external DNS domain servers by google? That sounds crazy after I typed it, heh.

[b]C:\Users\Interspace>nslookup -debug -q=A server02 192.168.10.11[/b]
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 1, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 0,  additional = 0

    QUESTIONS:
        11.10.168.192.in-addr.arpa, type = PTR, class = IN

------------
[b]Server:  UnKnown
Address:  192.168.10.11[/b]

------------
Got answer:
    HEADER:
        opcode = QUERY, id = 2, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        server02.interspacetech.net, type = A, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1174 (19 mins 34 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
*** UnKnown can't find server02: Non-existent domain

[b]C:\Users\Interspace>nslookup -debug -q=A server02 google.com[/b]
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 1, rcode = NOERROR
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 1,  authority records = 0,  additional = 0

    QUESTIONS:
        20.225.125.74.in-addr.arpa, type = PTR, class = IN
    ANSWERS:
    ->  20.225.125.74.in-addr.arpa
        name = ord08s05-in-f20.1e100.net
        ttl = 86400 (1 day)

------------
[b]Server:  ord08s05-in-f20.1e100.net
Address:  74.125.225.20[/b]

------------
Got answer:
    HEADER:
        opcode = QUERY, id = 2, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        server02.interspacetech.net, type = A, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1169 (19 mins 29 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
*** ord08s05-in-f20.1e100.net can't find server02: Non-existent domain

C:\Users\Interspace>

Open in new window

0
 
arnoldCommented:
Not sure why you are redirexcting the request to ns77.worldnic.net

The DNS should have the domain as authoritative which means it should respond with answers versus redirecting.
What DNS service is running on 192.168.10.10 is this a windows DNS or is it a caching dns server that does not include the remote IPs as authorized to query.
0
 
J-RodderAuthor Commented:
DNS server is Windows, Server 2008R2. Do I have to make my DNS server authoritative to the internet for interspacetech.net lookups even when the request is piped over a "local" VPN connection with internal addresses? Can you give me some advice as to how I should check?
0
 
arnoldCommented:
Is the domain interspacetech.net local on your system?
Check the global query block list to make sure you are not limiting the response to the local LAN only.

0
 
J-RodderAuthor Commented:
Yes, interspacetech.net is local on this network for our AD. It's *also* at Network Solutions for external internet lookups. Is that the problem? I went into one of the DNS servers here, 192.168.10.10 and issued a "dnscmd /config /enableglobalqueryblocklist 0" to test to see if that would make a difference. Doesn't appear so.
0
 
arnoldCommented:
Using a publicly accessible domain for the AD runs into the problems you've experienced.
Usually using private label suffixes .private, .local, etc. avoids this type of issue but has different issues when hosting DNS in house.

The 192.168.10.10 based on your -debug seems to refer the client to the external name server.
Check the local zone and what NS records it has.

nslookup -q=soa interspacetech.net locally on the LAN.
0
 
J-RodderAuthor Commented:
Yes, I usually use .local for new clients, I inherited this AD system when I started employment here. Here's my output from a corporate machine:

C:\Users\jclark>nslookup -q=soa interspacetech.net
Server:  artemis.interspacetech.net
Address:  192.168.10.10

interspacetech.net
        primary name server = artemis.interspacetech.net
        responsible mail addr = hostmaster.interspacetech.net
        serial  = 4071
        refresh = 900 (15 mins)
        retry   = 600 (10 mins)
        expire  = 86400 (1 day)
        default TTL = 3600 (1 hour)
artemis.interspacetech.net      internet address = 192.168.10.10

Open in new window

0
 
arnoldCommented:
what about the NS records in the local zone?
Note your zone expiry is 1 day.
0
 
J-RodderAuthor Commented:
I don't know if I understand that question, but I have in my local zone SOA of Artemis.interspacetech.net, 3 nameservers in the zone.

artemis.interspacetech.net 192.168.10.10
hercules.interspacetech.net 192.168.10.11
ithomeserver.interspacetech.net 192.168.14.3

I left the other values as default, but after some googling pertaining to your notice, I see that it's way too short.
0
 
arnoldCommented:
The problem I'm seeing is that when you generate a request for interspacetech.net from the remote location, you get responses/references to ns77.worldnic.net which is the external Authoritative host.
AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1174 (19 mins 34 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM

this looks as though the requests to 192.168.10.10 or the others do not make it through to the remote site and the lookup goes to external versus internal zone information.

Double check the zone configuration as well as the DNS server configuration dealing with forwarding requests (Forwarders).
0
 
J-RodderAuthor Commented:
That makes sense, but doesn't nslookup directly query the nameserver I choose via "lserver" or "server" command once opened? so by issuing a nslookup, and switching server to 10.10, and then entering a domain to lookup, it's directly asking that server for a response, regardless of how the issuing host DNS is configured? This is what led me to believe it was something to do with the ASA config, since as far as networking is concerned, those servers are completely available. interspacetech.net DNS servers use our local ISP DNS and google as a third for forwarding.
0
 
J-RodderAuthor Commented:
Ok, well read more on the manpages for nslookup, I guess I was inferring incorrectly. Looks like no matter what, the host DNS matters, it's not so simple to just make a request to the DNS server of my choice apparently. If it truly is something misconfigured on my local domain DNS setup, I must not know what I am looking for. I'll award points, I think I have taken enough of your time at this point. :)
0
 
J-RodderAuthor Commented:
I've requested that this question be closed as follows:

Accepted answer: 500 points for arnold's comment http:/Q_27425495.html#37084908
Assisted answer: 0 points for J-Rodder's comment http:/Q_27425495.html#37085253

for the following reason:

Maybe this thread will help someone in the future. If I find the final resolution in my case, I will be sure to append this for future reference.
0
 
arnoldCommented:
It looks as though the referrer redirect response is coming from the remote system 192.168.10.10 in the case above.

You could enable query logging and see what IP the 192.168.10.10 sees the request as coming from.
An internal caching DNS server, should not be responding with a redirect in the way yours has i.e. check with ns77.worldnic.net.
0
 
J-RodderAuthor Commented:
Ok, I am kind of glad you objected to closing the thread. I was making the assumption that my 10.10 server should have seen the traffic as internal, but you're right, there's a good chance it sees the traffic as external, coming from the external interface of the remote site firewall, and then getting rejected. I will look into how to set up the query logging and get back to you.
0
 
arnoldCommented:
It is not that it is coming over via an external interface, the issue is likely that you've limited your internal DNS (caching) to your internal LAN and the requests from the remote systems via the VPN fall outside that range.

Check the /etc/named.conf file to see what you are limiting the option  query-allow {Local LAN, remote LAN; or any;};
0
 
J-RodderAuthor Commented:
Silly question, I have enabled logging on the DNS server, going to the office later today to check it out. You told me to check the /etc/named.conf. I don't seem to have that on Windows Server DNS. Is that a BIND thing?
0
 
arnoldCommented:
Oh, sorry, when you referenced 10.10 jumped that it was a Ubuntu 10.10 version versus the 192.168.10.10.
0
 
J-RodderAuthor Commented:
Looking at my DNS debug logs on the 10.10 server, it looks like none of the external requests are even making it to the server. Every request there is coming from the main corporate subnet. So basically, even if I make my DNS servers on a remote host 10.10 and 10.11, those requests aren't making it over. Unless of course, rejected requests don't get logged? I turned on all the options I could when I was enabling the debug.

Starting to think it's a matter of me missing something pretty basic as far as what needs to be set in order for it to work.
0
 
arnoldCommented:
IP/netmask from the remote location?
0
 
J-RodderAuthor Commented:
Heh, well not *that* basic. Everything is /24, and all other routing and connectivity works. Specifically a DNS issue.
0
 
arnoldCommented:
Can a person on the remote site access any shared resources on the Corporate LAN side by name or IP?
0
 
J-RodderAuthor Commented:
Yep. I can do anything I want via IP. Full 2-way tunnel.
0
 
arnoldCommented:
try the following from the remote site:
nslookup
lserver 192.168.10.10
set debug
set nosearch
set querytype=ns
interspacetech.net.

It looks as though the issue is that for one reason or another, your DNS server is forwarding the request outbound.
and server02 does not exist on the external name servers.
0
 
J-RodderAuthor Commented:
I happen to be at home right now, so no Windows machines to try it from, but I am pretty sure that nslookup is a UNIX command anyway. My home is also a stub site for work, set up the same way as what I was testing from before, other than my local DNS server being a Zyntal box. (BIND) Here's that output, and I'll run it again from the same place we did the other testing to ensure conformity to the testing.

jrod@Nibiru:~$ nslookup
> lserver 192.168.10.10
Default server: 192.168.10.10
Address: 192.168.10.10#53
> set debug
> set nosearch
> set querytype=ns
> interspacetech.net.
Server:		192.168.10.10
Address:	192.168.10.10#53

------------
    QUESTIONS:
	interspacetech.net, type = NS, class = IN
    ANSWERS:
    ->  interspacetech.net
	nameserver = ns77.worldnic.com.
	ttl = 6807
    ->  interspacetech.net
	nameserver = ns78.worldnic.com.
	ttl = 6807
    AUTHORITY RECORDS:
    ADDITIONAL RECORDS:
------------
Non-authoritative answer:
interspacetech.net	nameserver = ns77.worldnic.com.
interspacetech.net	nameserver = ns78.worldnic.com.

Authoritative answers can be found from:
> 

Open in new window


0
 
arnoldCommented:
Stub zone to which local LAN settings or to the public zone?
note your output is directing the request outward to worldnic as well.
0
 
J-RodderAuthor Commented:
I'm sorry, I am not sure exactly what you are asking. I have no zones in my DNS with the remote subnet. The only remote subnet that has entries for a remote network is a remote site that is actually running a DC for the domain. I then just now tried to add a stub zone for our new remote site in question, but the wizard is telling me that my DNS servers are not authoritative?
0
 
arnoldCommented:
interspacetech.net is a public domain and you are using it as an internal AD which must have a DNS server with interspacetech.net zone.
When you were querying the domain from the remote site via the VPN, you were being redirected to ns*.worldnic.net (external Name servers)
This is the cause of your issues.
You need to determine why 192.168.10.10 responds one way when the query is local on the LAN or you have your internal interspacetech.net domain referencing worldnic's name servers within the zone.

0
 
arnoldCommented:
Check the security tab within the properties of your internal zone on the 192.168.10.10 name server, does it have everyone with read rights to this zone?
0
 
J-RodderAuthor Commented:
Yeah, I'm with you on the issue, as far as why it's trying to look up records on the external nameservers. What I don't understand is why the query wasn't looking to the server I have defined in the host DNS, when that internal DNS server appears to be fully available. "Everyone" has read rights for the interspacetech.net forward lookup zone.
0
 
arnoldCommented:
I do not see why the nslookup -debug -q=ns interspacetech.net 192.168.10.10 while you can ping and access resources, gives a response that ns*.worldnic.net are the name servers.

You could try using wireshark (wireshark.net) and capture port 53 traffic (requests/responses to and from 192.168.10.10) from the remote system to see what is going on and where the breakdown seems to occur.
0
 
J-RodderAuthor Commented:
Well I was doing that in a way, but not with Wireshark yet. I was using iftop from a linux monitoring machine, connected to a spanned port on the corporate network, with a port 53 filter. I can see that no traffic over port 53 is reaching my internal DNS servers from the remote networks. They *used* to, when I had that configuration being pushed over the easyvpn group policy config, but I have since broken that in order to ensure remote sites aren't dependent on the corporate DNS for lookups.
dns.png
0
 
arnoldCommented:
I'd suggest testing it from the client side versus the server side.

not sure what type of VPN you have i.e. whether each remote is a client/remote type of VPN. Or a site to site VPN where the remote LAN IPs.

Client side will also see fewer packets and would be easier to sift through.


your internal returns non-authoritative answers for interspacetech.net while it should return authoritative answers.
How is interspacetech.net configured on the 192.168.10.10 DNS server? Is this an AD integrated zone?
0
 
J-RodderAuthor Commented:
0
 
J-RodderAuthor Commented:
The VPN setup is utilizing corporate ASA5505 as a headend, and the remote sites are also using ASA5505, with EasyVPN configured to pull config from headend. I can't really do a true site to site VPN, since the remote sites change, and are dynamic IPs. The DNS here is AD integrated, and all works fine from the corporate site.
0
 
J-RodderAuthor Commented:
It's the "UnKnown" response I am getting from the remote site that bothers me. Like I said, I have seen that before even at main sites, and it was fixable by configuring RDNS. I mean DNS still worked regardless, but the DNS server never responded to a nslookup with its FQDN without it. I just don't know why the traffic doesn't seem to want to come over the VPN link. From the Wireshark, it does look like ARTEMIS.interspacetech.net (192.168.10.10) is trying to answer?
0
 
J-RodderAuthor Commented:
Here's a dnslint output:


dnslint.htm
0
 
arnoldCommented:
what about using nslookup -debug -q=ns -nosearch interspacetech.net 192.168.10.10
It looks like the requests and responses are going and coming from 192.168.10.10 but a configuration on the 192.168.10.10 provides a different response.

configure a filter
tcp.port==53 or udp.port=53
this should capture the dns traffic.
use a similar rule on your corporate LAN side and see whether the IP from which the requests are comming are NAtted easyVPN router assigned IPs.

Dnslint ran from which side corporate LAN or from the remote??
Do you have an option to run dnslint from the remote side?
0
 
J-RodderAuthor Commented:
dnslint and the wireshark from remote are attached. For the life of me, I can't seem to see any traffic from the remote site showing up on the spanned port of the corporate interface. I'll also attach a picture of the corporate LAN setup, that might help a bit. The SPAN is on the switch, monitoring the WAN port of ther ASA. That lets me see all the packets flowing into and out of the .10 network.

I can see the ICMP traffic I tested with using iftop on the same interface as I am sniffing with Wireshark on, yet in Wireshark with no filters applied I see no traffic coming from or going to remote host.

So basically, I am still having trouble seeing if it's a NAT issue, but I don't think it is, else iftop wouldn't be seeing the .17.4 address, it would be seeing those packets as the outside IP of my remote site, no?
wireshark-remote.png
dnslint.htm
Interspace.png
screenshot.png
0
 
arnoldCommented:
The wireshark remote clearly points out that the response from your internal 192.168.10.10 for NS record for interspacetech.net points the user to the worldnic.net servers versus pointing to the internal set on the 192.168. network.

regarding iftop, are you mirroring the port entirely or you have a filter/access-list that limits the traffic mirrored to icmp only? i.e. setup for troubleshooting while minimizing the load on the switch?

Presumably installing wireshark on the production 192.168.10.10 is out of the question.
wireshark filter
ip.addr==192.168.17.0/24 and ( tcp.port==53 or udp.port==53)
on the 192.168.10.10 should only reflect requests coming from this segment and limited to DNS requests.
0
 
J-RodderAuthor Commented:
I ran some captures using packet capture on the corporate ASA5505 as well. I can see the ICMP there, but no DNS activity over the link, after trying to run your debug query.
0
 
J-RodderAuthor Commented:
Basically I have a VMware server that I use here, one of those NICs is linked to my monitoring port. That physical port is a complete SPAN, and when I run iftop from the shell, I don't have any filters set. I do that manually with filters once it is running. In the case of my screenshot there, I had applied a screen filter of the remote IP address, so it would be clear.
0
 
arnoldCommented:
The remote side sees responses from 192.168.10.10 so I am not sure how it is handled on the corporate side.
0
 
J-RodderAuthor Commented:
I am seeing all the normal traffic I would expect on the LAN, but nothing from the remote site coming in. EXCEPT FOR the remote site .14.0/24, which is a remote site with the same setup, but that site has a domain controller and is in the domain. THAT traffic is showing up, but nothing from remote sites not in the domain. What is weird is not even ICMP is visible in wireshark, when there's no reason it shouldn't be. I am losing my mind now. :)
0
 
arnoldCommented:
I don't know, but you have VLAN so not sure whether it is somehow interferring.

What about wireshark on the 192.168.10.10 system?
You may have MS network tool that could capture packets as well.
0
 
J-RodderAuthor Commented:
I can install wireshark on the 10.10 server, not a big deal. I trust the program, and I can just pull it off later. Bunch of other stuff has cropped up, I will get back to this thread with some output of wireshark on the 10.10 while trying to make some requests to it.
0
 
J-RodderAuthor Commented:
I have wireshark on the 10.10 server itself. I feel a bit more sane now. I can see the ICMP requests now that I *should* have been seeing before, but for some reason was not. What I *don't* see however, is any DNS requests to 10.10 from the remote site .17 subnet. Any way you can drop me an email @ jclark@interspacetech.net? I'd like to give you a full packet capture file, but for obvious reasons would rather not post it publicly. If that's a faux-paux for this board, let me know.
0
 
arnoldCommented:
use the ip.addr==192.168.17.0/24 and (tcp.port==53 or udp.port==53)
 as the filter and export just these packets for transmission.
0
 
J-RodderAuthor Commented:
I did so already, and there were no DNS packets appearing on the 10.10 server from the remote site. I verified them again watching wireshark on the remote host though. That filter generated no visible packets. I'll try it again tomorrow with a clear head and see what happens, and show some documentation.
0
 
arnoldCommented:
I still think this is an issue with your configuration of your DNS server i.e. internal systems query and get authoritative answers about the local AD zone while non-AD computers get the caching functionality only.
Try the following if you could, use a laptop that is not joined to the domain on the LAN and query the local interspacetech.net zone to see if what you see with the remote site querying the local DNS is the same answer you get when using a non AD system on the local LAN querying the local DNS 192.168.10.10.
0
 
J-RodderAuthor Commented:
laptop that is not in the domain, hooked up to the internal LAN. Nslookup brings back the FQDN of the DNS server as it should, and responds to queries about internal and external hosts as expected.
0
 
arnoldCommented:
Do you have remote vpn that you can test with this laptop and see whether the response via VPN is the same?
Something is out of place, but I can not put my finger on it other than the response from the 192.168.10.10 that the remote system sees is a cached response pointing to external name servers.

Did you have a chance  to run dnslint from the remote site?

could you rerun dnslint locally with
dnslint /d interspacetech.net. /s 192.168.10.10

Adding the terminating period to make sure that there is no search domain that is set by the DHCP server that leads/alters the query from interspacetech.net to interspacetech.net.interspacetech.local. or private etc. that will be the cause for the distinguishing behavior.

if you still have the unjoined laptop, ipconfig /all
DNS specific search list
0
 
J-RodderAuthor Commented:
I am guessing you mean to try it with a regular VPN client to the headend ASA, and not the EasyVPN setup. Yes I can test that easily enough tomorrow. I ran dnslint earlier in the thread from the remote site, but tomorrow I will provide them from both sides.
0
 
arnoldCommented:
Ok, Make sure you terminate the domain interspacetech.net. to make sure there is no appending of a search domain the mocks the results.
0
 
J-RodderAuthor Commented:
I haven't gotten to the rest of it yet, but I just tried connecting from a public IP, using a VPN client to the headend. In that case, using nslookup pulls my first google DNS server, and issuing the lserver command and connecting to 192.168.10.10 let me resolve inside hosts as expected. It must be something with the way the tunnel is established using the EasyVPN then?
0
 
J-RodderAuthor Commented:
I verified again the behavior of the ASA Group Policy config. If I put in the corporate DNS server in the DNS server settings that get pushed when a VPN client connects, then it works, but with the caveat that ALL requests get sent to corporate first. Not the split setup I was going for. Same with the VPN client to an external host, all requests go to corporate.

I was thinking along these lines, but I don't see anything in GUI regarding split-dns, just split-tunneling. This is why I was wondering if it was even possible.

https://supportforums.cisco.com/thread/2015428
0
 
arnoldConnect With a Mentor Commented:
Any chance/option of putting a local DNS server at the remote location this way you could configure the forwarder for interspacetech.net to route to 192.168.10.10, but when the VPN is down, or access to the external site will be an issue unless your internal points to the public one.
0
 
J-RodderAuthor Commented:
I'm starting to think that might be my only choice. It's one of the reasons I planned on configuring the remote sites with a router/wireless AP running DD-WRT. I figured it would provide me with more options when something like this cropped up. I have no idea how to set that up, but it looks like through all this troubleshooting, that might be the only way it would even be possible. I think it uses dnsmasq, and there's some documentation floating around.

In the end, I don't mind all the DNS traffic routing through corporate, as long as it doesn't mean broken DNS at remote sites when VPN or DNS server is down. It's not like we are talking about a massive amount of traffic. In my mind, it's not as *clean* as a way to conditionally split, but it would certainly at least achieve the main goal.
0
 
arnoldCommented:
Do you have an old workstation that you can setup as a local caching DNS using a linux environment? Might be better to stick with the ASA versus trying to replace something that mostly working and start down a new troubleshooting road if something does not work right.
0
 
J-RodderAuthor Commented:
I could, but I'd rather figure it out in the DD-WRT system. These remote sites get migrated, so I need an easy plug and play solution. Essentially, an ASA/DD-WRT combo with which I can go to the site, plug it in and be done, same with taking it out. These remote sites are essentially just for Cisco phones, and once in a while access to some shared files at corporate.

http://www.dd-wrt.com/wiki/index.php/DNSMasq_-_DNS_for_your_local_network_-_HOWTO
0
 
arnoldCommented:
The ASA connects to switch and dd-wrt connects into the switch and any computer connects into the dd-wrt while the phones connect into the switch?
You run the problem of double NAT ASA NAT dd-wrt NAT

Linux based caching DNS
http://myhowtosandprojects.blogspot.com/2008/07/configure-linux-dns-server-cache-and.html
Have not looked at tinydns but might be an option to use instead of bind.
0
 
J-RodderAuthor Commented:
Well I only use 1 vlan on the DDWRT router, no using the WAN port at all. I basically turn it into a wireless switch, and I modified the DHCP server to hand out the ASA IP as a default gateway, and not the address of the DDWRT itself. So, no WAN port usage, everything is on the same subnet. The phones connect to the ASA, but only because that's where my PoE is.

It works now, other than the obvious DNS issues currently. I guess in my mind I figured I should have some kind of functionality with the way I wanted DNS lookups to happen, before I figured out how to get it done with the DDWRT. There were just too many variables in play for me to figure out what the heck was going on.
0
 
J-RodderAuthor Commented:
I never did get this to work as I liked, but in the end there's more than one way to skin a cat. I will either have to configure DNS servers at the remote sites, or just live with some static mappings in DDWRT using DNSMasq for the few resources that are actually even needing to be shared. Thanks for all the help, Arnold.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.