?
Solved

Remote sites, DNS lookups question

Posted on 2011-11-01
79
Medium Priority
?
2,594 Views
Last Modified: 2012-05-12
I have an issue I've been working with the last few days, tearing my hair out. I am guessing it's a matter of setting split-dns, maybe but at this point I'm not sure.

I have a main site, using ASA5505 as a VPN headend. Remote sites also use an ASA5505 with EasyVPN because they are all dynamic IPs. Networking works, I have split tunneling enabled and working. No real functionality issue other than me not wanting all the DNS queries to come back to corporate. I monitor the traffic using iftop, on a spanned interface from the main ASA, with a port filter of 53. On a remote site, I can do a nslookup, and it will query the DNS server of choice as assigned by the local DHCP server. (In this case, google 8.8.8.8) The lookup happens, but on my iftop I see the remote client has actually queried the corporate DNS servers. Miniscule traffic, about 700b or so per request.

I never even noticed this before, until a few weeks ago a server here died, happened to be the primary DNS server. All the remote sites stopped being able to do any DNS resolution until I changed the ASA at corporate to use a public DNS server under group policy.

That being said, what do I need to do configuration-wise to make all requests that aren't "mydomain.com" only ask the DNS server as configured at each remote site, and requests to the mydomain.com actually traverse the tunnel and query my corporate DNS server?

Most of the remote sites are not in the domain, they are just homes that will be using the VPN function to use Cisco IP phones while at home, and possibly connect to some shares. If they need to access shares, even then I'm not really worried about DNS, I can map by IP in those cases.
0
Comment
Question by:J-Rodder
  • 45
  • 34
79 Comments
 
LVL 81

Expert Comment

by:arnold
ID: 37067697
how are the remote Local DNS setup?  Do they link into the domain?
Do they replicate the mydomain.com so they have a local copy?
the issue you might have is that the settings on the mydomain.com are too short for the Expiry i.e. they replicate the zone, but the default expiry on the mydomain.com zone is 1day such that once the DNS server went down, the mydomain.com copies expired after a day.

You could configure the remote DNS servers to only query your corporate DNS when requests are for mydomain.com

Not sure what types of requests you were seeing.
0
 

Author Comment

by:J-Rodder
ID: 37069758
Well I managed to get the DNS lookups unlinked from coming in to corporate, however now I can't do lookups to the FQDN of mydomain.com sites. Less than ideal, but better than before.

Basically the remote sites will use whomever their ISP is (or google DNS) to do lookups from that site. They really have no need of doing lookups to the corporate DNS, but I wanted it to work for sake of completeness.

I don't have control of the remote DNS servers, as they will be public for internet lookups. I tried configuring my corporate DNS as a secondary, for FQDN lookups but that didn't work. I just looked at nslookup from a remote site, and changed to the corporate DNS server, but there's something being filtered or denied. Maybe DNS inspection or something? Instead of returning the FQDN of the corporate DNS server in the "Name:" field as it does when doing a nslookup directly from corporate subnet, it returns the same IP.

I can post my main ASA config here if that would help any. Basically in a nutshell my issue right now is, NSLOOKUP from corporate works completely fine. NSLOOKUP to same DNS server over the VPN does not. At least however, I have stopped the dependence on the corporate link for DNS lookups.

group-policy INTERVPN internal
group-policy INTERVPN attributes
 dns-server value 8.8.8.8 192.168.10.10
 vpn-tunnel-protocol ikev1 
 split-tunnel-policy tunnelspecified
 split-tunnel-network-list value INTERVPN_splitTunnelAcl
 default-domain value interspacetech.net
 ip-phone-bypass enable
 nem enable

Open in new window


0
 
LVL 81

Expert Comment

by:arnold
ID: 37069862
If there is no Local DNS at the remote sites, you wide up in the scenario you are in.
Either all DNS requests go through to the internal DNS server or no internal traffic is available.

Do the remote offices rely on the AD or are they isolated from corporate?
Having a local DNS servers would let you do what you want. i.e. forward all non mydomain.com requests to the ISP's DNS server while requests for mydomain.com will flow through the VPN to the corporate DNS servers if copies of the DNS zone are not available on the local DNS server.
0
Transaction-level recovery for Oracle database

Veeam Explore for Oracle delivers low RTOs and RPOs with agentless transaction log backup and transaction-level recovery of Oracle databases. You can restore the database to a precise point in time, even to a specific transaction.

 

Author Comment

by:J-Rodder
ID: 37069896
The remote sites are not in the corporate domain, no. So far for my Plug and Play deployment, I am pairing a DD-WRT enabled router, which is turned into basically a wireless switch and DHCP server. The DHCP server on the DDWRT hands out the IPs, and the DG of the ASA. It has DNS functionality there, I guess it's a matter of configuring that as a DNS server to do the forwarding I need? Probably overkill for my situation, but I hate to leave things unresolved in case I come up on them again in the future. :)
0
 

Author Comment

by:J-Rodder
ID: 37070385
I guess what is bothering me is, I *should* be able to force a query via nslookup to the DNS server of my choice and get a response, no?
0
 
LVL 81

Expert Comment

by:arnold
ID: 37070766
nslookup www.yourdomain.com dns_server_ip

You can not force anything.  If the VPN is down and you use the remote LAN IP the attempt might trigger the VPN connection attempt, but it might not complete before the lookup timeout.
0
 

Author Comment

by:J-Rodder
ID: 37071151
VPN is up. What I meant by "force" was forcing the issue of what DNS to query via nslookup. I can ping the internal IP of the corporate DNS server from the remote site. So from a remote host on the subnet of 192.168.17.0, I issue an nslookup. First server is loads is what I have configured, in this case 8.8.8.8. Fine, so remote hosts use google for now to do all lookups by default, not traversing the VPN link. I issue a "server 192.168.10.10" from nslookup. supposedly changes to that server. No requests can be made, but I question if it actually even made a connection to the server, as the requests aren't showing up on the wire via my iftop sniffing.

I guess I am wondering where the problem actually lies, since I can't "access" the corporate DNS server over the VPN. I mean it's there and ailve, but looks like not usuable for DNS functions without some changes *somewhere*.
0
 
LVL 81

Expert Comment

by:arnold
ID: 37072055
within nslookup lserver is the correct approach to set the dns server.
nslookup www.yourdomain.com IP_op_your_remote_LAN_DNS_SERVER
should give you the same information.
what is the local IP range?
You may have an IP overlap and while you think you are pining the corporate dns on 192.168.10.10 you are actually pinging a local system.

0
 

Author Comment

by:J-Rodder
ID: 37072176
I don't have an overlap. "nslookup www.yourdomain.com IP_op_your_remote_LAN_DNS_SERVER" is the same thing as

nslookup
server ip_of_new_dns_server
domain_to_query

subnets are simple. Corporate is 192.168.10.0/24, remote sites are 192.168.x.0/24. I am quite sure I connectable to the DNS server. I even opened up an RDP session to it over the VPN just for fun. My gut feeling is that the traffic is being either dropped or filtered via the VPN link. nslookup to the corporate DNS server from main site, by IP resolves to the FQDN inside of the nslookup prompt.

From 192.168.10.0/24 (corporate)
C:\Users\jclark>nslookup
Default Server:  hercules.interspacetech.net
Address:  192.168.10.11

> lserver 192.168.10.10
Default Server:  artemis.interspacetech.net
Address:  192.168.10.10

Open in new window


those are correct. From the remote site:

C:\Users\jclark>nslookup
Default Server: google-public-dns-a.google.com
Address: 8.8.8.8

>lserver 192.168.10.10
Default Server: [192.168.10.10]
Address: 192.168.10.10

Open in new window


nslookup to the same DNS server IP from the remote site will not respond with the FQDN anymore, just the IP.

I can still resolve to sites that aren't in my corporate domain from the remote site, when pointed to the 10.10 server, but anything that *should* be resolving internally, as in hosts within my corporate domain, are not. Does this make any sense?
0
 
LVL 81

Expert Comment

by:arnold
ID: 37072253
The domain you use is public and you're remote systems use ISP DNS servers which resolve the external hostname versus the internal one.

i.e. nslookup hostname.yourdomain.net hits the ISP dns and then provides an external IP rather than an internal one. The access to the external IP is not by way of the VPN and likely runs into the external firewall that does not allow port 53 access.

Can you check whether VPN access to port 53 is both TCP and UDP?

run the following to see what happens.
nslookup -debug hostname.mydomain.net 192.168.10.10
0
 

Author Comment

by:J-Rodder
ID: 37072306
I understand how external and internal DNS works. From my understanding, nslookup allows me to directly query the server of my choice, and get the resolution from that server... Here's the debug output.

C:\Users\Interspace>nslookup -debug service02.interspacetech.net 192.168.10.10
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 1, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 0,  additional = 0

    QUESTIONS:
        10.10.168.192.in-addr.arpa, type = PTR, class = IN

------------
Server:  UnKnown
Address:  192.168.10.10

------------
Got answer:
    HEADER:
        opcode = QUERY, id = 2, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net.interspacetech.net, type = A, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1799 (29 mins 59 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 3, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net.interspacetech.net, type = AAAA, class = IN

    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1799 (29 mins 59 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 4, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net, type = A, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1800 (30 mins)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 5, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net, type = AAAA, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1799 (29 mins 59 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
*** UnKnown can't find service02.interspacetech.net: Non-existent domain

Open in new window

0
 
LVL 81

Expert Comment

by:arnold
ID: 37072348
The issue is you have a search domain defined for your domain which is appended to the query host if not terminated
service02.interspacetech.net.interspacetech.net
instead of
service02.interspacetech.net

run nslookup service02.interspacetech.net. 192.168.10.10 and you should get the answer you are expecting.
0
 

Author Comment

by:J-Rodder
ID: 37072352
Before when I have seen the "UnKnown" via nslookup, was on domains that didn't have RDNS configured properly, or at all. I appreciate your help in attacking this, as I am close to pulling my hair out. :)
0
 

Author Comment

by:J-Rodder
ID: 37072381
I did notice the appending before I posted, so for fun I had run a query with just the name, letting it append. I also ran it your way, this was the output:

C:\Users\Interspace>nslookup -debug service02 192.168.10.10
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 1, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 0,  additional = 0

    QUESTIONS:
        10.10.168.192.in-addr.arpa, type = PTR, class = IN

------------
Server:  UnKnown
Address:  192.168.10.10

------------
Got answer:
    HEADER:
        opcode = QUERY, id = 2, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net, type = A, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1237 (20 mins 37 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 3, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net, type = AAAA, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1237 (20 mins 37 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
*** UnKnown can't find service02: Non-existent domain

C:\Users\Interspace>nslookup -debug service02.interspacetech.net. 192.168.10.10
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 1, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 0,  additional = 0

    QUESTIONS:
        10.10.168.192.in-addr.arpa, type = PTR, class = IN

------------
Server:  UnKnown
Address:  192.168.10.10

------------
Got answer:
    HEADER:
        opcode = QUERY, id = 2, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net, type = A, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1228 (20 mins 28 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 3, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        service02.interspacetech.net, type = AAAA, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1227 (20 mins 27 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
*** UnKnown can't find service02.interspacetech.net.: Non-existent domain

C:\Users\Interspace>

Open in new window

0
 
LVL 81

Expert Comment

by:arnold
ID: 37072550
Did not pay attention to the query, but your DNS server 192.168.10.10 is the cause for the issue, it does not seem to see the request as coming from an "authorized source" and is referring the request to query the worldnic.net servers for the domain.
this is either the DNS on 192.168.10.10 is not authoritative for the domain or something else is going on

Why are you using ipv6 (type=AAAA) for the lookup try
nslookup -debug -q=A server02 192.168.10.10
0
 

Author Comment

by:J-Rodder
ID: 37072645
Well that's something I didn't consider. I am pretty sure it's all configured properly on this end. (my reverse DNS isn't updating properly right now from DHCP but I don't think that's related to this issue, right?) Ran that debug, and same *non* result. UnKnown server response, telling me it's a non-existent domain. The worldnic DNS is Network Solutions iirc, and is where interspacetech.net is hosted. Maybe the nslookup never really connected to 192.168.10.10 and instead the requests are being handed off to the interspacetech.net external DNS domain servers by google? That sounds crazy after I typed it, heh.

[b]C:\Users\Interspace>nslookup -debug -q=A server02 192.168.10.11[/b]
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 1, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 0,  additional = 0

    QUESTIONS:
        11.10.168.192.in-addr.arpa, type = PTR, class = IN

------------
[b]Server:  UnKnown
Address:  192.168.10.11[/b]

------------
Got answer:
    HEADER:
        opcode = QUERY, id = 2, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        server02.interspacetech.net, type = A, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1174 (19 mins 34 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
*** UnKnown can't find server02: Non-existent domain

[b]C:\Users\Interspace>nslookup -debug -q=A server02 google.com[/b]
------------
Got answer:
    HEADER:
        opcode = QUERY, id = 1, rcode = NOERROR
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 1,  authority records = 0,  additional = 0

    QUESTIONS:
        20.225.125.74.in-addr.arpa, type = PTR, class = IN
    ANSWERS:
    ->  20.225.125.74.in-addr.arpa
        name = ord08s05-in-f20.1e100.net
        ttl = 86400 (1 day)

------------
[b]Server:  ord08s05-in-f20.1e100.net
Address:  74.125.225.20[/b]

------------
Got answer:
    HEADER:
        opcode = QUERY, id = 2, rcode = NXDOMAIN
        header flags:  response, want recursion, recursion avail.
        questions = 1,  answers = 0,  authority records = 1,  additional = 0

    QUESTIONS:
        server02.interspacetech.net, type = A, class = IN
    AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1169 (19 mins 29 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM
        serial  = 111090909
        refresh = 10800 (3 hours)
        retry   = 3600 (1 hour)
        expire  = 604800 (7 days)
        default TTL = 3600 (1 hour)

------------
*** ord08s05-in-f20.1e100.net can't find server02: Non-existent domain

C:\Users\Interspace>

Open in new window

0
 
LVL 81

Expert Comment

by:arnold
ID: 37073219
Not sure why you are redirexcting the request to ns77.worldnic.net

The DNS should have the domain as authoritative which means it should respond with answers versus redirecting.
What DNS service is running on 192.168.10.10 is this a windows DNS or is it a caching dns server that does not include the remote IPs as authorized to query.
0
 

Author Comment

by:J-Rodder
ID: 37075969
DNS server is Windows, Server 2008R2. Do I have to make my DNS server authoritative to the internet for interspacetech.net lookups even when the request is piped over a "local" VPN connection with internal addresses? Can you give me some advice as to how I should check?
0
 
LVL 81

Expert Comment

by:arnold
ID: 37076268
Is the domain interspacetech.net local on your system?
Check the global query block list to make sure you are not limiting the response to the local LAN only.

0
 

Author Comment

by:J-Rodder
ID: 37083796
Yes, interspacetech.net is local on this network for our AD. It's *also* at Network Solutions for external internet lookups. Is that the problem? I went into one of the DNS servers here, 192.168.10.10 and issued a "dnscmd /config /enableglobalqueryblocklist 0" to test to see if that would make a difference. Doesn't appear so.
0
 
LVL 81

Expert Comment

by:arnold
ID: 37084286
Using a publicly accessible domain for the AD runs into the problems you've experienced.
Usually using private label suffixes .private, .local, etc. avoids this type of issue but has different issues when hosting DNS in house.

The 192.168.10.10 based on your -debug seems to refer the client to the external name server.
Check the local zone and what NS records it has.

nslookup -q=soa interspacetech.net locally on the LAN.
0
 

Author Comment

by:J-Rodder
ID: 37084430
Yes, I usually use .local for new clients, I inherited this AD system when I started employment here. Here's my output from a corporate machine:

C:\Users\jclark>nslookup -q=soa interspacetech.net
Server:  artemis.interspacetech.net
Address:  192.168.10.10

interspacetech.net
        primary name server = artemis.interspacetech.net
        responsible mail addr = hostmaster.interspacetech.net
        serial  = 4071
        refresh = 900 (15 mins)
        retry   = 600 (10 mins)
        expire  = 86400 (1 day)
        default TTL = 3600 (1 hour)
artemis.interspacetech.net      internet address = 192.168.10.10

Open in new window

0
 
LVL 81

Expert Comment

by:arnold
ID: 37084478
what about the NS records in the local zone?
Note your zone expiry is 1 day.
0
 

Author Comment

by:J-Rodder
ID: 37084520
I don't know if I understand that question, but I have in my local zone SOA of Artemis.interspacetech.net, 3 nameservers in the zone.

artemis.interspacetech.net 192.168.10.10
hercules.interspacetech.net 192.168.10.11
ithomeserver.interspacetech.net 192.168.14.3

I left the other values as default, but after some googling pertaining to your notice, I see that it's way too short.
0
 
LVL 81

Expert Comment

by:arnold
ID: 37084908
The problem I'm seeing is that when you generate a request for interspacetech.net from the remote location, you get responses/references to ns77.worldnic.net which is the external Authoritative host.
AUTHORITY RECORDS:
    ->  interspacetech.net
        ttl = 1174 (19 mins 34 secs)
        primary name server = NS77.WORLDNIC.COM
        responsible mail addr = namehost.WORLDNIC.COM

this looks as though the requests to 192.168.10.10 or the others do not make it through to the remote site and the lookup goes to external versus internal zone information.

Double check the zone configuration as well as the DNS server configuration dealing with forwarding requests (Forwarders).
0
 

Author Comment

by:J-Rodder
ID: 37085080
That makes sense, but doesn't nslookup directly query the nameserver I choose via "lserver" or "server" command once opened? so by issuing a nslookup, and switching server to 10.10, and then entering a domain to lookup, it's directly asking that server for a response, regardless of how the issuing host DNS is configured? This is what led me to believe it was something to do with the ASA config, since as far as networking is concerned, those servers are completely available. interspacetech.net DNS servers use our local ISP DNS and google as a third for forwarding.
0
 

Author Comment

by:J-Rodder
ID: 37085253
Ok, well read more on the manpages for nslookup, I guess I was inferring incorrectly. Looks like no matter what, the host DNS matters, it's not so simple to just make a request to the DNS server of my choice apparently. If it truly is something misconfigured on my local domain DNS setup, I must not know what I am looking for. I'll award points, I think I have taken enough of your time at this point. :)
0
 

Author Comment

by:J-Rodder
ID: 37097207
I've requested that this question be closed as follows:

Accepted answer: 500 points for arnold's comment http:/Q_27425495.html#37084908
Assisted answer: 0 points for J-Rodder's comment http:/Q_27425495.html#37085253

for the following reason:

Maybe this thread will help someone in the future. If I find the final resolution in my case, I will be sure to append this for future reference.
0
 
LVL 81

Expert Comment

by:arnold
ID: 37086158
It looks as though the referrer redirect response is coming from the remote system 192.168.10.10 in the case above.

You could enable query logging and see what IP the 192.168.10.10 sees the request as coming from.
An internal caching DNS server, should not be responding with a redirect in the way yours has i.e. check with ns77.worldnic.net.
0
 

Author Comment

by:J-Rodder
ID: 37086616
Ok, I am kind of glad you objected to closing the thread. I was making the assumption that my 10.10 server should have seen the traffic as internal, but you're right, there's a good chance it sees the traffic as external, coming from the external interface of the remote site firewall, and then getting rejected. I will look into how to set up the query logging and get back to you.
0
 
LVL 81

Expert Comment

by:arnold
ID: 37086984
It is not that it is coming over via an external interface, the issue is likely that you've limited your internal DNS (caching) to your internal LAN and the requests from the remote systems via the VPN fall outside that range.

Check the /etc/named.conf file to see what you are limiting the option  query-allow {Local LAN, remote LAN; or any;};
0
 

Author Comment

by:J-Rodder
ID: 37087887
Silly question, I have enabled logging on the DNS server, going to the office later today to check it out. You told me to check the /etc/named.conf. I don't seem to have that on Windows Server DNS. Is that a BIND thing?
0
 
LVL 81

Expert Comment

by:arnold
ID: 37088120
Oh, sorry, when you referenced 10.10 jumped that it was a Ubuntu 10.10 version versus the 192.168.10.10.
0
 

Author Comment

by:J-Rodder
ID: 37088395
Looking at my DNS debug logs on the 10.10 server, it looks like none of the external requests are even making it to the server. Every request there is coming from the main corporate subnet. So basically, even if I make my DNS servers on a remote host 10.10 and 10.11, those requests aren't making it over. Unless of course, rejected requests don't get logged? I turned on all the options I could when I was enabling the debug.

Starting to think it's a matter of me missing something pretty basic as far as what needs to be set in order for it to work.
0
 
LVL 81

Expert Comment

by:arnold
ID: 37089460
IP/netmask from the remote location?
0
 

Author Comment

by:J-Rodder
ID: 37090241
Heh, well not *that* basic. Everything is /24, and all other routing and connectivity works. Specifically a DNS issue.
0
 
LVL 81

Expert Comment

by:arnold
ID: 37090515
Can a person on the remote site access any shared resources on the Corporate LAN side by name or IP?
0
 

Author Comment

by:J-Rodder
ID: 37090633
Yep. I can do anything I want via IP. Full 2-way tunnel.
0
 
LVL 81

Expert Comment

by:arnold
ID: 37092264
try the following from the remote site:
nslookup
lserver 192.168.10.10
set debug
set nosearch
set querytype=ns
interspacetech.net.

It looks as though the issue is that for one reason or another, your DNS server is forwarding the request outbound.
and server02 does not exist on the external name servers.
0
 

Author Comment

by:J-Rodder
ID: 37093533
I happen to be at home right now, so no Windows machines to try it from, but I am pretty sure that nslookup is a UNIX command anyway. My home is also a stub site for work, set up the same way as what I was testing from before, other than my local DNS server being a Zyntal box. (BIND) Here's that output, and I'll run it again from the same place we did the other testing to ensure conformity to the testing.

jrod@Nibiru:~$ nslookup
> lserver 192.168.10.10
Default server: 192.168.10.10
Address: 192.168.10.10#53
> set debug
> set nosearch
> set querytype=ns
> interspacetech.net.
Server:		192.168.10.10
Address:	192.168.10.10#53

------------
    QUESTIONS:
	interspacetech.net, type = NS, class = IN
    ANSWERS:
    ->  interspacetech.net
	nameserver = ns77.worldnic.com.
	ttl = 6807
    ->  interspacetech.net
	nameserver = ns78.worldnic.com.
	ttl = 6807
    AUTHORITY RECORDS:
    ADDITIONAL RECORDS:
------------
Non-authoritative answer:
interspacetech.net	nameserver = ns77.worldnic.com.
interspacetech.net	nameserver = ns78.worldnic.com.

Authoritative answers can be found from:
> 

Open in new window


0
 
LVL 81

Expert Comment

by:arnold
ID: 37093813
Stub zone to which local LAN settings or to the public zone?
note your output is directing the request outward to worldnic as well.
0
 

Author Comment

by:J-Rodder
ID: 37093884
I'm sorry, I am not sure exactly what you are asking. I have no zones in my DNS with the remote subnet. The only remote subnet that has entries for a remote network is a remote site that is actually running a DC for the domain. I then just now tried to add a stub zone for our new remote site in question, but the wizard is telling me that my DNS servers are not authoritative?
0
 
LVL 81

Expert Comment

by:arnold
ID: 37093958
interspacetech.net is a public domain and you are using it as an internal AD which must have a DNS server with interspacetech.net zone.
When you were querying the domain from the remote site via the VPN, you were being redirected to ns*.worldnic.net (external Name servers)
This is the cause of your issues.
You need to determine why 192.168.10.10 responds one way when the query is local on the LAN or you have your internal interspacetech.net domain referencing worldnic's name servers within the zone.

0
 
LVL 81

Expert Comment

by:arnold
ID: 37093976
Check the security tab within the properties of your internal zone on the 192.168.10.10 name server, does it have everyone with read rights to this zone?
0
 

Author Comment

by:J-Rodder
ID: 37094009
Yeah, I'm with you on the issue, as far as why it's trying to look up records on the external nameservers. What I don't understand is why the query wasn't looking to the server I have defined in the host DNS, when that internal DNS server appears to be fully available. "Everyone" has read rights for the interspacetech.net forward lookup zone.
0
 
LVL 81

Expert Comment

by:arnold
ID: 37094043
I do not see why the nslookup -debug -q=ns interspacetech.net 192.168.10.10 while you can ping and access resources, gives a response that ns*.worldnic.net are the name servers.

You could try using wireshark (wireshark.net) and capture port 53 traffic (requests/responses to and from 192.168.10.10) from the remote system to see what is going on and where the breakdown seems to occur.
0
 

Author Comment

by:J-Rodder
ID: 37094075
Well I was doing that in a way, but not with Wireshark yet. I was using iftop from a linux monitoring machine, connected to a spanned port on the corporate network, with a port 53 filter. I can see that no traffic over port 53 is reaching my internal DNS servers from the remote networks. They *used* to, when I had that configuration being pushed over the easyvpn group policy config, but I have since broken that in order to ensure remote sites aren't dependent on the corporate DNS for lookups.
dns.png
0
 
LVL 81

Expert Comment

by:arnold
ID: 37094116
I'd suggest testing it from the client side versus the server side.

not sure what type of VPN you have i.e. whether each remote is a client/remote type of VPN. Or a site to site VPN where the remote LAN IPs.

Client side will also see fewer packets and would be easier to sift through.


your internal returns non-authoritative answers for interspacetech.net while it should return authoritative answers.
How is interspacetech.net configured on the 192.168.10.10 DNS server? Is this an AD integrated zone?
0
 

Author Comment

by:J-Rodder
ID: 37094149
0
 

Author Comment

by:J-Rodder
ID: 37094164
The VPN setup is utilizing corporate ASA5505 as a headend, and the remote sites are also using ASA5505, with EasyVPN configured to pull config from headend. I can't really do a true site to site VPN, since the remote sites change, and are dynamic IPs. The DNS here is AD integrated, and all works fine from the corporate site.
0
 

Author Comment

by:J-Rodder
ID: 37094176
It's the "UnKnown" response I am getting from the remote site that bothers me. Like I said, I have seen that before even at main sites, and it was fixable by configuring RDNS. I mean DNS still worked regardless, but the DNS server never responded to a nslookup with its FQDN without it. I just don't know why the traffic doesn't seem to want to come over the VPN link. From the Wireshark, it does look like ARTEMIS.interspacetech.net (192.168.10.10) is trying to answer?
0
 

Author Comment

by:J-Rodder
ID: 37094261
Here's a dnslint output:


dnslint.htm
0
 
LVL 81

Expert Comment

by:arnold
ID: 37094505
what about using nslookup -debug -q=ns -nosearch interspacetech.net 192.168.10.10
It looks like the requests and responses are going and coming from 192.168.10.10 but a configuration on the 192.168.10.10 provides a different response.

configure a filter
tcp.port==53 or udp.port=53
this should capture the dns traffic.
use a similar rule on your corporate LAN side and see whether the IP from which the requests are comming are NAtted easyVPN router assigned IPs.

Dnslint ran from which side corporate LAN or from the remote??
Do you have an option to run dnslint from the remote side?
0
 

Author Comment

by:J-Rodder
ID: 37094906
dnslint and the wireshark from remote are attached. For the life of me, I can't seem to see any traffic from the remote site showing up on the spanned port of the corporate interface. I'll also attach a picture of the corporate LAN setup, that might help a bit. The SPAN is on the switch, monitoring the WAN port of ther ASA. That lets me see all the packets flowing into and out of the .10 network.

I can see the ICMP traffic I tested with using iftop on the same interface as I am sniffing with Wireshark on, yet in Wireshark with no filters applied I see no traffic coming from or going to remote host.

So basically, I am still having trouble seeing if it's a NAT issue, but I don't think it is, else iftop wouldn't be seeing the .17.4 address, it would be seeing those packets as the outside IP of my remote site, no?
wireshark-remote.png
dnslint.htm
Interspace.png
screenshot.png
0
 
LVL 81

Expert Comment

by:arnold
ID: 37095003
The wireshark remote clearly points out that the response from your internal 192.168.10.10 for NS record for interspacetech.net points the user to the worldnic.net servers versus pointing to the internal set on the 192.168. network.

regarding iftop, are you mirroring the port entirely or you have a filter/access-list that limits the traffic mirrored to icmp only? i.e. setup for troubleshooting while minimizing the load on the switch?

Presumably installing wireshark on the production 192.168.10.10 is out of the question.
wireshark filter
ip.addr==192.168.17.0/24 and ( tcp.port==53 or udp.port==53)
on the 192.168.10.10 should only reflect requests coming from this segment and limited to DNS requests.
0
 

Author Comment

by:J-Rodder
ID: 37095014
I ran some captures using packet capture on the corporate ASA5505 as well. I can see the ICMP there, but no DNS activity over the link, after trying to run your debug query.
0
 

Author Comment

by:J-Rodder
ID: 37095055
Basically I have a VMware server that I use here, one of those NICs is linked to my monitoring port. That physical port is a complete SPAN, and when I run iftop from the shell, I don't have any filters set. I do that manually with filters once it is running. In the case of my screenshot there, I had applied a screen filter of the remote IP address, so it would be clear.
0
 
LVL 81

Expert Comment

by:arnold
ID: 37095157
The remote side sees responses from 192.168.10.10 so I am not sure how it is handled on the corporate side.
0
 

Author Comment

by:J-Rodder
ID: 37095238
I am seeing all the normal traffic I would expect on the LAN, but nothing from the remote site coming in. EXCEPT FOR the remote site .14.0/24, which is a remote site with the same setup, but that site has a domain controller and is in the domain. THAT traffic is showing up, but nothing from remote sites not in the domain. What is weird is not even ICMP is visible in wireshark, when there's no reason it shouldn't be. I am losing my mind now. :)
0
 
LVL 81

Expert Comment

by:arnold
ID: 37095281
I don't know, but you have VLAN so not sure whether it is somehow interferring.

What about wireshark on the 192.168.10.10 system?
You may have MS network tool that could capture packets as well.
0
 

Author Comment

by:J-Rodder
ID: 37096076
I can install wireshark on the 10.10 server, not a big deal. I trust the program, and I can just pull it off later. Bunch of other stuff has cropped up, I will get back to this thread with some output of wireshark on the 10.10 while trying to make some requests to it.
0
 

Author Comment

by:J-Rodder
ID: 37097208
I have wireshark on the 10.10 server itself. I feel a bit more sane now. I can see the ICMP requests now that I *should* have been seeing before, but for some reason was not. What I *don't* see however, is any DNS requests to 10.10 from the remote site .17 subnet. Any way you can drop me an email @ jclark@interspacetech.net? I'd like to give you a full packet capture file, but for obvious reasons would rather not post it publicly. If that's a faux-paux for this board, let me know.
0
 
LVL 81

Expert Comment

by:arnold
ID: 37097401
use the ip.addr==192.168.17.0/24 and (tcp.port==53 or udp.port==53)
 as the filter and export just these packets for transmission.
0
 

Author Comment

by:J-Rodder
ID: 37097417
I did so already, and there were no DNS packets appearing on the 10.10 server from the remote site. I verified them again watching wireshark on the remote host though. That filter generated no visible packets. I'll try it again tomorrow with a clear head and see what happens, and show some documentation.
0
 
LVL 81

Expert Comment

by:arnold
ID: 37102663
I still think this is an issue with your configuration of your DNS server i.e. internal systems query and get authoritative answers about the local AD zone while non-AD computers get the caching functionality only.
Try the following if you could, use a laptop that is not joined to the domain on the LAN and query the local interspacetech.net zone to see if what you see with the remote site querying the local DNS is the same answer you get when using a non AD system on the local LAN querying the local DNS 192.168.10.10.
0
 

Author Comment

by:J-Rodder
ID: 37102707
laptop that is not in the domain, hooked up to the internal LAN. Nslookup brings back the FQDN of the DNS server as it should, and responds to queries about internal and external hosts as expected.
0
 
LVL 81

Expert Comment

by:arnold
ID: 37104071
Do you have remote vpn that you can test with this laptop and see whether the response via VPN is the same?
Something is out of place, but I can not put my finger on it other than the response from the 192.168.10.10 that the remote system sees is a cached response pointing to external name servers.

Did you have a chance  to run dnslint from the remote site?

could you rerun dnslint locally with
dnslint /d interspacetech.net. /s 192.168.10.10

Adding the terminating period to make sure that there is no search domain that is set by the DHCP server that leads/alters the query from interspacetech.net to interspacetech.net.interspacetech.local. or private etc. that will be the cause for the distinguishing behavior.

if you still have the unjoined laptop, ipconfig /all
DNS specific search list
0
 

Author Comment

by:J-Rodder
ID: 37104521
I am guessing you mean to try it with a regular VPN client to the headend ASA, and not the EasyVPN setup. Yes I can test that easily enough tomorrow. I ran dnslint earlier in the thread from the remote site, but tomorrow I will provide them from both sides.
0
 
LVL 81

Expert Comment

by:arnold
ID: 37104905
Ok, Make sure you terminate the domain interspacetech.net. to make sure there is no appending of a search domain the mocks the results.
0
 

Author Comment

by:J-Rodder
ID: 37107297
I haven't gotten to the rest of it yet, but I just tried connecting from a public IP, using a VPN client to the headend. In that case, using nslookup pulls my first google DNS server, and issuing the lserver command and connecting to 192.168.10.10 let me resolve inside hosts as expected. It must be something with the way the tunnel is established using the EasyVPN then?
0
 

Author Comment

by:J-Rodder
ID: 37107834
I verified again the behavior of the ASA Group Policy config. If I put in the corporate DNS server in the DNS server settings that get pushed when a VPN client connects, then it works, but with the caveat that ALL requests get sent to corporate first. Not the split setup I was going for. Same with the VPN client to an external host, all requests go to corporate.

I was thinking along these lines, but I don't see anything in GUI regarding split-dns, just split-tunneling. This is why I was wondering if it was even possible.

https://supportforums.cisco.com/thread/2015428
0
 

Accepted Solution

by:
J-Rodder earned 0 total points
ID: 37108218
I have attached the dnslint from corporate and remote.

This might be part of the problem here:

https://supportforums.cisco.com/thread/2071302

I did find the split-dns option under the split tunneling options for group policy, can't believe I missed that before. I guess part of it was knowing exactly where to drill down looking. But, as that poster says, it doesn't work for the hardware to hardware connections.

Split-dns should only works via vpn client on PC not hareware client like ASA5505 cause it is PC to initiate DNS query. A PC behind hareware VPN client don't know anything about this split-dns setup.

Can you change your DHCP setup on your 5505 to assign DNS server IP to the client in the following order

<corporate_DNS_IP> <public_DNS_IP>

In this way, PC behind ASA5505 should try corporate DNS server first and then your ISP DNS server.

Sure that works, but doesn't resolve the issue of me wanting to keep the DNS split unless it's specifically for domain traffic. Ah well, maybe it's just not possible with this setup?

I have played with it a bit more after configuring the split-dns option, and now at least it looks like there's attempted communication where before there was none on the wire. If I query a hostname, I'll get a "server failed" on the remote end, and can see the request packets showing up on the 10.10 server.
dnslint-corporate.htm
dnslint-remote.htm
0
 
LVL 81

Assisted Solution

by:arnold
arnold earned 2000 total points
ID: 37108340
Any chance/option of putting a local DNS server at the remote location this way you could configure the forwarder for interspacetech.net to route to 192.168.10.10, but when the VPN is down, or access to the external site will be an issue unless your internal points to the public one.
0
 

Author Comment

by:J-Rodder
ID: 37108509
I'm starting to think that might be my only choice. It's one of the reasons I planned on configuring the remote sites with a router/wireless AP running DD-WRT. I figured it would provide me with more options when something like this cropped up. I have no idea how to set that up, but it looks like through all this troubleshooting, that might be the only way it would even be possible. I think it uses dnsmasq, and there's some documentation floating around.

In the end, I don't mind all the DNS traffic routing through corporate, as long as it doesn't mean broken DNS at remote sites when VPN or DNS server is down. It's not like we are talking about a massive amount of traffic. In my mind, it's not as *clean* as a way to conditionally split, but it would certainly at least achieve the main goal.
0
 
LVL 81

Expert Comment

by:arnold
ID: 37108734
Do you have an old workstation that you can setup as a local caching DNS using a linux environment? Might be better to stick with the ASA versus trying to replace something that mostly working and start down a new troubleshooting road if something does not work right.
0
 

Author Comment

by:J-Rodder
ID: 37108770
I could, but I'd rather figure it out in the DD-WRT system. These remote sites get migrated, so I need an easy plug and play solution. Essentially, an ASA/DD-WRT combo with which I can go to the site, plug it in and be done, same with taking it out. These remote sites are essentially just for Cisco phones, and once in a while access to some shared files at corporate.

http://www.dd-wrt.com/wiki/index.php/DNSMasq_-_DNS_for_your_local_network_-_HOWTO
0
 
LVL 81

Expert Comment

by:arnold
ID: 37108844
The ASA connects to switch and dd-wrt connects into the switch and any computer connects into the dd-wrt while the phones connect into the switch?
You run the problem of double NAT ASA NAT dd-wrt NAT

Linux based caching DNS
http://myhowtosandprojects.blogspot.com/2008/07/configure-linux-dns-server-cache-and.html
Have not looked at tinydns but might be an option to use instead of bind.
0
 

Author Comment

by:J-Rodder
ID: 37108941
Well I only use 1 vlan on the DDWRT router, no using the WAN port at all. I basically turn it into a wireless switch, and I modified the DHCP server to hand out the ASA IP as a default gateway, and not the address of the DDWRT itself. So, no WAN port usage, everything is on the same subnet. The phones connect to the ASA, but only because that's where my PoE is.

It works now, other than the obvious DNS issues currently. I guess in my mind I figured I should have some kind of functionality with the way I wanted DNS lookups to happen, before I figured out how to get it done with the DDWRT. There were just too many variables in play for me to figure out what the heck was going on.
0
 

Author Closing Comment

by:J-Rodder
ID: 37194297
I never did get this to work as I liked, but in the end there's more than one way to skin a cat. I will either have to configure DNS servers at the remote sites, or just live with some static mappings in DDWRT using DNSMasq for the few resources that are actually even needing to be shared. Thanks for all the help, Arnold.
0

Featured Post

How to Use the Help Bell

Need to boost the visibility of your question for solutions? Use the Experts Exchange Help Bell to confirm priority levels and contact subject-matter experts for question attention.  Check out this how-to article for more information.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A 2007 NCSA Cyber Security survey revealed that a mere 4% of the population has a full understanding of firewalls. As business owner, you should be part of that 4% that has a full understanding.
On Feb. 28, Amazon’s Simple Storage Service (S3) went down after an employee issued the wrong command during a debugging exercise. Among those affected were big names like Netflix, Spotify and Expedia.
As a trusted technology advisor to your customers you are likely getting the daily question of, ‘should I put this in the cloud?’ As customer demands for cloud services increases, companies will see a shift from traditional buying patterns to new…
Both in life and business – not all partnerships are created equal. Spend 30 short minutes with us to learn:   • Key questions to ask when considering a partnership to accelerate your business into the cloud • Pitfalls and mistakes other partners…
Suggested Courses
Course of the Month15 days, 13 hours left to enroll

850 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question