Solved

Redundant ISP and DNS

Posted on 2010-09-21
10
1,047 Views
Last Modified: 2012-05-10
We have two ISP's coming into our facility.  We have two external DNS servers we host onsite, one being on one ISP and one on the other.  Network Solutions holds the records pointing the names of these servers to the respective ISP IP address.

What I want to get in place is some type of DNS redundancy if one ISP goes down.  Right now I have DNS1 hosting A records for our internal devices pointing to IP's from ISP1.  And DNS2 hosting A records for our internal devices pointing to IP's from ISP2.  Via NAT, I'm pointing the appropriate ISP IP's, whether from ISP1 or ISP2 to the same internal IP on our internal network.  

This all works fine, I can ping from outside and see that it resolves to both IP's, normally about 50% one and 50% the other.  The issue is is one ISP goes down then the corresponding DNS server is not available.  So users on the outside get bad lookups about 50% of the time.  They have no way of knowing that particular DNS server is down and since Network Solutions still holds the record, outside users can still make requests to it.

I'm wondering how I could have better results.  Is there something Network Solutions could do to monitor the health of the two records they hold for our DNS servers?  will Registrars do that and adjust the requests accordingly?  Or is there some other record manipulation I could do myself to help.

thanks for any insight into what others are doing for DNS redundancy.
0
Comment
Question by:suffolkdit
  • 6
  • 3
10 Comments
 
LVL 4

Expert Comment

by:HunterPine
ID: 33726481
Rather than have your hosts resolve to two separate A records, sign up for an account with dyndns.org (free) or a similar dynamic DNS provider, and use CNAME records to point your hosts to their hosted dynamic dns records.
Use the free dyndns client (or one built into your router if it has one) to automatically change your DNS records to an active line.
This won't allow you to "load balance" incoming connections, but ensures that incoming connections will always point to an active link.
0
 
LVL 20

Expert Comment

by:Daniel McAllister
ID: 33727245
Ouch... you have static IP addresses already from each ISP, and all you want is to have DNS point to either address.

The DNS functionality (whether ISC's BIND [the granddaddy of DNS servers], or Microsoft's DNS server service) is designed to provide for load balancing of multiple paths, but not really for "failover" (that is, report one address all the time, but if it goes down, report a different address).

Since you already have the redundant IP addresses, your BEST solution (IMHO) is to make the DNS servers each report BOTH ip addresses as valid A records for the domains in question.

Let's say you have:
 - DomainA.Com hosted by ISP1 on IP address 1.2.3.4
 - DomainB.Com hosted by ISP2 on IP address 2.3.4.5

Now you've already mentioned that the ACTUAL domain data for each is located on the SAME server NATed behind each router. I am assuming you have the server multi-homed, so that it knows that responses to packets FROM ISP1 go back out ISP1, and similarly responses FROM ISP2 go back out on ISP2. If not, you may want to look into a dual-port WAN router that can do this for you.

In any case, all you want to do now is make sure that the A records for BOTH DomainA.com and DomainB.com point to BOTH 1.2.3.4 AND 2.3.4.5.

The zone records might look similar to this:

$ORIGIN .
domaina.com   IN   A  1.2.3.4
domaina.com   IN   A  2.3.4.5

$ORIGIN domaina.com.
www               IN   A  1.2.3.4
www               IN   A  2.3.4.5

----

What will happen is that the DNS server will "LOAD BALANCE" between the two addresses. (NOTE: Unlike MX records, which have a "weight value" to prefer one (set of) server over another, A records have no "weight".  However, you can influence the routing by stacking the deck, so to speak... simply repeat the entry that is preferred... for example:

$ORIGIN domaina.com.
www               IN   A  1.2.3.4
www               IN   A  1.2.3.4
www               IN   A  1.2.3.4
www               IN   A  2.3.4.5

Will result in IP address 1.2.3.4 getting 75% of the "hits" and address 2.3.4.5 only 25%. NOTE that this doesn't mean 75% of the traffic -- the DNS server has no way of knowing WHY you want to resolve www.domaina.com, or what you'll do with the info when you get it... but this setting does mean that 75% of the time www.domaina.com will resolve to 1.2.3.4.

Next, you'll need to make sure that BOTH DNS servers are active for EACH domain. The EASY way to do that is to make each DNS server an authoritative master for its own domain -- and a slave server for the other domain.

Depending upon the DNS server software you're using, do a GOOGLE search for slave server (e.g.: google "bind slave server" & you'll get links to a FAQ about slave servers and a howto site (xytrax.com)).

Good Luck!

Dan
IT4SOHO
0
 

Author Comment

by:suffolkdit
ID: 33727948
thanks for the comments.  I understand dyndns concept I think, I don't like relying on that but maybe that is because I'm not educated enough on its reliability.  Dan, your comment seems like the direction I want to go.  thought I would try to explain a little better the current setup.

only one domain in question here....domainA.com

ISP1.....1.2.3.4
ISP2......5.6.7.8

DNS1.domainA.com.....1.2.3.5
DNS2.domainA.com.....5.6.7.9

Network Solutions is hosting the two records above for the DNS servers

DNS1  domainA.com zone file
www   IN A  1.2.3.6
blah      IN  A  1.2.3.7
foo        IN  A  1.2.3.8


DNS2  domainA.com zone file
www    IN A  5.6.7.10
blah       IN A  5.6.7.11
foo        IN  A  5.6.7.12

So then I have a Nortel Application Switch which sits in front of my firewall.  this device is kind of a complicated box but is capable of balancing traffic across multiple ISP's.  This is where the multiple ISP's plug in.  This device is doing the NAT.  So I have NAT statements like so in that device.

1.2.3.6       -> 192.168.1.5
5.6.7.10     -> 192.168.1.5


1.2.3.7      -> 192.168.1.6
5.6.7.11    ->  192.168.1.6


1.2.3.8      -> 192.168.1.7
5.6.7.12    ->  192.168.1.7

So to answer your question about multi homed DNS servers, No, the two DNS servers are not multi-homed.  So I'm more interested in how Network Solutions is balancing the requests to the two records they host for my two DNS servers.  I think they are just doing a round robin because currently I ping one of my internal records and I'll get a reply some from one ISP, then another connection replies from the other ISP.  If one ISP goes down then obviously that DNS server is no longer reachable, but requests still go to that record network solutions is hosting.
0
 
LVL 20

Accepted Solution

by:
Daniel McAllister earned 500 total points
ID: 33778664
suffolkdit,

Your DNS entries for DomainA.com are problematic, at best. You need to have a SINGLE set of entries for your domain for the "failover" you want to work. As it appears now, if I have contacted DNS1.domainA.com and resolved the address for www.domainA.com to be 1.2.3.6 -- and then that ISP fails, I will simply lose my connection to www.domainA.com because the "cached" DNS reply I received for www.domainA.com had a single entry: 1.2.3.6, and so I'll believe that there are no alternatives.

The issue here is that you appear to have what in DNS terms is called "inconsistent master zone files" -- that is, the "master" zone files for your domains don't match.

I urge you to confirm this by using the website "intodns.com", where their tool will evaluate your entire DNS setup (at least as it is visible from the outside world using the root-level DNS servers).

The "fix" for your current configuration is to make one of these DNS servers the "master", the other a "slave server" for the domainA.com zone (this way you can update the master and the changes will automatically propagate to the slave server(s).) Then, in the MASTER zone file, you can specify BOTH A records for www.domainA.com (1.2.3.6 & 5.6.7.10) as I showed you above.

Also, if you want something other than a 50/50 split, you can use the method I showed above.... for example, to get a 70/30 split try:

www.domainA.com.  IN  A 1.2.3.6
www.domainA.com.  IN  A 1.2.3.6
www.domainA.com.  IN  A 1.2.3.6
www.domainA.com.  IN  A 1.2.3.6
www.domainA.com.  IN  A 1.2.3.6
www.domainA.com.  IN  A 1.2.3.6
www.domainA.com.  IN  A 1.2.3.6
www.domainA.com.  IN  A 5.6.7.10
www.domainA.com.  IN  A 5.6.7.10
www.domainA.com.  IN  A 5.6.7.10

... but ALL of these entries would be put into the ONE master zone file on the ONE master DNS server, which would then replicate these entries to any (ALL) slave DNS servers so that they would ALL report the same info.

In this case, if ISP1 failed and I had tried to access www.domainA.com using 1.2.3.6 and it timed out, I would already know that I could turn to 5.6.7.10 as an alternative because the DNS server (any of them) would have given me the alternative IP as part of the original A record lookup!

I hope these instructions are clear enough to lead you to the right solution... again, use the tools at intodns.com to test your implementations...

Good Luck!

Dan
IT4SOHO
0
 

Author Comment

by:suffolkdit
ID: 33779000
Dan,

thanks for the reply.  I think this is the direction I will go in.  The only issue I may have is getting network solutions to make one a master and oen a slave.  I haven't talked to them about it yet but I'm guessing maybe they can do that for me.  It would make sense.

If I cannot make one a master and one a slave if I put both A records in both DNS server manually i could accomplish the same, correct?

Thanks again.  I'll let you know how it goes after some testing.  

Charlie
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 20

Expert Comment

by:Daniel McAllister
ID: 33781626
The DNS servers (DNS1.domainA.com & DNS2.domainA.com) are your local systems, no?

Network Solutions is hosting your domain and has informed the TLD DNS Servers (the .com servers) that your DNS servers are DNS1 & DNS2 (with IP addresses of 1.2.3.5 and 5.6.7.9 respectively. That's about as far as Network Solutions should be taking it, unless they're also hosting your servers.

So the changes should be on YOUR DNS servers, not Network Solutions' servers (unless they are one and the same).

Dan
IT4SOHO
0
 

Author Comment

by:suffolkdit
ID: 33789923
Dan,

That is correct.  What I was wondering was if Network Solutions would be relaying to the TLD servers that DNS1 is primary and DNS2 is secondary or if one has a precedence over another.

Charlie
0
 

Author Closing Comment

by:suffolkdit
ID: 33789936
Dan helped out tremendously.
0
 

Author Comment

by:suffolkdit
ID: 33790328
Dan,

Do you know if there is a way to see what alternative A records a computer is receiving as part of its DNS request?

Charlie
0
 

Author Comment

by:suffolkdit
ID: 33790447
Nevermind...see that I can do it with NSLOOKUP....

thanks again.
0

Featured Post

Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

Join & Write a Comment

One of the most often confused topics in the area DNS is the idea of GLUE records. Specifically, what they are, when they are needed, when they are provided, and how they are created. First, WHAT IS GLUE? To understand GLUE, you must first under…
Occasionally you run into the website or two that will not resolve properly using your own DNS servers.  Some people simply set up global forwarders for their DNS server.  I don’t recommend doing this because it can cause problems resolving addresse…
When you create an app prototype with Adobe XD, you can insert system screens -- sharing or Control Center, for example -- with just a few clicks. This video shows you how. You can take the full course on Experts Exchange at http://bit.ly/XDcourse.
This tutorial demonstrates a quick way of adding group price to multiple Magento products.

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now