Solved

DNS TTL VALUE

Posted on 2010-09-14
21
5,356 Views
Last Modified: 2012-05-10
I need an expert advice about DNS TTL Values.
My understanding about TTL value is the max time the the record can be cached in all other non-authoritative DNS Servers.
If I am changing an entry and making the TTL the min, is it has any impact in propagaition time of the record to other DNS Servers. I will explain - if I need to refect the new entry as fast as possible, any workaround like reducing the TTL?  How it is working.

Thanks,
Peter
0
Comment
Question by:anishpeter
  • 6
  • 6
  • 4
  • +3
21 Comments
 
LVL 3

Expert Comment

by:blaslett
ID: 33669660
Minimum TTL is usually 500 (seconds).
Best practice is to set MX and A records with minimum expiry, that way changes can be updated in a reasonable timeframe.
As far as I know there is nopw downside to reducing you TTL value.
In terms of the cahanges you have just/are making, unfortunately you cannot change the current value of the TTL to make it expire quicker, you will just have to wait it out. Howvere future records with short TTL values will enable you to make changes quicker.
0
 
LVL 3

Expert Comment

by:sayed_maher
ID: 33669706
anish,

IMHO the best practice for DNS, _IF_ you have the bandwidth available to constantly go online and get the records, is to set the time to live to a half day. If you are using unbound for example, and you are caching dns records, unbound takes into consideration round robin records; So if you have a site like CNN.com which has like 30 records, which it rotates, a single cache will be good for a long time. If you are trying to save bandwidth aswell as speedup your dns resolve, the best idea is to cache the record for atleast 4 days, because after an indepth study (which I conducted for work, at an ISP) i found that records dont change all that often, but to be on the safe side, I only cache for 4-6 days). Furthermore, I run a script on the 4th or 6th day, (depends on what I set my TTL to) to go online and update the records for the most visited sites. That way when our users request a lookup, its already _fresh_ and in the cache, which means faster response times, and happier people. I would personally suggest unbound, since we are having such success with it. http://www.unbound.net/

Good luck.
0
 
LVL 1

Author Comment

by:anishpeter
ID: 33669749
Hi.. Blaslett,
I agree with you.
But since the reduced TTL value  will give more burden on DNS Server ,most of the providers have min TTL. Yahoo now not allowing you to change TTL. Network Solutions min TTL 3600s.
About the new changes for records, I agree that for reducing the propogation delay, we cannot do anything. But I heard, even experts saying, if we reduce the TTL  after changing the record, the propogation time will be reduced. Please check

Peter
0
 
LVL 1

Author Comment

by:anishpeter
ID: 33669822
Hi.. Sayed,
But you dont mention about relation of TTL for changed record. How propogation time related to TTL

Peter
0
 
LVL 3

Expert Comment

by:sayed_maher
ID: 33670034
Peter,

In reality the propogation time is not static, it is different for every record on the internet. Some records are set to be valid for 1 year and they were set 364 days ago, that means if you are caching for 4 days and you cache today, tomorrow that record is invalid. What unbound does is once a lot of invalid request are returned, it automatically goes online and gets the new record. How much is alot you might ask, the answer to that is more then 5 I believe. I modified the source and I cant remember the original value. Now for _most_ of the internet, 4-6 days is safe TTL and your propogation time most likely will be half of that, since it takes 24-72 hours for the DNS to be updated internationally anyways. Its a mathematical probability equation really. In reality I think that 4-6 days is safe for an amount greater then 78% of the websites on the internet. One thing I noticed when you answered blaslett though is I sort of got the drift that you are a web master and not necessarily a network admin. If that is the case, Im not sure if my answer is still valid because you are not in control of how long the ISP's will cache your DNS for. As for how propogation time is related to TTL not complicated or anything, its rather unbalanced and unstatic. For example if you update your records today, and it requires 24 hours for them to be updated internationally, and i decided that at exactly 23 hours and 59 minutes to update my records, and my TTL is 500 seconds say, that means that for 7 minutes your site will not be accessible to my users. Furthermore, If its 72 hours before they update internationally, that means that it may not be accessible for 1 day and 7 minutes, etc. etc. Unbound fixes this problem like I mentioned above. Keep us posted!

Good luck,
0
 
LVL 11

Expert Comment

by:Ben Personick
ID: 33670060
@Alaslett, I'm pretty sure the Minimum TTL is either 300 seconds or 60 seconds, at least in Microsoft's implementation of DNS.

@Anishpeter you have confused two DNS terms which are completely unrelated. Let me see if I might clear this up for you:
TTL (Time To Live) is the time that a DNS Client (such as your local computer) Should** catch the record before considering it stale and querying the record again.

  A very short TTL on a DNS record tells the computer which queried DNS that the record is likely to change more frequently so it woudl be a good Idea to check back soon.  (of course a client can always force a refresh)
 
 
**  Note: I say "Should" because a TTL is only a suggestion of how long to keep the DNS record before the device which is requesting the DNS record shodul request it again.  because of the way DNS Works the device which is requesting the record can be separately configured to ignore the TTL value and instead check DNS More Frequently or less frequently.

 Propagation of DNS Records: This ONLY relates to the records for the DNS servers themselves which are being linked to the domain you are working with.

  Propagation ONLY takes place when you add, chance, or remove a Domain controller record from the root domain servers through your registrar, and has nothing to do with the host records you setup for your website or mail server etc. on your DNS servers or even in your registrar's DNS servers.  Those host entries are governed by "Time To Live" only.

  This is done through the registrar who holds your domain validation (the people you bought the domain from such as Go Daddy, or Network Solutions)

  The Propagation period for changes to your DNS Servers is generally 24 to 48 hours, and is almost never an issue that you will need to worry about after you initially setup your domain controller records.  The Propagation time is the time it takes to be certain that all the major root DNS Servers have appropriately picked up your DNS servers as being the owners of YourDomain.com, varies because there are many Root domain servers and they are all configured to check very infrequently for updates from registrars as to what domain names are currently available and where the DNS servers for their sub domains and hosts exist so that the chance for DNS Cache Poisoning is reduced.

0
 
LVL 70

Expert Comment

by:Chris Dent
ID: 33670528

> (of course a client can always force a refresh)

A client uses a server. Most DNS clients on computers are stub-resolvers, they cannot find records without a server to help out (incapable of perforning Iterative queries).

Each resolver in a chain will typically cache an answer. Including:

1. The client (client-side cache)
2. The local DNS Server (the server the client is configured to use)
3. A Forwarder (if one is used)

In each case, TTLs will be inherited for cached entries. That is, if you have this system:

  Client  <-->  Local DNS  <-->  Forwarder

Local DNS will inherit TTL values from Forwarder, and the client from Local DNS. To clear out an entry from Client you would also have to remove it from Local DNS and Forwarder.

> Propagation ONLY takes place when you add, chance, or remove a Domain controller record from the root
> domain servers through your registrar

Propagation isn't a DNS specific label, in context it refers to the gradual dissemination of information. That term can be applied to both new DNS records as well as changes to existing records.

This process is *not* limited to "root servers".

> The Propagation time is the time it takes to be certain that all the major root DNS Servers have appropriately picked up
> your DNS servers as being the owners of YourDomain.com

No, it really isn't.

The Root Servers hold directions to TLD (Top Level Domain) Servers. YourDomain.com will *not* be listed on the Root Servers at all.

Top Level Domain Servers are, for the most part, geograpically dispersed collections of AnyCast DNS clusters run by nominated organisations. They do hold records for YourDomain.com. In the case of .com you're looking at Verisign (responsible for a-m.gtld-servers.net).

There's a list of those here:

http://www.iana.org/domains/root/db/

Propagation of updates within that system (the TLD) is typically very fast. It's not unreasonable to see changes propagate there in less than a couple of hours. However, that depends on the organisation in use.

Propagation of a change to other servers on the Internet is dependant on two things:

1. The Record TTL
2. A minimum cache TTL

Servers like MS DNS don't offer the ability to set a minimum cache TTL, but others do. You typically see minimum cache times in use on very large service providers (AOL, for example).

In short:

Propagation time is driven by the TTL in the majority of cases. If you set a very low TTL a change will propagate more quickly. This statement is correct:

> My understanding about TTL value is the max time the the record can be cached in all other non-authoritative DNS Servers

Just remember that TTL changes are not retroactive, you must set a new TTL in advance; allowing time for the current TTL to expire anywhere that might be holding on to it.

Chris
0
 
LVL 70

Expert Comment

by:Chris Dent
ID: 33670674

I wanted to add a few commands that let you explore TTL values (both on authoritative and non-authoritative servers).

To see the remaining TTL on a non-authoritative answer:

nslookup -d experts-exchange.com.

The -d flag (debug) is the key here, it shows us debug level output (there's a verbose form, -d2). And note the trailing "." after the name. That's to prevent nslookup appending a suffix search list, simplifies our output.

The output for that should include something like this at the end:

------------
------------
Got answer:
    ....
    ANSWERS:
    ->  experts-exchange.com
        internet address = 64.156.132.140
        ttl = 406 (6 mins 46 secs)

The TTL here is the TTL remaining on the server we asked. Once that has expired the server will have to go off and ask another DNS server about the record (either as a recursive query to a Forwarder, or an Interative query process via the root servers). If the record has updated we "should" see the updated data.

I want to introduce two different tools at this point, Dig and DnsShell. The first is a DNS tool distributed with BIND and typically found on Unix / Linux systems. You can grab a Windows version of it here:

http://members.shaw.ca/nicholas.fong/dig/

DnsShell is my own tool set, but it will also allow us to go a bit deeper than nslookup does. It really doesn't matter if you use this or Dig above.

http://code.msdn.microsoft.com/dnsshell

We can follow the public hierarchy from root to the domain record as follows:

dig experts-exchange.com a +trace

Or:

Get-Dns experts-exchange.com a -Trace

In both cases we're performing an Iterative query for the A (Host) record, querying each DNS server in the chain individually to get an answer. The output format differs, but you should see that the final Answer includes a TTL, and that the value is 3600 (seconds), or one hour.

This is the basis for the propagation time. At any point in time a remote DNS server may have just picked up a copy. If the record subsequently changes the remote server will not see the change until the TTL has expired.

DNS is fun :)

Chris
0
 
LVL 57

Expert Comment

by:giltjr
ID: 33671545
As QCubed the TTL for a DNS entry is more of a "suggestion" than a hard and fast rule.

Most places setup their DNS server's to honor the TTL, but I have seen some places where a caching only DNS server will be setup to ignore the TTL and cache entries for as long as 72 hours.

0
 
LVL 70

Expert Comment

by:Chris Dent
ID: 33671654

I don't disagree that it is technically possible to implement a minimum cache TTL type setting. However, very few DNS servers do (in many cases simply because no such option is exposed), and attempting to support and account for those would seem to be pretty futile.

The TTL should be taken as an indication of total propagation time if only because that's how it's supposed to work. Servers that choose to ignore TTLs and apply their own minimum do so at their own risk.

In my opinion, of course :)

Chris
0
What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

 
LVL 11

Expert Comment

by:Ben Personick
ID: 33672016
@Chris, of course The Root DNS servers Pont to the TLD servers and so on and so forth.

  It is only the iteration your DNS servers fall under that have to be propagated within.  I was simply trying to paraphrase instead of getting too nitty-gritty for the Author.

  Yes, I agree TTL affects the catch on any resolver (AKA DNS client), Please don't mistake the usage of DNS client to mean only client PCs.  Any point in the resolution chain of DNS is considered a DNS Client when it queries recursively and will catch either according to TTL, or it's own parameters. (I had even written "or your DNS server's catch, etc." In the asside but struck that portion as I felt it would confuse the differentiation I was trying to convey to the author in the simplest terms possible)

  TTL can only affect the propagation process of your connection to yourdomain.com to your DNS server when you have access to the DNS where your servers have been inserted, generally the registrar does not allow that level of access.

  Even if they did, generally the servers will only be checking in to the registrar for new entries at set intervals to minimize traffic and complexity of the system, and would probably ignore a TTL.


  Where as if we are speaking about a child domain then he can completely skip this process by forcing DNS transfers from his parent servers to his replica servers and TTL would play little involvement in this process, because TTL is only applicable to each step of the opperation.  A host www.My.Domain.com. may have a different TTL at every level, in the case of the resolver catch for www only the final hop plAys a direct Role in the original requesting client's Evaluation of TTL.  That is to say . Has a TTL for com, com has a TTL for Domain, domain has a TTL for My, my has a TTL for www.

Etc. Etc. Etc. Ad infinitum ad nauseum

I'm posting this because I think you were under the impression that I did not understand DNS at all, while in fact I am attempting to simplify the 'solution' to be at a level the author will feel more comfortable with, and try to keep the responce as short as possible.

^^

-Q
0
 
LVL 70

Expert Comment

by:Chris Dent
ID: 33672550
I'm sorry, but I can't respond to anything but your last statement, I'm having a lot of difficulty making sense of most of your reply. I can only really reference the last part.

> I am attempting to simplify the 'solution'

This is the simple / short response:

Changing the TTL on the Record is the correct way of speeding up propagation time for that record.  You only need to allow enough time for the existing TTL to expire.

Chris
0
 
LVL 1

Author Comment

by:anishpeter
ID: 33672700
Hi.. All,
I Hope The discussion became knolegable for all. Thanks for everyone. I want to Clarify one of the comments of Chris
"Just remember that TTL changes are not retroactive, you must set a new TTL in advance; allowing time for the current TTL to expire anywhere that might be holding on to it."
I mean setting a reduced TTL value at the time of changing a record wont benift for reducing the propogation delay.(almost 48-72 hours) If we think to change a record after let say 2 days, reduce the TTL to min possible today itself.
Can evryone agree with this point?

Thanks,
Peter
0
 
LVL 70

Accepted Solution

by:
Chris Dent earned 300 total points
ID: 33672964
Hi Peter,

> I mean setting a reduced TTL value at the time of changing a record wont benift for reducing the propogation delay

Correct.

Imagine we had this record and I wanted to change the IP:

Name: www.bob.com
IP: 1.2.3.4
TTL: 2 Days

I would follow this process (based on todays date):

14/09/2010 - Reduce TTL to 15 minutes (900 seconds, example value only)
16/09/2010 - Change the IP address
16/09/2010 - Increase the TTL back to its original value

Allowing 2 days for the current TTL to expire ensures that everyone has the copy of the record with the 15 minute TTL. After the change the TTL can go up again, we only needed the old version to flush out quickly.

Chris
0
 
LVL 11

Expert Comment

by:Ben Personick
ID: 33673165
hey sorry if I'm ush-mouthed first thing in the morning.

My point was that I was trying to write in simple terms that there is a difference between propagation of the canonical  name servers and Time to Live that you will set on your records in those name servers.

I believe the author is confusing these two items.

  As he will likely not be able to directly control 'propagation' as the term generally refers to the propagation of the Name Servers for a particular domain.

  he will be able to suggest to any DNS Client to catch his host record for only the time specified in the time to live.


I think where you and I are having a miss communication is that I am a very visual person I can quite clearly picture the process, but perhaps my description is leaving something to be desired, my attempts to were to underscore the differences between why one woudl matter more than the other and why propagation is a term used for the holder of sub elements not a sub element itself.

  Does that help clear things up?

 I imagine not, but I can hardly draw a diagram and speak to you so I'm making my best attempt in the form of words alone.  it's particularly hard because DNS resolution is both an iterative and recursive process.

  Lets see if this helps get my point across?

(Please not I'm not posting this because I think you don't understand it, but instead because I assume you're having a "who the hell is this guy" moment about myself)

Cleint-->hosts-->locally configured dns server-->forwarding server-->...-->Resolving Server-->(Recursive solution path)


(simplified)
client-->server-->(Path)

(path expands as:)
Client(WWW.Subdomain.domain.com.)-->Server(ComNS?)-->Root(Com=IP)-->Server(DomainNS?)-->COM(Domain=IP)-->Server(SubDomainNS?)-->Domain(SubDomain=IP)-->Server(Host)-->Subdomain(Host=IP)-->Server(WWW=IP)-->Client

-- However root only catches com/net/org etc com only catches top level domains in com.  the Author's Domain likely exists as one of these.

-- Also most  DNS servers in the chain through to the DNS Server which acts as the resolver should have catching of queries disabled to mitigate bad requests

-- The only likely exception to this will be the locally configured DNS server on the client unless this is the user's ISP's DNS Server or some other server not in his control.

-- Do only the client machine and the DNS catch at the next hop actually listen to "TTL" on the host record. and they only "should" they do not "HAVE" to

-- TTL has nothing to do with secondary servers catching the values of the DNS records of the master server. Replica (Secondary) servers are configured to query record sets from the master at a set interval, and the master is usually set to force a DNS replication when a change is made to it's set on the level of the user's domain.

--Propagation is actually what occurs when the primary COM servers query the registrars to see if there are changes to what sub-domains of com are registered as valid to exist, and any changes to who holds which sub domains of COM at that particular registrar, and then transfers that data to each of it's secondary servers.

propagation in general is always in reference to (a/several) primary servers updating their secondary servers and secondaries updating tertiaries and the like.  However inn specific it is used primarily in the function of the registration of the DNS servers which are canonical to Top Level domain (COM/NEt/ORG/CO/Etc).

The process of Propogation in terms off the DNS servers associated with the domains below COM is lengthy because of the size of the digest needing to be processed, and in order to forestall any malicious or accidental poisoning the DNS record sets of the TLDs (Top Level Domains)

I believe that will better outline my point for everyone but it is far more verbose.  put simply.  TTL and Propagation have practically nothing to do with each other..




0
 
LVL 11

Assisted Solution

by:Ben Personick
Ben Personick earned 100 total points
ID: 33673506
@qtljr  I'm getting too caught up in trying to define the difference between propagation and TTL Catching so here goes a simple answer to your Q instead:

Your host records TTL is a suggestion to whoever will need o use them on how long to hold onto them.  you cannot be assured 100% of hosts following it but as a rule they generally do, so we will assume they do for our purposes.

Generally I set TTL on public domains in the 4 to 8 hour region to keep the queries for them to a minimum but keep the sight IP changing flexible as kin my experiences you often need to make changes on short notice.

assume you have MX Record of SMTP1.YourDomain.Com for yourdomain.com and an A Record of 192.168.172.5 for SMTP1.YourDomain.com

  right now for this site your TTL for the MX address may be 48 hours, and for the A Record it may be 4 hours.  which means that a mail server looking to send you email will likely not bother to check for your email address for 48 hours.  Obviously in the case of email this can bee a large problem if you need to change your sending or receiving server.

So if you change your TTL today for the MX record to be 5 minutes then, in 48 hours every mail server looking to send emails to you should check back to see if the URL of your SMTP server has cchanged from SMTP1.Yourdomain.com IF it HAS NOT CHANGED then they will only query the IP address of your A Record for changes every 4 hours.

the above is a great way to go when you may want to change WHICH SMTP server mail will flow to.

Likewise if you leave the TTL of the SMTP server at 48 hours, and change the A Record to 5 minutes, then the mail server sending to your domain will only check to see if the URL of your SMTP server has changed about once in 48 hours if it needs to send you mail at that time.  however every five minutes if the mail server needs to send mail to you it will check the A Record for SMTP1.YourDomain.com to see if the IP adddress has changed.

the precedence method is very useful if you plan to change the IP of your existing mail server, or if you want to change the IP address of any A Record such as www
by using the above methods you can tell client machines to handle their DNS Resolver Caches in a way which will bee useful to you should you make changes.

Once your change is made you can set your DNS records to higher value again.  I suggest setting the MX record to 24 hours and the host record to 4 hours because you arre likely to have at least 24 hours notice before you will move a mail server to a differentt URL  so that you can change the TTL to 5 minutes and wait 24 hours for that change to be the configured TTL held by all DNS clients before changing your URL and setting things back while still want to be able to keep light on your feet about changes to the IP adddress of your mail server on a short notice if you are doing some sort of DR which may require an IP change.

Note: I use the Mail server as an example because I'm guessing that's where your query comes from because most administrators keep their host records in a smaller time period but become worried at the mere thought of needing to move their mail servers in anyway.


0
 
LVL 70

Expert Comment

by:Chris Dent
ID: 33673851

If you prefer we can set up a private discussion, or indeed another thread to discussion the use of terms. However, I'm not going to pursue the issue further within this thread, I suspect you're correct that it's nothing more than a misunderstanding :)

I believe it is enough to say that reducing the TTL is the right course of action as it is the only metric that allows a DNS administrator to influence caching time on third-party servers.

Chris
0
 
LVL 11

Expert Comment

by:Ben Personick
ID: 33674189
Cool, agreed on all accounts :)
0
 
LVL 3

Assisted Solution

by:blaslett
blaslett earned 100 total points
ID: 33678042
@Chris-Dent - you are correct. As I tried to keep simple in my original replay.
@everyone else - Jeez, way to complicate a simple question!

Reduced TTL allows for quicker future propogation.

You have to wait out for the the current TTL to expire for the initial change to take effect.

Problem solved!
0
 
LVL 1

Author Comment

by:anishpeter
ID: 33679013
Thanks for Chris and Blaslett for summarising the ideas and I think we can close the discussion.  
Many Thanks for QCubed for sicere efferts to make this discussion Knoledgable for everyone.

Thanks,
Peter
0
 
LVL 11

Expert Comment

by:Ben Personick
ID: 33684565
you're welcome and many thanks for the points!
0

Featured Post

6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

Join & Write a Comment

Suggested Solutions

Creating an OSPF network that automatically (dynamically) reroutes network traffic over other connections to prevent network downtime.
Even if you have implemented a Mobile Device Management solution company wide, it is a good idea to make sure you are taking into account all of the major risks to your electronic protected health information (ePHI).
Viewers will learn how to connect to a wireless network using the network security key. They will also learn how to access the IP address and DNS server for connections that must be done manually. After setting up a router, find the network security…
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now