I have faced a weird problem few times in past year, which I do not understand exactly.
I have server "farm" for webhosting, where all domains rely on 3 DNS servers. 2 of them are "real" servers, running each in separate datacenter for redundancy, while 3rd DNS is meant as last resort, in case of ....something. So I thought it is not so important where it is located, and I put it onto my home-server as a virtual machine. It runs 24/7/365 and has more than 99,9% uptime.
All 3 DNS servers are on Server 2008R2 technology.
All domains have ALL 3 DNS registered as their DNS servers at registrar.
Now, if ONE of those DNS should fail, all domains should still resolve fine.
Even if 2 of 3 DNS fail, domains should still resolve. Right?
All this seems fine, but HUGE problems arrive upon rebooting or reinstalling some app on my home-server, and for some reason virtual machine does not start. As it starts most of times, I do not check it...because it 98% of times starts up normally.
But in those rare 2% cases, DNS 3 dose not start up...
...and after domain TTL expires, almost half of domains DO NOT RESOLVE ANYMORE! Not all of them, but random significant part.
I check DNS zones and records on primary DNS1 and secondary DNS2, and all records are there and manual resolving fine, if I query DNS server directly.
But most of internet DNS cache around the world is simply empty, without a clue about those domains.
Why is that happening?
Why do not public DNS cache get filled with zone records from the rest of working DNS1 and DNS2 servers?