Determining the domain names within a certain TLD

Hi Omnranawad,

I don't think you understood the question. I don't need to know how to perform any of the tools offered by the URL you specified. All the tools provided there assume that you already know the name of the domain. What I am looking for is a tool that tells me what all the subdomains of a given top level domain are.

For example, I am looking for a tool that will give me a list of all .com domains, not one that tells me more about a specific .com domain.

The tool Does not exist - DNS has no provision for such a request.

Kinsey,

Thanks for that comment. Of course not really what I want to hear, but if that's the case then that's the case.

However, there certainly should be a way to determine this. After all, imagine I set up a DNS server as a secondary server for a TLD. For the secondary to do its job, certainly it must request details of ALL the subdomains - at least the SOA and NS records. So while a definite tool for this may not exist, the mere existence of secondary nameservers means that a mechanism for doing this must actually exist. So even if my question then changes to "How do I extract internal data from a running DNS Server", there has to be a way.

Secondary servers are idenified on the primary server and normaly only those servers are allowed access to the donwnload the doamin information. In fact they may be restriced to only downloading some subdomains.
'nslookup' has a command 'ls' to do what you want, IF you can get command prompt acces to the DNS server (primary or secondary)

It's called AXFR (full-zone transfer), and it's the same thing a secondary does when it needs to transfer full domain info from a primary.  It's what nslookup uses when doing an 'ls'.

Contrary to kinsey's comments, you do *not* need "command prompt" access (I take it that means "shell access" or something).  You *do* need explicit permission in /etc/named.conf (or wherever the DNS config files live).

Good luck obtaining such permission from ICANN.  Let me know how it goes.  I am understandably skeptical, but who knows - I never heard of anyone who tried.

You may want to look into all the wackos who advocate "alternate dns roots".  Surely they have a way to get this info, since they're supposedly running root servers that are not part of the cabal (or are they just set up to recurse onto the existing roots?)

What kind of research project are you doing exactly?  I bet there are sites theat are collecting the kind of stats you want, but I can't help until I know exactly what your project is.

Cheers,
-Jon

The Captain,

Bascically, I'm doing a research project for a legal concern that wishes to find domain registration pirates. As you know, these are guys that purposefully register domains of known trademarks and then try to sell them to the trademark owner. Essentially, the question being asked is as follows:

(1) Given Pirate X that is known to have registered domain xxx.yyy.com, what other domains has this person registered?

Now all I can think of is to check the whois databases for matching administrative contacts. The problem is that I am not aware of a WHOIS capable of searching by administative contact e-mail address, which leaves me no option but crawling the WHOIS databases for all known domains within the TLD of interest.

This is why I need to know how to retrieve the SLD's within a specific TLD.

Can you help ?

from Matt Zito on NANOG:

"I'm also not sure when it started, but its worth noting that ICANN requires that all registrars provide companies with bulk whois information for a fee which (IIRC) cannot be more than $10,000 US - its part of the registrar accreditation agreement."

Unfortunately, this excludes anyone who has:

bulk_whois_optout: N

in the registry entry for the domain.

There you have it - hope your research budget is accordingly fat.

-Jon

oops - that shuld read:

bulk_whois_optout: Y

I misread "optout" at "output"

-Jon

Uhmm cool - but what does this mean ?

It meand you can prob get what you want from the various registrars (NSI/Verisign, register.com, opensrs, etc.), but they are going to make you pay an arm and a leg for it.

Here is a bit of history, some of it dealing with bulk whois:

http://www.dnso.org/dnso/notes/20020122.rc01.4.html

These folks welcome inquiries about their bulk whois service, apparently...

http://www.easydomains.net/main/contactus.asp

I imagine most registars require a direct inquiry via phone or email.

-Jon

You may be able to get a break if you are associated with an accredited university.

-Jon

OK, I get this - I didn't even know there was a thing such a bulk whois. And it seems that the US$10,000 is really there to prevent people from easily harvesting the whois information for spamming purposes. Pity that spammers affect people just doing honest and relevant research :(
Thanks for this info.

Of course, at US$10000 annually!, I'm not likely to want to follow this route. So I really am back to the approach of where I need the list of registered domains. I've been able to AXFR a couple of second level domains and then use ls -t NS sld. So now the question is, how do I get an AXFR for the big tld's like .com, .net and so on. I realise that this is controlled through named.conf (the permissions section), but is there no other way to get this info ?

Survey says?