mrwhipple
asked on
is there a robots.txt equivalent for the "no archive" meta tag
I need to stop Google from caching a lot of pages on a website. Is there a way I can tell google not to archive a whole directory or even the entire website without having to add the following meta tag to each page: <META content="noarchive" NAME="robots">. I would like to use the robots.txt file for this if possible.
ASKER
That would solve my issue but create another. I still want to allow google to crawl and index the entire site, but I don't want them to be able to cache it. i.e. there would be no "cached" link next to the search results for the site.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
User-agent: *
Disallow: /
Now, this means Googlebot won't index any of the pages in that site, so none of your internal pages will have a pagerank unless links exist to them from outside your website.