How to make site invisible to search engines

ald413
ald413 used Ask the Experts™
on
Hello Experts,

I was wondering if there was a way to make a site not indexed in search engines and to not show up on google searches, without making the page private.

Thank you for your help
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Simply put the robots.txt file in the root.  Create a text file named robots.txt and insert the following:

User-agent: *
Disallow: /



More info at:  http://www.robotstxt.org/
The best way to accomplish this is with a meta "noindex,nofollow" command
<META NAME="ROBOTS" CONTENT="NOINDEX, FOLLOW">

This is better than the robots.txt because while search engines will still access your pages, they won't be indexed and won't show up in search results. Typically if you do a robots.txt file, search engines won't index your pages but they will still show your site for some search results. This is because they know the pages are there (through links pointing to the pages - they can get a sense of what is on the page through anchor text); you are telling them not to access them with the robots.txt. The meta robots command explicitly tells robots not to index/show the page

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial