• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 275
  • Last Modified:

Keeping a page away from Google

I like to put up a page I'm working on with a client so they can see the progress and we can have something to talk about.  I usually build the site initially in some obscure subfolder that no one is likely to stumble onto it by chance by entering www.mysite/theobscuresitefoldername and see it. But I know search engines crawl over everything on the web so it's possible, it seems, that the site in progress could still be found by Google and get indexed.  Is there something I can put on the site that Google, and othe rsearch engines will respect, and leave the site alone?  I know about nofollow but I t hink that's just something you can put on a link to keep Google from following it.  I need something that keeps Google out of the whole page.

Thanks.  
0
steva
Asked:
steva
3 Solutions
 
florjanCommented:
www.mysite place a file robots.txt and put in
User-agent: *
Disallow: /theobscuresitefoldername

Open in new window

0
 
MajorBigDealCommented:
Even if you tell search engines not to include something, they might still make a copy of it.  Also, there are search engines that provide the "feature" of allowing searches on things that are specially marked as disallow.   So when I want to keep something out of of a web page, I  use  a server side script that will not serve the page unless the user has provided a correct password.  
0
 
jaxbrianCommented:
you could also do this in the head of your page that you do not want to be indexed.
<meta name="robots" content="noindex">

if it is a folder you are trying to block with robots.txt then you need to have a
User-agent: *
Disallow: /theobscuresitefoldername/       //// the trailing slash is important to have
0
 
stevaAuthor Commented:
You guys all gave great information.  i split the points.

Thanks.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Get 10% Off Your First Squarespace Website

Ready to showcase your work, publish content or promote your business online? With Squarespace’s award-winning templates and 24/7 customer service, getting started is simple. Head to Squarespace.com and use offer code ‘EXPERTS’ to get 10% off your first purchase.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now