Hi,
i am hosted by bluehost.
I have a number of add-on-domain names such as "example.com" and the files for these domains are kept in subdirectories of my account such as "myaccount.com/example"
i create websites for people and sometimes search engines are finding their way to the subdirectories and subdomains such as "myaccount.com/example" and "example.myaccount.com" instead of simply "example.com"
is there a sure fire way to disallow search engines from indexing these subdirectories and subdomains?
will a robot.txt placed in the "myaccount.com/robot.txt" directory, that contains code to disallow search engines from indexing subdirectories (such as "myaccount.com/example") work? Or will that only stop search engines from following links from the root folder (myaccount.com).
Will i need some sort of .htaccess script. for example putting this in "myaccount.com/example"
"RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.thedomainyouwanttor
RewriteRule (.*) http://www.example.com/dir
?
Thanks for your help
Simon
This Question has been solved and asker verified All Experts Exchange premium technology solutions are available to subscription members.
Experts Exchange has been collecting answers to technology questions since 1996…3 million and counting! If you have a question, chances are we already have your answer.
If you can't find the exact answer you're looking for, ask our exclusive community of 50,000 experts. You’ll get a personalized answer from a trusted professional.
Thousands of free tech tips, tricks, how-to’s and tutorials are available in our peer reviewed articles section. See for yourself how smart our experts are, no login required.
Save solutions to your questions, answers you’ve discovered through searching plus helpful articles in your personal knowledgebase for easy future access.
Access the answers to your technology questions today.
30-day free trial. Register in 60 seconds.
Members of the expert community talk about why the experience at Experts Exchange is different than what you will find anywhere else.
Try it out and discover for yourself.
30-day free trial. Register in 60 seconds.
Join the community of experts here and help other tech pros by answering question in your area of expertise. You can earn FREE access to all Experts Exchange's premium features and resources.
As long as the robots.txt file is in the root folder of your website then search engines will find it.
The robots.txt file does not seem to work 100%. In particualr it can take some time before pages that are already indexed are removed.
Just to try and clarify your situation
http://myaccount.com = your main website
http://example.myaccount.c
http://myaccount.com/examp
And you want to stop search engines indexing the last one. Then your robots.txt file should be here:
http://myaccount.com/robot
and contain
Business Accounts
Answer for Membership
by: TiggeritoPosted on 2009-05-19 at 03:39:49ID: 24420332
If spiders are finding those urls there must be links to them. So the first thing is remove those links.
Your robots.txt file idea will help.
You could also register with Google Webmaster Tools where you can instruct Google to remove links that it has indexed.
https://www.google.com/web