Solved

using robots.txt on subdomain

Posted on 2011-02-28
6
481 Views
Last Modified: 2012-05-11
How do I set up a robots.txt on a subdomain for Google, Bing, Yahoo? I set up the subdomain outside the htdocs folder. I'm on a Linux server. So where htdocs is set at:
/htdocs

I set the subdomain at:
/subdomain-test

Do I simply place the robots.txt at:
/subdomain-test

Thanks,
Vkimura
0
Comment
Question by:Victor Kimura
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
6 Comments
 
LVL 2

Expert Comment

by:PortableTech
ID: 34996847
You are correct, just place the robots.txt in the root folder of the subdomain and you are all set.  
0
 

Author Comment

by:Victor Kimura
ID: 35002707
Hi, thanks.

So do I use this code:

User-agent: *
Disallow: /


or this one:

User-agent: *
Disallow: /subdomain-test
0
 
LVL 2

Assisted Solution

by:PortableTech
PortableTech earned 500 total points
ID: 35003136
Well, if it is a subdomain such as

http://subdomain.testdomain.com

then you would use

User-agent: *
Disallow: /

Otherwise, if it is actually a path name in the URL, you would want to specify as such.
0
MS Dynamics Made Instantly Simpler

Make Your Microsoft Dynamics Investment Count  & Drastically Decrease Training Time by Providing Intuitive Step-By-Step WalkThru Tutorials.

 

Author Comment

by:Victor Kimura
ID: 35003506
So could we use a path name like?

User-agent: *
Disallow: http://subdomain.testdomain.com/testpage.html

0
 
LVL 2

Accepted Solution

by:
PortableTech earned 500 total points
ID: 35021297
My apologies for the lack of response, this one slid under my radar for some reason.  What I meant in regards to a subdomain path was something like

http://subdomain.testdomain.com/test123

would need to be

User-agent: *
Disallow: /test123

If you only wanted that specific folder.  As far as the robots.txt is concerned, an indenepdant subdomain is the root start location so you would specify the path relative to the start of the subdomain.

Let me know if that does not make sense.


0
 

Author Closing Comment

by:Victor Kimura
ID: 35027186
Thanks, PortableTech. Yes, that makes sense. =)
0

Featured Post

The Orion Papers

Are you interested in becoming an AWS Certified Solutions Architect?

Discover a new interactive way of training for the exam.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

If you are a web developer, you would be aware of the <iframe> tag in HTML. The <iframe> stands for inline frame and is used to embed another document within the current HTML document. The embedded document could be even another website.
Owning a franchise can be the dream of a lifetime. It provides a chance for economic growth. You can be as successful as you want.  To make your franchise successful, you need to market it successfully. Here are six of the best marketing strategies …
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
An overview of how to create reports in Adobe Analytics (formerly Omniture Site Catalyst) using pageNames, events, eVars and props. This video will show you how to install the Omniture Debugger tool so can see (and test) what is being passed int…

688 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question