Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

using robots.txt on subdomain

Posted on 2011-02-28
6
Medium Priority
?
503 Views
Last Modified: 2012-05-11
How do I set up a robots.txt on a subdomain for Google, Bing, Yahoo? I set up the subdomain outside the htdocs folder. I'm on a Linux server. So where htdocs is set at:
/htdocs

I set the subdomain at:
/subdomain-test

Do I simply place the robots.txt at:
/subdomain-test

Thanks,
Vkimura
0
Comment
Question by:Victor Kimura
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
6 Comments
 
LVL 2

Expert Comment

by:PortableTech
ID: 34996847
You are correct, just place the robots.txt in the root folder of the subdomain and you are all set.  
0
 

Author Comment

by:Victor Kimura
ID: 35002707
Hi, thanks.

So do I use this code:

User-agent: *
Disallow: /


or this one:

User-agent: *
Disallow: /subdomain-test
0
 
LVL 2

Assisted Solution

by:PortableTech
PortableTech earned 2000 total points
ID: 35003136
Well, if it is a subdomain such as

http://subdomain.testdomain.com

then you would use

User-agent: *
Disallow: /

Otherwise, if it is actually a path name in the URL, you would want to specify as such.
0
Sign your company up to try the MB 660 headset now

Take control and stay focused in noisy open office environments with the MB 660. By reducing background noise, you can revitalize your office and improve concentration.

 

Author Comment

by:Victor Kimura
ID: 35003506
So could we use a path name like?

User-agent: *
Disallow: http://subdomain.testdomain.com/testpage.html

0
 
LVL 2

Accepted Solution

by:
PortableTech earned 2000 total points
ID: 35021297
My apologies for the lack of response, this one slid under my radar for some reason.  What I meant in regards to a subdomain path was something like

http://subdomain.testdomain.com/test123

would need to be

User-agent: *
Disallow: /test123

If you only wanted that specific folder.  As far as the robots.txt is concerned, an indenepdant subdomain is the root start location so you would specify the path relative to the start of the subdomain.

Let me know if that does not make sense.


0
 

Author Closing Comment

by:Victor Kimura
ID: 35027186
Thanks, PortableTech. Yes, that makes sense. =)
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this blog, I will share you some basic tips for content marketing and to rank your website on Google.
Starting your own business is always a daunting process, and for most people it is brand new experience. Avoid the common pitfalls by following these tips to start on the road to success.
Use Wufoo, an online form creation tool, to make powerful forms. Learn how to choose which pages of your form are visible to your users based on their inputs. The page rules feature provides you with an opportunity to create if:then statements for y…
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial
Suggested Courses
Course of the Month11 days, 16 hours left to enroll

636 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question