?
Solved

using robots.txt on subdomain

Posted on 2011-02-28
6
Medium Priority
?
490 Views
Last Modified: 2012-05-11
How do I set up a robots.txt on a subdomain for Google, Bing, Yahoo? I set up the subdomain outside the htdocs folder. I'm on a Linux server. So where htdocs is set at:
/htdocs

I set the subdomain at:
/subdomain-test

Do I simply place the robots.txt at:
/subdomain-test

Thanks,
Vkimura
0
Comment
Question by:Victor Kimura
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
6 Comments
 
LVL 2

Expert Comment

by:PortableTech
ID: 34996847
You are correct, just place the robots.txt in the root folder of the subdomain and you are all set.  
0
 

Author Comment

by:Victor Kimura
ID: 35002707
Hi, thanks.

So do I use this code:

User-agent: *
Disallow: /


or this one:

User-agent: *
Disallow: /subdomain-test
0
 
LVL 2

Assisted Solution

by:PortableTech
PortableTech earned 2000 total points
ID: 35003136
Well, if it is a subdomain such as

http://subdomain.testdomain.com

then you would use

User-agent: *
Disallow: /

Otherwise, if it is actually a path name in the URL, you would want to specify as such.
0
Visualize your virtual and backup environments

Create well-organized and polished visualizations of your virtual and backup environments when planning VMware vSphere, Microsoft Hyper-V or Veeam deployments. It helps you to gain better visibility and valuable business insights.

 

Author Comment

by:Victor Kimura
ID: 35003506
So could we use a path name like?

User-agent: *
Disallow: http://subdomain.testdomain.com/testpage.html

0
 
LVL 2

Accepted Solution

by:
PortableTech earned 2000 total points
ID: 35021297
My apologies for the lack of response, this one slid under my radar for some reason.  What I meant in regards to a subdomain path was something like

http://subdomain.testdomain.com/test123

would need to be

User-agent: *
Disallow: /test123

If you only wanted that specific folder.  As far as the robots.txt is concerned, an indenepdant subdomain is the root start location so you would specify the path relative to the start of the subdomain.

Let me know if that does not make sense.


0
 

Author Closing Comment

by:Victor Kimura
ID: 35027186
Thanks, PortableTech. Yes, that makes sense. =)
0

Featured Post

WordPress Tutorial 3: Plugins, Themes, and Widgets

The three most common changes you will make to your website involve the look (themes), the functionality (plugins), and modular elements (widgets).

In this article we will briefly define each again, and give you directions on how to install them.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

For many of us, the  holiday season kindles the natural urge to give back to our friends, family members and communities. While it's easy for friends to notice the impact of such deeds, understanding the contributions of businesses and enterprises i…
Choosing a core focus or particular set of features and options can be tough. To help out, we’re going to highlight a handful of things your business needs on one of your social media pages. In other words, if one of these is missing, you should imp…
Learn how to set-up custom confirmation messages to users who complete your Wufoo form. Include inputs from fields in your form, webpage redirects, and more with Wufoo’s confirmation options.
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.
Suggested Courses
Course of the Month9 days, 2 hours left to enroll

764 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question