Link to home
Start Free TrialLog in
Avatar of strongd
strongdFlag for United States of America

asked on

Virtual Hosting and Robot.txt

I have 2 websites running on my web server.   I am running Linux and Apache.

My question is:  How do I get my robots.txt files working for both domains?

Do I just both a robot.txt in each of the website root directories?  If so, I have done that.   But my second website still is not coming up.  My first website is still showing old stuff for 6 months ago when I search in google for it.  I have updated my site since then with major changes.

But the main one I am concerned about is the second website.   Here is a week, it will be fully functional and I want it to come up in google or other.

Thanks...



Avatar of smidgie82
smidgie82
Flag of United States of America image

As for your initial question, you need to have robots.txt in the root of each website.  However, robots.txt will in no way help you get indexed faster.  Go to Google Sitemaps and register your sites.  Then download a utility to create sitemaps and submit them.  That WILL help you get indexed faster.
Avatar of strongd

ASKER

How does the spider know that there is 2 websites or to different directory to go to?
ASKER CERTIFIED SOLUTION
Avatar of smidgie82
smidgie82
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of strongd

ASKER

Thanks alot for your help.   Yes, I have 2 websites running on the same machine using virtual hosting.   So I guess I need to go to google and do the sitemap thing.

Thanks again for your help.  I think you answered my question.

My pleasure.  Good luck!
Avatar of strongd

ASKER

Thanks, hopefully I can get this thing going so I can get people to my second site.   The first site was more or less a play toy/ learning thing...  

Thanks again.