strongd
asked on
Virtual Hosting and Robot.txt
I have 2 websites running on my web server. I am running Linux and Apache.
My question is: How do I get my robots.txt files working for both domains?
Do I just both a robot.txt in each of the website root directories? If so, I have done that. But my second website still is not coming up. My first website is still showing old stuff for 6 months ago when I search in google for it. I have updated my site since then with major changes.
But the main one I am concerned about is the second website. Here is a week, it will be fully functional and I want it to come up in google or other.
Thanks...
My question is: How do I get my robots.txt files working for both domains?
Do I just both a robot.txt in each of the website root directories? If so, I have done that. But my second website still is not coming up. My first website is still showing old stuff for 6 months ago when I search in google for it. I have updated my site since then with major changes.
But the main one I am concerned about is the second website. Here is a week, it will be fully functional and I want it to come up in google or other.
Thanks...
As for your initial question, you need to have robots.txt in the root of each website. However, robots.txt will in no way help you get indexed faster. Go to Google Sitemaps and register your sites. Then download a utility to create sitemaps and submit them. That WILL help you get indexed faster.
ASKER
How does the spider know that there is 2 websites or to different directory to go to?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks alot for your help. Yes, I have 2 websites running on the same machine using virtual hosting. So I guess I need to go to google and do the sitemap thing.
Thanks again for your help. I think you answered my question.
Thanks again for your help. I think you answered my question.
My pleasure. Good luck!
ASKER
Thanks, hopefully I can get this thing going so I can get people to my second site. The first site was more or less a play toy/ learning thing...
Thanks again.
Thanks again.