?
Solved

Google and file placement

Posted on 2003-03-23
5
Medium Priority
?
189 Views
Last Modified: 2010-04-27
Hi

I have been developing my site whilst it is up and running, I thought I'd finish it and then submit it to the search engines thinking that no-one would be able to see it unless they guessed the url.

However, what I didn't realise is that google seems to have already picked up on it but only on one page which isn't the index and if a user clicks on that its right in the middle of a process and will make no sense.

Are there rules for where the index page and other pages should be placed on the server to prevent this happening?
0
Comment
Question by:jhazard
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
5 Comments
 
LVL 24

Expert Comment

by:duz
ID: 8193829
jhazard -

I assume this is because the search engine spider has followed an external link to your page and then indexed it. Is there an external link to that page?

The more pages of your site that are indexed the better - so don't worry about it.  Just remember Search Engines like Google don't rank web sites, they rank web pages.

- duz
0
 

Author Comment

by:jhazard
ID: 8193949
no, there is no external link to my site, no-one knows about it yet as it isn't finished.

How is it that other sites all seem to have their main pages listed and not a page from deep within their site.  How can I avoid that?
0
 
LVL 24

Expert Comment

by:duz
ID: 8194287
jhazard - No external link!? then I would have to look at the page that was indexed - post the url if you want to.

"How is it that other sites all seem to have their main pages listed and not a page from deep within their site."

Either because they don't bother with or understand SEO which is the vast majority. Or each page has been optimized for a different search term so that for a single search on key-word only the most relevant page on their site will show high in the rankings. Also Google tends to limit itself to two entries from the same site for any given search.

"How can I avoid that?"

Optimize each page for a different key-word (phrase).

- duz


0
 
LVL 24

Expert Comment

by:duz
ID: 8194405
jhazard - No external link!? then I would have to look at the page that was indexed - post the url if you want to.

"How is it that other sites all seem to have their main pages listed and not a page from deep within their site."

Either because they don't bother with or understand SEO which is the vast majority. Or each page has been optimized for a different search term so that for a single search on key-word only the most relevant page on their site will show high in the rankings. Also Google tends to limit itself to two entries from the same site for any given search.

"How can I avoid that?"

Optimize each page for a different key-word (phrase).

- duz


0
 

Accepted Solution

by:
BlaM earned 80 total points
ID: 8194811
I'll just post some links that might help you finding out more about your problem and maybe to "solve" it (in fact, I'm always happy if Google links to me - preferably to every sub-page. You could check the HTTP_REFERER to prevent any problems caused by this).


To prevent google (and any other search engine) from indexing parts of your page, you have to put an "robots.txt"-file to the root of your webserver:

Find out more here:
http://www.robotstxt.org/wc/robots.html

There's a good FAQ on google.com itself:
http://www.google.com/bot.html#robotsinfo

They even answer your exact question:
http://www.google.com/bot.html#secretserver
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

If you don’t want your company's site to fail on the web, you’d do well to observe these best web design practices and make sure you implement them when applicable.
Starting your own business is always a daunting process, and for most people it is brand new experience. Avoid the common pitfalls by following these tips to start on the road to success.
This tutorial walks through the best practices in adding a local business to Google Maps including how to properly search for duplicates, marker placement, and inputing business details. Login to your Google Account, then search for "Google Mapmaker…
Use Wufoo, an online form creation tool, to make powerful forms. Learn how to selectively show certain fields based on user input using rules to gather relevant information and data from your forms. The rules feature provides you with an opportunity…
Suggested Courses
Course of the Month9 days, 6 hours left to enroll

765 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question