Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 301
  • Last Modified:

search listing GOOGLE

How long does it take for googlebot to get your site listed / indexed ?

The bot has been 4 (apparently) times but never gone further than the index page. The site is dynamicallty driven PHP = MySQL and I have been reading a lot of informatuion regarding OTHER bots not searching URL strings containing ? and  or &

Can anyone provide any help on this.

Greatly appreciated.

1 Solution
For all the fastest information on Search Engine Optimization and google. It is best to go to http://www.webmasterworld.com

They are the world's best forum for experts in SEO and Search Engines to meet and discuss everything from Google, to Overture and all that.

Also searchenginewatch.com is a great resource as well.

Yes it depends, on a how many variables you have on the url.

There are ways around it depending on the web server you have, and your language.

The trend is to go away from index.php?x=2&y=3
towards index.php/x/2/y/3 or index.php/x=2/y=3
depends on the code used.

and how it can interact with the web server in terms of what data is available.

Google will index page.php?argument quite well... But won't go into too much details for page.php?argument=something&another=whatever.

Example can be seen here (for my site):

I get that result only after clicking view all similar pages. So basically it proves that Google does cache dynamic pages very well, however it chokes on the more important pages with more than two arguments (eg.: file.php?arg1=something&arg2=whatever&arg3=stuff won't get indexed - 3 arguments).

A good way to still index your PHP pages with more than two arguments is to use mod_rewrite to simulate a static page to the client. Most hosts enable mod_rewrite.

Documentation on mod_rewrite can be obtained here:
There are three popular methods for changing your URL's to help search engines which are all discussed at the following site:
What I have once read, is that Google will not follow links from those dynamically generated pages. This is quite logical since Session-IDs and such stuff would cause an endless amount of pages.

Because of this I have one "non-dynamic" page as my start page (/index.html) which redirects normal browsers to the real start page via "meta"-command, javascript or (if both noth available) a click of the user. This page is rather small (about 1k) and does not affect normal users that much (anyway, they normally start at "subpages", coming from search engines - so that's okay). Normally you don't even see that page (check out http://www.4cheaters.de/index.html to see that). This page contains links to some other "hardcoded" pages. These are automatically generated every night and contain links to all dynamically generates subpages of my site (http://www.4cheaters.de/games.html - note that this page also contains the redirect, so don't really expect something else to show up ;).

These pages will not confuse any search engine (even if they try to access it with another Browser-ID (to find out if I'm spamming)) and they will redirect them to every subpage.

I'm quite successful with this.
Hello Jim,

I have a site also that is written entirely with php and have looked up a lot of information on the subject. In regards to Google, the session id is the main issue from what I have found. You will find a lot of information in regards to rewriting the url string, but I have never done this myself and have never had a problem with Google crawling my entire site, which is only about 1,400 pages.

For a bigger example of this you might want to search google for phpbb.com, which is a php forum and the entire site is dynamic and the url is not rewritten. At google search for allinurl:phpbb.com and this will list all pages indexed by google. You will notice the results are over 19,000 pages, which would be more than most sites would ever need indexed. I would suggest looking into removing the session id's and seeing if that fixes the problem. An excellent resource I found was also on the phpbb site. It details how to allow sessions for regular users but exclude them for search bots. I use this framework to remove the id's from the other scripts I have running.

I hope this helps!

Featured Post

[Webinar] Database Backup and Recovery

Does your company store data on premises, off site, in the cloud, or a combination of these? If you answered “yes”, you need a data backup recovery plan that fits each and every platform. Watch now as as Percona teaches us how to build agile data backup recovery plan.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now