[Webinar] Streamline your web hosting managementRegister Today

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 310
  • Last Modified:

Google to get results from internal database

Hi There,

I am developing a site that has loads of products in a database [stored on our dedicated server]
The client has requested that the products be returned when a user does a google search... I know this is posible as I have seen it done on many sites [including this one] i.e. if I where to go to google and search for "Google to get results from internal database" it would return this page.... somehow.....

Does it have to do with the way the site is developed?
Databse setup? Special Submission to the search engines?

Any help will be greatly appreciated.
0
d2kagw
Asked:
d2kagw
1 Solution
 
passmarkCommented:

Search engine spiders follow links that you have on your web site. The scans web pages as if it was a normal user browsing your site. Provided all your database content is accessible via web pages (ie. there is a URL to each record in your database), the spider mode indexing would work, regardless of the database backend you are using (MySQL, Postgres, MS Access, Oracle, etc.).

However, if there are portions of your database which can not be accessed via your web pages, then you may need to create a list of URLs to refer to these records in your database.

For example, you may have a database of cooking recipe. Each of these recipe are a record in the database, and are accessed via a PHP webpage which displays the recipe selected such as the following:

http://mysite.com/recipe.php?id=12
http://mysite.com/recipe.php?name=pavalova
etc.

We develop our own search engine spider software (for searching within a website) and give this advice to our customers, see here,
http://www.wrensoft.com/zoom/support.html#sql

David
0
 
gvmdevelopmentCommented:
Many search engines will not index dynamic pages (.php, .asp, cgi, etc or URLs that have ? in them) The best solution to get these type of pages indexed into search engines is to output them into .html pages and create links to them. These static pages would only be the entry point into the site and then everything else would be dynamic. It is then up to you how often you create or update these static pages.

You can either write an application to create these pages, or there are applications available which will do the dynamic to static page creation for you on a regular basis.
0
 
passmarkCommented:

> Many search engines will not index dynamic pages...

Well maybe. I don't know of any however.

This is not correct. Google *does* index dymanic pages. Doing a search for just about any topic in Google returns lots of dymanic pages. Google also indexes URLs with a '?'.

For example, doing a Google search for "Highlighters Pencils Pens", returns this link as number #1
www.4imprint.com/Products/ViewSuperGroup.aspx?sgrp=2
and this link as #3
http://eshop.msn.com/category.aspx?catId=987

Both are dymanic database driven sites with '?' in their URLs.

Maybe what gvmdevelopment means is that that these type of dymanic URLs are not always ranked as highly as other 'plain' URLs in some search engines. But this is often becuase the key words don't appear in the URL.

For example,
www.mysite.com/Highlighters-Pencils-Pens.htm

is going to look more relevent to search engine than

www.mysite.com/product.php?ProductId=123

David
0
The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

 
gvmdevelopmentCommented:
You are correct, Google does index these pages, but in many cases it looks down on such pages. The most important part like David said is that there be links to each and every product page, so that the search engine can find them.
0
 
d2kagwAuthor Commented:
Good Stuff, two more questions then:
[I have increased points to 400 too]

01.
Do you guys know of a good spider/robot that I could use to test the site [I need to make sure that particular areas are definatly found and others not found at all]

02.
David, you mentioned "creating a list of URLs to refer to these records in your database", would this mean just making a page that is filled with links like:
www.mysite.com/product.php?ProductId=123
www.mysite.com/product.php?ProductId=124
www.mysite.com/product.php?ProductId=125
www.mysite.com/product.php?ProductId=126
www.mysite.com/product.php?ProductId=127
if that was the case, how would you get the spider/robot to look at that page, without it being a hard link?

Thanks again guys !
0
 
passmarkCommented:

1/
My company sells the Zoom Search Engine that you could use to spider you site. But it is only free for small sites.
See, http://www.wrensoft.com and http://www.wrensoft.com/zoom/screen1.html
You just point it at your home page and hit the start button.

Otherwise there are some free alternatives on the web which may not be as easy to use (in my rather bias opinion).

2/
Yes. But here is your real problem.
Just because Google *could* index your site, doesn't mean it will. You will have the same problem as many other web designers. Namely, how do you optimize your site to get good placement (or even any placement at all) in Google? Google might also penalise or ignore pages that look like they have no significant content. So I don't think a straight list of links is going to look very interesting for Google. Whole books (and websites) have been written about placement optimization. Do a search for "Search Engine Optimization".

David
0
 
duzCommented:
d2kagw -

>Do a search for "Search Engine Optimization".

Don't do that because 99% of what you will read will be complete rubbish or out of date information. Search the "Online Marketing" channel here on Experts Exchange for more reliable and up to date answers.

- duz
0

Featured Post

The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now