Solved

Google to get results from internal database

Posted on 2004-08-11
7
304 Views
Last Modified: 2007-02-13
Hi There,

I am developing a site that has loads of products in a database [stored on our dedicated server]
The client has requested that the products be returned when a user does a google search... I know this is posible as I have seen it done on many sites [including this one] i.e. if I where to go to google and search for "Google to get results from internal database" it would return this page.... somehow.....

Does it have to do with the way the site is developed?
Databse setup? Special Submission to the search engines?

Any help will be greatly appreciated.
0
Comment
Question by:d2kagw
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
7 Comments
 
LVL 3

Expert Comment

by:passmark
ID: 11779786

Search engine spiders follow links that you have on your web site. The scans web pages as if it was a normal user browsing your site. Provided all your database content is accessible via web pages (ie. there is a URL to each record in your database), the spider mode indexing would work, regardless of the database backend you are using (MySQL, Postgres, MS Access, Oracle, etc.).

However, if there are portions of your database which can not be accessed via your web pages, then you may need to create a list of URLs to refer to these records in your database.

For example, you may have a database of cooking recipe. Each of these recipe are a record in the database, and are accessed via a PHP webpage which displays the recipe selected such as the following:

http://mysite.com/recipe.php?id=12
http://mysite.com/recipe.php?name=pavalova
etc.

We develop our own search engine spider software (for searching within a website) and give this advice to our customers, see here,
http://www.wrensoft.com/zoom/support.html#sql

David
0
 
LVL 1

Expert Comment

by:gvmdevelopment
ID: 11780372
Many search engines will not index dynamic pages (.php, .asp, cgi, etc or URLs that have ? in them) The best solution to get these type of pages indexed into search engines is to output them into .html pages and create links to them. These static pages would only be the entry point into the site and then everything else would be dynamic. It is then up to you how often you create or update these static pages.

You can either write an application to create these pages, or there are applications available which will do the dynamic to static page creation for you on a regular basis.
0
 
LVL 3

Accepted Solution

by:
passmark earned 400 total points
ID: 11780459

> Many search engines will not index dynamic pages...

Well maybe. I don't know of any however.

This is not correct. Google *does* index dymanic pages. Doing a search for just about any topic in Google returns lots of dymanic pages. Google also indexes URLs with a '?'.

For example, doing a Google search for "Highlighters Pencils Pens", returns this link as number #1
www.4imprint.com/Products/ViewSuperGroup.aspx?sgrp=2
and this link as #3
http://eshop.msn.com/category.aspx?catId=987

Both are dymanic database driven sites with '?' in their URLs.

Maybe what gvmdevelopment means is that that these type of dymanic URLs are not always ranked as highly as other 'plain' URLs in some search engines. But this is often becuase the key words don't appear in the URL.

For example,
www.mysite.com/Highlighters-Pencils-Pens.htm

is going to look more relevent to search engine than

www.mysite.com/product.php?ProductId=123

David
0
Why Off-Site Backups Are The Only Way To Go

You are probably backing up your data—but how and where? Ransomware is on the rise and there are variants that specifically target backups. Read on to discover why off-site is the way to go.

 
LVL 1

Expert Comment

by:gvmdevelopment
ID: 11780502
You are correct, Google does index these pages, but in many cases it looks down on such pages. The most important part like David said is that there be links to each and every product page, so that the search engine can find them.
0
 
LVL 3

Author Comment

by:d2kagw
ID: 11780699
Good Stuff, two more questions then:
[I have increased points to 400 too]

01.
Do you guys know of a good spider/robot that I could use to test the site [I need to make sure that particular areas are definatly found and others not found at all]

02.
David, you mentioned "creating a list of URLs to refer to these records in your database", would this mean just making a page that is filled with links like:
www.mysite.com/product.php?ProductId=123
www.mysite.com/product.php?ProductId=124
www.mysite.com/product.php?ProductId=125
www.mysite.com/product.php?ProductId=126
www.mysite.com/product.php?ProductId=127
if that was the case, how would you get the spider/robot to look at that page, without it being a hard link?

Thanks again guys !
0
 
LVL 3

Expert Comment

by:passmark
ID: 11780952

1/
My company sells the Zoom Search Engine that you could use to spider you site. But it is only free for small sites.
See, http://www.wrensoft.com and http://www.wrensoft.com/zoom/screen1.html
You just point it at your home page and hit the start button.

Otherwise there are some free alternatives on the web which may not be as easy to use (in my rather bias opinion).

2/
Yes. But here is your real problem.
Just because Google *could* index your site, doesn't mean it will. You will have the same problem as many other web designers. Namely, how do you optimize your site to get good placement (or even any placement at all) in Google? Google might also penalise or ignore pages that look like they have no significant content. So I don't think a straight list of links is going to look very interesting for Google. Whole books (and websites) have been written about placement optimization. Do a search for "Search Engine Optimization".

David
0
 
LVL 24

Expert Comment

by:duz
ID: 11781421
d2kagw -

>Do a search for "Search Engine Optimization".

Don't do that because 99% of what you will read will be complete rubbish or out of date information. Search the "Online Marketing" channel here on Experts Exchange for more reliable and up to date answers.

- duz
0

Featured Post

How Do You Stack Up Against Your Peers?

With today’s modern enterprise so dependent on digital infrastructures, the impact of major incidents has increased dramatically. Grab the report now to gain insight into how your organization ranks against your peers and learn best-in-class strategies to resolve incidents.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Because your company can’t afford for you to make SEO mistakes, you’ll want to ensure you’re taking the right steps each and every time you post a new piece of content. This list of optimization do’s and don’ts can help you become an SEO wizard.
Australian government abolished Visa 457 earlier this April and this article describes how this decision might affect Australian IT scene and IT experts.
This tutorial walks through the best practices in adding a local business to Google Maps including how to properly search for duplicates, marker placement, and inputing business details. Login to your Google Account, then search for "Google Mapmaker…
Any person in technology especially those working for big companies should at least know about the basics of web accessibility. Believe it or not there are even laws in place that require businesses to provide such means for the disabled and aging p…

751 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question