Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

Google to get results from internal database

Posted on 2004-08-11
7
Medium Priority
?
308 Views
Last Modified: 2007-02-13
Hi There,

I am developing a site that has loads of products in a database [stored on our dedicated server]
The client has requested that the products be returned when a user does a google search... I know this is posible as I have seen it done on many sites [including this one] i.e. if I where to go to google and search for "Google to get results from internal database" it would return this page.... somehow.....

Does it have to do with the way the site is developed?
Databse setup? Special Submission to the search engines?

Any help will be greatly appreciated.
0
Comment
Question by:d2kagw
7 Comments
 
LVL 3

Expert Comment

by:passmark
ID: 11779786

Search engine spiders follow links that you have on your web site. The scans web pages as if it was a normal user browsing your site. Provided all your database content is accessible via web pages (ie. there is a URL to each record in your database), the spider mode indexing would work, regardless of the database backend you are using (MySQL, Postgres, MS Access, Oracle, etc.).

However, if there are portions of your database which can not be accessed via your web pages, then you may need to create a list of URLs to refer to these records in your database.

For example, you may have a database of cooking recipe. Each of these recipe are a record in the database, and are accessed via a PHP webpage which displays the recipe selected such as the following:

http://mysite.com/recipe.php?id=12
http://mysite.com/recipe.php?name=pavalova
etc.

We develop our own search engine spider software (for searching within a website) and give this advice to our customers, see here,
http://www.wrensoft.com/zoom/support.html#sql

David
0
 
LVL 1

Expert Comment

by:gvmdevelopment
ID: 11780372
Many search engines will not index dynamic pages (.php, .asp, cgi, etc or URLs that have ? in them) The best solution to get these type of pages indexed into search engines is to output them into .html pages and create links to them. These static pages would only be the entry point into the site and then everything else would be dynamic. It is then up to you how often you create or update these static pages.

You can either write an application to create these pages, or there are applications available which will do the dynamic to static page creation for you on a regular basis.
0
 
LVL 3

Accepted Solution

by:
passmark earned 1600 total points
ID: 11780459

> Many search engines will not index dynamic pages...

Well maybe. I don't know of any however.

This is not correct. Google *does* index dymanic pages. Doing a search for just about any topic in Google returns lots of dymanic pages. Google also indexes URLs with a '?'.

For example, doing a Google search for "Highlighters Pencils Pens", returns this link as number #1
www.4imprint.com/Products/ViewSuperGroup.aspx?sgrp=2
and this link as #3
http://eshop.msn.com/category.aspx?catId=987

Both are dymanic database driven sites with '?' in their URLs.

Maybe what gvmdevelopment means is that that these type of dymanic URLs are not always ranked as highly as other 'plain' URLs in some search engines. But this is often becuase the key words don't appear in the URL.

For example,
www.mysite.com/Highlighters-Pencils-Pens.htm

is going to look more relevent to search engine than

www.mysite.com/product.php?ProductId=123

David
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
LVL 1

Expert Comment

by:gvmdevelopment
ID: 11780502
You are correct, Google does index these pages, but in many cases it looks down on such pages. The most important part like David said is that there be links to each and every product page, so that the search engine can find them.
0
 
LVL 3

Author Comment

by:d2kagw
ID: 11780699
Good Stuff, two more questions then:
[I have increased points to 400 too]

01.
Do you guys know of a good spider/robot that I could use to test the site [I need to make sure that particular areas are definatly found and others not found at all]

02.
David, you mentioned "creating a list of URLs to refer to these records in your database", would this mean just making a page that is filled with links like:
www.mysite.com/product.php?ProductId=123
www.mysite.com/product.php?ProductId=124
www.mysite.com/product.php?ProductId=125
www.mysite.com/product.php?ProductId=126
www.mysite.com/product.php?ProductId=127
if that was the case, how would you get the spider/robot to look at that page, without it being a hard link?

Thanks again guys !
0
 
LVL 3

Expert Comment

by:passmark
ID: 11780952

1/
My company sells the Zoom Search Engine that you could use to spider you site. But it is only free for small sites.
See, http://www.wrensoft.com and http://www.wrensoft.com/zoom/screen1.html
You just point it at your home page and hit the start button.

Otherwise there are some free alternatives on the web which may not be as easy to use (in my rather bias opinion).

2/
Yes. But here is your real problem.
Just because Google *could* index your site, doesn't mean it will. You will have the same problem as many other web designers. Namely, how do you optimize your site to get good placement (or even any placement at all) in Google? Google might also penalise or ignore pages that look like they have no significant content. So I don't think a straight list of links is going to look very interesting for Google. Whole books (and websites) have been written about placement optimization. Do a search for "Search Engine Optimization".

David
0
 
LVL 24

Expert Comment

by:duz
ID: 11781421
d2kagw -

>Do a search for "Search Engine Optimization".

Don't do that because 99% of what you will read will be complete rubbish or out of date information. Search the "Online Marketing" channel here on Experts Exchange for more reliable and up to date answers.

- duz
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Although a lot of people devote their energy toward marketing for specific industries, there are some basic principles that can be applied to any sector imaginable. We’ll look at four steps to take and examine how those steps were put into action fo…
CTAs encourage people to do something specific to show interest in your company, product or service. Keep reading to learn why CTAs should always be thought of as extremely important, albeit small, sections of websites.
The viewer will learn how to dynamically set the form action using jQuery.
The viewer will learn how to count occurrences of each item in an array.
Suggested Courses

971 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question