?
Solved

How does Google get such fast response times?

Posted on 2016-10-01
5
Medium Priority
?
258 Views
Last Modified: 2016-10-02
I had an interview Friday and was asked that question and was told this firm uses this same method for near-instant response against millions of patient records. I was told to "think about it" before my next interview.

So I am curious how search results can be grouped when you have various elements inputted, such as:

First Name
Age
Sex

The Google example involved multiple search words.

I am curious to hear a few different ideas on this.

Thanks
0
Comment
Question by:newbieweb
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
5 Comments
 
LVL 35

Assisted Solution

by:johnsone
johnsone earned 500 total points
ID: 41825022
Fast response time is done with massively parallel searching.  There isn't one machine searching billions of records.  There are thousands of machines searching thousands of records.
0
 
LVL 42

Accepted Solution

by:
pcelba earned 1000 total points
ID: 41825027
Beside the large computing capacity Google also uses fulltext indexing so even several words are searched very quickly in the index file and then it can provide fast preview of the results. More details about full text searching is e.g. here: https://en.wikipedia.org/wiki/Full_text_search or here: http://www.ideaeng.com/database-full-text-search-0201

Google also knows what terms are searched most often and optimizes these searches to be more efficient and provide results more quickly...

That's not all. Google can also "narrow" search results by your previous preferences and by paid advertisements... etc. etc. I would guess the company where you've been interviewed does not use such techniques.

And now what's important:
You need nothing from the above to search million of patient records based on known  First Name, Age, and Sex.  For such a simple query you don't need fulltext index but just two standard indexes on First Name and Age. The Sex does not need index because such index would not speed the searching up because it does not reduce the result set significantly.
0
 
LVL 77

Assisted Solution

by:slightwv (䄆 Netminder)
slightwv (䄆 Netminder) earned 500 total points
ID: 41825031
Why ask us?  They tell you:   https://www.google.com/insidesearch/howsearchworks/thestory/

>> and was told this firm uses this same method for near-instant response against millions of patient records. I was told to "think about it" before my next interview.

So, they bought a Google appliance?  I mean if they use the "same" search as Google, they sort of had to but it from them...
https://enterprise.google.com/search/products/gsa.html

>>So I am curious how search results can be grouped

As mentioned above:  Parallel on top of parallel.

>>The Sex does not need index because such index would not speed the searching up because it does not reduce the result set significantly.

Depends on your data.  A BTree index, I agree but a bitmap index might make sense.
0
 

Author Closing Comment

by:newbieweb
ID: 41825063
thanks
0
 
LVL 35

Expert Comment

by:johnsone
ID: 41825264
Using the "same method", doesn't mean using the same algorithms.  You don't need a Google appliance to do searching, in fact to do it the way that Google does it, you really need several of them.

Purpose built data warehouse systems (think the IBM Netezza product and its similar competitors), do massively parallel operations.  They don't have any indexes in their database.  Rather than one database server going through all the data, it breaks the database up into many pieces and each piece is searched by its own server.

I'm not sure if Oracle's Exadata product does it exactly this way, but it is possible.  I don't think it has the levels of parallelism that the other products do, they just throw more hardware at it.
0

Featured Post

Automating Terraform w Jenkins & AWS CodeCommit

How to configure Jenkins and CodeCommit to allow users to easily create and destroy infrastructure using Terraform code.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article shows how to get a list of available printers for display in a drop-down list, and then to use the selected printer to print an Access report or a Word document filled with Access data, using different syntax as needed for working with …
In this article, we’ll look at how to deploy ProxySQL.
Internet Business Fax to Email Made Easy - With  eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, f…
In this video, Percona Director of Solution Engineering Jon Tobin discusses the function and features of Percona Server for MongoDB. How Percona can help Percona can help you determine if Percona Server for MongoDB is the right solution for …

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question