Solved

How does Google get such fast response times?

Posted on 2016-10-01
5
229 Views
Last Modified: 2016-10-02
I had an interview Friday and was asked that question and was told this firm uses this same method for near-instant response against millions of patient records. I was told to "think about it" before my next interview.

So I am curious how search results can be grouped when you have various elements inputted, such as:

First Name
Age
Sex

The Google example involved multiple search words.

I am curious to hear a few different ideas on this.

Thanks
0
Comment
Question by:newbieweb
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
5 Comments
 
LVL 35

Assisted Solution

by:johnsone
johnsone earned 125 total points
ID: 41825022
Fast response time is done with massively parallel searching.  There isn't one machine searching billions of records.  There are thousands of machines searching thousands of records.
0
 
LVL 42

Accepted Solution

by:
pcelba earned 250 total points
ID: 41825027
Beside the large computing capacity Google also uses fulltext indexing so even several words are searched very quickly in the index file and then it can provide fast preview of the results. More details about full text searching is e.g. here: https://en.wikipedia.org/wiki/Full_text_search or here: http://www.ideaeng.com/database-full-text-search-0201

Google also knows what terms are searched most often and optimizes these searches to be more efficient and provide results more quickly...

That's not all. Google can also "narrow" search results by your previous preferences and by paid advertisements... etc. etc. I would guess the company where you've been interviewed does not use such techniques.

And now what's important:
You need nothing from the above to search million of patient records based on known  First Name, Age, and Sex.  For such a simple query you don't need fulltext index but just two standard indexes on First Name and Age. The Sex does not need index because such index would not speed the searching up because it does not reduce the result set significantly.
0
 
LVL 77

Assisted Solution

by:slightwv (䄆 Netminder)
slightwv (䄆 Netminder) earned 125 total points
ID: 41825031
Why ask us?  They tell you:   https://www.google.com/insidesearch/howsearchworks/thestory/

>> and was told this firm uses this same method for near-instant response against millions of patient records. I was told to "think about it" before my next interview.

So, they bought a Google appliance?  I mean if they use the "same" search as Google, they sort of had to but it from them...
https://enterprise.google.com/search/products/gsa.html

>>So I am curious how search results can be grouped

As mentioned above:  Parallel on top of parallel.

>>The Sex does not need index because such index would not speed the searching up because it does not reduce the result set significantly.

Depends on your data.  A BTree index, I agree but a bitmap index might make sense.
0
 

Author Closing Comment

by:newbieweb
ID: 41825063
thanks
0
 
LVL 35

Expert Comment

by:johnsone
ID: 41825264
Using the "same method", doesn't mean using the same algorithms.  You don't need a Google appliance to do searching, in fact to do it the way that Google does it, you really need several of them.

Purpose built data warehouse systems (think the IBM Netezza product and its similar competitors), do massively parallel operations.  They don't have any indexes in their database.  Rather than one database server going through all the data, it breaks the database up into many pieces and each piece is searched by its own server.

I'm not sure if Oracle's Exadata product does it exactly this way, but it is possible.  I don't think it has the levels of parallelism that the other products do, they just throw more hardware at it.
0

Featured Post

Online Training Solution

Drastically shorten your training time with WalkMe's advanced online training solution that Guides your trainees to action. Forget about retraining and skyrocket knowledge retention rates.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Learn how the use of a bunch of disparate tools requiring a lot of manual attention led to a series of unfortunate backup events for one company.
A Stored Procedure in Microsoft SQL Server is a powerful feature that it can be used to execute the Data Manipulation Language (DML) or Data Definition Language (DDL). Depending on business requirements, a single Stored Procedure can return differe…

739 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question