Solved

How does Google get such fast response times?

Posted on 2016-10-01
5
188 Views
Last Modified: 2016-10-02
I had an interview Friday and was asked that question and was told this firm uses this same method for near-instant response against millions of patient records. I was told to "think about it" before my next interview.

So I am curious how search results can be grouped when you have various elements inputted, such as:

First Name
Age
Sex

The Google example involved multiple search words.

I am curious to hear a few different ideas on this.

Thanks
0
Comment
Question by:newbieweb
5 Comments
 
LVL 34

Assisted Solution

by:johnsone
johnsone earned 125 total points
ID: 41825022
Fast response time is done with massively parallel searching.  There isn't one machine searching billions of records.  There are thousands of machines searching thousands of records.
0
 
LVL 42

Accepted Solution

by:
pcelba earned 250 total points
ID: 41825027
Beside the large computing capacity Google also uses fulltext indexing so even several words are searched very quickly in the index file and then it can provide fast preview of the results. More details about full text searching is e.g. here: https://en.wikipedia.org/wiki/Full_text_search or here: http://www.ideaeng.com/database-full-text-search-0201

Google also knows what terms are searched most often and optimizes these searches to be more efficient and provide results more quickly...

That's not all. Google can also "narrow" search results by your previous preferences and by paid advertisements... etc. etc. I would guess the company where you've been interviewed does not use such techniques.

And now what's important:
You need nothing from the above to search million of patient records based on known  First Name, Age, and Sex.  For such a simple query you don't need fulltext index but just two standard indexes on First Name and Age. The Sex does not need index because such index would not speed the searching up because it does not reduce the result set significantly.
0
 
LVL 76

Assisted Solution

by:slightwv (䄆 Netminder)
slightwv (䄆 Netminder) earned 125 total points
ID: 41825031
Why ask us?  They tell you:   https://www.google.com/insidesearch/howsearchworks/thestory/

>> and was told this firm uses this same method for near-instant response against millions of patient records. I was told to "think about it" before my next interview.

So, they bought a Google appliance?  I mean if they use the "same" search as Google, they sort of had to but it from them...
https://enterprise.google.com/search/products/gsa.html

>>So I am curious how search results can be grouped

As mentioned above:  Parallel on top of parallel.

>>The Sex does not need index because such index would not speed the searching up because it does not reduce the result set significantly.

Depends on your data.  A BTree index, I agree but a bitmap index might make sense.
0
 

Author Closing Comment

by:newbieweb
ID: 41825063
thanks
0
 
LVL 34

Expert Comment

by:johnsone
ID: 41825264
Using the "same method", doesn't mean using the same algorithms.  You don't need a Google appliance to do searching, in fact to do it the way that Google does it, you really need several of them.

Purpose built data warehouse systems (think the IBM Netezza product and its similar competitors), do massively parallel operations.  They don't have any indexes in their database.  Rather than one database server going through all the data, it breaks the database up into many pieces and each piece is searched by its own server.

I'm not sure if Oracle's Exadata product does it exactly this way, but it is possible.  I don't think it has the levels of parallelism that the other products do, they just throw more hardware at it.
0

Featured Post

NAS Cloud Backup Strategies

This article explains backup scenarios when using network storage. We review the so-called “3-2-1 strategy” and summarize the methods you can use to send NAS data to the cloud

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Most of the applications these days are on Cloud. Cloud is ubiquitous with many service providers in the market. Since it has many benefits such as cost reduction, software updates, remote access, disaster recovery and much more.
Cloud-based technologies and services will continue to grow in popularity in 2017 thanks to the simple, scalable and cost-effective solutions they deliver. Here are three areas where cloud adoption is poised to really take off.

766 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question