Solved

How does Google get such fast response times?

Posted on 2016-10-01
5
124 Views
Last Modified: 2016-10-02
I had an interview Friday and was asked that question and was told this firm uses this same method for near-instant response against millions of patient records. I was told to "think about it" before my next interview.

So I am curious how search results can be grouped when you have various elements inputted, such as:

First Name
Age
Sex

The Google example involved multiple search words.

I am curious to hear a few different ideas on this.

Thanks
0
Comment
Question by:newbieweb
5 Comments
 
LVL 34

Assisted Solution

by:johnsone
johnsone earned 125 total points
ID: 41825022
Fast response time is done with massively parallel searching.  There isn't one machine searching billions of records.  There are thousands of machines searching thousands of records.
0
 
LVL 41

Accepted Solution

by:
pcelba earned 250 total points
ID: 41825027
Beside the large computing capacity Google also uses fulltext indexing so even several words are searched very quickly in the index file and then it can provide fast preview of the results. More details about full text searching is e.g. here: https://en.wikipedia.org/wiki/Full_text_search or here: http://www.ideaeng.com/database-full-text-search-0201

Google also knows what terms are searched most often and optimizes these searches to be more efficient and provide results more quickly...

That's not all. Google can also "narrow" search results by your previous preferences and by paid advertisements... etc. etc. I would guess the company where you've been interviewed does not use such techniques.

And now what's important:
You need nothing from the above to search million of patient records based on known  First Name, Age, and Sex.  For such a simple query you don't need fulltext index but just two standard indexes on First Name and Age. The Sex does not need index because such index would not speed the searching up because it does not reduce the result set significantly.
0
 
LVL 76

Assisted Solution

by:slightwv (䄆 Netminder)
slightwv (䄆 Netminder) earned 125 total points
ID: 41825031
Why ask us?  They tell you:   https://www.google.com/insidesearch/howsearchworks/thestory/

>> and was told this firm uses this same method for near-instant response against millions of patient records. I was told to "think about it" before my next interview.

So, they bought a Google appliance?  I mean if they use the "same" search as Google, they sort of had to but it from them...
https://enterprise.google.com/search/products/gsa.html

>>So I am curious how search results can be grouped

As mentioned above:  Parallel on top of parallel.

>>The Sex does not need index because such index would not speed the searching up because it does not reduce the result set significantly.

Depends on your data.  A BTree index, I agree but a bitmap index might make sense.
0
 

Author Closing Comment

by:newbieweb
ID: 41825063
thanks
0
 
LVL 34

Expert Comment

by:johnsone
ID: 41825264
Using the "same method", doesn't mean using the same algorithms.  You don't need a Google appliance to do searching, in fact to do it the way that Google does it, you really need several of them.

Purpose built data warehouse systems (think the IBM Netezza product and its similar competitors), do massively parallel operations.  They don't have any indexes in their database.  Rather than one database server going through all the data, it breaks the database up into many pieces and each piece is searched by its own server.

I'm not sure if Oracle's Exadata product does it exactly this way, but it is possible.  I don't think it has the levels of parallelism that the other products do, they just throw more hardware at it.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
Homework Help 5 51
MSSQL Convert Char To Date 4 43
recursion example 16 69
Need Distinct Value from Joined Tables in MSSQL 5 27
If your business is like most, chances are you still need to maintain a fax infrastructure for your staff. It’s hard to believe that a communication technology that was thriving in the mid-80s could still be an essential part of your team’s modern I…
Exchange server is not supported in any cloud-hosted platform (other than Azure with Azure Premium Storage).
An introduction to basic programming syntax in Java by creating a simple program. Viewers can follow the tutorial as they create their first class in Java. Definitions and explanations about each element are given to help prepare viewers for future …
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now