Solved

# Best Data algorithm book for the experts.

Posted on 2013-11-06
389 Views
I planning to invent a new algorithm which should read the 25 tb data in less than a minute. I know it's tough job ,but like to start my R&D on this.
0

LVL 27

Expert Comment

ID: 39627943

What is this 25 TB of data, and how it stored?
What hardware and interface are you using to read it?
What do you hope to do with it once you have read it?
What is so special about 1 minute?  Is 59 seconds really much better than 61?

You might need to develop new massively parallel hardware before you can start to work on the algorithm.

You could start looking here:
http://www.amazon.com/Introduction-Algorithms-Thomas-H-Cormen/dp/0262033844
0

LVL 33

Expert Comment

ID: 39628033
the maximum ssd size to date is 2 tb (http://www.thessdreview.com/our-reviews/solidata-k8-1920e-2tb-ssd-review/)  it reads 230 MB per second. so you need a rack of 13 sata drives and - given you are the only user of the system - you theoretically could read 25 tb sequentially within two minutes. reading parallel you could achieve your goal if you overcome the bottlenecks current hardware surely has. as you can't store all the data in memory (beside you use hundreds or thousands of units what arises new issues) you would need to evaluate the data while reading.

Sara
0

LVL 84

Expert Comment

ID: 39628107
You could have 25 thousand Raspberry Pi's, each reading 1gb per minute.
What does that have to do with algorithms?
0

LVL 40

Accepted Solution

evilrix earned 500 total points
ID: 39628172
By far the best book I've read, related to algorithm design.

The Algorithm Design Manual, 2nd Edition, Steven Skiena
http://www.algorist.com/
0

## Featured Post

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The greatest common divisor (gcd) of two positive integers is their largest common divisor. Let's consider two numbers 12 and 20. The divisors of 12 are 1, 2, 3, 4, 6, 12 The divisors of 20 are 1, 2, 4, 5, 10 20 The highest number among the c…
Container Orchestration platforms empower organizations to scale their apps at an exceptional rate. This is the reason numerous innovation-driven companies are moving apps to an appropriated datacenter wide platform that empowers them to scale at a …
The goal of this video is to provide viewers with basic examples to understand opening and writing to files in the C programming language.
The viewer will learn how to use the return statement in functions in C++. The video will also teach the user how to pass data to a function and have the function return data back for further processing.