• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 456
  • Last Modified:

Best Data algorithm book for the experts.

I planning to invent a new algorithm which should read the 25 tb data in less than a minute. I know it's tough job ,but like to start my R&D on this.
0
VIVEKANANDHAN_PERIASAMY
Asked:
VIVEKANANDHAN_PERIASAMY
1 Solution
 
d-glitchCommented:
An advanced design needs to start with a rigorous specification.

What is this 25 TB of data, and how it stored?
What hardware and interface are you using to read it?
What do you hope to do with it once you have read it?
What is so special about 1 minute?  Is 59 seconds really much better than 61?

You might need to develop new massively parallel hardware before you can start to work on the algorithm.

You could start looking here:
     http://www.amazon.com/Introduction-Algorithms-Thomas-H-Cormen/dp/0262033844
0
 
sarabandeCommented:
the maximum ssd size to date is 2 tb (http://www.thessdreview.com/our-reviews/solidata-k8-1920e-2tb-ssd-review/)  it reads 230 MB per second. so you need a rack of 13 sata drives and - given you are the only user of the system - you theoretically could read 25 tb sequentially within two minutes. reading parallel you could achieve your goal if you overcome the bottlenecks current hardware surely has. as you can't store all the data in memory (beside you use hundreds or thousands of units what arises new issues) you would need to evaluate the data while reading.

Sara
0
 
ozoCommented:
You could have 25 thousand Raspberry Pi's, each reading 1gb per minute.
What does that have to do with algorithms?
0
 
evilrixSenior Software Engineer (Avast)Commented:
By far the best book I've read, related to algorithm design.

The Algorithm Design Manual, 2nd Edition, Steven Skiena
http://www.algorist.com/
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Cloud Class® Course: C++ 11 Fundamentals

This course will introduce you to C++ 11 and teach you about syntax fundamentals.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now