Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Best Data algorithm book for the experts.

Posted on 2013-11-06
4
Medium Priority
?
425 Views
Last Modified: 2013-11-10
I planning to invent a new algorithm which should read the 25 tb data in less than a minute. I know it's tough job ,but like to start my R&D on this.
0
Comment
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
4 Comments
 
LVL 27

Expert Comment

by:d-glitch
ID: 39627943
An advanced design needs to start with a rigorous specification.

What is this 25 TB of data, and how it stored?
What hardware and interface are you using to read it?
What do you hope to do with it once you have read it?
What is so special about 1 minute?  Is 59 seconds really much better than 61?

You might need to develop new massively parallel hardware before you can start to work on the algorithm.

You could start looking here:
     http://www.amazon.com/Introduction-Algorithms-Thomas-H-Cormen/dp/0262033844
0
 
LVL 35

Expert Comment

by:sarabande
ID: 39628033
the maximum ssd size to date is 2 tb (http://www.thessdreview.com/our-reviews/solidata-k8-1920e-2tb-ssd-review/)  it reads 230 MB per second. so you need a rack of 13 sata drives and - given you are the only user of the system - you theoretically could read 25 tb sequentially within two minutes. reading parallel you could achieve your goal if you overcome the bottlenecks current hardware surely has. as you can't store all the data in memory (beside you use hundreds or thousands of units what arises new issues) you would need to evaluate the data while reading.

Sara
0
 
LVL 84

Expert Comment

by:ozo
ID: 39628107
You could have 25 thousand Raspberry Pi's, each reading 1gb per minute.
What does that have to do with algorithms?
0
 
LVL 40

Accepted Solution

by:
evilrix earned 2000 total points
ID: 39628172
By far the best book I've read, related to algorithm design.

The Algorithm Design Manual, 2nd Edition, Steven Skiena
http://www.algorist.com/
0

Featured Post

How to Use the Help Bell

Need to boost the visibility of your question for solutions? Use the Experts Exchange Help Bell to confirm priority levels and contact subject-matter experts for question attention.  Check out this how-to article for more information.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

One of Google's most recent algorithm changes affecting local searches is entitled "The Pigeon Update." This update has dramatically enhanced search inquires for the keyword "Yelp." Google searches with the word "Yelp" included will now yield Yelp a…
Prime numbers are natural numbers greater than 1 that have only two divisors (the number itself and 1). By “divisible” we mean dividend % divisor = 0 (% indicates MODULAR. It gives the reminder of a division operation). We’ll follow multiple approac…
The goal of this video is to provide viewers with basic examples to understand how to create, access, and change arrays in the C programming language.
The viewer will learn additional member functions of the vector class. Specifically, the capacity and swap member functions will be introduced.

604 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question