Link to home
Create AccountLog in
Avatar of turbot_yu
turbot_yuFlag for Singapore

asked on

Blade or normal pc, which is good for big data

Trying to setup a demo big data system, it can be several blades or normal pcs. Normal pc are cheap but not sure, if it need to linkup by fibre.

Some friend said hadoop are distributed calculation, not parallel calculation, so normal rj45 cable is good enough, is it true.

Or they said can DIY servers by self to save money. Or any cheapest blades.

Any suggestion, thanks.
SOLUTION
Avatar of Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)
Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)
Flag of United Kingdom of Great Britain and Northern Ireland image

Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
SOLUTION
Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
Avatar of turbot_yu

ASKER

Thanks a lot skullnobrains, is there any detailed example or link of setup the blades myself or linkup the pc by the fiber.

Really need to do it now.
SOLUTION
Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
SOLUTION
Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
Any link for cheap fibre storage device.

Hi skullnobrains,it difficult for me to find such person, can you recommend.

Or I just quickly setup by some PCs, then come to the hadoop, map-reduce and hive.

When demo is ready, may show it and see if can get some project from power grid or traffic video monitoring.
SOLUTION
Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
The data coming in by stream, such as video stream or data flow.

Each data point about 100M in full mode or serveral hundred k in filtering mode.
another question is how is the storage.

if using PCs, every PC has a hard-disk?

if using blade, is it a centralized storage?
SOLUTION
Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
SOLUTION
Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
actually, I also not very sure how much power is needed, just want to make the system has the capability to extend.

Hadoop is the most stable big data os, so want to try it. The raw data for one sensor may be 100M/s, after filtering, it may less than 1M, every station may have <100 sensor. Theoretically, may have 150k sensors.

eliyart(at)hotmail.com
SOLUTION
Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
It is like the description, why hadoop is needed is data mining is needed and big storage is needed.
there are 2 demos one for sensors, one for video.
SOLUTION
Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
ASKER CERTIFIED SOLUTION
Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
Thanks a lot,  it is quite clear, I am trying to buy some small factor hp pc now, see if they can work.
Hi  skullnobrains, is it possible to learn your background.
SOLUTION
Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account