Linux Cluster

What is the best hardware for setting up a linux cluster?  Is the myrinet switch and network adaptors the way to go?  What else is needed or what other solutions are there?
kevinsmailAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

giltjrCommented:
What will this cluster be used for?  How mission critical is it?  Are you using it as a "active-active" cluster or a "active-failover/passive" customer?

0
kevinsmailAuthor Commented:
It will be used for some simulation software.  The cluster will be active active I am guess.  I mean I am looking at hooking up like 6 1U servers with 2 CPU's each to create a 12 CPU processor box.  It will be important that it stays up and running.  
0
giltjrCommented:
Umm, ideas, ideas, and ideas, lead to questions, questions, questions.

Do you need the speed of myrinet?  This normally means LOTS of data transfer between something?

Have you thought about 4-ways instead of 2-ways?  Yes, 2-ways are much cheaper, however you need to look at the big picture.  If you have 3 computers instead of 6 you have fewer things to buy and fewer things to break.  Yes, you have less capacity if something breaks.  The cost of going from 2 to 4 way is not as big of a jump as going to 8-ways and up.  

Is the application cluster aware?  Does the software vendor for the simulation software have any recommendations?

You may want to look at blade servers, there are some companies that make them specifically for cluster purposes, http://www.egenera.com and http://www.appro.com/ are two.
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

kevinsmailAuthor Commented:
The software is cluster aware and the vendor did have a recommendation for hardware.  It was a cluster using 4 way servers but not the brand that the customer likes so they asked us to configure something for them.  I haven't done one so that is why I am here.  I would say we need the most speed possible to transfer data between nodes.  I will look into 4 ways also.  So basically to do the myrinet you put the myrinet card in the server and a myrinet swtich in?  Do you need any extra Network interfaces besides that?  Server has 2 onboard 1 Gb alreday.  
0
kevinsmailAuthor Commented:
How would you do it without the myrinet switch?
0
giltjrCommented:
You can use LAN connection.  Depending on the exact details of what you are doing you can use at least two NIC's in each computer.  One NIC will be used for for the "intra-cluster" communctions instead of myrinet swtich the other NIC will be use for the other traffic.

Myrinet, IIRC, is about 10 Gbps.  Although this is the same speed as 10 Gbps Ethernet, Myrinet is supposed to be better for clustering, but requires unique switches.  For Ethernet you can use most any Enterprise or Carrier class switch and just put 10 Gbps adapters.  That is if you need 10 Gbps thoughput.

If you are not doing a lot of heavy data transfers then 1 Gbps connections may work for you.
0
kevinsmailAuthor Commented:
Thanks a bunch.  This really helps.
0
publicCommented:
Myrinet has very low latency. Ethernet has far greater latency because packet headers need to be opened.
Myrinet is completely incompatible with anybody else, but it is the perfect solution for cluster message passing.
0
NetTechDudeCommented:
Stay away from 1U rack units if you can.
The 1U cases are so overloaded with fans because getting air to flow is damn difficult with so little space. Thus, noise from the cluster computers becomes a concern. And the cooling is not even up to what you are used from any desktop computer.
And especially your harddisks will like the heat less with every generation.
I'd stay with old-style 4U cases. You can even hang the discs vertically so they'd cool even better.
0
giltjrCommented:
I would suggest just putting in two IDE's in a RAID 0 on the servers and then use SAN externally.  This you do not have to worry about the heat inside the servers and you may reduce you total DISK space requirments if you can share files on a SAN rather than storing every on each server in the cluster.
0
NetTechDudeCommented:
Raid0 is plainly crazy.
Raid0 is so error prone. The question is not if it fails but when it fails.
Even if there's only the os on it, it my be nasty to stop the cluster every now and then and resetup one machine because one of it's raid0 disks has failed.

OK, Raid0 is a little faster but do you need high disk-i/o or do you need lots of processing power ?
Put in two IDE's as RAID-1 for security and use an external SAN if you bathe in money. If you don't want no SAN but need high storage capacity anyways, go for ide raid 5 controllers with 4 or more disks each. Use one disk as hot spare, then it's rather error proof. If you need more disk i/o thruput, use a SCSI raid instead of ide with bunches of fast 15000rpm server discs.
0
giltjrCommented:
Opps, I meant RAID1 so that a single hard drive faliure would not cause problems.

I would assume that an external SAN would be cheaper than 6 sets of RAID controllers and 20+ drives and it definitly would make expanding the cluster cheaper.  If you say you need 6, you are going to end up with 12. :)

0
publicCommented:
There seems to be some confusion about the goals.
Linux cluster usually means high performance computing with self contained nodes added as needed.
Reliability and redundancy are secondary.
The cluster interconnect is used just for the cluster message passing.
Is this what you are designing?
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Storage Software

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.