Solved

What is maximum number of disc recommanded in RAID 5 Array?

Posted on 2009-04-09
15
1,250 Views
Last Modified: 2013-11-14
I'm having 14 * 1 TB HDD.

I would like to make RAID 5 Array using those 14 discs.

What is the recommanded number of discs can be used in Single Array?

Like can we use all 14 discs and make it as a single Array or can we use less number of discs and make it as mulitple arrays?

Which one is recommanded?

Which will give good performance?

Please help me.
0
Comment
Question by:erteam
  • 6
  • 4
  • 2
  • +2
15 Comments
 
LVL 13

Expert Comment

by:Randy_Bojangles
ID: 24107651
Generally speaking the more spindles (disks) you have in a RAID5 the better your performance is.

The number of diks you can put into a single array depends on the specific hardware (controller and enclosure) you are using
0
 
LVL 55

Expert Comment

by:andyalder
ID: 24108494
HP recommend not to have more than 7 or 8, reason being if a disk fails the performance goes through the floor and the chance of a second disk failing before the array is rebuilt gets too high. For more than 8 they would suggest using RAID 10 or RAID 6 depending on whether you want capacity or performance. Another option to tackle it would be to use RAID 50.
0
 
LVL 70

Expert Comment

by:garycase
ID: 24110389
As noted above, the number of disks you can use in a RAID-5 array is limited by your controller.  If your controller supports it, you can easily use 14 drives in the array.

However, with 14 disks, the concern Andy noted is very real -- the likelihood of a 2nd drive failing during a rebuild is a real concern.   I'd recommend you get a controller that supports RAID-6 and use these drives in a RAID-6 array => you'll get the same level of performance, and your array will tolerate two drive failures with no data loss.   This provides superb fault tolerance; and makes it very unlikely you'll ever lose data during a rebuild.
0
 
LVL 55

Expert Comment

by:andyalder
ID: 24111614
erteam could do an experiment for us, build a 14 disk RAID 5 with 1TB SATAs (HP were talking about 10K SCSI when they said max 8 for safety) and then yank one of them out - I presume they are hotplug. Shove it back in again and record how long it takes to rebuild.

What's the duty-cycle of the disks? HP's 8 disk max was for 100% duty-cycle disks but SATAs (ignoring 10K ones with SCSI/SAS capacities like raptors) are rated at only 40% duty-cycle and rebuilding is a full-time job. I expect them to go into slow read-after-write mode to protect themselves from overheating but as yet I have no proof. So I'd really like erteam to do the experiment.

There's a good rebuild/performance whitepaper from HP (you can guess I work for an HP house) but it only covers RAID 5, RAID 6 rebuild/performance time would be interesting to see.
0
 
LVL 70

Expert Comment

by:garycase
ID: 24111920
The rebuild time is very much a function of the controller.   The newer controllers with the 1680 chips can rebuild a drive in an array like that in a matter of 2-3 hours.  The older 12xx series chips would take ~ 25 hours.

These numbers are based on a similar system detailed on the AVS forum with 20 1TB drives.
0
 
LVL 16

Expert Comment

by:Gerald Connolly
ID: 24125242
As Randy said the basic premise is that the more disks in a RAID stripe (RAID-5 is striping with redundancy) the better the performance, certainly where IOPS are concerned, and that is where HP's EVA scores with its ability to have over 150 disks in a single VRAID5 RAIDset Its really a RAID-50 implementation (striped RAID-5 sets, but with lots of extra smarts (what HP StorageWorks Division calls Secret-Sauce)) But conventional RAID controllers usually have a limit of 15 to 30 odd spindles per RAIDset, and as Andy and Gary have already said, 7-8 spindles is the usual recomendation due to rebuild times (although what controller Gary is talking about that can rebuild a 7 spindle RAIDset in a couple of hours, when 1TB spindle are becoming the standard, sounds a bit optimistic, especially if its supposed to being real work at the same time [1TB in 3 hours works out to be 100MB/s]).
Another reason that 7-8 spindles is often considered a maximum is down to your personal paranoia about the reliability of the shelves/trays/etc that your disks are plugged into. You have two choices -only have one spindle from your RAIDset in a shelf and therefore survive a shelf failure, or allow your RAIDset to wrap round and then unless you are using RAID-6 you won't! Now i realise its an unlikely scenario as shelves don't fail very often, but its a risk to consider.

I would also concur with Andy and Gary and recomend that you use RAID-6, although if your Write to Read ratio is high you will need to consider the penalty that running RAID-6 brings as very very few RAID controllers have 0% RAID-6 overhead (i only know of one, and its very expensive)

Hopefully you have already taken into account the IOPS, MB's/Sec and the SATA duty cycle constraints that come with using these big disks, and that you are not expecting this setup to be high performance.
0
 
LVL 55

Expert Comment

by:andyalder
ID: 24125337
None of us remembered to mention the possibility of unrecoverable read errors either, if a spindle fails what's the chance of one of the remaining disks also having a UWE on one sector in which case backup/restore is normally the only way to regain parity since the array can't be rebuilt. Bad that happen even on fairly good SCSI disks.

(to be honest I think they dropped the server because there was a whole bunch of UWEs at about the same sector number making it look like a partial head crash. And that was with a controller that does background parity scrubbing.)
0
Scale it in WD Gold

With up to ten times the workload capacity of desktop drives, WD Gold hard drives employ advanced technology to deliver among the best in reliability, capacity, power efficiency and performance.

 
LVL 55

Expert Comment

by:andyalder
ID: 24125367
I mean I had that happen, not bad that it happens - typo.
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 24136456
To better understand the very dangerous scenario of RAID 5/6 large arrays, I used the below spec sheet for WD RE3 Enterprise SATA 1TB drive
   http://www.wdc.com/en/library/spec/2879-701281.pdf

Those are the numbers:
-Unrecoverable read error per bits read : 1 bits per 10exp15 bits read
-1 RE3 1TB drive : 1953525168*512*8 bits
-At rebuild time, you have (N-1) drive to be read at once to rebuild a spare drive
-That is (N-1)x 1953525168*512*8 / 10exp15 probability for another unrecoverable read error
==> A raid 5 array using 14x 1TB drives would have a 10% probability to have another drive failure at rebuild time

Using the below numbers, you can choose your risk level...or decide to go with a RAID 10 array (which is at 0,8% level for 1TB drive).
 1-2      0,8%
 3      1,6%
 4      2,4%
 5      3,2%
 6      4,0%
 7      4,8%
 8      5,6%
 9      6,4%
 10      7,2%
 11      8,0%
 12      8,8%
 13      9,6%
 14      10,4%
 15      11,2%
 16      12,0%

NB: Some large SATA drives are claiming for a 1 unrecoverable bits read per 10exp14 bits read...that would be 10 times WORSE in a raid Array
0
 
LVL 70

Expert Comment

by:garycase
ID: 24140438
The possibillity of a unrecoverable bit error during a rebuild is indeed fairly high with large arrays; that's why I noted that with RAID-5 arrays "... the likelihood of a 2nd drive failing during a rebuild is a real concern."   But RAID-6 dramatically mitigates this, since it will recover from a single bit error even after one drive has failed.  This makes it very unlikely you'd have an issue during a rebuild unless a 2nd drive physically failed -- random unrecoverable bit errors would simply be corrected.
0
 
LVL 55

Expert Comment

by:andyalder
ID: 24141002
Gimme a Savvio 15K.2 any day, 1 URE in 10^16. (Mind you they do cost just a tad more).
0
 
LVL 70

Expert Comment

by:garycase
ID: 24141500
Your definition of "... a tad ..." is different than mine !! :-)
(But Seagate's Savvio series are VERY nice drives)
0
 
LVL 18

Accepted Solution

by:
BigSchmuh earned 125 total points
ID: 24141835
The final choices are:
a)RAID 60 using 2x6 drives + 2 spare (=8TB on a single GPT partition or 4x 2TB MBR partitions)
==> Requires a fine tuning (stripe size = 4KB for CIFS fileserving) and the parity problem (read old parity/recompute/write data and parity) may make the write io very slow

b) RAID 6 using 13 drives + 1 spare (=11TB on a single GPT partition or 6x MBR partitions)
==> Largest usable space, requires a fine tuning but the parity problem (read old parity/recompute/write data and parity) may make the write io very very slowwwww

c) RAID 10 using 7x2drives (=7TB on a single GPT partition or 4x MBR partitions)
==> Fastest one and most efficient without tuning. The only one mode to support write intensive io (more than 10% write)


Anyway :
-Please align any partition in your arrays to the stripe size (or full stripe size)
-In multiuser context, please consider the stripe size to be equal to the client io size to optimize the array IOPS
-In monouser context with raid 5/6, please consider the full stripe size to be equal to the client io size to turn off the write io problem
0
 
LVL 55

Expert Comment

by:andyalder
ID: 24142085
unsub.
0

Featured Post

Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

Join & Write a Comment

In this article, I provide some information on storage disks which go into calculations that will help you figure out how much Input/output Operations Per Second (IOPS) your disk subsystem can deliver. To effectively size & tune up applications l…
Ever notice how you can't use a new drive in Windows without having Windows assigning a Disk Signature?  Ever have a signature collision problem (especially with Virtual Machines?)  This article is intended to help you understand what's going on and…
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

760 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now