Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions

What is maximum number of disc recommanded in RAID 5 Array?

Posted on 2009-04-09
Last Modified: 2013-11-14
I'm having 14 * 1 TB HDD.

I would like to make RAID 5 Array using those 14 discs.

What is the recommanded number of discs can be used in Single Array?

Like can we use all 14 discs and make it as a single Array or can we use less number of discs and make it as mulitple arrays?

Which one is recommanded?

Which will give good performance?

Please help me.
Question by:erteam
  • 6
  • 4
  • 2
  • +2
LVL 13

Expert Comment

ID: 24107651
Generally speaking the more spindles (disks) you have in a RAID5 the better your performance is.

The number of diks you can put into a single array depends on the specific hardware (controller and enclosure) you are using
LVL 55

Expert Comment

ID: 24108494
HP recommend not to have more than 7 or 8, reason being if a disk fails the performance goes through the floor and the chance of a second disk failing before the array is rebuilt gets too high. For more than 8 they would suggest using RAID 10 or RAID 6 depending on whether you want capacity or performance. Another option to tackle it would be to use RAID 50.
LVL 70

Expert Comment

ID: 24110389
As noted above, the number of disks you can use in a RAID-5 array is limited by your controller.  If your controller supports it, you can easily use 14 drives in the array.

However, with 14 disks, the concern Andy noted is very real -- the likelihood of a 2nd drive failing during a rebuild is a real concern.   I'd recommend you get a controller that supports RAID-6 and use these drives in a RAID-6 array => you'll get the same level of performance, and your array will tolerate two drive failures with no data loss.   This provides superb fault tolerance; and makes it very unlikely you'll ever lose data during a rebuild.
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

LVL 55

Expert Comment

ID: 24111614
erteam could do an experiment for us, build a 14 disk RAID 5 with 1TB SATAs (HP were talking about 10K SCSI when they said max 8 for safety) and then yank one of them out - I presume they are hotplug. Shove it back in again and record how long it takes to rebuild.

What's the duty-cycle of the disks? HP's 8 disk max was for 100% duty-cycle disks but SATAs (ignoring 10K ones with SCSI/SAS capacities like raptors) are rated at only 40% duty-cycle and rebuilding is a full-time job. I expect them to go into slow read-after-write mode to protect themselves from overheating but as yet I have no proof. So I'd really like erteam to do the experiment.

There's a good rebuild/performance whitepaper from HP (you can guess I work for an HP house) but it only covers RAID 5, RAID 6 rebuild/performance time would be interesting to see.
LVL 70

Expert Comment

ID: 24111920
The rebuild time is very much a function of the controller.   The newer controllers with the 1680 chips can rebuild a drive in an array like that in a matter of 2-3 hours.  The older 12xx series chips would take ~ 25 hours.

These numbers are based on a similar system detailed on the AVS forum with 20 1TB drives.
LVL 17

Expert Comment

by:Gerald Connolly
ID: 24125242
As Randy said the basic premise is that the more disks in a RAID stripe (RAID-5 is striping with redundancy) the better the performance, certainly where IOPS are concerned, and that is where HP's EVA scores with its ability to have over 150 disks in a single VRAID5 RAIDset Its really a RAID-50 implementation (striped RAID-5 sets, but with lots of extra smarts (what HP StorageWorks Division calls Secret-Sauce)) But conventional RAID controllers usually have a limit of 15 to 30 odd spindles per RAIDset, and as Andy and Gary have already said, 7-8 spindles is the usual recomendation due to rebuild times (although what controller Gary is talking about that can rebuild a 7 spindle RAIDset in a couple of hours, when 1TB spindle are becoming the standard, sounds a bit optimistic, especially if its supposed to being real work at the same time [1TB in 3 hours works out to be 100MB/s]).
Another reason that 7-8 spindles is often considered a maximum is down to your personal paranoia about the reliability of the shelves/trays/etc that your disks are plugged into. You have two choices -only have one spindle from your RAIDset in a shelf and therefore survive a shelf failure, or allow your RAIDset to wrap round and then unless you are using RAID-6 you won't! Now i realise its an unlikely scenario as shelves don't fail very often, but its a risk to consider.

I would also concur with Andy and Gary and recomend that you use RAID-6, although if your Write to Read ratio is high you will need to consider the penalty that running RAID-6 brings as very very few RAID controllers have 0% RAID-6 overhead (i only know of one, and its very expensive)

Hopefully you have already taken into account the IOPS, MB's/Sec and the SATA duty cycle constraints that come with using these big disks, and that you are not expecting this setup to be high performance.
LVL 55

Expert Comment

ID: 24125337
None of us remembered to mention the possibility of unrecoverable read errors either, if a spindle fails what's the chance of one of the remaining disks also having a UWE on one sector in which case backup/restore is normally the only way to regain parity since the array can't be rebuilt. Bad that happen even on fairly good SCSI disks.

(to be honest I think they dropped the server because there was a whole bunch of UWEs at about the same sector number making it look like a partial head crash. And that was with a controller that does background parity scrubbing.)
LVL 55

Expert Comment

ID: 24125367
I mean I had that happen, not bad that it happens - typo.
LVL 18

Expert Comment

ID: 24136456
To better understand the very dangerous scenario of RAID 5/6 large arrays, I used the below spec sheet for WD RE3 Enterprise SATA 1TB drive

Those are the numbers:
-Unrecoverable read error per bits read : 1 bits per 10exp15 bits read
-1 RE3 1TB drive : 1953525168*512*8 bits
-At rebuild time, you have (N-1) drive to be read at once to rebuild a spare drive
-That is (N-1)x 1953525168*512*8 / 10exp15 probability for another unrecoverable read error
==> A raid 5 array using 14x 1TB drives would have a 10% probability to have another drive failure at rebuild time

Using the below numbers, you can choose your risk level...or decide to go with a RAID 10 array (which is at 0,8% level for 1TB drive).
 1-2      0,8%
 3      1,6%
 4      2,4%
 5      3,2%
 6      4,0%
 7      4,8%
 8      5,6%
 9      6,4%
 10      7,2%
 11      8,0%
 12      8,8%
 13      9,6%
 14      10,4%
 15      11,2%
 16      12,0%

NB: Some large SATA drives are claiming for a 1 unrecoverable bits read per 10exp14 bits read...that would be 10 times WORSE in a raid Array
LVL 70

Expert Comment

ID: 24140438
The possibillity of a unrecoverable bit error during a rebuild is indeed fairly high with large arrays; that's why I noted that with RAID-5 arrays "... the likelihood of a 2nd drive failing during a rebuild is a real concern."   But RAID-6 dramatically mitigates this, since it will recover from a single bit error even after one drive has failed.  This makes it very unlikely you'd have an issue during a rebuild unless a 2nd drive physically failed -- random unrecoverable bit errors would simply be corrected.
LVL 55

Expert Comment

ID: 24141002
Gimme a Savvio 15K.2 any day, 1 URE in 10^16. (Mind you they do cost just a tad more).
LVL 70

Expert Comment

ID: 24141500
Your definition of "... a tad ..." is different than mine !! :-)
(But Seagate's Savvio series are VERY nice drives)
LVL 18

Accepted Solution

BigSchmuh earned 125 total points
ID: 24141835
The final choices are:
a)RAID 60 using 2x6 drives + 2 spare (=8TB on a single GPT partition or 4x 2TB MBR partitions)
==> Requires a fine tuning (stripe size = 4KB for CIFS fileserving) and the parity problem (read old parity/recompute/write data and parity) may make the write io very slow

b) RAID 6 using 13 drives + 1 spare (=11TB on a single GPT partition or 6x MBR partitions)
==> Largest usable space, requires a fine tuning but the parity problem (read old parity/recompute/write data and parity) may make the write io very very slowwwww

c) RAID 10 using 7x2drives (=7TB on a single GPT partition or 4x MBR partitions)
==> Fastest one and most efficient without tuning. The only one mode to support write intensive io (more than 10% write)

Anyway :
-Please align any partition in your arrays to the stripe size (or full stripe size)
-In multiuser context, please consider the stripe size to be equal to the client io size to optimize the array IOPS
-In monouser context with raid 5/6, please consider the full stripe size to be equal to the client io size to turn off the write io problem
LVL 55

Expert Comment

ID: 24142085

Featured Post

Back Up Your Microsoft Windows Server®

Back up all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
San to San Replication Slowness/Issues 3PAR 7000 5 41
Disk Cloning 9 52
SDD and HDD laptops 22 62
SAS Storage with Vcenter 6.0 4 38
When we purchase storage, we typically are advertised storage of 500GB, 1TB, 2TB and so on. However, when you actually install it into your computer, your 500GB HDD will actually show up as 465GB. Why? It has to do with the way people and computers…
Many businesses neglect disaster recovery and treat it as an after-thought. I can tell you first hand that data will be lost, hard drives die, servers will be hacked, and careless (or malicious) employees can ruin your data.
This video Micro Tutorial explains how to clone a hard drive using a commercial software product for Windows systems called Casper from Future Systems Solutions (FSS). Cloning makes an exact, complete copy of one hard disk drive (HDD) onto another d…
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

789 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question