Solved

RAID 6 vs 10

Posted on 2011-03-07
24
961 Views
Last Modified: 2012-05-11
Hello experts, I need some advice. I have done some research on the different RAID configurations, but would like your advice for my specific situation. First - some background: Due to a bad batch of hard drives (I can only assume) I have suffered 3 catastrophic failures of RAID 5 arrays in the last 18 months, so I am kinda jumpy when it comes to my servers dropping their RAID array.

I am in the position where I am preparing to rebuild a server that will be my organization's main storage server. This server will have Windows Server 2008 on a separate (RAID 1) array. This server will ONLY be a storage server - no other services will be running on it. I have about 200 "heavy" users that access the files frequently and another 1500 "light" users that only access files occasionally. My server has six 500GB hard drives available - storage capacity is not as big a concern as reliability, but I do have about 850GB of data to store (at this point in time).

Which RAID configuration should I use. From what I have read I see the following options:
 - RAID 10 (4 drives in the array, 2 hot spares): I could lose 1 drive, and maybe 2 (depending on which drives failed) at the same time. Performance would be best
 - RAID 6 (5 drives in the array, 1 hot spare): I could lose 2 drives at the same time - regardless of which drives fail, but performance would be decreased. I would also have more capacity.

I am also open to other suggestions. Thank you.
0
Comment
Question by:Lotocus
  • 12
  • 6
  • 4
  • +1
24 Comments
 
LVL 11

Expert Comment

by:Plantwiz
Comment Utility
What type of drives are you using?

Scsi or sata?

What is the platter size?  2.5 or 3.5"?

which brand of drive are you using?
0
 

Author Comment

by:Lotocus
Comment Utility
Seagate Barracuda 7200.11 SATA drives. I'm not sure on the platter size.

Here is a link to the manufacturer specs: http://www.seagate.com/ww/v/index.jsp?vgnextoid=c89ef141e7f43110VgnVCM100000f5ee0a0aRCRD#tTabContentSpecifications
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
RAID6 has highest degree of protection.  

As you are obviously buying SATA instead of SAS disks, then you certainly won't have a performance issue if you go SAS.  Were you even using enterprise class SATA, or cheap desktop disks?

Anyway, pay the money and get SAS drives, I recommend SAS-2, and a quality SAS-2 controller with a battery backup unit for cache.  YOu will not have any data reliability or performance issues with that.
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
The 'cudas are DESKTOP class disks.  No wonder you have the problems you do. They are totally unsuitable for RAID5, let alone 24x7.   That is root cause.  You did the equivalent of overclocking your disk drives.
0
 

Author Comment

by:Lotocus
Comment Utility
I just realized that... I am VERY unhappy with the vendor who sold me this server. I was not experienced with purchasing servers at the time and didn't know the difference.

I'm stuck with these for the time being, as I can't afford to purchase $2,000 in new hard drives, so I need to get the most reliability that I can from what I have. Suggestions?
0
 
LVL 55

Accepted Solution

by:
andyalder earned 42 total points
Comment Utility
No wonder they keep dropping out, those disks don't have Error Recovery Control so whenever they suffere a dodgy block they go into deep recovery taking up to 30 seconds to read the data in which time the controller has decided they are faulty and thrown them out of the array. You need to swap them for Barracuda ES which do have ERC.

I'd use RAID 10, not just for write performance but for the speed of rebuilding after a disk is replaced.
0
 
LVL 55

Expert Comment

by:andyalder
Comment Utility
Sorry dlethe, it looks like I copied your post, truth is I type slow!
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
I would first go to the original vendor.  If they are an authorized reseller for a server system and sold you an invalid configuration, then you may have a little bit of leverage.

Your alternatives are:

1. Use Solaris & zfs.  Their RAIDn tolerates consumer-class disks and adds the necessary redundancy and data integrity within zfs & the kernel.  This will absolutely work with desktop drives.
2. RAID1 + software RAID.  This is not as good as solaris, but native windows RAID will tolerate desktop drives.
3. Some 3WARE/AMCC controllers tolerate desktop-class RAID.  I don't keep up with what models work, but a few are architected to handle it.
4. Live with it and backup often.

Other than that, sorry you can't afford the right kind of disks for the job and your supplier scr*wed you.  But if you value your data you are just going to have to do something such as above.
0
 

Author Comment

by:Lotocus
Comment Utility
Thanks for the suggestions. I'm going to have to calm down a bit before I pen an email to the vendor. Needless to say any hope of future orders from me have dissolved.

I appreciate all your comments. I'll award you both points in a little while.
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
RAID5, 6, and to a lesser-extent, RAID10 is just too risky.  The more disks you have depending upon each other in a RAID group, the higher the probability it will fail.  Reliability, in your case is inversely proportional to the number of disks you have in an array, so the least dangerous for you is RAID1.
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
Curious, is this a national vendor, or a local shop?  For the benefit of all who may follow, feel free to announce to the world who those bozos are that either knowingly sold you an unsupportable configuration, or just as bad, didn't know any  better.
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
One other thing ... even IF you get a RAID configuration that "tolerates" those disks, it still does not change the fact that those drives are architected for a whole 2400 hours use per year, on a 8 hour per day x 300 days per year duty cycle.   This information is buried in the product manuals (at least the older versions).  

So find this little nugget and you could very well have basis for small claims court lawsuit, or at least the threat of one may be enough for them to do the right thing :)
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 11

Expert Comment

by:Plantwiz
Comment Utility
Wow, sorry it took a bit ot get to a full keyboard.


@Lotocus:
I had asked about the type of drives and sizes of platters and such as these do play a role in the reliability of theRAID.

I echo dlethe and andyalder comments.

I would say that the drives you have are not bad drives, however, they are not likely the best for the volume of use you stated they are receiving.

Take care in reaming your vendor on the drives (especially if budget was a concern at the time of purchase).  I have seen clients opt for the less costly SATA drives over SAS or older SCSI (and SAS are a new type of SCSI) despite being given the facts of the types of drive...sometimes price drives the project.  Performance drives are very expensive in comparision to SATA drives.  Nothing bad, except when they start to fail, the cost savings is quickly chewed up with down time and replacement drives.

In my experience the 2.5" platters cost a good deal more than the 3.5" and seem to fail more frequently in high usage environments.  So, when given the option, I'll select the 3.5" untl the 2.5" begin to show better results.  However, both are adequate if the proper drive type is utilized for the purpose.

I understand if you are 'stuck' for now, but as you are able to upgrade, or possibly work out the ROI of changing the drives out sooner, look toward the SAS models.

Good luck
0
 

Author Comment

by:Lotocus
Comment Utility
@Plantwiz

Thanks for the warning. I am waiting a while to send my letter to the vendor - just so I calm down a little and make sure I get all my facts straight. I am pretty sure I just asked them to quote me a price - and I know there was no discussion about the type of hard drives that would be included. This was one of the first servers I ever purchased (my prior jobs didn't involve purchasing or specing servers), so I didn't really know what to look for, but I still consider it bad practice to equip a server with a hard drive that is not designed to take the work load.
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
Here is the ammo you need.   (But in light of you issuing more of a purchase order and not having them configure something for you, then certainly you bear the responsibility unless they somehow certified or built the config for you).

Link to the manual that references the QUOTED text below is: http://www.seagate.com/ww/v/index.jsp?locale=en-US&name=barracuda-support&vgnextoid=dadc63a0d92b5210VgnVCM1000001a48090aRCRD

======================================
On general reliability and duty cycle

AFR and MTBF specifications are based on the following assumptions for desktop personal computer environments:
•2400 power-on-hours per year.
•10,000 average motor start/stop cycles per year.
•Operations at nominal voltages.
•Temperatures outside the specifications in Section 2.8 may reduce the product reliability.
•Normal I/O duty cycle for desktop personal computers. Operation at excessive I/O duty cycle may degrade product reliability.
The desktop personal computer environment of power-on-hours, temperature, and I/O duty cycle affect the product AFR and MTBF. The AFR and MTBF will be degraded if used in an enterprise application

===========

http://www.seagate.com/docs/pdf/whitepaper/mb538-drive-selection-guide.pdf

Link above says, among other things that the 'cuda is "On as needed, 8 hrs/day 5 days a week", and mentions other disks designed for always on 24x
It also specifically states "Desktop RAID 0+1/JBOD with less than five drives".  (This is consistent with what I said about windows software-based RAID).  But you'll notice, no RAID5 or RAID10.


0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
The suitability beyond just duty cycle, i.e, the deal-killer is that as Andy pointed out, the lack of error control, which is sometimes referred to TLER.  This bit of magic is a firmware feature required by most RAID controllers.  This is why you need to go to RAID1 if you value your data .. the inherent incompatibility that results in disks going offline.  The drive selection sheet does not have a tick in the "error control" column for the cudas.

Beyond that, the error rate is awful 1xE14, and the rotation vibration tolerance.  Even if the disks had the right firmware, you're throwing away a lot of performance because in a mult-enclosure array they shake so much I guarantee you are getting enough errors to adversely affect performance.

Well, I think this horse has been beaten dead.  Good luck, and let us know if you have any luck with the vendor.
0
 
LVL 11

Assisted Solution

by:Plantwiz
Plantwiz earned 42 total points
Comment Utility
@Lotocus:

Definately take it as something learned for future specs.

I understand your frustation with this matter.  Because pricing is so critical, I have had client come to me complaining that a particular vendor is 'sticking it to them' in price.  However, when I go over their quote and explain to them what they are getting compared to what they could have with cheaper parts...they are a little less ticked off, but too frequently (in my experience) I see clients that despite their needs...focus on cost.

Even the best hard drives, if and when you can find the engineering specs, won't be rated for 24 hour x 7 day a week usage.  From experience and peer conversations we all learn how other use these drives and learn over time which tend to have the best performance long-term.

Generally, I've found that it works best to prepare clients that their hard drives will fail at some point.  Ideally we'll get 2-3 years but plan for upgrading in 18-months (for budgeting purposes) but typically we stretch servers 4-6 years depending on the environment and their overall size.

I have a couple clients where we have planned to upgrade drives every 12-14 months and because of their environement and needs, this works in their business model.  This is NOT ideal for everyone, but this is a client who must have servers online nearly 100% of the time and therefore we have lots of redundancy built into their design.

SATA drives work well for a cheap RAID.  Occassionally, one gets 'lucky' and has great success with a cheaper RAID set inside a higher end environement...just know, it is not the norm.

Don't beat yourself up too much.  Quoting accurately takes practice from both the end-user and the client in gathering/providing of information.  Sometimes the end-user just doesn't know what they need, which makes it challenging for the vendor to recommend something.  I've sold a lot more servers that were 'lower' end because they do meet the needs of most of the environments.  However, from experience, I present my clients with their options and different scenerios as well as ROI on if they buy too low of a grade, they'll be paying me to 'fix' more problems and long term...this costs them more money and downtime.

There is a lot to know.  Be patient.  And now you'll be better prepared.    However, this doesn't mean I wouldn't question the vendor about their product.  I simply caution that they may not have had the best information to make a recommendation, so be careful on the accusations.   They may offer you a deal on an upgrade for discussing this matter with them or they may be able to work out a better deal when you are in a position to replace the whole server.  Keep an eye on the long-term relationship IF this is the first bad experience.   YMMV
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
I disagree with you a bit, Platwiz.  I assure you that some disks are architected for 24x7x365x5.  I know of sites with thousands of Seagate FC disk drives (mostly in the oil & gas biz in Houston) that have been running racks upon racks of disks for over 5 years.  Only reason they stopped after 5 years is to get a technology refresh because the 5-year manufacturer's warranty expired so it makes sense to replace the drives and go with higher density, lower power.

Once you get into enterprise SAS, SCSI, and fibre channel, then you have a superior product that really is designed to provide trouble-free operation for years to come.  
0
 
LVL 47

Assisted Solution

by:dlethe
dlethe earned 41 total points
Comment Utility
On the flip side, I do work with a Dept. of Defense contractor that deploys  disk drives in interesting places that involve a lot of sand.  On place, in particular, has this type of sticky sandy dust that gets everywhere.  Average life is about 3 months, and they use triple redundancy I.e, RAID7.   Interestingly enough they go with the cheapest disks they can find because all disks pretty much burn out at the same rate due to the extreme environmental conditions.  

Google also uses cheapo desktop class drives, and they have a type of distributed software-based RAID that is highly redundant.   The lesson here is that there is a place for all storage technologies, even the cheap desktop class.  Get the right kind of RAID, i.e, software-based RAID1 and you'll be OK.  (Or seriously consider using that server with solaris for the storage farm, then using a lower-end PC for the Win2K8 server functionality, less file serving, then you can use the RAIDZ1 or RAIDZ2 to replace RAID5 or RAID6.)

If  you have a fear of solaris + zfs.  Please do yourself a favor and read up on it.  Compatibility is there, it has hot snapshot, iSCSI, de-dup.  You can mix & match any make/models, even with different sizes to make any RAID level you want, and expand hot.   You can even just stick in a SSD in a RAID set, of ANY size, and it will be used as cache to speed up performance.  
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
(In fact, a lot of people just use vmware or virtualbox and run BOTH solaris for the back-end file server, and Win2K8 for the "services" front end.  There are right and wrong ways to set this up, but you already have everything you need, and Sun Solaris 11 is free.  I would log a new question in EE and find some people that have done this and get their opinions for yourself, or go to Vmware or virtualbox sites and research.   You could even use some migration tools to automate this.

I think you'll be quite happy with this solution.  Please read up.  
0
 

Author Comment

by:Lotocus
Comment Utility
You all have been very helpful... What I thought was a simple question has certainly given me a lot to consider. @dlethe, I'll have to spend some time looking to the options you suggest,
0
 
LVL 11

Expert Comment

by:Plantwiz
Comment Utility
@dlethe:

>I disagree with you a bit, Platwiz.  I assure you that some disks are architected for 24x7x365x5.  I >know of sites with thousands of Seagate FC disk drives (mostly in the oil & gas biz in Houston) that >have been running racks upon racks of disks for over 5 years.

I have clients who run disks easily this long as I've stated above.  However, I've read through engineering docs from some of the HDD mfgs and they would not committ to their drives working for 24x7.  However, experience has shown that they do.  Paint shops and auto places are as greasy and dusty so, these are about as intense environments as oil/gas.  I don't disagree with you, but I am clearing up what you may have interpreted what I stated.  I've read the documents.

If they did not work this well, Seagate for one would not warranty drives for 5 years.

I've had great success with Seagate.  My point to the OP is that SATA drives (as with all HDD) can and do fail.  

I think he/she has what he needs.  

I'd prefer he doesn't error on ripping off a vendor's head (even if it seems the right thing to do today), but look to find the opportunity for the future with that vendor.  He/she may find there will be more gained by approaching the conversation with knowledge rather than telling the vendor they sold him/her crap.
0
 

Author Comment

by:Lotocus
Comment Utility
I am glad I have waited to contact the vendor. There are some things I will re-word when I speak to the vendor, but I am still frustrated that they didn't even talk to me about the hard drives. I went back and checked my email traffic. I asked them to quote me a server with certain specs (quad core processor, 6 500GB SATA hard drives in a RAID5, and some other stuff). I never asked for a specific drive - depending on their expertise to fill in the blanks with appropriate hardware.

This was in April of 2009. Aren't there SATA drives that are more appropriate for servers?
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
We're all on same page.  I was there working for RAID manufacturers when we were laughing and joking about no way would SATA disks ever get put into desktop systems.  (When SATA appeared on market, Seagate told us this was strictly nearline storage, like what you would use for tape libraries, and not suitable for day-to-day or even desktop).

Boy, were we wrong :)

Actually at least at one time, the fibre channel had 7 year warranty.  5 years for warranty replacement to end-user, 7 years to the OEM.
0

Featured Post

Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

Join & Write a Comment

If you have a USB Drive that is not recognized by Windows the problem is usually that you have too many network drives or other drives that occupy all the drive letters D: E: or F: which is the normal drive letter of a usb drive. The way to correct …
The article will include the best Data Recovery Tools along with their Features, Capabilities, and their Download Links. Hope you’ll enjoy it and will choose the one as required by you.
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now