Hardware RAID vs OS RAID

Still trying to make an informed decision. Rindi I am really not trying to pick on you or make your life miserable. I am trying to make an informed decision and your knowledge of OS RAID is unmatched and your input greatly appreciated. I set up a Server 2012 with two Intel 540s SSD drives (480MB on the motherboard SATA ports using AHCI. I then mirrored the volumes.

On the good side TRIM was still supported on this RAID 1 configuration. TRIM was not supported on the Adaptec 8405 in a RAID 5 configuration (same SSD drives)

Now the downside (benchmarking with HDTune Pro):

Motherboatd SATA Port. Single SSD. Not yet mirrored:
       Min Transfer : 218MB/s
       Max Transfer: 225MB/s
Average Transfer: 222MB/s
                     Burst: 185MB/s

Motherboatd SATA Port. Single SSD. Mirrored:
       Min Transfer : 192MB/s
       Max Transfer: 224MB/s
Average Transfer: 204MB/s
                     Burst: 174MB/s

Adaptec 8405 RAID 5 (Same SSD Drives):
       Min Transfer : 547MB/s
       Max Transfer: 643MB/s
Average Transfer: 587/s
                     Burst: 570MB/s

It appears as though the throughput is about two to three times better with a hardware RAID controller. The downside is that the RAID controller with battery backup and a third drive for RAID 5 is another $600 (but the usable space does increase 480GB in RAID 5). Do those transfer numbers seem real to you? Thoughts?

Lastly LOL! I pulled drive 0 to see what would happen. No visible or audible notification. Can you set notification via email?

Now disk manager shows Drive an unknow drive as "missing". I plugged the drive back in thinking it would automatically remirror and it didn't. Drive 0 now shows up as Foreign. How do I re-establish the mirror?

I am going to keep playing with OS RAID. I want to give it a fair shot.
LVL 15
Who is Participating?
Your benchmark results are rather interesting.  Have you tried RAID 1 on the hardware RAID controller?  That would be very informative.

I have stayed away from RAID 5 for quite a while now.  Recovery from failure can be much more problematic.  This gets confirmed frequently here by Rindi and others.

With RAID 1, the drives are still readable (and recoverable) from non-RAID controllers.  Yes, you lose more disk space, but I consider the recoverability a very big bonus.  I've been successful at recovering RAID 5 failures with Raid Reconstructor, but it is time consuming and not always successful.
QlemoBatchelor, Developer and EE Topic AdvisorCommented:
It is quite normal that a replaced or re-inserted drive does not sync automatically - the OS cannot know the exact state and purpose (could be a reserve drive for hot swapping, or you might want to change the config, or ...). Just reconfigure the mirror set.

RAID 5 is slower than a mirror set (if the latter can make use of the disks in parallel). I would not expect the figures you've seen to be realistic, but I don't have an explanation unless you only copied small amount of data.
LockDown32OwnerAuthor Commented:
I have been running RAID 5 in servers since Netware 3.X and I have to tell you it has never missed. I even had a customer pull two of the three drives and when she put them back in and rebooted the Adaptec Card figured it out and it was back up and running in 10 minutes. Same on a failed drive with rebuilding. Never, ever had a problem. Maybe it is simply because I have never used anything but Adaptec RAID controllers. I have just never had RAID 5 do anything but work. Don't know what to tell you.

   Yes RAID 5 is slower (writing) then RAID 1 but you make up for it in fault tolerance and available space. Every RAID Levels has its advantages and disadvantages. As far as the figures being accurate? I would believe them. Same reasons I have always been given to go Hardware RAID. 1) The controller relieves the CPU from the burden 2) The 8405 has a 1GB hardware cache 3) PCIE 3.0. I have been told to go Hardware RAID for as long as I can remember for those reasons but as I said I really want to give OS RAID a fair shake.

   @CompProbSolv I will tried RAID 1 on the Adaptec 8405 when I have more time. Your comment "With RAID 1, the drives are still readable (and recoverable) from non-RAID controllers.  Yes, you lose more disk space, but I consider the recoverability a very big bonus.  I've been successful at recovering RAID 5 failures with Raid Reconstructor, but it is time consuming and not always successful." is that with respect to OS RAID or Hardware RAID?
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

My comment was specifically about RAID 1 vs. RAID 5.  I should have been clearer.

I would generally agree with a preference for hardware vs. software RAID, though cost considerations may tilt one toward software in some cases.

As far as fault tolerance goes, I'll respectfully disagree.  If you look at RAID 1 vs. RAID 5 with hot spare disks, RAID 1 (or 10) should be more tolerant.

I agree that RAID 1 will cost more because of lost disk space.
Jack RiderCommented:
RAID 5 with hot spare is a reasonable choice on high speed SAS drives (10K/15K) but not on a large (multi-TB), slow (7.2K) SATA or NL-SAS drive where the rebuild times are so long that a mult-disk failure becomes a real concern.
LockDown32, In NetWare 3.1 days the disk were 4.7GB so there was very little chance of a bad block on one disk causing a failed rebuild when another disk failed. Nowadays the disks can be 1000 times the capacity and therefore 1000 times more chance of a bad block on one disk and another one failing. Take a read of http://www.zdnet.com/article/why-raid-5-stops-working-in-2009/ - bear in mind that the disks in that article have a URE rate of 1 in 1^14 but it still applies. Bear in mind also though that disk manufacturers don't flag UREs as S.M.A.R.T errors and even enterprise disks can go flakey with a very high URE rate without giving a pre-failure alert. For high capacity low speed systems it's far safer to use RAID 6 or RAID 60.

Regarding that Adaptec card with SSDs there's a review at http://www.storagereview.com/adaptec_series_8_raid_controllers_review , There certainly was a time when SSDs on RAID controllers performed badly because reading directly from an SSD was faster than reading from the SSD into the controller and then out to the motherboard. RAID card manufacturers introduced a licensed bypass mechanism, you actually paid more to have fastpath etc. bypass the RoC but that seems to be inbuilt nowadays.

A proper test of software Vs hardware RAID would not use the onboard SATA ports, you can put your Adaptec card into HBA mode instead to benefit from the 12Gb SAS interface and the PCIe-3 connectivity.
LockDown32OwnerAuthor Commented:
My reference to Netware 3.X was intended to show how long I have been running RAID 5 without a problem. Even articles you read today say you won't run in to a problem until you start using multi-terabyte drives. Those 500GB SSDs I referred to above took only a little over an hour to rebuild. The articles also state that if you keep the drives under 1TB that RAID 5 is still and acceptable, cost effective approach.

I disagree with your suggestion of a proper test Software vs Hardware. I am trying to decide whether or not a Hardware RAID Controller is worth the cost vs no Hardware RAID Controller. Why would I want to put a Hardware RAID Controller in and essentially not use it? That would kind of be a waste of money.....

Right now the benefits if OS RAID on SATA ports are TRIM and cost...

The benefits of Hardware RAID look like speed and easy of use.  I have yet to encounter a problem with RAID 5 in all the years I have been using it and the same controller will also support RAID 6 should the need arise.....
You already have the card but aren't prepared to test it in HBA mode just out of interest? What a surprising lack of curiosity.
LockDown32OwnerAuthor Commented:
What would be the point? I am curious about things if there is ever a chance of using or implementing it but putting a $600 RAID Controller in a computer simply for HBA mode isn't one of them :)
Part of the reason why the 5 is so much faster the 1 is you are using a caching controller which makes all the difference in the world.

Disable the cache and you will see a world of difference in benchmarks.
He's not interested in disabling the cache to see what difference it makes, he's not curious about things that aren't beneficial to him even though it would only take a few minutes to do the experiment. Fortunately not all people think like that or we would never have put men on the Moon.
LockDown32OwnerAuthor Commented:
Deal with it andyalder. Not what I am after. I have better things to do with my time than to play with things that will never benefit me or that I will never use. Glad you have so much free time on your hands. What don't you get a card and try it if you are so interested in it?

Anyway I answered your question previously, Rindi's (presumed) previous statement that software RAID with SSDs may be faster than hardware RAID with SSDs comes from the days when it was true because those old controllers were designed for disks and couldn't read the SSDs fast enough.

Whether those 540s stand up to working consistently on a hardware controller without TRIM is debatable though, they're not enterprise drives that have lots of over-provisioning to do their own garbage collection in background.
LockDown32OwnerAuthor Commented:
Yes. That would be one of the downsides to using Consumer SSDs on a RAID Controller. The other would be power loss. I looked at the pluses and minus's. A recent article (I forget by who) said a Consumer SSD writing 50GB/day, every day,  would last 12 years (they didn't say with or without TRIM) and if you don't write 50GB /day it goes up from there. That is about twice as long as I leave servers in service.

   As far as power loss... I like it to a battery cache on a RAID Controller. Nice but a good APC is nice too. Not a complete answer but added insurance. It is always a game of bang for the buck......
Looking at your spec on a sata and ssd benchmark,no raid or other stuff involve,it appears as if it's very slow.
Could be an older chipset ,but I'm running a 5 year old Fujitsu lap(Intel core duo 2.2 ghz) and a 1 tb Patriot SSD  with 8 gb ram and my speed is way faster.
LockDown32OwnerAuthor Commented:
Good Catch. The above benchmarks were in an actual Intel S1200BTLR motherboard. C206 (Series 6?) chipset. Just re-ran on a Dell Optiplex 3040 which has I believe a Series 8 Chipset (H110) and it was much better. Still half the Adaptec but much better:

Motherboatd SATA Port. Single SSD. Series 8 Chipset:
       Min Transfer : 277MB/s
       Max Transfer: 377MB/s
Average Transfer: 341MB/s
                     Burst: 185MB/s
How much data was transferred in the tests?  More specifically, how does it compare to the 1G cache on the controller?

Also.... according to this:

the H110 supports 6Gb/s SATA rate.

According to this:
the Adaptec 8405 supports up to 12Gb/s SATA rate.

Could the difference in the native transfer rate be the real issue here?
LockDown32OwnerAuthor Commented:
As far as the amount of data transferred in the tests... you'll have to call HD Tune :) As is always the case no one uses the same benchmark program so the results are skewed.

The Intel 540s Series SSD is 6Gb/s so it will never put out 12Gb/s no matter which controller you use. If what I have been told about RAID Controllers is true that by using multiple drives (3 in a RAID 5) you pump more in to the controller (3 times more) you you get better throughput. That coupled with a 1GB cache should make the Adaptec a lot faster.
A reasonable conclusion at this point would be that the RAID 5 arrangement achieves faster transfer rates by the factor of reading multiple drives at once when drives are the slow part of the chain.

As far as 3 drives in RAID 5 goes, I'd expect that you could only get double the raw transfer speed and not three.  Remember that one of the three at any moment is used for error checking, not data.  Nonetheless, double the rate is significant!

I would think that the last test is to see how hardware and software (OS) RAID 1 perform.  If they take advantage of the ability to read both drives at once then the read throughput should be about double that of a single drive.  This would put it in line with what you saw with RAID 5.

I don't see any mention (may have missed it) as to whether your HDTune results are for reads or for writes.  That is rather important.

As far as "you'll have to call HD Tune".... last time I used it, I believe that there were settings for how much data you were transferring.  That's a critical question, especially when there is cache involved.  If you are doing reads and writes that stay in the cache, the RAID controller will be especially fast.  If that matches your real-world needs, then it is a meaningful test.  If it does not, then a test more like your real-world needs would be appropriate.
David Johnson, CD, MVPOwnerCommented:
When it comes to benchmarking there are so many variables involved so one always has to compare apples vs apples.  Benchmarks may not reflect actual use.  In general avoid the fake raid controllers that are on the motherboard like the plague. If you have hardware raid with battery backed cache use it over Software Raid.

Right now the benefits if OS RAID on SATA ports are TRIM and cost... Agreed

The benefits of Hardware RAID look like speed and easy of use.  I have yet to encounter a problem with RAID 5 in all the years I have been using it and the same controller will also support RAID 6 should the need arise..... if the total data in the array is >12TB (doesn't matter on the individual disk size) then the URE problem could bite you. Better to side on the conservative side.

What you should take into consideration is the # of IOPS available and the number of IOPS that you feel are required for the the expected load on the server. Items like Remote Desktop Sessions, Roaming Profiles, SQL Databases will affect the # of IOPS required.  

Always remember that RAID is not backup
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.