Solved

Does traditional HDD RAID lose performance over time?

Posted on 2013-05-27
8
481 Views
Last Modified: 2013-05-29
So I've been noticing slowdown of my 4-disk RAID10 setup on my ICH10R.  I've never actually benchmarked it, but after recently installing an SSD I started doing so for comparison.

I was getting abysmal performance metrics of my RAID10 volume.  Using CrystalDiskMark 3.0.2 x64, I was seeing the following:

------------------------------------------------------------------------
Volume:  Raid10 at 64k stripe
Test : 500 MB [D: 61% (363.0/596.0 GB)] (x5)
OS : Windows 8 Professional [6.2 Build 9200] (x64)
------------------------------------------------------------------------
Sequential Read :  76.67 MB/s                  Sequential Write :  133.9 MB/s
Random Read 512KB :   26.85 MB/s            Random Write 512KB :  57.95 MB/s
Random Read 4KB (QD=1) :  0.317 MB/s            Random Write 4KB (QD=1) :  2.273 MB/s
Random Read 4KB (QD=32) :  2.040 MB/s      Random Write 4KB (QD=32) :  2.371 MB/s


I intended to rebuild into a RAID0 configuration as my backup scheme has become far more robust, but I wanted to test the RAID10 again after blowing it away and recreating it.  After doing so, my performance increased nearly 3 fold for seq. read, and up to 2 fold for other metrics.  

------------------------------------------------------------------------
Volume:  Raid10 at 64k stripe
Test : 500 MB [D: 0.0% (0.2/596.0 GB)] (x5)
OS : Windows 8 Professional [6.2 Build 9200] (x64)
------------------------------------------------------------------------
Sequential Read :  212.549 MB/s            Sequential Write :  190.535 MB/s
Random Read 512KB :  44.323 MB/s            Random Write 512KB :   80.019 MB/s
Random Read 4KB (QD=1) :  0.652 MB/s            Random Write 4KB (QD=1) :  2.240 MB/s
Random Read 4KB (QD=32) :  4.302 MB/s      Random Write 4KB (QD=32) :  3.216 MB/s


At first I figured it may be due to fragmentation, but Windows did report 0 fragmentation before the first test.  Could it be the disk usage being 61% on the first test and only 0% on the second?  Or does RAID actually lose performance over time?  I've never heard of that.

Outside of that, these numbers don't reflect what I would expect from a 4 drive RAID10.  It is my understanding that the onboard ICH10R chipset has some issues with RAID10, per some searching I found.  Too bad SSD is expensive/GB, otherwise I'd replace this RAID entirely.

For comparison, my SSD:

------------------------------------------------------------------------
Drive:  Samsung 840 PRO SSD
Test : 500 MB [D: 66.0% (79/119.0 GB)] (x5)
OS : Windows 8 Professional [6.2 Build 9200] (x64)
------------------------------------------------------------------------
Sequential Read :  467.2 MB/s            Sequential Write :  333.6 MB/s
Random Read 512KB :  406.2 MB/s            Random Write 512KB :   178.1 MB/s
Random Read 4KB (QD=1) :  24.07 MB/s            Random Write 4KB (QD=1) :  60.98 MB/s
Random Read 4KB (QD=32) :  328.4 MB/s      Random Write 4KB (QD=32) :  176.9 MB/s



Same 4 HDD above in Raid0 128K  (I have large files)

------------------------------------------------------------------------
Volume:  Raid0 at 128k stripe
Test : 500 MB [D: 0.0% (0.2/1192.0 GB)] (x5)
OS : Windows 8 Professional [6.2 Build 9200] (x64)
------------------------------------------------------------------------
Sequential Read :  409.5 MB/s            Sequential Write :  299.6 MB/s
Random Read 512KB :  55.06 MB/s            Random Write 512KB :   124.7 MB/s
Random Read 4KB (QD=1) :  0.702 MB/s            Random Write 4KB (QD=1) :  4.726 MB/s
Random Read 4KB (QD=32) :  4.405 MB/s      Random Write 4KB (QD=32) :  7.088 MB/s
0
Comment
Question by:mcdonamw79
  • 4
  • 4
8 Comments
 
LVL 47

Expert Comment

by:dlethe
ID: 39200180
Well, first, that controller is crap. It doesn't even do read load balancing.  If you want to double read performance (in perfect world), turn off that RAID controller and let the O/S do 2 x RAID1 instead of a single RAID10.  

Your performance hit is most likely bad block recovery, which that controller , (if you want to call that a controller), doesn't handle well. It locks up and doesn't get the data from the other HDD in RAID set in parallel with remapping a bad block.
0
 

Author Comment

by:mcdonamw79
ID: 39200236
Yeah I recently found out about the lack of read load balancing, which was rather discerning.  Crap controller aside, would the bad block recovery you speak of come into play in all of my testing scenarios?  I'm using the same controller in each test, so I would expect its shortcomings to exist regardless.  

Ultimately I'm not understanding why recreating the RAID10 netted me such greater performance.  That's all I'm really looking to understand here.
0
 
LVL 47

Expert Comment

by:dlethe
ID: 39200291
Well bad blocks only happen once per block, as each one is remapped.  Only way to know for sure is to get some decent diagnostic software that maps out all blocks (without attempting recovery) before you run a test.

that software isn't cheap.   Performance has two major metrics, throughput and I/Os per second. They are mutually exclusive and all have different performance characteristics based on block size, reads vs. writes, random vs. sequential, and permutations involving combinations of all of the above.    Suffice to say, you're wasting your time tuning with this fakeraid config. No way can you get better performance no matter what you do with that controller vs. just using native host-based software raid.

Also benchmarking is rarely done right, and don't get hung up by synthetic loads that they generate.  Even many storage pros screw them up.  If your O/S does journaling and if you are benching a mounted filesystem or even a boot device and aren't looking at recovered/unrecovered errors then you can't possibly nail the results.

Spend your time dumping that fakeraid and going to software RAID. It is a better use of your time.
0
 

Author Comment

by:mcdonamw79
ID: 39203208
I concede that trying to get better performance from the ICH10R is pointless. I am not trying to do this. I am merely trying to understand why breaking an existing array and recreating it exactly the same yielded 2-3 times performance increase. Am I merely misunderstanding you?
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 
LVL 47

Expert Comment

by:dlethe
ID: 39203261
Because you likely had RECOVERABLE read errors.  A rebuild forces full reads of all blocks, and it could take up to 30 secs for a single block.   This can be confirmed (well, could have before you began the rebuild).
0
 

Author Comment

by:mcdonamw79
ID: 39205712
Gotcha. Perhaps that is what it was. I do know I ran a verify on the Raid array using the Intel RST before I blew it away. It reported no errors. Perhaps I will never know. Either way I think my primary question is answered. That was: do RAID arrays lose performance over time. Outside of block errors I am assuming this is not possible. Correct?
0
 
LVL 47

Accepted Solution

by:
dlethe earned 500 total points
ID: 39205747
Correct, allowing also for a statistically insignificant factors such as entropy, heat due to crud that builds up everywhere, etc ..
0
 

Author Closing Comment

by:mcdonamw79
ID: 39206269
Thank you for your time.
0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

Ever notice how you can't use a new drive in Windows without having Windows assigning a Disk Signature?  Ever have a signature collision problem (especially with Virtual Machines?)  This article is intended to help you understand what's going on and…
I previously wrote an article addressing the use of UBCD4WIN and SARDU. All are great, but I have always been an advocate of SARDU. Recently it was suggested that I go back and take a look at Easy2Boot in comparison.
This video Micro Tutorial explains how to clone a hard drive using a commercial software product for Windows systems called Casper from Future Systems Solutions (FSS). Cloning makes an exact, complete copy of one hard disk drive (HDD) onto another d…
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now