Solved

Does traditional HDD RAID lose performance over time?

Posted on 2013-05-27
8
483 Views
Last Modified: 2013-05-29
So I've been noticing slowdown of my 4-disk RAID10 setup on my ICH10R.  I've never actually benchmarked it, but after recently installing an SSD I started doing so for comparison.

I was getting abysmal performance metrics of my RAID10 volume.  Using CrystalDiskMark 3.0.2 x64, I was seeing the following:

------------------------------------------------------------------------
Volume:  Raid10 at 64k stripe
Test : 500 MB [D: 61% (363.0/596.0 GB)] (x5)
OS : Windows 8 Professional [6.2 Build 9200] (x64)
------------------------------------------------------------------------
Sequential Read :  76.67 MB/s                  Sequential Write :  133.9 MB/s
Random Read 512KB :   26.85 MB/s            Random Write 512KB :  57.95 MB/s
Random Read 4KB (QD=1) :  0.317 MB/s            Random Write 4KB (QD=1) :  2.273 MB/s
Random Read 4KB (QD=32) :  2.040 MB/s      Random Write 4KB (QD=32) :  2.371 MB/s


I intended to rebuild into a RAID0 configuration as my backup scheme has become far more robust, but I wanted to test the RAID10 again after blowing it away and recreating it.  After doing so, my performance increased nearly 3 fold for seq. read, and up to 2 fold for other metrics.  

------------------------------------------------------------------------
Volume:  Raid10 at 64k stripe
Test : 500 MB [D: 0.0% (0.2/596.0 GB)] (x5)
OS : Windows 8 Professional [6.2 Build 9200] (x64)
------------------------------------------------------------------------
Sequential Read :  212.549 MB/s            Sequential Write :  190.535 MB/s
Random Read 512KB :  44.323 MB/s            Random Write 512KB :   80.019 MB/s
Random Read 4KB (QD=1) :  0.652 MB/s            Random Write 4KB (QD=1) :  2.240 MB/s
Random Read 4KB (QD=32) :  4.302 MB/s      Random Write 4KB (QD=32) :  3.216 MB/s


At first I figured it may be due to fragmentation, but Windows did report 0 fragmentation before the first test.  Could it be the disk usage being 61% on the first test and only 0% on the second?  Or does RAID actually lose performance over time?  I've never heard of that.

Outside of that, these numbers don't reflect what I would expect from a 4 drive RAID10.  It is my understanding that the onboard ICH10R chipset has some issues with RAID10, per some searching I found.  Too bad SSD is expensive/GB, otherwise I'd replace this RAID entirely.

For comparison, my SSD:

------------------------------------------------------------------------
Drive:  Samsung 840 PRO SSD
Test : 500 MB [D: 66.0% (79/119.0 GB)] (x5)
OS : Windows 8 Professional [6.2 Build 9200] (x64)
------------------------------------------------------------------------
Sequential Read :  467.2 MB/s            Sequential Write :  333.6 MB/s
Random Read 512KB :  406.2 MB/s            Random Write 512KB :   178.1 MB/s
Random Read 4KB (QD=1) :  24.07 MB/s            Random Write 4KB (QD=1) :  60.98 MB/s
Random Read 4KB (QD=32) :  328.4 MB/s      Random Write 4KB (QD=32) :  176.9 MB/s



Same 4 HDD above in Raid0 128K  (I have large files)

------------------------------------------------------------------------
Volume:  Raid0 at 128k stripe
Test : 500 MB [D: 0.0% (0.2/1192.0 GB)] (x5)
OS : Windows 8 Professional [6.2 Build 9200] (x64)
------------------------------------------------------------------------
Sequential Read :  409.5 MB/s            Sequential Write :  299.6 MB/s
Random Read 512KB :  55.06 MB/s            Random Write 512KB :   124.7 MB/s
Random Read 4KB (QD=1) :  0.702 MB/s            Random Write 4KB (QD=1) :  4.726 MB/s
Random Read 4KB (QD=32) :  4.405 MB/s      Random Write 4KB (QD=32) :  7.088 MB/s
0
Comment
Question by:mcdonamw79
  • 4
  • 4
8 Comments
 
LVL 47

Expert Comment

by:dlethe
ID: 39200180
Well, first, that controller is crap. It doesn't even do read load balancing.  If you want to double read performance (in perfect world), turn off that RAID controller and let the O/S do 2 x RAID1 instead of a single RAID10.  

Your performance hit is most likely bad block recovery, which that controller , (if you want to call that a controller), doesn't handle well. It locks up and doesn't get the data from the other HDD in RAID set in parallel with remapping a bad block.
0
 

Author Comment

by:mcdonamw79
ID: 39200236
Yeah I recently found out about the lack of read load balancing, which was rather discerning.  Crap controller aside, would the bad block recovery you speak of come into play in all of my testing scenarios?  I'm using the same controller in each test, so I would expect its shortcomings to exist regardless.  

Ultimately I'm not understanding why recreating the RAID10 netted me such greater performance.  That's all I'm really looking to understand here.
0
 
LVL 47

Expert Comment

by:dlethe
ID: 39200291
Well bad blocks only happen once per block, as each one is remapped.  Only way to know for sure is to get some decent diagnostic software that maps out all blocks (without attempting recovery) before you run a test.

that software isn't cheap.   Performance has two major metrics, throughput and I/Os per second. They are mutually exclusive and all have different performance characteristics based on block size, reads vs. writes, random vs. sequential, and permutations involving combinations of all of the above.    Suffice to say, you're wasting your time tuning with this fakeraid config. No way can you get better performance no matter what you do with that controller vs. just using native host-based software raid.

Also benchmarking is rarely done right, and don't get hung up by synthetic loads that they generate.  Even many storage pros screw them up.  If your O/S does journaling and if you are benching a mounted filesystem or even a boot device and aren't looking at recovered/unrecovered errors then you can't possibly nail the results.

Spend your time dumping that fakeraid and going to software RAID. It is a better use of your time.
0
 

Author Comment

by:mcdonamw79
ID: 39203208
I concede that trying to get better performance from the ICH10R is pointless. I am not trying to do this. I am merely trying to understand why breaking an existing array and recreating it exactly the same yielded 2-3 times performance increase. Am I merely misunderstanding you?
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 
LVL 47

Expert Comment

by:dlethe
ID: 39203261
Because you likely had RECOVERABLE read errors.  A rebuild forces full reads of all blocks, and it could take up to 30 secs for a single block.   This can be confirmed (well, could have before you began the rebuild).
0
 

Author Comment

by:mcdonamw79
ID: 39205712
Gotcha. Perhaps that is what it was. I do know I ran a verify on the Raid array using the Intel RST before I blew it away. It reported no errors. Perhaps I will never know. Either way I think my primary question is answered. That was: do RAID arrays lose performance over time. Outside of block errors I am assuming this is not possible. Correct?
0
 
LVL 47

Accepted Solution

by:
dlethe earned 500 total points
ID: 39205747
Correct, allowing also for a statistically insignificant factors such as entropy, heat due to crud that builds up everywhere, etc ..
0
 

Author Closing Comment

by:mcdonamw79
ID: 39206269
Thank you for your time.
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Ever notice how you can't use a new drive in Windows without having Windows assigning a Disk Signature?  Ever have a signature collision problem (especially with Virtual Machines?)  This article is intended to help you understand what's going on and…
How to update Firmware and Bios in Dell Equalogic PS6000 Arrays and Hard Disks firmware update.
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

23 Experts available now in Live!

Get 1:1 Help Now