RAID Level recommendation?

Hi Guys,

I was wondering what do you think about wich would be the best way to configure a server with two Intel SRCS28X Raid controllers (8 ports each) and 16x 500GB drives, which will be running Open-e on top.

Storage capacity in not that important to me as anything over 2TB will be more than enough, the idea of having so many drives is to improve performance without compromising redundancy...

- The SRCS28X cards have a 2TB limit on each logical drive.
- I have 2x spare drives on my desk so having hot spares is not that important.

These are the options:
1 - 1x RAID 50 on each controler made of 2x 3 drive RAID 5 and a RAID 0 on top, plus two hot spares (this is due to the 2TB limit, 4 drive RAID 5s would be too big). Then RAID 0 on top of the two RAID 50 using Open-e Software RAID.

2- 1x RAID 10 on each controller made of 4x 2 drive RAID 1 and a RAID 0 on top. Then RAID 0 on top of the two RAID 10 using Open-e Software RAID.

3- 4x RAID 1 on each controller made of 2 drive each. Then RAID 0 on top of the eight RAID 1 using Open-e Software RAID.

All 3 conf end with 4TB usable space and are more less very redundant and fast... but which one would be better?, in case of going with RAID 10, do you think there will be an advantage of the intermedium hardware level on conf 2 over conf 3, or will it cause trouble instead?

Bu the way, this is to hold 10 servers with Virtual Iron with different functions with FC.

Thanks in advance!
Who is Participating?
BigSchmuhConnect With a Mentor Commented:
I can not evaluate the IO pattern for your dbms here...

But assuming you will install all dbms on 64bit Win2008 edition, this allows to raise their dedicated cache to gain sufficient performance.

I can complete my previous recos below:
1x 2TB array build using a 64KB stripe size (256KB full stripe) for low IOPS high throughput io clients :
-Disk backup,
-dbms logs,
-large files,
-index files,
-all apps,
==> Try using only Win2008 editions because Win2003 reserves 31.5KB at the beginning of its partition...which is terrible for your 256KB aligned array. Although try to use a customized installation to format you NTFS using 64KB cluster size to fully benefit from the large block array

-1x 2TB array build using a 4KB stripe size (16KB full stripe) for high IOPS io clients :
-dbms data,
-office files,
-Exchange data
-Any data expecting concurrency access
==> Try using only Win2008 editions because Win2003 reserves 31.5KB at the beginning of its partition...which is terrible for your 16KB aligned array.

NB: Does Virtual Iron kernel use some disk space for itself in your arrays ? If it does, the "alignment" problem may come again...
I personally try to run Raid 10 when ever i get a chance.
Rodney BarnhardtServer AdministratorCommented:
One thing to keep in mind, using any type of software RAID could cause adverse effects on the performance of your server depending on what else it is doing. I have never had any issue using just hardware RAID and have been doing this for 20 years. I would do the RAID 10, but doing the RAID 0 on top of that with software I think is over kill.
Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

LumuteAuthor Commented:
Thanks rbarnhardt, well, this server is to be the storage server only, nothing running on it other than Open-e, it has 2x Xeon 3.0GHz and 6GB Ram, so doing software RAID should not affect performance on anything else... the idea of the software RAID 0 on top is to actually inprove performance by writting and reading from drives on both controllers at the same time, thus using all 16 drive at the same time...
Because you are using 500GB disks which must be slow SATA ones I wouldn't worry too much about performance of the RAID level because you aren't going to get much performance in a random read/write environment out of the disks anyway.
What are this storage to be used for ?

There is mainly only 2 io pattern of interest :
1) Concurrency
Concurrency requires some IOPS capability.
==> Using 7200 rpm drives (SATA except SSD and WD*raptors) allows up to about 180x 4KB io per second per drive
==> Using 10 krpm drives (SATA WD*raptors, SAS/SCSI) allows up to about 320x 4KB io per second per drive
==> Using 15 krpm drives (SAS/SCSI only) allows up to about 400x 4KB io per second per drive
==> Using SSD (Intel X25-E) allows up to about 3200x 4KB io per second per drive

2) <10% write io context
When you expect more than 10% write with concurrency : the answer is RAID 10.

Otherwise, RAID 10/50/60 are usable with 2 array configurations
a) Concurrency io context or RAID 10
Use a stripe size equivalent to your io client size (Ex: 4KB by default and up to 64KB on NTFS for XP/Win2003)

b) Workstation io context
Use a FULL stripe size equivalent to your io client size to avoid the parity write problem (read old parity -> compute new parity -> write new parity and data)

Assuming your goal is to build a storage server used concurrently by many writers, I would go with your a 4th solution based on a RAID 10 arrays of 4x RAID 1 per card:
-1x 2TB array build using a 64KB stripe size (256KB full stripe) for low IOPS high throughput io clients (Disk backup, dbms logs, large files, index files, apps)
-1x 2TB array build using a 4KB stripe size (16KB full stripe) for high IOPS io clients (Temp, dbms data, office files)

A better advice would be to let you evaluate the expected io patterns of all your client apps and post the results here ;-)
I think we have a good idea of the IO profile, they say in the last paragraph "Bu the way, this is to hold 10 servers with Virtual Iron with different functions" so it's likely to be pretty random I/O. It's going to have to seek between the 10 or more server partitions or however Virtual Iron splits the server VMs up.
I read the same words...but was confused by the "different functions".

Some of those "functions" may be "Backuping" or "Video editing" for example thus claiming for large throughput instead of IOPS
LumuteAuthor Commented:
Thank you for you responses...

andyalder: I am using 7200RPM SATA drives. I don't get your logic, one would think that if the drives are slow, then one more reason to try to get as much performance as possible from the RAID...

BigSchmuh: Thank you, very good and informative post, I got a similar answer from another very experienced IT about using two separate RAID 10 or maybe a 10 and a 5, and you confirmed it...

My idea for the soft RAID 0 on top was to improve performance by using all 16 disks at the same time, but he mentioned something very important: it will be way too complicate to recover from a RAID failure if I have software RAID over hardware RAID...

This storage server and the VM servers are going to be used to consolidate all out internal IT servers, no workstations... so a couple of domain controllers, one file server (around 80GB of users data), one exchange server, one database server with a couple of small databases, one BES and 4 or 5 other different application servers. All of them either Windows 2003 or 2008.

How can I evaluate the IO patterns of my apps as you suggest?
Lumute, I was just making suer you know the poor performance of 500GB SATA disks, with an average seek time of about 10ms you're down to about 80 IOPS on random data. A 15K SAS disk you're looking at nearer 200 IOPS.
LumuteAuthor Commented:
Hi andyalder, I understand what you mean...

The drives are RAID optimized drives and have an average seek time of 8.0ms with very good reviews, of course, not as good as a 15K SATA, but that is what I count with, no money to buy better drives :-(...
180 IOPS on a 7.2K SATA? I'm sorry, I should keep out of this because it's likely to cause a flame war but 70 IOPS is more like what you can get unless it's coming out of RAM; 180 IOPS is what you expect from a 15K SAS/SCSI/FC disk. Just add the time for a random seek to the time to spin the platter half way around to get an average I/O. If you can get 400 random IOPS from a 15K disk I'll eat my hat, try 180 IOPS tops for the best disk in the world and then boost it a bit through battery backed write cache.

I'm sorry to interupt, please ignore my post and continue pretending disks are three times as fast as they are in real life.
LumuteAuthor Commented:
Hi BigSchmuh, cool, that is actually very useful advice... not sure about the Virtual Iron disk usage for the kernel but I will try to find that out... also I did not know that Windows 2008 does not reserve that space, I do have available 2008 licenses but need to confirm all apps are compatible first, may have to keep one or two 2003's.

Hi andyalder, not sure what your point is as you already stablished the drives are slow and I don't see anyone saying you can do 180 IOPS on a 7.2K SATA... the thing is, I could probably do better with a Tera-RamSam, but I do not have 1 million dollars to spend nor the question is what I could do if I had better hardware, but how should I configure what I do have to get the most out of it... Thanks for them imput anyway.
In a previous comment, I wrote that a SATA 7200 rpm can reach 180 IOPS...

Those numbers comes from the performance is not up to date but it is still very serious numbers.
From the below URL, look at "IOMeter File Server - 64 I/O" and you get my above won't find recent drive here, they are all about 2-3 years old :-(
BigSchmuh, It's the IOMeter File Server - 1 I/O number you have to go by, not the 64 or 128 outstanding I/O value when you talk about disk IOPS, another way would be to use the low level suite random acces time, dividing this number by 1000 and you get 75 for the 1TB disk which is its true performance.
@andyalder : Using metrics from a 1 io depth is equivalent to disabling the NCQ feature, I totally disagree this way of accounting drives for a server usage.
andyalderConnect With a Mentor Commented:
@BigSchmuh, Whether you like it or not it is what is used throughout the storage industry. For you to quote IOPS which differ from what the industry quotes is misleading.

Now, back to the original question. Since there are various undefined I/O profiles that are going to hit this storage the principle of SAME should be used, 
Although the white paper is from Oracle it applies to just about everything with the exception of backup to disk. For this you need one big RAID 10 array with a large stripe size, stripe element of 64K works quite well. You then slice this up either through the controller or by partitioning under the OS.

With dual controllers there are 2 options to create the array, create two RAID 0 sets on the controllers and mirror them under the OS or create two RAID 10 sets on the controllers and stripe them under the OS. The latter is preferable because it is more tolerant of multiple disks failures and performs better with a disk down. The first is RAID 0+1 the second better option is RAID 1+0 (RAID 10).
@andyalder: I agree that "IOPS" is mainly seen as a benchmark with fixed parameters...but my "This disable NCQ" point is to be accounted !

"Some hard drives will improve in performance as the number of outstanding IO's (i.e. queue depth) increases."

I already read this (old) Oracle white paper.
It does not focus on how efficient an IO is regarding to the disk usage.

For example, if a io client always issues random 4KB io and the stripe size is 1MB, you will loose latency (then IOPS)...for nothing.

Regarding the 1MB stripe size reco from Oracle, I agree that the first 2TB raid 10 array may have a small benefit from using 128KB stripe (512KB full stripe) instead of 64KB.
>@andyalder: I agree that "IOPS" is mainly seen as a benchmark with fixed parameters...but my "This disable NCQ" point is to be accounted !

Well then put it as a seperate statement, something along the lines of "you get about 75 IOPS from a 7.2K vertically encoded disk, you may see significant improvements up to 180 IOPS if you can maintain a high queue count".
LumuteAuthor Commented:
Very good information and suggestions from both of you.

Thank you.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.