Solved

RAID Level recommendation?

Posted on 2009-04-06
20
943 Views
Last Modified: 2013-11-14
Hi Guys,

I was wondering what do you think about wich would be the best way to configure a server with two Intel SRCS28X Raid controllers (8 ports each) and 16x 500GB drives, which will be running Open-e on top.

Storage capacity in not that important to me as anything over 2TB will be more than enough, the idea of having so many drives is to improve performance without compromising redundancy...

Notes:
- The SRCS28X cards have a 2TB limit on each logical drive.
- I have 2x spare drives on my desk so having hot spares is not that important.

These are the options:
1 - 1x RAID 50 on each controler made of 2x 3 drive RAID 5 and a RAID 0 on top, plus two hot spares (this is due to the 2TB limit, 4 drive RAID 5s would be too big). Then RAID 0 on top of the two RAID 50 using Open-e Software RAID.

2- 1x RAID 10 on each controller made of 4x 2 drive RAID 1 and a RAID 0 on top. Then RAID 0 on top of the two RAID 10 using Open-e Software RAID.

3- 4x RAID 1 on each controller made of 2 drive each. Then RAID 0 on top of the eight RAID 1 using Open-e Software RAID.

All 3 conf end with 4TB usable space and are more less very redundant and fast... but which one would be better?, in case of going with RAID 10, do you think there will be an advantage of the intermedium hardware level on conf 2 over conf 3, or will it cause trouble instead?

Bu the way, this is to hold 10 servers with Virtual Iron with different functions with FC.

Thanks in advance!
0
Comment
Question by:Lumute
  • 7
  • 6
  • 5
  • +2
20 Comments
 
LVL 23

Expert Comment

by:ComputerTechie
ID: 24080956
I personally try to run Raid 10 when ever i get a chance.
CT
 
0
 
LVL 32

Expert Comment

by:Rodney Barnhardt
ID: 24081097
One thing to keep in mind, using any type of software RAID could cause adverse effects on the performance of your server depending on what else it is doing. I have never had any issue using just hardware RAID and have been doing this for 20 years. I would do the RAID 10, but doing the RAID 0 on top of that with software I think is over kill.
0
 

Author Comment

by:Lumute
ID: 24081177
Thanks rbarnhardt, well, this server is to be the storage server only, nothing running on it other than Open-e, it has 2x Xeon 3.0GHz and 6GB Ram, so doing software RAID should not affect performance on anything else... the idea of the software RAID 0 on top is to actually inprove performance by writting and reading from drives on both controllers at the same time, thus using all 16 drive at the same time...
0
 
LVL 55

Expert Comment

by:andyalder
ID: 24085884
Because you are using 500GB disks which must be slow SATA ones I wouldn't worry too much about performance of the RAID level because you aren't going to get much performance in a random read/write environment out of the disks anyway.
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 24088040
What are this storage to be used for ?


There is mainly only 2 io pattern of interest :
1) Concurrency
Concurrency requires some IOPS capability.
==> Using 7200 rpm drives (SATA except SSD and WD*raptors) allows up to about 180x 4KB io per second per drive
==> Using 10 krpm drives (SATA WD*raptors, SAS/SCSI) allows up to about 320x 4KB io per second per drive
==> Using 15 krpm drives (SAS/SCSI only) allows up to about 400x 4KB io per second per drive
==> Using SSD (Intel X25-E) allows up to about 3200x 4KB io per second per drive


2) <10% write io context
When you expect more than 10% write with concurrency : the answer is RAID 10.

Otherwise, RAID 10/50/60 are usable with 2 array configurations
a) Concurrency io context or RAID 10
Use a stripe size equivalent to your io client size (Ex: 4KB by default and up to 64KB on NTFS for XP/Win2003)

b) Workstation io context
Use a FULL stripe size equivalent to your io client size to avoid the parity write problem (read old parity -> compute new parity -> write new parity and data)


Assuming your goal is to build a storage server used concurrently by many writers, I would go with your a 4th solution based on a RAID 10 arrays of 4x RAID 1 per card:
-1x 2TB array build using a 64KB stripe size (256KB full stripe) for low IOPS high throughput io clients (Disk backup, dbms logs, large files, index files, apps)
-1x 2TB array build using a 4KB stripe size (16KB full stripe) for high IOPS io clients (Temp, dbms data, office files)

A better advice would be to let you evaluate the expected io patterns of all your client apps and post the results here ;-)
0
 
LVL 55

Expert Comment

by:andyalder
ID: 24088314
I think we have a good idea of the IO profile, they say in the last paragraph "Bu the way, this is to hold 10 servers with Virtual Iron with different functions" so it's likely to be pretty random I/O. It's going to have to seek between the 10 or more server partitions or however Virtual Iron splits the server VMs up.
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 24088368
I read the same words...but was confused by the "different functions".

Some of those "functions" may be "Backuping" or "Video editing" for example thus claiming for large throughput instead of IOPS
0
 

Author Comment

by:Lumute
ID: 24088515
Thank you for you responses...

andyalder: I am using 7200RPM SATA drives. I don't get your logic, one would think that if the drives are slow, then one more reason to try to get as much performance as possible from the RAID...

BigSchmuh: Thank you, very good and informative post, I got a similar answer from another very experienced IT about using two separate RAID 10 or maybe a 10 and a 5, and you confirmed it...

My idea for the soft RAID 0 on top was to improve performance by using all 16 disks at the same time, but he mentioned something very important: it will be way too complicate to recover from a RAID failure if I have software RAID over hardware RAID...

This storage server and the VM servers are going to be used to consolidate all out internal IT servers, no workstations... so a couple of domain controllers, one file server (around 80GB of users data), one exchange server, one database server with a couple of small databases, one BES and 4 or 5 other different application servers. All of them either Windows 2003 or 2008.

How can I evaluate the IO patterns of my apps as you suggest?
0
 
LVL 55

Expert Comment

by:andyalder
ID: 24088707
Lumute, I was just making suer you know the poor performance of 500GB SATA disks, with an average seek time of about 10ms you're down to about 80 IOPS on random data. A 15K SAS disk you're looking at nearer 200 IOPS.
0
 

Author Comment

by:Lumute
ID: 24089025
Hi andyalder, I understand what you mean...

The drives are RAID optimized drives and have an average seek time of 8.0ms with very good reviews, of course, not as good as a 15K SATA, but that is what I count with, no money to buy better drives :-(...
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 18

Accepted Solution

by:
BigSchmuh earned 125 total points
ID: 24089177
I can not evaluate the IO pattern for your dbms here...

But assuming you will install all dbms on 64bit Win2008 edition, this allows to raise their dedicated cache to gain sufficient performance.

I can complete my previous recos below:
1x 2TB array build using a 64KB stripe size (256KB full stripe) for low IOPS high throughput io clients :
-Disk backup,
-dbms logs,
-large files,
-index files,
-all apps,
-OS
==> Try using only Win2008 editions because Win2003 reserves 31.5KB at the beginning of its partition...which is terrible for your 256KB aligned array. Although try to use a customized installation to format you NTFS using 64KB cluster size to fully benefit from the large block array

-1x 2TB array build using a 4KB stripe size (16KB full stripe) for high IOPS io clients :
-Temp,
-dbms data,
-office files,
-Exchange data
-Any data expecting concurrency access
==> Try using only Win2008 editions because Win2003 reserves 31.5KB at the beginning of its partition...which is terrible for your 16KB aligned array.


NB: Does Virtual Iron kernel use some disk space for itself in your arrays ? If it does, the "alignment" problem may come again...
0
 
LVL 55

Expert Comment

by:andyalder
ID: 24092701
180 IOPS on a 7.2K SATA? I'm sorry, I should keep out of this because it's likely to cause a flame war but 70 IOPS is more like what you can get unless it's coming out of RAM; 180 IOPS is what you expect from a 15K SAS/SCSI/FC disk. Just add the time for a random seek to the time to spin the platter half way around to get an average I/O. If you can get 400 random IOPS from a 15K disk I'll eat my hat, try 180 IOPS tops for the best disk in the world and then boost it a bit through battery backed write cache.

I'm sorry to interupt, please ignore my post and continue pretending disks are three times as fast as they are in real life.
0
 

Author Comment

by:Lumute
ID: 24093206
Hi BigSchmuh, cool, that is actually very useful advice... not sure about the Virtual Iron disk usage for the kernel but I will try to find that out... also I did not know that Windows 2008 does not reserve that space, I do have available 2008 licenses but need to confirm all apps are compatible first, may have to keep one or two 2003's.

Hi andyalder, not sure what your point is as you already stablished the drives are slow and I don't see anyone saying you can do 180 IOPS on a 7.2K SATA... the thing is, I could probably do better with a Tera-RamSam, but I do not have 1 million dollars to spend nor the question is what I could do if I had better hardware, but how should I configure what I do have to get the most out of it... Thanks for them imput anyway.
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 24094403
In a previous comment, I wrote that a SATA 7200 rpm can reach 180 IOPS...

Those numbers comes from the StorageReview.com performance database...it is not up to date but it is still very serious numbers.
From the below URL, look at "IOMeter File Server - 64 I/O" and you get my above numbers...you won't find recent drive here, they are all about 2-3 years old :-(

http://www.storagereview.com/php/benchmark/suite_v4.php?typeID=10&testbedID=4&osID=6&raidconfigID=1&numDrives=1&devID_0=321&devID_1=326&devID_2=366&devID_3=368&devCnt=4
0
 
LVL 55

Expert Comment

by:andyalder
ID: 24095047
BigSchmuh, It's the IOMeter File Server - 1 I/O number you have to go by, not the 64 or 128 outstanding I/O value when you talk about disk IOPS, another way would be to use the low level suite random acces time, dividing this number by 1000 and you get 75 for the 1TB disk which is its true performance.
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 24095105
@andyalder : Using metrics from a 1 io depth is equivalent to disabling the NCQ feature, I totally disagree this way of accounting drives for a server usage.
0
 
LVL 55

Assisted Solution

by:andyalder
andyalder earned 125 total points
ID: 24095238
@BigSchmuh, Whether you like it or not it is what is used throughout the storage industry. For you to quote IOPS which differ from what the industry quotes is misleading.

Now, back to the original question. Since there are various undefined I/O profiles that are going to hit this storage the principle of SAME should be used, www.oracle.com/technology/deploy/availability/pdf/oow2000_same.pdf
Although the white paper is from Oracle it applies to just about everything with the exception of backup to disk. For this you need one big RAID 10 array with a large stripe size, stripe element of 64K works quite well. You then slice this up either through the controller or by partitioning under the OS.

With dual controllers there are 2 options to create the array, create two RAID 0 sets on the controllers and mirror them under the OS or create two RAID 10 sets on the controllers and stripe them under the OS. The latter is preferable because it is more tolerant of multiple disks failures and performs better with a disk down. The first is RAID 0+1 the second better option is RAID 1+0 (RAID 10).
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 24095947
@andyalder: I agree that "IOPS" is mainly seen as a benchmark with fixed parameters...but my "This disable NCQ" point is to be accounted !

"Some hard drives will improve in performance as the number of outstanding IO's (i.e. queue depth) increases."

     http://en.wikipedia.org/wiki/IOPS

I already read this (old) Oracle white paper.
It does not focus on how efficient an IO is regarding to the disk usage.

For example, if a io client always issues random 4KB io and the stripe size is 1MB, you will loose latency (then IOPS)...for nothing.

Regarding the 1MB stripe size reco from Oracle, I agree that the first 2TB raid 10 array may have a small benefit from using 128KB stripe (512KB full stripe) instead of 64KB.
0
 
LVL 55

Expert Comment

by:andyalder
ID: 24096343
>@andyalder: I agree that "IOPS" is mainly seen as a benchmark with fixed parameters...but my "This disable NCQ" point is to be accounted !

Well then put it as a seperate statement, something along the lines of "you get about 75 IOPS from a 7.2K vertically encoded disk, you may see significant improvements up to 180 IOPS if you can maintain a high queue count".
0
 

Author Closing Comment

by:Lumute
ID: 31567200
Very good information and suggestions from both of you.

Thank you.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Suggested Solutions

A bootable USB key can be very handy now-a-days. My favorite USB key consists of our Windows 7 image, network card drivers (to connect up to a Ghost server), the latest BIOS updates for all of our PCs and CopyWipe (to erase a retired PC) Creating…
This article is an update and follow-up of my previous article:   Storage 101: common concepts in the IT enterprise storage This time, I expand on more frequently used storage concepts.
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now