zeusindc
asked on
RAID 5 with hot spare or hot swap
I have a DELL PE2650 with 5 Drives. RAID controller PERC 3
planning to do RAID with 2 disks for OS and RAID 5 with other 3 for DATA.
Can I add another same size disks and use that as hot spare for RAID 5.
Also what is the difference between hot spare and hot swap. If hot swap is that i can have a spare drive sitting on my closet and if RAID 1 fails I can just swap the failed drive manually and RAID controller will rebuilt the RAID 1 with OS on it.
Why it is not recommended to have the OS on RAID 5?
planning to do RAID with 2 disks for OS and RAID 5 with other 3 for DATA.
Can I add another same size disks and use that as hot spare for RAID 5.
Also what is the difference between hot spare and hot swap. If hot swap is that i can have a spare drive sitting on my closet and if RAID 1 fails I can just swap the failed drive manually and RAID controller will rebuilt the RAID 1 with OS on it.
Why it is not recommended to have the OS on RAID 5?
It's easier to handle raid1 than raid5. For the OS you don't need much disk space, so raid 1 is enough for that. It is recommended to separate the OS from the rest of the system, to use a raid 5 for the OS would be a bit overkill.
Yes, you can have a hot spare on the array, and as leew says, if another disk fails, the hot spare will automatically take over. This of course is better than if you first have to change the disk. You might not notice the failed disk until a 2nd one also fails, while with a hot swap the system would be running again. Hot swap is again as leew says, when you can change a HD while the system is running. This has nothing directly to do with raid.
Yes, you can have a hot spare on the array, and as leew says, if another disk fails, the hot spare will automatically take over. This of course is better than if you first have to change the disk. You might not notice the failed disk until a 2nd one also fails, while with a hot swap the system would be running again. Hot swap is again as leew says, when you can change a HD while the system is running. This has nothing directly to do with raid.
Just to add one more voice - Since the page file is usually on the OS volume (depending on how conscientious the sysadmin is) - RAID 1 for the OS is preferable as writes to a RAID 1 set are 4 times faster than writes to a RAID 5 set...
meyersd - could you post some supporting evidence to that statement? are you saying that RAID 5 is 4x slower than a single disk?
Sorry, I don't see how WRITES could possibly be THAT much faster on a RAID 1 rather than a RAID 5. Maybe a little faster, but 4x? maybe 4%. Reads MIGHT be faster, depending on how the RAID 1 works on that controller, but but writes 4x faster?
Sorry, I don't see how WRITES could possibly be THAT much faster on a RAID 1 rather than a RAID 5. Maybe a little faster, but 4x? maybe 4%. Reads MIGHT be faster, depending on how the RAID 1 works on that controller, but but writes 4x faster?
Good spot leew, I now realise I missed the all-important qualifier "up to".
>are you saying that RAID 5 is 4x slower than a single disk?
On write operations, RAID 5 will be *up to* 4 times slower than a single disc. Read operations will be significantly faster - how much depends on how smart your RAID controller is, what sort of data you're reading (large sequential v. small random) and how many discs are in the RAID 5 set.
Which leads me to the RAID 5 write penalty (assume no write caching on the RAID controller):
1. Read original parity
2. read original data
Generate new parity (usually 2 XOR ops - but this does not significantly impact performance)
3. write new parity
4. write new data
As you can see, each write operation to a RAID 5 set requires two reads, a couple of calculations and two write operations. The slow parts are the disc operations - you have to wait for rotational latency, seek time, settling time and so on. Once you cache write operations things get better. If the write cache is lightly utilised then the RAID 5 write penalty should become largely invisible as write caching masks what's going on at the discs. The problem is that on many systems, the write cache does get hammered and fills up pretty quickly (because of the RAID 5 write penalty) - so the RAID 5 write penalty does affect the server. This'll be particularly true of the page file if many swaps are going on.
My experience with SAN arrays has been that a RAID 5 set takes 4 times longer to rebuild than a RAID 1/0 set or a straight mirror - so for critical apps, I recommend RAID 1/0 as much for the rebuild times as for performance.
>are you saying that RAID 5 is 4x slower than a single disk?
On write operations, RAID 5 will be *up to* 4 times slower than a single disc. Read operations will be significantly faster - how much depends on how smart your RAID controller is, what sort of data you're reading (large sequential v. small random) and how many discs are in the RAID 5 set.
Which leads me to the RAID 5 write penalty (assume no write caching on the RAID controller):
1. Read original parity
2. read original data
Generate new parity (usually 2 XOR ops - but this does not significantly impact performance)
3. write new parity
4. write new data
As you can see, each write operation to a RAID 5 set requires two reads, a couple of calculations and two write operations. The slow parts are the disc operations - you have to wait for rotational latency, seek time, settling time and so on. Once you cache write operations things get better. If the write cache is lightly utilised then the RAID 5 write penalty should become largely invisible as write caching masks what's going on at the discs. The problem is that on many systems, the write cache does get hammered and fills up pretty quickly (because of the RAID 5 write penalty) - so the RAID 5 write penalty does affect the server. This'll be particularly true of the page file if many swaps are going on.
My experience with SAN arrays has been that a RAID 5 set takes 4 times longer to rebuild than a RAID 1/0 set or a straight mirror - so for critical apps, I recommend RAID 1/0 as much for the rebuild times as for performance.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
>a dedicated SINGLE disk might be the best route for a pagefile
Absolutely! I guess it all comes down to budget...
But a quarter of the performance is (unfortunately) about right if there is no write cache - and there is reasonably heavy random I/O loads. If the RAID controller is one of the smarter modern ones that have RAID 5 write optimisations then the performance for sequential writes can be vastly improved.
I like this document very much: http://www.emc.com/products/systems/pdf/H1049_emc_clariion_fibre_channel_storage_fundamentals_ldv.pdf. Although it is from EMC and it'd be fair to say that a fair bit is EMC specific, much of it is excellent background information on disc subsystem performance and performance tuning.
Who said we don't offer value for money here at EE?
:-)
Absolutely! I guess it all comes down to budget...
But a quarter of the performance is (unfortunately) about right if there is no write cache - and there is reasonably heavy random I/O loads. If the RAID controller is one of the smarter modern ones that have RAID 5 write optimisations then the performance for sequential writes can be vastly improved.
I like this document very much: http://www.emc.com/products/systems/pdf/H1049_emc_clariion_fibre_channel_storage_fundamentals_ldv.pdf. Although it is from EMC and it'd be fair to say that a fair bit is EMC specific, much of it is excellent background information on disc subsystem performance and performance tuning.
Who said we don't offer value for money here at EE?
:-)
I would point out that although there are 4 I/Os per write on a RAID 5 array there are also more disks to do these I/Os with so a 4 drive RAID 5 array writes at the same speed as a single disk. Also with 2 I/Os per write on a mirrored pair but two disks to do these I/Os with RAID 1 writes at similar speed.
Without wanting to start a stoush on this - that isn't correct.
The problem is that the read and write operations don't (and cannot) happen simultaneously. A RAID 5 set *without write caching* runs at up to a quarter of the speed of a single disc depending on many factors - but typically between a two thirds and a quarter of the performance of a single disc. Read operation occur at up to (number of discs -1) * single disc - again depending on many factors, not least the smarts of the RAID controller, type of I/O and how well skewed the benchmark is toward the results you want. A mirrored pair will write at the same speed as a single disc (the mirror operation is handled by the RAID controlelr and happens in the background) and reads at up to twice the speed of a single disc.
As noted above, once you add in write caching, things change. The write cache masks the activity at the discs and so insulates the server from the RAID 5 write penalty to a degree. Once the write cache fills (which can happen with a single burst of write activity) then you are exposed to the full performance hit of the RAID 5 write penalty.
Another note on RAID 5 writes - the two disc write operations have to occur sequentially otherwise the controller could not then determine if a the parity or data sectors had been successfully written in the event of an outage. The process is usually referred to as a two-stage commit.
The problem is that the read and write operations don't (and cannot) happen simultaneously. A RAID 5 set *without write caching* runs at up to a quarter of the speed of a single disc depending on many factors - but typically between a two thirds and a quarter of the performance of a single disc. Read operation occur at up to (number of discs -1) * single disc - again depending on many factors, not least the smarts of the RAID controller, type of I/O and how well skewed the benchmark is toward the results you want. A mirrored pair will write at the same speed as a single disc (the mirror operation is handled by the RAID controlelr and happens in the background) and reads at up to twice the speed of a single disc.
As noted above, once you add in write caching, things change. The write cache masks the activity at the discs and so insulates the server from the RAID 5 write penalty to a degree. Once the write cache fills (which can happen with a single burst of write activity) then you are exposed to the full performance hit of the RAID 5 write penalty.
Another note on RAID 5 writes - the two disc write operations have to occur sequentially otherwise the controller could not then determine if a the parity or data sectors had been successfully written in the event of an outage. The process is usually referred to as a two-stage commit.
We shouldn't discuss this inside a user's thread; I think we all agree that with the 5 or 6 disks zeusindc has they are best off using two for the OS and the rest in RAId 5 for the data, it's the standard spec for a low performance high reliability and maintainable srver.
You can post a discussion thread for 20 points or I'll post one tomorrow on our difference of performance thoughts.
You can post a discussion thread for 20 points or I'll post one tomorrow on our difference of performance thoughts.
Soooooooooooooooooo zeusindc....
We got waaaaaaay off topic here - but does all the above answer your question?
:-)
We got waaaaaaay off topic here - but does all the above answer your question?
:-)
1) no, because the pe2650 will only hold 5 drives, but do keep another drive on hand, and you can swap it out without having to down the system because the 80 pin scsi drives are hot swappable.
2) hotspare (also called "failover") is a drive that sits in your system doing nothing but waiting for a drive to fail, when a drive does fail, it should automatically rebuild to the hotspare (failover) drive.
3) see #1
PS:
I recommend splitting your backplane by using the backplane daughter board, this will allow you to have your OS raid 1 on channel 0 of the scsi raid controller and your data raid 5 on channel 1 of your scsi raid controller.
see this for information on adding a daughterboard (you might even have one already, it's very easy to find out):
http://support.dell.com/support/edocs/systems/pe2650/multlang/4p220/4p220.pdf
jeff
2) hotspare (also called "failover") is a drive that sits in your system doing nothing but waiting for a drive to fail, when a drive does fail, it should automatically rebuild to the hotspare (failover) drive.
3) see #1
PS:
I recommend splitting your backplane by using the backplane daughter board, this will allow you to have your OS raid 1 on channel 0 of the scsi raid controller and your data raid 5 on channel 1 of your scsi raid controller.
see this for information on adding a daughterboard (you might even have one already, it's very easy to find out):
http://support.dell.com/support/edocs/systems/pe2650/multlang/4p220/4p220.pdf
jeff
sorry leew, I kinda reiterated what you said
ASKER
thanks to all for getting back....since this was my first question on EE, i realize i assign too many points for this question :)
but thanks meyersd and leww for providing the answers. I will accept the answers and give points but can i add few more things so that will justify points ?? :)
As you can see in my question i kinda knew what hot spare n hot swap was and wanted to verify....
can i get some documents on best practices for Creating RAID. I have 50 servers and will be managing that. I get new servers and have to design RAID for it. The requirement varies and is mostly database. What will be best way to manage these servers "RAID configuration", disk space( most of them DELL 2850,2650,2450 ) and HP Proliant. I guess what i mean was managing from one console maybe. As one drives fail and if there is no hot spare how can i find out?
regardless if u answer i will give points
but thanks meyersd and leww for providing the answers. I will accept the answers and give points but can i add few more things so that will justify points ?? :)
As you can see in my question i kinda knew what hot spare n hot swap was and wanted to verify....
can i get some documents on best practices for Creating RAID. I have 50 servers and will be managing that. I get new servers and have to design RAID for it. The requirement varies and is mostly database. What will be best way to manage these servers "RAID configuration", disk space( most of them DELL 2850,2650,2450 ) and HP Proliant. I guess what i mean was managing from one console maybe. As one drives fail and if there is no hot spare how can i find out?
regardless if u answer i will give points
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Problem with databases that you really want 2 disks for the OS, 2 for the transaction logs and the rest for the data and in general servers don't have enough internal disk bays to achieve this. You end up putting the logs on the same spindle as the OS and pagefile which is OK when it's just sitting there running the database but goes slow when anyone uses a console app like SQL enterprise manager and it suddenly starts paging and slows the log file down.
You can often get extra external drive containers for your servers so you can use more disks in the arrays.
ASKER
Alright Rindi, I will open another question for you to explain CIM. I couldnt find much about that.
Back to RAID 5
when it says Create RAID 5 via either Scrub or Clear. What is that suppose to mean? I know scrubbing takes a little longer cause it does some consistency checks and clear just wipes the disk and rebuild. But what is recommended? when a disk fails and controller starts rebuilding via scrub does the controller goes off means the machine is down
Back to RAID 5
when it says Create RAID 5 via either Scrub or Clear. What is that suppose to mean? I know scrubbing takes a little longer cause it does some consistency checks and clear just wipes the disk and rebuild. But what is recommended? when a disk fails and controller starts rebuilding via scrub does the controller goes off means the machine is down
ASKER
when I used Create RAID 5 via scrub, It is taking for ever to resync the RAID. And i havent even install anything on the disks yet. I am still on Array Manager.
If this is the case then it will take for ever with data on the disks. It actually boots up and say both the logical drives (RAID1 n RAID 5) are Scrubbing. what is this ad how long should this take?
If this is the case then it will take for ever with data on the disks. It actually boots up and say both the logical drives (RAID1 n RAID 5) are Scrubbing. what is this ad how long should this take?
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Is this scrubbing only done by Adaptec PERC (DELL) or other controllers do this too. It took 2 hours for a RAID 1 to complete (36GB drives each) and right now is still scrubbing the 3x 73GB drives with RAID 5. I guess my questions is I configured a HP server last week with 2 RAID 1's and a hotspare and right after configuring I install the OS using HP application. There was nothing I saw on the HP array manager where there was some scrubbing or anything going on. So HP doesnt do scrubbing? or it does i didnt see it maybe i just installed the OS?
If I start the OS installation process while the controller is doing scrubbing, how does that affect anything?
also if it is taking (overnight) to scrub basically empty disks, how long would it take to scrub disks with data? or does it not really matter
If I start the OS installation process while the controller is doing scrubbing, how does that affect anything?
also if it is taking (overnight) to scrub basically empty disks, how long would it take to scrub disks with data? or does it not really matter
>If I start the OS installation process while the controller is doing scrubbing, how does that affect anything?
Good question. You get a warning from the BIOS that suggests that you won't get full redundancy until the scrub is complete. If you're building a new server and you lose a disc whilst your installing Windows and you have to reload - so what? I have routinely kicked off an OS installation whilst the scrub is in process with no problems. In short, go ahead and do whatever you need - you're not affecting data integrity in any way.
>also if it is taking (overnight) to scrub basically empty disks, how long would it take to scrub disks with data? or does it not really matter
It'll take the same amount of time - probably around 6 hours for the RAID 5 set.
The scrub thing seems to be unique to Adaptec. Cards from what was AMI (now LSI) and Mylex don't exhibit this behaviour (alright - to be strictly accurate - AMI cards used to do an initialisation on a new array that took **hours**, but they haven't done that for probably 5 years or so).
Good question. You get a warning from the BIOS that suggests that you won't get full redundancy until the scrub is complete. If you're building a new server and you lose a disc whilst your installing Windows and you have to reload - so what? I have routinely kicked off an OS installation whilst the scrub is in process with no problems. In short, go ahead and do whatever you need - you're not affecting data integrity in any way.
>also if it is taking (overnight) to scrub basically empty disks, how long would it take to scrub disks with data? or does it not really matter
It'll take the same amount of time - probably around 6 hours for the RAID 5 set.
The scrub thing seems to be unique to Adaptec. Cards from what was AMI (now LSI) and Mylex don't exhibit this behaviour (alright - to be strictly accurate - AMI cards used to do an initialisation on a new array that took **hours**, but they haven't done that for probably 5 years or so).
Man, this thread is getting long :-)
It's preferred that it's on a Mirror (RAID 1) but there's nothing wrong with a RAID 5 - you really just want a form of RAID that's redundant.
Hot Swap - you can remove a hard drive with the system running:
Hot Spare - If a disk fails in the system, there's another UNUSED drive waiting and the system will take that disk and AUTOMATICALLY repair the RAID using this "spare" disk.