Mike R.
asked on
Combination of ZFS and Hardware to gain Raid 51
I'm looking into RAID solutions for a very large file server (70+TB, ONLY serving NFS and CIFS). I know using ZFS raid on top of hardware raid is generally contraindicated, however I find myself in an unusual situation.
My personal preference would be to setup large `RAID-51` virtual disks. I.e Two mirrored RAID5, with each RAID5 having 9 data + 1 hotspare (so we don't lose TOO much storage space). This eases my administrative paranoia by having the data mirrored on two different drive chassis, while allowing for 1 disk failure in each mirror set before a crisis hits.
HOWEVER this question stems from the fact, that we have existing hardware RAID controllers (LSI Megaraid integrated disk chassis + server), licensed ONLY for RAID5 and 6. We also have an existing ZFS file system, which is intended (but not yet configured) to provide HA using RFS-1.
The suggestion is to use the hardware raid to create two, equally sized RAID5 virtual disks on each chassis. These two RAID5 virtual disks are then presented to their respective servers as /dev/sdx.
Then use ZFS + RFS-1 to mirror those two virtual disks as an HA mirror set (see image)
Is this a good idea, a bad idea, or just an ugly (but usable) configuration.
Are there better solutions?
My personal preference would be to setup large `RAID-51` virtual disks. I.e Two mirrored RAID5, with each RAID5 having 9 data + 1 hotspare (so we don't lose TOO much storage space). This eases my administrative paranoia by having the data mirrored on two different drive chassis, while allowing for 1 disk failure in each mirror set before a crisis hits.
HOWEVER this question stems from the fact, that we have existing hardware RAID controllers (LSI Megaraid integrated disk chassis + server), licensed ONLY for RAID5 and 6. We also have an existing ZFS file system, which is intended (but not yet configured) to provide HA using RFS-1.
The suggestion is to use the hardware raid to create two, equally sized RAID5 virtual disks on each chassis. These two RAID5 virtual disks are then presented to their respective servers as /dev/sdx.
Then use ZFS + RFS-1 to mirror those two virtual disks as an HA mirror set (see image)
Is this a good idea, a bad idea, or just an ugly (but usable) configuration.
Are there better solutions?
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
You know, one thing I'm trying to solve is getting the data duplicated between two individual controller daisy chains, since the three chassis in our current one controller chain (that we have been discussing in multiple threads :D ) is starting to have significant hardware issues.
I want to add a new (completely separate) controller + 24 drive slot chassis, WITH a daisy chained second 24 drive slot chassis. At minimum I'm looking at a 0+1 (or possible RAID-10) solution - but (as I understand it) with both 0+1 and 10, if one drive in each of the mirrors goes bad, you still have data loss (probably not the SAME loss of data in both mirrors, but you'd still have to do a restore).
If you look at the diagram, what I'm actually thinking about is having two hardware RAID-5 virtual drives (each with 10 disks, intentionally large - for reasons I'll explain).
THEN using ZFS on top to mirror those two RAID-5 virtual disks (creating a defacto RAID 51 - because neither our controller, NOR ZFS support true RAID 51).
Raid 0+1 is generally considered "safe" because the odds of losing one disk in each mirror is normally low. So I figured making the virtual disks large RAID 5's gives the ability to lose two disks in BOTH sides of the mirror before data loss occurs.
I hope that makes sense :)