Nested RAID 50 Linux Raid + hardware raid question

Hi all...

I need help to figure out something...

Before banging my head on the wall with a problem I want to know if anybody would throw in a bit of advice on this one...

I'm killing an old win 2003 backup server. reinstalling in linux of course :P ... here is the problem I have...

I have 2 RAID controllers in there with 3X1TB in RAID 5 on each...
I did put those 2 raid 5 in stripe using software RAID...

I know that this easy and possible with Linux here is the real question...

I will be receiving 4 extra drives in a couple of weeks to add to this server... 2 drives for each Raid 5 pool... extending those with the hp cards will be easy as hell... I'm just not sure of how to do the rest in linux... BTW the server is not converted yet...

My plan was: 2X raid5 in nested Stripe with linux raid (Raid 50) -> then LVM over that -> then ext3 or xfs on top...

Thanks in advance for the comments!!!!

Who is Participating?
Well, there are a lot of online articles,  this points to a few threads that are as good as any, but let me add my own advice before you even start ..

1.  Upgrade kernel.  LOTS of bugs always being found with md, so might as well make sure you are current.
2.  Do full backup first.
3.  Do not expand RAID5 by 2 disks, I strongly recommend going to a RAID6 config, which means effectively adding only one usable disk for data.  It could very well take 3-4 days, or longer to rebuild those arrays, you don't want to be exposed that long.  So best way to do that is to boot to a non-RAID disk, or USB flash drive. Use the 4 new disks to create a 4-drive RAID6, copy from the existing RAID5.

It takes maybe 1/3rd as long to copy from the old array to a new array as it takes to expand an existing array.  You could do this overnight if you build the new array, then copied data to it.  Then when the disks are freed up from the first source array, you would probably have enough free space on that new array to save a compressed copy of the data from the other array, then repeat the process.

But here are some links for doing it the way you wanted to originally ..
What, exactly, is the question?

ismog_techAuthor Commented:
ya sorry for that I just re-read myself that's not that clear...

basically once my setup is made with the original hardware... what would be the steps I take to add the additionnal 4 drives of storage let's say past the expansion on the 2 raid cards (thios I know how to do)... How do I expand the raid volumes in linux then expand the LVM then expand the xfs or ext3...

better??? :P
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

ismog_techAuthor Commented:
ok I think you didn't figure out what I meant...

first the setup insn't important it's only transitory storage for D2D2T...

2nd ...

2 raids cards with 2 raid 5 so how do you expect to build a raid 6 with the 4 news drives since those will be plugged on 2 different raid cards???

so RAID5 array #1 =ra1
RAID5 array #2 =ra2

final storage at startup = final

final = stripe (ra1+ ra2) initial setup

what I need to know is....

final = stripe ( (ra1+ expansion) + (ra2 + expansion) )

how will I expand the 2 raid partition composing my stripe so they reflect the RAID array expansions?

So you are using md driver to stripe the 2 hardware-based RAID devices?   (Well, actually you could still use md to do this, as the LUN presented by each controller could still be part of a RAID6, but obviously that is not what you want).

First, you have to expand ra1 and ra2 to use the new drives per each raid controller's documentation.   Do all of this when system is booted to another device so ra1, ra2 and the md0 (assuming md0 = f(ra1,ra2).

Then you have to tell mdadm to discover the resized devices, and expand the volume ...  look at 2.2 here
Having said that, unless you have premium controllers with cache battery backup and channel load balancing, then you are most likely creating a big bottleneck with your design.  Every write is going to require at least one I/O on every disk drive.

If you care about performance, you should consider just using md RAID.
Adding to the above discussion depends on your initial setup.

I.e. do you use MD to completely RAID i.e. first RAID5 is /dev/sda and the second /dev/sdb
are you using software raid as /dev/md0 which is made up of /dev/sda /dev/sdb
Or are you first breaking each array and mirroring the individual partitions?
/dev/md0 /dev/sda1 /dev/sdb1 etc.

The per-partition software raid provides additional flexibility dealing with shifting partitions if/when issues arise with the underlying RAID controller.

As dlethe pointed out.
You would have to go through the steps.
expand each of the HW RAID groups
adjust the md create software raid
extend the physical volume to reflect the added space
adjust the logical volume/volumes to take advantage of the new space or create additional logical partitions.

ismog_techAuthor Commented:
but is that possible???

since the expansion of the 1st array will be right in the middle of the stripe....

another thing I don't understand how does my nested Raid 50 setup is underperforming compared to a raid6 software??? my card are hp P600 with batteries and 256 cache each...
Can't see how the shrink and grow link helps since it's talking about RAID 1 and just breaks the mirror, resizes and then re-mirrors. Not valid for this setup as you want to extend software RAID 0.

From what I read mdadm --grow isn't supported for RAID 0, but since it's just D2D2T you might as well blow it away and start again when you get the new disks.
On second thoughts says you can expand RAID 0.
ismog_techAuthor Commented:
Point for all ... good examples and links but finally took the terminator approach... kill and rebuild and rekill and rebuild... like I said it's temporary storage... and I like to have my weekends off :P but overall that was a good brain twister. thanks to all.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.