Solved

Nested RAID 50 Linux Raid + hardware raid question

Posted on 2010-11-21
10
810 Views
Last Modified: 2012-05-10
Hi all...

I need help to figure out something...

Before banging my head on the wall with a problem I want to know if anybody would throw in a bit of advice on this one...

I'm killing an old win 2003 backup server. reinstalling in linux of course :P ... here is the problem I have...

I have 2 RAID controllers in there with 3X1TB in RAID 5 on each...
I did put those 2 raid 5 in stripe using software RAID...

I know that this easy and possible with Linux here is the real question...

I will be receiving 4 extra drives in a couple of weeks to add to this server... 2 drives for each Raid 5 pool... extending those with the hp cards will be easy as hell... I'm just not sure of how to do the rest in linux... BTW the server is not converted yet...

My plan was: 2X raid5 in nested Stripe with linux raid (Raid 50) -> then LVM over that -> then ext3 or xfs on top...

Thanks in advance for the comments!!!!

Marco
0
Comment
Question by:ismog_tech
  • 4
  • 3
  • 2
  • +1
10 Comments
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
What, exactly, is the question?

0
 
LVL 1

Author Comment

by:ismog_tech
Comment Utility
ya sorry for that I just re-read myself that's not that clear...

basically once my setup is made with the original hardware... what would be the steps I take to add the additionnal 4 drives of storage let's say past the expansion on the 2 raid cards (thios I know how to do)... How do I expand the raid volumes in linux then expand the LVM then expand the xfs or ext3...

better??? :P
0
 
LVL 47

Accepted Solution

by:
dlethe earned 167 total points
Comment Utility
Well, there are a lot of online articles,  this points to a few threads that are as good as any, but let me add my own advice before you even start ..

1.  Upgrade kernel.  LOTS of bugs always being found with md, so might as well make sure you are current.
2.  Do full backup first.
3.  Do not expand RAID5 by 2 disks, I strongly recommend going to a RAID6 config, which means effectively adding only one usable disk for data.  It could very well take 3-4 days, or longer to rebuild those arrays, you don't want to be exposed that long.  So best way to do that is to boot to a non-RAID disk, or USB flash drive. Use the 4 new disks to create a 4-drive RAID6, copy from the existing RAID5.

It takes maybe 1/3rd as long to copy from the old array to a new array as it takes to expand an existing array.  You could do this overnight if you build the new array, then copied data to it.  Then when the disks are freed up from the first source array, you would probably have enough free space on that new array to save a compressed copy of the data from the other array, then repeat the process.

But here are some links for doing it the way you wanted to originally ..
http://ubuntuforums.org/archive/index.php/t-517282.html


http://ubuntuforums.org/archive/index.php/t-517282.html
0
 
LVL 1

Author Comment

by:ismog_tech
Comment Utility
ok I think you didn't figure out what I meant...

first the setup insn't important it's only transitory storage for D2D2T...

2nd ...

2 raids cards with 2 raid 5 so how do you expect to build a raid 6 with the 4 news drives since those will be plugged on 2 different raid cards???

so RAID5 array #1 =ra1
RAID5 array #2 =ra2

final storage at startup = final

final = stripe (ra1+ ra2) initial setup

what I need to know is....

final = stripe ( (ra1+ expansion) + (ra2 + expansion) )

how will I expand the 2 raid partition composing my stripe so they reflect the RAID array expansions?

0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
So you are using md driver to stripe the 2 hardware-based RAID devices?   (Well, actually you could still use md to do this, as the LUN presented by each controller could still be part of a RAID6, but obviously that is not what you want).

First, you have to expand ra1 and ra2 to use the new drives per each raid controller's documentation.   Do all of this when system is booted to another device so ra1, ra2 and the md0 (assuming md0 = f(ra1,ra2).


Then you have to tell mdadm to discover the resized devices, and expand the volume ...  look at 2.2 here

http://www.howtoforge.com/how-to-resize-raid-partitions-shrink-and-grow-software-raid
==
Having said that, unless you have premium controllers with cache battery backup and channel load balancing, then you are most likely creating a big bottleneck with your design.  Every write is going to require at least one I/O on every disk drive.

If you care about performance, you should consider just using md RAID.
0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 
LVL 76

Assisted Solution

by:arnold
arnold earned 166 total points
Comment Utility
Adding to the above discussion depends on your initial setup.

I.e. do you use MD to completely RAID i.e. first RAID5 is /dev/sda and the second /dev/sdb
are you using software raid as /dev/md0 which is made up of /dev/sda /dev/sdb
Or are you first breaking each array and mirroring the individual partitions?
/dev/md0 /dev/sda1 /dev/sdb1 etc.

The per-partition software raid provides additional flexibility dealing with shifting partitions if/when issues arise with the underlying RAID controller.

As dlethe pointed out.
You would have to go through the steps.
expand each of the HW RAID groups
adjust the md create software raid
extend the physical volume to reflect the added space
adjust the logical volume/volumes to take advantage of the new space or create additional logical partitions.


0
 
LVL 1

Author Comment

by:ismog_tech
Comment Utility
but is that possible???

since the expansion of the 1st array will be right in the middle of the stripe....

another thing I don't understand how does my nested Raid 50 setup is underperforming compared to a raid6 software??? my card are hp P600 with batteries and 256 cache each...
0
 
LVL 55

Expert Comment

by:andyalder
Comment Utility
Can't see how the shrink and grow link helps since it's talking about RAID 1 and just breaks the mirror, resizes and then re-mirrors. Not valid for this setup as you want to extend software RAID 0.

From what I read mdadm --grow isn't supported for RAID 0, but since it's just D2D2T you might as well blow it away and start again when you get the new disks.
0
 
LVL 55

Assisted Solution

by:andyalder
andyalder earned 167 total points
Comment Utility
On second thoughts http://man-wiki.net/index.php/8:raidreconf says you can expand RAID 0.
0
 
LVL 1

Author Closing Comment

by:ismog_tech
Comment Utility
Point for all ... good examples and links but finally took the terminator approach... kill and rebuild and rekill and rebuild... like I said it's temporary storage... and I like to have my weekends off :P but overall that was a good brain twister. thanks to all.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Introduction We as admins face situation where we need to redirect websites to another. This may be required as a part of an upgrade keeping the old URL but website should be served from new URL. This document would brief you on different ways ca…
This article is an update and follow-up of my previous article:   Storage 101: common concepts in the IT enterprise storage This time, I expand on more frequently used storage concepts.
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now