Software RAID 1 setup on CentOS using Grub boot loader

> Basically, I want a complete RAID 1 setup so if one of my hard disk fails the other one will take over and during a disaster
/boot in "Software" RAID 1 is not allowed.

The easier and better way is buy a IDE RAID controller (hardware RAID 1), such as Promise IDE RAID card.

Or you can install Cent)S 4.2 on one IDE drive and do a disk image on another IDE drive.
Put your IDE drive on two different IDE channel as master drive (/dev/hda and /dev/hdc).

Then configure your "/" partition as "software" RAID 1.
The swap partition leave it untouch (Software RAID 1 on swap partition will slow down your performance).

Then edit the following file on the second IDE drive:
/boot2/grub/menu.lst  or  /boot2/grub/grub.conf
and change all "/dev/hda --> /dev/hdc" or change "root=LABEL=/" --> "root=/dev/hdc2

There is one problem is /etc/fstab for the following line:
LABEL=/                 /                       ext3    defaults        1 1

Since you mirror / partition, so /etc/fstab will be the same on both drive.
But you need to change /etc/fstab on the second drive to:
/dev/hdc2                 /                       ext3    defaults        1 1


So the conclusion is still to buy the IDE RAID card.........

> /boot in "Software" RAID 1 is not allowed.
bullshit, I have many system with that config, and works fine.

> /dev/md1      2000MB       Swap
Don't use software raid for swap partitions - it's pointless
instead in /etc/fstab:
/dev/hda3               swap                    swap    defaults,pri=1  0 0
/dev/hdc3               swap                    swap    defaults,pri=1  0 0
(and make the partitions of type swap(with fdisk), and make them to be swap(mkswap /dev/hd[ac]3))

OK, it's safe to make /boot or root partition to be sodtware raid1. If about grub, it's just You have to install it on both mirrored partitions (but those must be synced before the operation, chek it with 'cat /proc/mdstat')
It's advised to install grub in MBR. The installer would install the grub only on first disk (as the command: grub-install /dev/hda)
Next, You have to install the grub on the second disk by hand
run: 'grub', then enter commands (note the command root (hd0,0) - it assumes that /boot is on /dev/hdc1, use (hd0,1) for /dev/hdc2 etc.)
device (hd0) /dev/hdc
root (hd0,0)
setup (hd0)

Hi ravenpl,

According to your method, after you tweak the grub for /boot partition on the second drive, then you can not sync the /boot
partition any more. Otherwise, the files under /boot/grub/ on the second disk will be overwritten by RAID 1 syncing.
If you do not sync /boot, then the content of /boot partition on both disk are not sync or out of sync.
In this case, /boot is not in software RAID 1, right?

I'm not sure whether kernel 2.6.x or new version of GRUB make software RAID 1 on /boot work or not. But I tried on kernel 2.4.x and
/boot on software RAID 1 is not allowed.

Besides, based on the original request, if one drive fails, then he can boot from the other drive.
Then swap partition on both drive in /etc/fstab will cause boot issue if one of drive fail.

Correct me if I were wrong.

> then you can not sync the /boot
No, You should sync /boot, as changes to /boot/grub/menu.lst should be synced on both drives, right ?
It's just You have to install grub on the second disk AFTER the sync finished.
Same applies to kernel images, You copy one only to the MD device once - it's spread automatically across all partitions.
Of course, if You install new version of grub, You have to install it on the second drive too.

> But I tried on kernel 2.4.x
I use it on both, 2.6 and 2.4 kernels - works well.

> Besides, based on the original request, if one drive fails, then he can boot from the other drive.

The only problem with /boot on software raid is, that if first disk fails(eg. badblock on the kernel or grub image), but it's still visible in bios, then the system will not start. But then, it's enough to turn the disk off in bios or plug it off from the machine to get the system back booting.
But if the /boot was not on the mirror, You would have to use liveCD to get Your system back working.

> Then swap partition on both drive in /etc/fstab will cause boot issue if one of drive fail
Not really, swap partitions are not mounted, therefore booting is not stopping becouse of that. Checked on FC2/4/gentoo.

> It's just You have to install grub on the second disk AFTER the sync finished.
Thank ravenpl for the explanation. I got it.

Thanks for all of your comments above - I am still little puzzled!

Ravenpl, Thank you for your idea especially. As you suggested, I did NOT make my swap partion as raid 1. I made /boot and / partion both raid 1 (/dev/md0 /dev/md1 respectively).  So, my current setup looks like this:

/dev/md0      196MB         ext3                  /boot
/dev/md1      76096MB     ext3                  /
/dev/hda
    hda1          196MB        SoftwareRaid    
    hda2          76096MB    SoftwareRaid
    hda3          2000MB      swap
/dev/hdb
    hdb1          196MB        SoftwareRaid
    hdb2          76096MB    SoftwareRaid
    hdb3          2000MB      swap

With the above setup I installed CentOS and I rebooted the computer the 'grub' did hang on me... but I managed to boot to the rescue mode using the installation CD and I ran the following commands from shell (and the response I got):

grub> device (hd0) /dev/hda
grub> root (hd0,0)
            Filesystem type is ext2fs, partion type 0xfd
grub> setup (hd0)
          checking if "/boot/grub/stage1" exists....no
          checking if "/grub/stage1" exists....yes
          checking if "/grub/stage2" exists....yes
          checking if "/grub/e2fs_stage1.5" exists....yes
          Running "embed /grub/e2fs_stage1_5 (hd0)".... 16 sectors are embedded. succeeded.
          Running "install /grub/stage1 (hd0) (hd0)1+16p (hd0,0) /grub/stage2 /grub/grub.conf" succeded.

Now, running all the above commands made my system bootable(I am happy). My understanding is that all the above commands made the boot loader to install on the boot partion of my first hard disk - right???

Do I have to do the same for my second hard disk now? so that, the /boot partion is installed on my second hard drive as well which will *hopefully* let me boot if the first hard disk fails?

If it helps, I did a 'cat /proc/mdstat' :

     personalities: [raid0][raid1][raid5][raid6]
     md1: active raid hdb2[1] hda2[0]
     md0: active raid hdb1[1] hda1[0]

Also, I am still not clear about the swap partion setup from Ravenpl's post.  Sorry, guys:-)

Thanks in advance for any help you can provide.  Ravenpl: I am hoping to hear from you especially:-)

> Now, running all the above commands made my system bootable(I am happy). My understanding is that all the above commands made the boot loader to install on the boot partion of my first hard disk - right???
the grub was installed in the MBR, so changing active partition couse nothing :)

> Do I have to do the same for my second hard disk now? so that, the /boot partion is installed on my second hard drive as well which will *hopefully* let me boot if the first hard disk fails?
Yes, If You want to be able to boot from the second disk (in case of the failure of primary), the do same commands except first (device (hd0) /dev/hdc)

> Also, I am still not clear about the swap partion setup from Ravenpl's post.  Sorry, guys:-)
Have You configured those in the /etc/fstab as I suggested? cat /proc/swaps

Hello ravenpl,
    Okay... Before I get to the swap partition part I have to explain what's happening with my system currently.

As I mentioned in my last post, this is what I did originally after taking your advice.
grub> device (hd0) /dev/hda
grub> root (hd0,0)
            Filesystem type is ext2fs, partion type 0xfd
grub> setup (hd0)
          checking if "/boot/grub/stage1" exists....no
          checking if "/grub/stage1" exists....yes
          checking if "/grub/stage2" exists....yes
          checking if "/grub/e2fs_stage1.5" exists....yes
          Running "embed /grub/e2fs_stage1_5 (hd0)".... 16 sectors are embedded. succeeded.
          Running "install /grub/stage1 (hd0) (hd0)1+16p (hd0,0) /grub/stage2 /grub/grub.conf"

Now... just for fun (really) I unplugged my primary hard disk and re-booted the system. Guess what? The system booted fine with the second hard disk... ( I didn't have to run any more command to make it to do this as you suggested in your last post). Okay, that's cool now. Now back to the original setup (both hard disks are plugged in) - I created a directory under /home and called it "test".

At this point, I shutdown the sytem, unplugged the primary hard disk again rebooted it with the second hard disk. I go into the /home directory and I don't see the "test" directory I just created  - Now this confuses the heck out of me:-(  With the RAID 1 setup shouldn't I have the exact copy of my first hard disk in the second one??

Okay.. One more test.. With the second hard disk in place (primary one is still unplugged) - I created another new directory called "newtest" under /home and shutdown the system. Plugged the primary drive back on re-booted the system did an "ls" under /home and guess what I found?  The "newtest" directory that I created when the primary hard disk wasn't plugged in.

So, what is going on here?? Is my system really running off the second hard disk even though the primary hard disk is plugged in??  How come things are not getting copied over when I create something under primary disk and I don't see it when I booted it from the second disk - shouldn't RAID 1 suppose to take care of that??

Please explain what's going on!!!  I am glad I tested by creating this temporary directories. If I decided that the mirroring was working once I was able to boot from the second hard disk and continue to setup everything a big shock might have been waited for me down the road when I actually had to boot from the second hard drive and not finding any of the files or directories I have created:-(

Thanks in advace... You are being really helpful and I really want this thing to be properly setup by the end of this weekend:-) so, I am working hard at it!

> With the RAID 1 setup shouldn't I have the exact copy of my first hard disk in the second one?
If the array is in sync state. When You unplug one disk, start system, stop, plug the disk again, and start system - the array is in unsynced state and only copy is used (for both read/write). You can check it in /proc/mdstat - which copy is used.

> The "newtest" directory that I created when the primary hard disk wasn't plugged in.
As I don't belive in miracles - You messing something up... What You mean by 'unplugged'? I hope cut off the power (not disabled in bios)

Ravelpl,
   I will have time to work on the system when I get back home this evening after my work.  I will do /proc/mdstat and find out which copy it is using currently.

But first, how do I make the array to be in sync state again? run the commands that I originally did under 'grub'?

    By 'unplugged' I mean cutting the power off to the hard disk.  So are you saying that the "newtest" directory that I created (when the primary disk wasn't plugged in) will not be shown until the hard disks are synchronized again??

Thanks.

ravenpl,
    Thanks very much for your help. I got everything sorted out:-)