Solved

CentOS 4 Software RAID 1 Configuration

Posted on 2006-11-04
46
1,570 Views
Last Modified: 2013-12-15
Hello,

I have attempted to install CentOS 4 using software RAID but utterly failed. The Goal is to have a set of SATA hard drives completely redundant so that if one fails, nothing has to be done for the second to continue operating. I was also experiancing this error durring the install:

"There was an error installing *RANDOM PACKAGE NAME*. This can
indicate media failure, lack of disk space, and/or hardware problems.
This is fatal error and your install will be aborted. Please verify your
media and try your install again."
(http://www.experts-exchange.com/Operating_Systems/Linux/Linux_Setup/Q_21395656.html?query=This+can+indicate+a+media+failure%2C+lack+of+disk+space&clearTAFilter=true)

But I believe I have determined that the cause of this is not enough RAM for the complex installation (192mb, 64 reserved for video that I can't disable?...)

Anyways, the issue is that after I reboot after completing a seeminly successful install I recieve GRUB errors (usually, "GRUB GRUB GRUB") or another that I cannot recall. My objectives are basically covered in:
http://www.experts-exchange.com/Operating_Systems/Linux/Linux_Administration/Q_21647293.html?query=CentOS+Software+RAID&clearTAFilter=true

However, as I am a complete "newbie" to linux was wondering if there is any other way than the complicated procedures listed in the above solution. As well, I wanted to create a "logical" volume so a text install will not work (Correct?).

This is the partitioning I wish to have in exact replication on both drives (Software RAID 1): (The drives are 160GB SATA)

100mb: /boot
1GB: Swap
10GB: "/"
The rest is a logical volume: /Content


*Note, there is also a 40GB IDE drive for backup purposes
*Note, all drives are formatted as ext3

Any assistance would be much appriciated as I have been trying to get this webserver working for a long time now.

Thank You,
Max
0
Comment
Question by:Maxwellb
  • 23
  • 21
46 Comments
 
LVL 14

Expert Comment

by:ygoutham
ID: 17879278
as far as i believe, i think it should be RAID first and then create a Logical Volume.  if you have created the LVM first, then i am not sure as to how the RAID 1 wud function!!!

g
0
 

Author Comment

by:Maxwellb
ID: 17887302
ygoutham,

I have created the RAID first and then laid LVM over top of it... but again, the major issue here is getting the GRUB bootloader to function on a RAID 1 array.

-Max
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 17887439
is /boot a part of the LVM???  if so, then i would give up as i am not sure how to handle that.
0
 

Author Comment

by:Maxwellb
ID: 17887448
No, it is not. Only /Content is a logical volume.

Thank you, Max
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 17887450
sorry, i see that the /boot is not a part of LVM.  i think you need to create identical partitions, then copy (with tar) into the newly created partitions, modify grub then it should work.

0
 

Author Comment

by:Maxwellb
ID: 17887455
Hi,

Thanks for your quick response. Unfortunately, as I would be classified as a Level 1 newbie, I have not the slightest idea of how to go about doing that...and how would I then create the logical volume?

Thank You, Max
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 17887466
make a /boot on sdb. then copy the sda1 to sdb1. once that is done, you can test if it works by modifying grub to read off sdb wherever it says sda.  

check to see if mdadm is running with one (_U) atleast before and once booted from sdb, convert the sda1 also to a "fd" type on fdisk and do a mdadm again to see if you get a (UU) in /proc/mdstats.  once you are successful with /boot alone, then you can try out the rest of the primary and extended partitions.

i believe that you will need to create a similar partition table on sdb (for /content) and then do the above to see if it works.
0
 

Author Comment

by:Maxwellb
ID: 17887479
Hello,

I'm afraid (being only ever in the windows world) that I do not know what a:

-sdb
-sda
-mdadm
-(_U)
-(UU)
-"fd"
-/proc/mdstats
 
is. Must I run the install in text mode to accomplish this?

Thank You, Max
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 17887572
http://www.linuxhomenetworking.com/wiki/index.php/Quick_HOWTO_:_Ch26_:_Linux_Software_RAID

sda and sdb are the two sata disks that you have

mdadm is to create a software raid

once mdadm is run successfully then you see the number of disks syncing with

cat /proc/mdstats

"fd" is a linux autoraid partition type.  normally you would have 82 or 83 as the ext3 type (one is for swap that i am forgetting as to which one)

no you do not need to run anything in text mode to do anything.  check out the above site.
0
 

Author Comment

by:Maxwellb
ID: 17928025
Questions regarding the above link:

-So I install the OS first on one of the SATA drives (leaving the otherone untouched) then use the process described in the linked article to copy the contents of that drive to the other in the process creating RAID 1?

-Is this entire issue occuring because I am using SATA drives?

-Is there a GUI interface for setting up RAID after the install?

-Why am I unable to create the RAID partition at install and is there any distro that will allow me to do so?

-Is "single user mode" a synonym for root?

Thanks for your time,
Max
0
 

Author Comment

by:Maxwellb
ID: 17972322
ygoutham?
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 17977546
correct in most of your assumptions.

no issues because you are using SATA disks.

GUI interface, i am not aware of one for using post install.

again i am not aware of one for doing so at install time. even though fedora and redhat installs give you the option, i find it simpler to do it at run time.

single user mode is not a synonym for root.  super user is synonym for root.  
you can type a "init 1" at command prompt and land in single user mode.  as most of the activity involves the hard disks it is not advisable to have other users to be in a position to be able to log on when you are formating or changing file system tables and other details.
0
 

Author Comment

by:Maxwellb
ID: 17983958
"again i am not aware of one for doing so at install time. even though fedora and redhat installs give you the option, i find it simpler to do it at run time"

-Does this mean that in Fedora and Redhat  I am able to set up a complete software RAID 1 configuration in GUI (including /boot) durring the install?

-Max
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 17985093
yes
0
 

Author Comment

by:Maxwellb
ID: 17985164
Isn't CentOS supposed to be a clone of RedHat, was this feature disabled in CentOS? Would you be able to give me a link to the section of the manual that describes doing this in either Redhat or fedora?

Thank You,
Max
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 17985190
http://kbase.redhat.com/faq/FAQ_43_2812.shtm

there is also a section at the bottom where people have voiced their woes of having done it at install time and the system not recognising the same.  i do not know about centos install screen, but i know for a fact that it is rhel 4 with some minor modifications.  i have seen the options on fedora.  

i have posted the link on redhat site and you see that they also advise only doing it at run time than install time.

read through the related sections for better clarity.  my suggestion is to do it post install where you know that nothing would change.
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 17985195
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 17985198
0
 

Author Comment

by:Maxwellb
ID: 17985300
In the redhat article it recomends that RAID 1 be set up durring installation...not after on a live system?

"Converting an existing 'live' filesytem on a single disk to a software RAID 1 mirror is not recommended and could potentially cause you to lose all your data. The best time to set up software RAID is either at installation time or before you intend using the filesystem."

When it mentions "Raid 1", is it reffering to a single partition or the entire drive, failsafe in case one drive fails.

So basically, which one is recommended: Durring install in fedora or redhat, or post install at the command line.

Thank you,
Max
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 17985379
the article is right and correct.  when you are installing the system and after reboot when you log back in is when you create your raid partitions and set up raid.  the system is fresh from install and therefore it is safe.

you have the confidence that no user is going to log in as you have not created any additional users apart from yourself.  you as root have the option of unmounting the disk and repartitioning / reformatting it as "Linux Raid Auto-Detect" after UNMOUNTING it.

not when the disks are mounted which might cause lose of data and unstable output.

raid 1 is for a partition and in most cases, they intend to make it for the entire disk as you might be required to pull out the disk and send it for repair / replacement.  therefore the advice for "failsafe" drives (read diskettes).

as i am comfortable with post-install, i would say post-install is recommended.  fedora creates LVM as default options.  i would say do RAID first and LVM next, so that the changes are effected on both diskettes.  in case of a LVM additions, you need to add two diskettes to the array. one for the LVM and one for Raid 1.
0
 

Author Comment

by:Maxwellb
ID: 17985451
Ok, tomorrow I will post any questions I have on the articles.

Thank you,
Max
0
 

Author Comment

by:Maxwellb
ID: 18259447
Hi,

Sorry everyone for the long wait time, I am currently exploring my Hardware RAID options in another post and am waiting to see if that will work out or not - if it doesn't, then I plan to continue with this post and use software RAID. If anyone is interested in helping out there, the address is:
"http://www.experts-exchange.com/Operating_Systems/Linux/Q_22065616.html"

Thank you for your patience,
Max
0
Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

 
LVL 25

Expert Comment

by:Cyclops3590
ID: 18401043
Maxwellb,

Any update.

Cyclops3590
EE Cleanup Volunteer
0
 

Author Comment

by:Maxwellb
ID: 18402452
No. the other post is progressing at an agonizing pace. Will inform ASAP either way.

Thanks,
Max
0
 

Author Comment

by:Maxwellb
ID: 18424368
Hi everyone,

Well, Hardware RAID, after all that, is not going to work because the drivers provided by the maufacturer only work with kernel 2.4 - so back to software RAID. But first, an update on the setup. I have since upgraded the RAM to 768mb but am still getting this message durring the install when trying to do RAID durring setup:

"There was an error installing *RANDOM PACKAGE NAME*. This can
indicate media failure, lack of disk space, and/or hardware problems.
This is fatal error and your install will be aborted. Please verify your
media and try your install again."


So back to doing RAID after the setup (The hard drive's, to the best of my knowledge, checked out OK in the manufacturer's utility). I don't know why that error message occurs, but I'm led to believe that it is a problem with linux RAID. Right now I am at the partitioning screen durring the install and am wondering the best way to partition the drives so that  "http://kbase.redhat.com/faq/FAQ_43_2812.shtm" will work. It will not allow me to create two of "/boot" or "/", etc. What is the best way to do this so there is little editing as possible later and that there is complete redundancy if one of the drives is pulled. Also, it mentioned at the end of the artice, "Once the RAID device has been created change all references from /dev/hda3 to /dev/md0 in files such as /etc/fstab." That implies that there are more of such files (that I would not happen to know about) .

Thank you for your assistance,
Max
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 18425239
hi i do know that linux has a problem with hardware raid. you can try the following.

select one disk and make the partitions and select the file system to be raid.

select second disk and make identical partitions to the first disk and select all file systems to be RAID.

now from the first disk, click on the partition /boot (or anything) and click on create raid. select the appropriate partition on the second disk and select a raid level.  proceed for the rest of the partitions.  

for every pair the system auto creates a /dev/mdx (x being the partition number raid-ed).  it takes care of altering all the entries in fstab and gives you a raid working set.

BE SURE TO DISABLE HARDWARE RAID FROM BIOS.
0
 

Author Comment

by:Maxwellb
ID: 18425341
That's the problem though, whenever I try to setup software RAID durring the install, it gives me this error message:

"There was an error installing *RANDOM PACKAGE NAME*. This can
indicate media failure, lack of disk space, and/or hardware problems.
This is fatal error and your install will be aborted. Please verify your
media and try your install again."

But not when I do a normal install. When I press <alt+F3(or F4, et)> I see errors about "resynch" and like "Needed mdadm for none" and other such cryptic messages.

Thank You,
Max
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 18425445
rsync and mdadm are packages that are required to keep the disks sync-ed with each other.  mdadm is the package which controlls all the raid arrays when you have a working system.

is there a package called rsync-xxxxx.rpm in your cd??? then probably that particular file might be corrupt.

options:  redownload the iso and burn a new cd to see if problem persists.

option 2:  proceed ahead with the install first without a raid (but keep the file systems as RAID) and see if the installation goes through without any troubles.  in which case i would say that the cd is good but something else is the matter.  if you still get an error message during install which says "rsync" and the like blah blah option 1 might solve it.

option3:  if the vanilla install goes through without any hassles, then you can use a fdisk to partition the second disk in an identical manner and create the raid by hand post install.  If you do not select the file system as raid when you install then you will have to partition the second disk make an exact copy of all the files from disk 1, use fdisk to change the filesystem type and then proceed to create the raid arrays.

so simple would be create partitions and make them raid file systems (which are journalised ext3) at format time.
0
 

Author Comment

by:Maxwellb
ID: 18425461
Hi,

Thanks for replying so quick. I have already re-downloaded (and md5 checksum'ed) all of the ISO, burnt them on three different brands of recordable CDs and verified the install media before the installation each time and it always comes up as pass. New CDROM drive, new RAM, and all new cables. I am at the format screen right now. I will format everything as RAID devices and then see how the install goes as I do not look forward to having to use fdisk to partition everything on the second drive :)

Thanks,
Max
0
 

Author Comment

by:Maxwellb
ID: 18425487
Hi,

Without the RAID devices, the installer will not let me proceed because it says that the system needs a "/" partition, and with all the partitions on the system being RAID partitions (which cannot be assigned a mount point) it will not proceed...

Any ideas?

Thanks,
Max
0
 

Author Comment

by:Maxwellb
ID: 18425573
Does this mean that it is not possible to complete "option 2" and I should simply make the partitions the same size to lessen the amount of work that will havbe to be done later to prepare the partitions of the second disk (All then I would have to do is rename them...?)?

Thank you,
Max
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 18425756
we can try option 2.  make the partitions the same size and leave out the disk2 from being mounted anywhere.  
we can subsequently use fdisk to change the filesystem to 16 or raid

then do a copy with tar and preserve permissions from disk1.  once done we can change the disk1 and change the filesystem type again to 16 there

my experiments shows that it is possible

##########
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14         274     2096482+  82  Linux swap / Solaris
/dev/sda3             275        4427    33358972+  83  Linux
 
Disk /dev/sdb: 36.4 GB, 36420075008 bytes
255 heads, 63 sectors/track, 4427 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
 
   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1          13      104391   fd  Linux raid autodetect
/dev/sdb2              14         274     2096482+  fd  Linux raid autodetect
/dev/sdb3             275        4427    33358972+  fd  Linux raid autodetect
 
#############

once we see that this is successfully done and disk2 has a copy of all disk1 data (leaving out swap) which i do not think is relevant as we can have the same not raid-ed and use it from fstab. alter the /boot/grub.conf to make the system boot from disk2.  once having booted from disk2 we can change the disk1 filesystem and then use

mdadm

to create the raid.
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 18425758
it is not 16 but fd. sorry about the confusion in the post.  fd is for Linux raid autodetect and 16 is ext3
0
 

Author Comment

by:Maxwellb
ID: 18425768
"leave out the disk2 from being mounted anywhere"

Does this mean durring the install to just make the partitions the same size, but with different names and as ext 3?
0
 

Author Comment

by:Maxwellb
ID: 18425772
"
Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1          13      104391   fd  Linux raid autodetect
/dev/sdb2              14         274     2096482+  fd  Linux raid autodetect
/dev/sdb3             275        4427    33358972+  fd  Linux raid autodetect
 "

I noticed that the ID for the second disk was all "fd", what is "fd" and why does the first disk use numbers as the ID instead?
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 18425808
ld happens to be the file system type that you select when using a fdisk.  when you say it is 82 then it means that it is linux swap partition. 83 would mean a ext3 partition. similarly "fd" means that it is a raid partition. i just used fdisk to format my disk and repartition them to look like the first disk (in terms of size notice the start and end block numbers).

>>>Does this mean durring the install to just make the partitions the same size, but with different names and as ext 3?
yes and no.  make similar size partitions but select the file system to be linux raid autodetect which is equal to "FD"

EXT3 = 83
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 18425814
list taken from fdisk to show file system types

################
0  Empty           1e  Hidden W95 FAT1 80  Old Minix       be  Solaris boot
 1  FAT12           24  NEC DOS         81  Minix / old Lin bf  Solaris
 2  XENIX root      39  Plan 9          82  Linux swap / So c1  DRDOS/sec (FAT-
 3  XENIX usr       3c  PartitionMagic  83  Linux           c4  DRDOS/sec (FAT-
 4  FAT16 <32M      40  Venix 80286     84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
 5  Extended        41  PPC PReP Boot   85  Linux extended  c7  Syrinx
 6  FAT16           42  SFS             86  NTFS volume set da  Non-FS data
 7  HPFS/NTFS       4d  QNX4.x          87  NTFS volume set db  CP/M / CTOS / .
 8  AIX             4e  QNX4.x 2nd part 88  Linux plaintext de  Dell Utility
 9  AIX bootable    4f  QNX4.x 3rd part 8e  Linux LVM       df  BootIt
 a  OS/2 Boot Manag 50  OnTrack DM      93  Amoeba          e1  DOS access
 b  W95 FAT32       51  OnTrack DM6 Aux 94  Amoeba BBT      e3  DOS R/O
 c  W95 FAT32 (LBA) 52  CP/M            9f  BSD/OS          e4  SpeedStor
 e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a0  IBM Thinkpad hi eb  BeOS fs
 f  W95 Ext'd (LBA) 54  OnTrackDM6      a5  FreeBSD         ee  EFI GPT
10  OPUS            55  EZ-Drive        a6  OpenBSD         ef  EFI (FAT-12/16/
11  Hidden FAT12    56  Golden Bow      a7  NeXTSTEP        f0  Linux/PA-RISC b
12  Compaq diagnost 5c  Priam Edisk     a8  Darwin UFS      f1  SpeedStor
14  Hidden FAT16 <3 61  SpeedStor       a9  NetBSD          f4  SpeedStor
16  Hidden FAT16    63  GNU HURD or Sys ab  Darwin boot     f2  DOS secondary
17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fd  Linux raid auto
18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fe  LANstep
1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid ff  BBT
1c  Hidden W95 FAT3 75  PC/IX
Hex code (type L to list codes):
##############
0
 

Author Comment

by:Maxwellb
ID: 18425825
In the CentOS Disk Druid GUI partitioning setup screen during the install, there is not option to use "linux raid autodetect"

Does this mean I have to do partitioning before starting the installation process or after?

Thanks,
Max
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 18425886
do a simple install using only disk1.  do not bother anything about raid autodetect. finish off the install. select all partitions to be ext3 during install.  once done then after first boot, we will start with disk2 and use

fdisk /dev/sdb

to create linux raid autodetect partitions of the same size as disk1.  once the disk is formatted and partitions created,

then we copy contents from disk1 to disk2.

copy /boot to the appropriate partition in sdb (my guess /dev/sdb1)
similarly copy all the rest of the partitions to disk2. once done we do a grub-install on disk2 to make it the primary disk to boot out of.  once booted from disk2 we

fdisk /dev/sda

to change the filesystem type
0
 

Author Comment

by:Maxwellb
ID: 18428445
OK,

Thank you, I will proceed to install the OS and notify you when it completes.


-Max
0
 

Author Comment

by:Maxwellb
ID: 18442411
Amazingly enough, by some fluke, I decided to try one more time to install CentOS with RAID configuration at install time (this time only selecting the graphical environment and not all the other packages - the install was about 1400mb), and it worked. Well, this is in theory. How do I know that it actually worked? How do I know that my system is stable and that software RAID1 is mirroring all data between the drives? What do I do if one of the drives fails?

Thank you,
Max
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 18442477
[root@xxx /]# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sdb1[1] sda2[0]
      40957568 blocks [2/2] [UU]
       
md2 : active raid1 sdb2[1] sda6[0]
      25559296 blocks [2/2] [UU]
       
md0 : active raid1 sdb3[1] sda3[0]
      10482304 blocks [2/2] [UU]
       
unused devices: <none>

this is how it should look. Note the [UU] which means disk 1 in USE and disk2 in USE.  if one device were not to be present then you have only [_U]

welll if you have one disk which is not working or fails then you have the option of removing the disk and take it to a different system to see if the disk is corrupt or data is corrupt.  data being corrupt is handled quite simply by taking away all the content and copying the relevant info into the partition and plug the disk back.

disk bad, then buy new one and start from step 1.

that is what i do
0
 
LVL 14

Accepted Solution

by:
ygoutham earned 500 total points
ID: 18442594
there is a way to make a forced fail of one of the disks in an array

mdadm -f /dev/sda1

will cause one of the disks to stop and if you still see the files in /boot then you know that it is functioning properly

you can test with that and and reboot to get back to the original mode.  i do not know how to reenable a failed disk in an array.  try going through "man -a mdadm" to get more info on that.  

useful before deploying machine in production
0
 

Author Comment

by:Maxwellb
ID: 18585084
Ygoutham,

Thank you very much for your assistance and patience in this matter. I award you full pounts.

Thanks,
Max
0
 
LVL 14

Expert Comment

by:ygoutham
ID: 18587682
am grateful to you too.  glad that i could be of assistance.  

Cheers and Good Luck!!!

Goutham
0

Featured Post

Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

Join & Write a Comment

In this tutorial I will explain how to make squid prevent malwares in five easy steps: Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP, and more. It reduces bandwidth and improves response times by caching and reusing frequently-…
SSH (Secure Shell) - Tips and Tricks As you all know SSH(Secure Shell) is a network protocol, which we use to access/transfer files securely between two networked devices. SSH was actually designed as a replacement for insecure protocols that sen…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now