Solved

Centos 7, grub2, problems putting root on raid1 md array

Posted on 2014-11-12
43
3,236 Views
Last Modified: 2014-11-22
Hi! I have just installed Centos 7 in text mode and I have created md raid1 arrays for /boot /home and / - respectively /dev/md0, /dev/md2 and /dev/md1. I have copied the filesystems to the arrays and the arrays work fine...
The system boots correctly from /dev/md0 (/boot) and the /dev/md3 array for /home also works, but after booting root is still on /dev/sda3 and I cannot find the way to put root on /dev/md1. When I try to boot with root on /dev/md1 the boot process stops and journalctl says that /dev/md1 does not exists - the same happens with UUID reference to the device...

After creating the md arrays I have done the following:

mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.old
dracut --mdadmconf --force /boot/initramfs-$(uname -r).img $(uname -r)

and

grub2-mkconfig -o /boot/grub2/grub.cfg


After booting with root on /dev/sda3 we have:

[root@filemon1 ~]# cat /proc/mdstat

Personalities : [raid1]
md1 : active raid1 sdb3[0]
      52395904 blocks super 1.2 [2/1] [U_]

md2 : active raid1 sda5[2] sdb5[0]
      911293248 blocks super 1.2 [2/1] [U_]
      [====>................]  recovery = 21.8% (199387776/911293248) finish=98.7min speed=120184K/sec

md0 : active raid1 sda1[1] sdb1[0]
      511936 blocks [2/2] [UU]

unused devices: <none>

============================================================================
[root@filemon1 ~]# blkid
/dev/sda1: UUID="164b8030-9374-544c-ea0a-9c266e71ac0d" TYPE="linux_raid_member"
/dev/sda2: UUID="5283dd61-f307-4713-aefc-5ae20be3d360" TYPE="swap"
/dev/sda3: UUID="ffd854fb-2873-42e2-a24f-418ce9403c06" TYPE="xfs"
/dev/sda5: UUID="18f7f4c8-f906-02a9-2b37-af7ae9376c77" UUID_SUB="0078a81e-59f6-82d8-d005-af55366d09a0" LABEL="filemon1:2" TYPE="linux_raid_member"
/dev/sdb1: UUID="164b8030-9374-544c-ea0a-9c266e71ac0d" TYPE="linux_raid_member"
/dev/sdb2: UUID="2b1f40b5-ba77-49df-aa5c-9a0a443a1797" TYPE="swap"
/dev/sdb3: UUID="966e1308-ae15-ed8b-1efd-7893b75eee13" UUID_SUB="1e967fb7-ab10-f1bb-353c-78954b74b312" LABEL="filemon1:1" TYPE="linux_raid_member"
/dev/sdb5: UUID="18f7f4c8-f906-02a9-2b37-af7ae9376c77" UUID_SUB="4c7a102a-1180-20f8-4d0f-cf542c91a315" LABEL="filemon1:2" TYPE="linux_raid_member"
/dev/md0: UUID="d3508712-dd77-4481-a2b8-614dc7e1df35" TYPE="xfs"
/dev/md2: UUID="f2f78b42-b663-4a29-8a71-2e3f44e71b46" TYPE="xfs"
/dev/md1: UUID="9c4cbd48-9319-45e4-8492-185551d3a955" TYPE="xfs"

============================================================================
[root@filemon1 ~]# cat /etc/fstab
UUID=9c4cbd48-9319-45e4-8492-185551d3a955 /                       xfs     defaults        1 1
UUID=d3508712-dd77-4481-a2b8-614dc7e1df35 /boot                   xfs     defaults        1 2
UUID=f2f78b42-b663-4a29-8a71-2e3f44e71b46 /home                   xfs     defaults        1 2
UUID=2b1f40b5-ba77-49df-aa5c-9a0a443a1797 swap                    swap    defaults        0 0
UUID=5283dd61-f307-4713-aefc-5ae20be3d360 swap                    swap    defaults        0 0

============================================================================

[root@filemon1 ~]# mount
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=12220856k,nr_inodes=3055214,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,seclabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/sda3 on / type xfs (rw,relatime,seclabel,attr2,inode64,noquota)        
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=34,pgrp=1,timeout=300,minproto=5,maxproto=5,direct)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
mqueue on /dev/mqueue type mqueue (rw,relatime,seclabel)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)
/dev/md2 on /home type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
/dev/md0 on /boot type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
[root@filemon1 ~]#

==================================================================================================

[root@filemon1 ~]# cat /boot/grub2/grub.cfg
#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub2-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
set pager=1

if [ -s $prefix/grubenv ]; then
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="${saved_entry}"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}

function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

terminal_output console
if [ x$feature_timeout_style = xy ] ; then
  set timeout_style=menu
  set timeout=5
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
  set timeout=5
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/09_swraid1_setup ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/09_swraid1_setup ###

### BEGIN /etc/grub.d/10_linux ###
menuentry 'CentOS Linux, with Linux 3.10.0-123.el7.x86_64' --class centos --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux
-3.10.0-123.el7.x86_64-advanced-ffd854fb-2873-42e2-a24f-418ce9403c06' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_msdos
        insmod diskfilter mdraid09
        insmod xfs
        set root='mduuid/164b80309374544cea0a9c266e71ac0d'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/164b80309374544cea0a9c266e71ac0d'  d3508712-dd77-4481-a2b8-614dc7e1df35
        else
          search --no-floppy --fs-uuid --set=root d3508712-dd77-4481-a2b8-614dc7e1df35
        fi
        linux16 /vmlinuz-3.10.0-123.el7.x86_64 root=UUID=ffd854fb-2873-42e2-a24f-418ce9403c06 ro vconsole.keymap=us crashkernel=auto  vconsole.font=latarcyrheb-
sun16 rhgb quiet
        initrd16 /initramfs-3.10.0-123.el7.x86_64.img
}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/20_ppc_terminfo ###
### END /etc/grub.d/20_ppc_terminfo ###

### BEGIN /etc/grub.d/30_os-prober ###
menuentry 'CentOS Linux release 7.0.1406 (Core)  (on /dev/md1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-simple-9c4cbd48
-9319-45e4-8492-185551d3a955' {
        insmod part_msdos
        insmod diskfilter mdraid09
        insmod xfs
        set root='mduuid/164b80309374544cea0a9c266e71ac0d'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/164b80309374544cea0a9c266e71ac0d'  d3508712-dd77-4481-a2b8-614dc7e1df35
        else
          search --no-floppy --fs-uuid --set=root d3508712-dd77-4481-a2b8-614dc7e1df35
        fi
        linux /vmlinuz-3.10.0-123.el7.x86_64 root=/dev/md1
        initrd /initramfs-3.10.0-123.el7.x86_64.img
}
submenu 'Advanced options for CentOS Linux release 7.0.1406 (Core)  (on /dev/md1)' $menuentry_id_option 'osprober-gnulinux-advanced-9c4cbd48-9319-45e4-8492-1855
51d3a955' {
        menuentry 'CentOS Linux release 7.0.1406 (Core)  (on /dev/md1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-/vmlinu
z-3.10.0-123.el7.x86_64--9c4cbd48-9319-45e4-8492-185551d3a955' {
                insmod part_msdos
                insmod diskfilter mdraid09
                insmod xfs
                set root='mduuid/164b80309374544cea0a9c266e71ac0d'
                if [ x$feature_platform_search_hint = xy ]; then
                  search --no-floppy --fs-uuid --set=root --hint='mduuid/164b80309374544cea0a9c266e71ac0d'  d3508712-dd77-4481-a2b8-614dc7e1df35
                else
                  search --no-floppy --fs-uuid --set=root d3508712-dd77-4481-a2b8-614dc7e1df35
                fi
                linux /vmlinuz-3.10.0-123.el7.x86_64 root=/dev/md1
                initrd /initramfs-3.10.0-123.el7.x86_64.img
        }
}

### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg;
fi
### END /etc/grub.d/41_custom ###

I would like to have / filesystem on /dev/md1 after booting.

Thank You for any suggestion!
0
Comment
Question by:Lelio Michele Lattari
  • 28
  • 15
43 Comments
 
LVL 61

Expert Comment

by:gheist
Comment Utility
why dont you mount using device names instead?
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
Hi! How is it possible that nobody tries to help me !? :-((
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
Evertthing mounts consistently:
mnttab
/dev/sda3 on / type xfs (rw,relatime,seclabel,attr2,inode64,noquota)  

blkid
/dev/sda3: UUID="ffd854fb-2873-42e2-a24f-418ce9403c06" TYPE="xfs"

grub:
root=UUID=ffd854fb-2873-42e2-a24f-418ce9403c06
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
Hi guys! As I said, I would like to have / filesystem on /dev/md1 after booting :-) Currently when I boot with filesystem / mounted on md1 ( menuentry 'CentOS Linux release 7.0.1406 (Core)  (on /dev/md1)' ) the system halts and, as I also said, journalctl says that /dev/md1 does not exists! Menuentry with / mounted on sda3 is the only that works, althought the /boot filesystem on md0 mounts correctly :-)
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
The same happens with Scientific Linux 7, installed in text mode, after creating md arrays. I think grub2 misses some modules at the boot time. This happens only trying to mount / filesystem from md array but boot and home can be mounted from md arrays at boot time. Pls help
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
Should I understand that nobody can help me? :-(((
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
You can rebuild initrd with dracut -f
And add md modules in /etc/dracut.conf if they are still missing.
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
As said earlier, after creating the md arrays I have done the following:

mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.old
dracut --mdadmconf --force /boot/initramfs-$(uname -r).img $(uname -r)

and

grub2-mkconfig -o /boot/grub2/grub.cfg
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
dracut -v | less

and check if DM files are properly included.
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
gheist: dracut -v | less produces nothing.... This does not solve my problem....

Executing: /usr/sbin/dracut -v
Will not override existing initramfs (/boot/initramfs-3.10.0-123.el7.x86_64.img) without --force

Message from syslogd@filemon1 at Nov 14 13:34:58 ...
 dracut:Will not override existing initramfs (/boot/initramfs-3.10.0-123.el7.x86_64.img) without --force
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
dracut -v -f // should display what is packaged in initrd
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
[r[root@filemon1 ~]# dracut --add "mdraid dmraid" --add-drivers "linear raid0 raid1" --add-device /dev/md1 --mdadmconf --force -v /boot/initramfs-$(uname -r).img
$(uname -r)
Executing: /usr/sbin/dracut --add "mdraid dmraid" --add-drivers "linear raid0 raid1" --add-device /dev/md1 --mdadmconf --force -v /boot/initramfs-3.10.0-123.el7
.x86_64.img 3.10.0-123.el7.x86_64
*** Including module: bash ***
*** Including module: i18n ***
*** Including module: network ***
*** Including module: ifcfg ***
*** Including module: drm ***
*** Including module: plymouth ***
*** Including module: dm ***
Skipping udev rule: 64-device-mapper.rules
Skipping udev rule: 60-persistent-storage-dm.rules
Skipping udev rule: 55-dm.rules
*** Including module: dmraid ***
*** Including module: kernel-modules ***
*** Including module: mdraid ***
Skipping udev rule: 63-md-raid-arrays.rules
Skipping udev rule: 64-md-raid-assembly.rules
*** Including module: resume ***
*** Including module: rootfs-block ***
*** Including module: terminfo ***
*** Including module: udev-rules ***
Skipping udev rule: 91-permissions.rules
*** Including module: biosdevname ***
*** Including module: systemd ***
*** Including module: usrmount ***
*** Including module: base ***
*** Including module: fs-lib ***
*** Including module: shutdown ***
*** Including modules done ***
*** Installing kernel module dependencies and firmware ***
*** Installing kernel module dependencies and firmware done ***
*** Resolving executable dependencies ***
*** Resolving executable dependencies done***
*** Hardlinking files ***
*** Hardlinking files done ***
*** Stripping files ***
*** Stripping files done ***
*** Generating early-microcode cpio image ***
*** Creating image file ***
*** Creating image file done ***


[root@filemon1 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-123.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-123.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-7915074ded504cad8c1a20fe385f98a0
Found initrd image: /boot/initramfs-0-rescue-7915074ded504cad8c1a20fe385f98a0.img
Found Scientific Linux release 7.0 (Nitrogen) on /dev/md1
/usr/sbin/grub2-probe: error: cannot find a GRUB drive for /dev/block/9.  Check your device.map.
/usr/sbin/grub2-probe: error: cannot find a GRUB drive for /dev/block/9.  Check your device.map.
done

[root@filemon1 ~]# cat /boot/grub2/grub.cfg
#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub2-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
insmod load_video
insmod gzio
insmod linear
insmod raid
insmod mdraid
insmod part_msdos
insmod xfs
insmod raid0
insmod raid1
insmod mdraid09
insmod mdraid1x
set pager=1

if [ -s $prefix/grubenv ]; then
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="${saved_entry}"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}

function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

terminal_output console
if [ x$feature_timeout_style = xy ] ; then
  set timeout_style=menu
  set timeout=5
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
  set timeout=5
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/10_linux ###
menuentry 'Scientific Linux, with Linux 3.10.0-123.el7.x86_64' --class scientific --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option '
gnulinux-3.10.0-123.el7.x86_64-advanced-1d0048ef-11dc-48a7-8c03-91551b328724' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_msdos msdos
        insmod diskfilter mdraid09
        insmod xfs
        set root='mduuid/e8c6b4661c551378ea0a9c266e71ac0d'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/e8c6b4661c551378ea0a9c266e71ac0d'  0551df1f-991f-4b65-9cc9-69c7b17db57a
        else
          search --no-floppy --fs-uuid --set=root 0551df1f-991f-4b65-9cc9-69c7b17db57a
        fi
        linux16 /vmlinuz-3.10.0-123.el7.x86_64 root=UUID=1d0048ef-11dc-48a7-8c03-91551b328724 ro vconsole.keymap=us crashkernel=auto  vconsole.font=latarcyrheb-
sun16 rhgb quiet
        initrd16 /initramfs-3.10.0-123.el7.x86_64.img
}
menuentry 'Scientific Linux, with Linux 0-rescue-7915074ded504cad8c1a20fe385f98a0' --class scientific --class gnu-linux --class gnu --class os --unrestricted $m
enuentry_id_option 'gnulinux-0-rescue-7915074ded504cad8c1a20fe385f98a0-advanced-1d0048ef-11dc-48a7-8c03-91551b328724' {
        load_video
        insmod gzio
        insmod part_msdos msdos
        insmod diskfilter mdraid09
        insmod xfs
        set root='mduuid/e8c6b4661c551378ea0a9c266e71ac0d'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/e8c6b4661c551378ea0a9c266e71ac0d'  0551df1f-991f-4b65-9cc9-69c7b17db57a
        else
          search --no-floppy --fs-uuid --set=root 0551df1f-991f-4b65-9cc9-69c7b17db57a
        fi
        linux16 /vmlinuz-0-rescue-7915074ded504cad8c1a20fe385f98a0 root=UUID=1d0048ef-11dc-48a7-8c03-91551b328724 ro vconsole.keymap=us crashkernel=auto  vconso
le.font=latarcyrheb-sun16 rhgb quiet
        initrd16 /initramfs-0-rescue-7915074ded504cad8c1a20fe385f98a0.img
}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/20_ppc_terminfo ###
### END /etc/grub.d/20_ppc_terminfo ###

### BEGIN /etc/grub.d/30_os-prober ###
menuentry 'Scientific Linux release 7.0 (Nitrogen) (on /dev/md1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-simple-f0d225
55-199a-4278-8bc2-9218201da07e' {

        linux  /boot/initramfs-0-rescue-7915074ded504cad8c1a20fe385f98a0.img:root=/dev/md1
        initrd /vmlinuz-0-rescue-7915074ded504cad8c1a20fe385f98a0
}
submenu 'Advanced options for Scientific Linux release 7.0 (Nitrogen) (on /dev/md1)' $menuentry_id_option 'osprober-gnulinux-advanced-f0d22555-199a-4278-8bc2-92
18201da07e' {
        menuentry '0 (on /dev/md1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux---f0d22555-199a-4278-8bc2-9218201da07e' {

                linux  /boot/initramfs-0-rescue-7915074ded504cad8c1a20fe385f98a0.img:root=/dev/md1
                initrd /vmlinuz-0-rescue-7915074ded504cad8c1a20fe385f98a0
        }
        menuentry '0 (on /dev/md1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux---f0d22555-199a-4278-8bc2-9218201da07e' {

                linux  /boot/initramfs-3.10.0-123.el7.x86_64.img:root=/dev/md1
                initrd /vmlinuz-3.10.0-123.el7.x86_64
        }
}

### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg;
fi
### END /etc/grub.d/41_custom ###
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
And I tried this:

[root@filemon1 ~]# dracut --add "mdraid dmraid" --add-drivers "linear raid0 raid1" --mdadmconf --fstab --force -v /boot/initramfs-$(uname -r).img $(uname -r)
Executing: /usr/sbin/dracut --add "mdraid dmraid" --add-drivers "linear raid0 raid1" --mdadmconf --fstab --force -v /boot/initramfs-3.10.0-123.el7.x86_64.img 3.
10.0-123.el7.x86_64
*** Including module: bash ***
*** Including module: i18n ***
*** Including module: network ***
*** Including module: ifcfg ***
*** Including module: drm ***
*** Including module: plymouth ***
*** Including module: dm ***
Skipping udev rule: 64-device-mapper.rules
Skipping udev rule: 60-persistent-storage-dm.rules
Skipping udev rule: 55-dm.rules
*** Including module: dmraid ***
*** Including module: kernel-modules ***
*** Including module: mdraid ***
Skipping udev rule: 63-md-raid-arrays.rules
Skipping udev rule: 64-md-raid-assembly.rules
*** Including module: resume ***
*** Including module: rootfs-block ***
*** Including module: terminfo ***
*** Including module: udev-rules ***
Skipping udev rule: 91-permissions.rules
*** Including module: biosdevname ***
*** Including module: systemd ***
*** Including module: usrmount ***
*** Including module: base ***
*** Including module: fs-lib ***
*** Including module: shutdown ***
*** Including modules done ***
*** Installing kernel module dependencies and firmware ***
*** Installing kernel module dependencies and firmware done ***
*** Resolving executable dependencies ***
*** Resolving executable dependencies done***
*** Hardlinking files ***
*** Hardlinking files done ***
*** Stripping files ***
*** Stripping files done ***
*** Generating early-microcode cpio image ***
*** Creating image file ***
*** Creating image file done ***
[root@filemon1 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-123.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-123.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-7915074ded504cad8c1a20fe385f98a0
Found initrd image: /boot/initramfs-0-rescue-7915074ded504cad8c1a20fe385f98a0.img
Found Scientific Linux release 7.0 (Nitrogen) on /dev/md1
done

And I have:

grub2-probe -t abstraction -d /dev/md0
diskfilter mdraid09
[root@filemon1 ~]# grub2-probe -t abstraction -d /dev/md1
diskfilter mdraid1x
[root@filemon1 ~]# grub2-probe -t abstraction -d /dev/md2
diskfilter mdraid1x
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
But the problem persists at boot time and the device /dev/md1 for root filesystem cannot be found....
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
I would say that every time you mount one mirror member superblocks desynchronize and linux is right to deny mounting.
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
But the /boot filesystem is mounted correctly from /dev/md0
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
blkid

blkid
/dev/md1 does not exist!

/dev/md1 does not exist!
No md devices on system boot !

No md devices on system boot !
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
Emergency mode - mdadm

Emergency mode mdadm
Emergency mode - devices in initramfs

Emergency mode disk devices present
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
Why nobody can give me an advice? :-(
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
Ok... I found that the array md1 is not being assembled at boot time but I can assemble it in emergency mode and mount root filesystem from it but then, when I exit dracut, mdmonitor.service fails to load.....and I am again in emergency mode
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
I just know how to create new md disk, not how to convert existing one....
You must have local_mdadmconf=yes in /etc/dracut.conf to add md configuration to initrd. Also you need to resynchronize mirror from boot disk, otherwise members are out of sync.
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 

Author Comment

by:Lelio Michele Lattari
Comment Utility
I added email to mdadm.conf and the service starts. I assembled raid for / in emergency mode but when I exit emergency mode then I cannot login
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
Array md1 with the / filesystem is not being assembled at boot time. I can assemble it and run but then, when exit from emergency mode ,  my password for root is not accepted
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
I see no evidence mdadm.conf is packaged in initrd.
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
Sorry but You dont read what I write :-)
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
did you change dracut.conf so that it really copies mdadm.conf to initrd?
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
I did not change dracut.conf. I've added modules and drivers to to initrams manually:

dracut --mdadmconf --fstab --add="dm dmraid mdraid" --add-drivers="raid0 raid1" --force /boot/initramfs-$(uname -r).img $(uname -r) -M

I said earlier that I assembled the md array in emergency mode, that's using mdadm from initramfs.
But when I exit from emergency mode ( as said earlier ;-) ) I cannot login, althought the password is correct.

I can only boot with / mounted from sda3:

[root@filemon1 ~]# mount -t xfs
/dev/sda3 on / type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md2 on /home type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md0 on /boot type xfs (rw,relatime,attr2,inode64,noquota)
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
Now I have done some progress. I can assemble the array for root fs in emergency mode , exit and normally boot! Then the root fs is mounted from md1. But why shall I do it manually?

In emergency mode the device /dev/md1 does not exists. But I do the following: mdadm --assemble --run /dev/md1. The array is assembled and I exit from emergency  mode and continue normal boot process. Then is OK. But why shall I manually assemble the array for / ?
Last login: Sun Nov 16 13:42:10 2014
[root@filemon1 ~]# mount -t xfs
/dev/md1 on / type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md2 on /home type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md0 on /boot type xfs (rw,relatime,attr2,inode64,noquota)
[root@filemon1 ~]#
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
Once you booted with dm active you need to assemble initrd (dracut should find that md is active like in EL6), so it boots right way in future.
/ is mounted from kernel parameters, only rest is from fstab.
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
Try these:
check /etc/sysconfig/grub
check /etc/dracut.conf
run grub2-mkconfig and check if root=md
run dracut - check that dm is added
cross fingers and reboot.
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
As I wrote earlier :-)  I included dm manually in initramfs

dracut --mdadmconf --fstab --add="dm dmraid mdraid" --add-drivers="raid0 raid1" --force /boot/initramfs-$(uname -r).img $(uname -r) -M

In grub2 menuentry root IS /dev/md1
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
I know :) I read it...
When root is /dev/md* or /dev/vol/* dracut and grub-mkconfig should catch up with that.
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
All the tools that are required for assembling the array for / are present in initramfs but I have to do it manually in emergency mode at every boot. Why?
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
Normally (EL6) once lvm.conf is included and root=/dev/lvm/root dracut somehow catches up and builds right LVM-capable initrd.
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
I KNOW. I have never had problems with this using earlier version of grub or using lilo...
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
The array for the / filesystem is present in initrafs but is not being assembled and started by the system. Why?

mdadm.conf from initrams:

CREATE owner=root group=disk mode=0660 auto=yes
DEVICE /dev/sd*[a-z][1-9]
ARRAY /dev/md0 UUID=d94999a4:4d706676:ea0a9c26:6e71ac0d
ARRAY /dev/md/1  metadata=1.2 UUID=a9189950:e3ce0155:ca5edd38:927ead3c name=filemon1:1
ARRAY /dev/md/2  metadata=1.2 UUID=a444c20b:a0673616:f94081dc:f85e56f0 name=filemon1:2
MAILADDR l.lattari@euroinfissi.eu
HOMEHOST <system>
AUTO +0.90 +1.x homehost -all
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
ouch
/dev/md/0 or /dev/md0 ?
hmmm...
mdadm documentation does not know either...
you can call them lvm style like /dev/md/root etc.
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
I have tried this too... ;-) I have tried with UUID also... and the effect is the same: I must assemble the / array manually in emergency mode and then continue booting normally. The arrays for /boot (md0) and /home (md2) are assembled by the system...
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
Tomorrow I will try the rootdelay kernel parameter and will install newest mdmadm from rpm... No other ideas!
0
 

Author Comment

by:Lelio Michele Lattari
Comment Utility
After long time searching on the net I have probably found the solution to my problem:

https://www.kernel.org/pub/linux/utils/boot/dracut/dracut.html#_md_raid

==============================================
" rd.auto rd.auto=1
    enable autoassembly of special devices like cryptoLUKS, dmraid, mdraid or lvm. Default is off as of dracut version >= 024. "
================================================
0
 

Accepted Solution

by:
Lelio Michele Lattari earned 0 total points
Comment Utility
This is the answer to my question.

In order to mount the / filesystem from an md array in my specific environment I had to add the kernel command rd.auto rd.auto=1. This command causes that the existent arrays are automatically ASSEMBLED at boot time. Early I had to manually assemble the / array in emergency mode, then exit the e mode and continue booting.

I am working with Centos 7 and Scientific Linux 7.

Now, after booting I have all the filesystems mounted correctly from RAID 1 disk arrays:

[root@filemon1 ~]# mount -t xfs
/dev/md1 on / type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md2 on /home type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md0 on /boot type xfs (rw,relatime,attr2,inode64,noquota)

The arrays (degraded for the moment) look like this:

[root@filemon1 ~]# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdb1[0]
      511936 blocks [2/1] [U_]

md2 : active raid1 sdb5[0]
      911293248 blocks super 1.2 [2/1] [U_]

md1 : active raid1 sdb3[0]
      52395904 blocks super 1.2 [2/1] [U_]

unused devices: <none>

It was also necessary do add some modules and drivers for the creation of a correct initrams:

 dracut --mdadmconf --fstab --add="dm dmraid mdraid" --add-drivers="raid0 raid1" --force /boot/initramfs-$(uname -r).img $(uname -r) -M

I have also changed the /etc/default/grub file for preloading important modules:

GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT='Scientific Linux, with Linux 3.10.0-123.9.2.el7.x86_64'
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="rd.auto rd.auto=1 vconsole.keymap=us crashkernel=auto  vconsole.font=latarcyrheb-sun16 rhgb quiet"
GRUB_DISABLE_RECOVERY="true"
GRUB_PRELOAD_MODULES="part_gpt raid mdraid raid5rec raid6rec dm_nv mdraid09 mdraid1x xfs"

The menuentry that I use for booting:

menuentry 'Scientific Linux, with Linux 3.10.0-123.9.2.el7.x86_64' --class scientific --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_opti
on 'gnulinux-3.10.0-123.9.2.el7.x86_64-advanced-a0c1f14f-f88c-4a95-91f7-4894377fab2d' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_msdos
        insmod diskfilter mdraid09
        insmod xfs
        set root='mduuid/d94999a44d706676ea0a9c266e71ac0d'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/d94999a44d706676ea0a9c266e71ac0d'  fa43cdb6-5bdd-40c3-be67-693b7155dcbf
        else
          search --no-floppy --fs-uuid --set=root fa43cdb6-5bdd-40c3-be67-693b7155dcbf
        fi
        linux16 /vmlinuz-3.10.0-123.9.2.el7.x86_64 root=UUID=a0c1f14f-f88c-4a95-91f7-4894377fab2d ro rd.auto rd.auto=1 vconsole.keymap=us crashkernel=auto  vcon
sole.font=latarcyrheb-sun16 rhgb quiet
        initrd16 /initramfs-3.10.0-123.9.2.el7.x86_64.img
}

Thank You friends, anyway, for any help!
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
Thanks a million for information.
0
 

Author Closing Comment

by:Lelio Michele Lattari
Comment Utility
The reason is than HERE nobody could help me and I have found the answer researching the net! How it is possible that I pay for getting help from the EXPERTS but there is not valid help here!?
0

Featured Post

How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

Join & Write a Comment

Suggested Solutions

This document is written for Red Hat Enterprise Linux AS release 4 and ORACLE 10g.  Earlier releases can be installed using this document as well however there are some additional steps for packages to be installed see Metalink. Disclaimer: I hav…
Introduction We as admins face situation where we need to redirect websites to another. This may be required as a part of an upgrade keeping the old URL but website should be served from new URL. This document would brief you on different ways ca…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now