Avatar of Lelio Michele Lattari
Lelio Michele Lattari
 asked on

Centos 7, grub2, problems putting root on raid1 md array

Hi! I have just installed Centos 7 in text mode and I have created md raid1 arrays for /boot /home and / - respectively /dev/md0, /dev/md2 and /dev/md1. I have copied the filesystems to the arrays and the arrays work fine...
The system boots correctly from /dev/md0 (/boot) and the /dev/md3 array for /home also works, but after booting root is still on /dev/sda3 and I cannot find the way to put root on /dev/md1. When I try to boot with root on /dev/md1 the boot process stops and journalctl says that /dev/md1 does not exists - the same happens with UUID reference to the device...

After creating the md arrays I have done the following:

mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.old
dracut --mdadmconf --force /boot/initramfs-$(uname -r).img $(uname -r)

and

grub2-mkconfig -o /boot/grub2/grub.cfg


After booting with root on /dev/sda3 we have:

[root@filemon1 ~]# cat /proc/mdstat

Personalities : [raid1]
md1 : active raid1 sdb3[0]
      52395904 blocks super 1.2 [2/1] [U_]

md2 : active raid1 sda5[2] sdb5[0]
      911293248 blocks super 1.2 [2/1] [U_]
      [====>................]  recovery = 21.8% (199387776/911293248) finish=98.7min speed=120184K/sec

md0 : active raid1 sda1[1] sdb1[0]
      511936 blocks [2/2] [UU]

unused devices: <none>

============================================================================
[root@filemon1 ~]# blkid
/dev/sda1: UUID="164b8030-9374-544c-ea0a-9c266e71ac0d" TYPE="linux_raid_member"
/dev/sda2: UUID="5283dd61-f307-4713-aefc-5ae20be3d360" TYPE="swap"
/dev/sda3: UUID="ffd854fb-2873-42e2-a24f-418ce9403c06" TYPE="xfs"
/dev/sda5: UUID="18f7f4c8-f906-02a9-2b37-af7ae9376c77" UUID_SUB="0078a81e-59f6-82d8-d005-af55366d09a0" LABEL="filemon1:2" TYPE="linux_raid_member"
/dev/sdb1: UUID="164b8030-9374-544c-ea0a-9c266e71ac0d" TYPE="linux_raid_member"
/dev/sdb2: UUID="2b1f40b5-ba77-49df-aa5c-9a0a443a1797" TYPE="swap"
/dev/sdb3: UUID="966e1308-ae15-ed8b-1efd-7893b75eee13" UUID_SUB="1e967fb7-ab10-f1bb-353c-78954b74b312" LABEL="filemon1:1" TYPE="linux_raid_member"
/dev/sdb5: UUID="18f7f4c8-f906-02a9-2b37-af7ae9376c77" UUID_SUB="4c7a102a-1180-20f8-4d0f-cf542c91a315" LABEL="filemon1:2" TYPE="linux_raid_member"
/dev/md0: UUID="d3508712-dd77-4481-a2b8-614dc7e1df35" TYPE="xfs"
/dev/md2: UUID="f2f78b42-b663-4a29-8a71-2e3f44e71b46" TYPE="xfs"
/dev/md1: UUID="9c4cbd48-9319-45e4-8492-185551d3a955" TYPE="xfs"

============================================================================
[root@filemon1 ~]# cat /etc/fstab
UUID=9c4cbd48-9319-45e4-8492-185551d3a955 /                       xfs     defaults        1 1
UUID=d3508712-dd77-4481-a2b8-614dc7e1df35 /boot                   xfs     defaults        1 2
UUID=f2f78b42-b663-4a29-8a71-2e3f44e71b46 /home                   xfs     defaults        1 2
UUID=2b1f40b5-ba77-49df-aa5c-9a0a443a1797 swap                    swap    defaults        0 0
UUID=5283dd61-f307-4713-aefc-5ae20be3d360 swap                    swap    defaults        0 0

============================================================================

[root@filemon1 ~]# mount
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=12220856k,nr_inodes=3055214,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,seclabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/sda3 on / type xfs (rw,relatime,seclabel,attr2,inode64,noquota)        
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=34,pgrp=1,timeout=300,minproto=5,maxproto=5,direct)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
mqueue on /dev/mqueue type mqueue (rw,relatime,seclabel)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)
/dev/md2 on /home type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
/dev/md0 on /boot type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
[root@filemon1 ~]#

==================================================================================================

[root@filemon1 ~]# cat /boot/grub2/grub.cfg
#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub2-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
set pager=1

if [ -s $prefix/grubenv ]; then
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="${saved_entry}"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}

function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

terminal_output console
if [ x$feature_timeout_style = xy ] ; then
  set timeout_style=menu
  set timeout=5
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
  set timeout=5
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/09_swraid1_setup ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/09_swraid1_setup ###

### BEGIN /etc/grub.d/10_linux ###
menuentry 'CentOS Linux, with Linux 3.10.0-123.el7.x86_64' --class centos --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux
-3.10.0-123.el7.x86_64-advanced-ffd854fb-2873-42e2-a24f-418ce9403c06' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_msdos
        insmod diskfilter mdraid09
        insmod xfs
        set root='mduuid/164b80309374544cea0a9c266e71ac0d'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/164b80309374544cea0a9c266e71ac0d'  d3508712-dd77-4481-a2b8-614dc7e1df35
        else
          search --no-floppy --fs-uuid --set=root d3508712-dd77-4481-a2b8-614dc7e1df35
        fi
        linux16 /vmlinuz-3.10.0-123.el7.x86_64 root=UUID=ffd854fb-2873-42e2-a24f-418ce9403c06 ro vconsole.keymap=us crashkernel=auto  vconsole.font=latarcyrheb-
sun16 rhgb quiet
        initrd16 /initramfs-3.10.0-123.el7.x86_64.img
}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/20_ppc_terminfo ###
### END /etc/grub.d/20_ppc_terminfo ###

### BEGIN /etc/grub.d/30_os-prober ###
menuentry 'CentOS Linux release 7.0.1406 (Core)  (on /dev/md1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-simple-9c4cbd48
-9319-45e4-8492-185551d3a955' {
        insmod part_msdos
        insmod diskfilter mdraid09
        insmod xfs
        set root='mduuid/164b80309374544cea0a9c266e71ac0d'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/164b80309374544cea0a9c266e71ac0d'  d3508712-dd77-4481-a2b8-614dc7e1df35
        else
          search --no-floppy --fs-uuid --set=root d3508712-dd77-4481-a2b8-614dc7e1df35
        fi
        linux /vmlinuz-3.10.0-123.el7.x86_64 root=/dev/md1
        initrd /initramfs-3.10.0-123.el7.x86_64.img
}
submenu 'Advanced options for CentOS Linux release 7.0.1406 (Core)  (on /dev/md1)' $menuentry_id_option 'osprober-gnulinux-advanced-9c4cbd48-9319-45e4-8492-1855
51d3a955' {
        menuentry 'CentOS Linux release 7.0.1406 (Core)  (on /dev/md1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-/vmlinu
z-3.10.0-123.el7.x86_64--9c4cbd48-9319-45e4-8492-185551d3a955' {
                insmod part_msdos
                insmod diskfilter mdraid09
                insmod xfs
                set root='mduuid/164b80309374544cea0a9c266e71ac0d'
                if [ x$feature_platform_search_hint = xy ]; then
                  search --no-floppy --fs-uuid --set=root --hint='mduuid/164b80309374544cea0a9c266e71ac0d'  d3508712-dd77-4481-a2b8-614dc7e1df35
                else
                  search --no-floppy --fs-uuid --set=root d3508712-dd77-4481-a2b8-614dc7e1df35
                fi
                linux /vmlinuz-3.10.0-123.el7.x86_64 root=/dev/md1
                initrd /initramfs-3.10.0-123.el7.x86_64.img
        }
}

### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg;
fi
### END /etc/grub.d/41_custom ###

I would like to have / filesystem on /dev/md1 after booting.

Thank You for any suggestion!
Linux DistributionsLinux

Avatar of undefined
Last Comment
Lelio Michele Lattari

8/22/2022 - Mon
gheist

why dont you mount using device names instead?
Lelio Michele Lattari

ASKER
Hi! How is it possible that nobody tries to help me !? :-((
gheist

Evertthing mounts consistently:
mnttab
/dev/sda3 on / type xfs (rw,relatime,seclabel,attr2,inode64,noquota)  

blkid
/dev/sda3: UUID="ffd854fb-2873-42e2-a24f-418ce9403c06" TYPE="xfs"

grub:
root=UUID=ffd854fb-2873-42e2-a24f-418ce9403c06
All of life is about relationships, and EE has made a viirtual community a real community. It lifts everyone's boat
William Peck
Lelio Michele Lattari

ASKER
Hi guys! As I said, I would like to have / filesystem on /dev/md1 after booting :-) Currently when I boot with filesystem / mounted on md1 ( menuentry 'CentOS Linux release 7.0.1406 (Core)  (on /dev/md1)' ) the system halts and, as I also said, journalctl says that /dev/md1 does not exists! Menuentry with / mounted on sda3 is the only that works, althought the /boot filesystem on md0 mounts correctly :-)
Lelio Michele Lattari

ASKER
The same happens with Scientific Linux 7, installed in text mode, after creating md arrays. I think grub2 misses some modules at the boot time. This happens only trying to mount / filesystem from md array but boot and home can be mounted from md arrays at boot time. Pls help
Lelio Michele Lattari

ASKER
Should I understand that nobody can help me? :-(((
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
gheist

You can rebuild initrd with dracut -f
And add md modules in /etc/dracut.conf if they are still missing.
Lelio Michele Lattari

ASKER
As said earlier, after creating the md arrays I have done the following:

mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.old
dracut --mdadmconf --force /boot/initramfs-$(uname -r).img $(uname -r)

and

grub2-mkconfig -o /boot/grub2/grub.cfg
gheist

dracut -v | less

and check if DM files are properly included.
Experts Exchange has (a) saved my job multiple times, (b) saved me hours, days, and even weeks of work, and often (c) makes me look like a superhero! This place is MAGIC!
Walt Forbes
Lelio Michele Lattari

ASKER
gheist: dracut -v | less produces nothing.... This does not solve my problem....

Executing: /usr/sbin/dracut -v
Will not override existing initramfs (/boot/initramfs-3.10.0-123.el7.x86_64.img) without --force

Message from syslogd@filemon1 at Nov 14 13:34:58 ...
 dracut:Will not override existing initramfs (/boot/initramfs-3.10.0-123.el7.x86_64.img) without --force
gheist

dracut -v -f // should display what is packaged in initrd
Lelio Michele Lattari

ASKER
[r[root@filemon1 ~]# dracut --add "mdraid dmraid" --add-drivers "linear raid0 raid1" --add-device /dev/md1 --mdadmconf --force -v /boot/initramfs-$(uname -r).img
$(uname -r)
Executing: /usr/sbin/dracut --add "mdraid dmraid" --add-drivers "linear raid0 raid1" --add-device /dev/md1 --mdadmconf --force -v /boot/initramfs-3.10.0-123.el7
.x86_64.img 3.10.0-123.el7.x86_64
*** Including module: bash ***
*** Including module: i18n ***
*** Including module: network ***
*** Including module: ifcfg ***
*** Including module: drm ***
*** Including module: plymouth ***
*** Including module: dm ***
Skipping udev rule: 64-device-mapper.rules
Skipping udev rule: 60-persistent-storage-dm.rules
Skipping udev rule: 55-dm.rules
*** Including module: dmraid ***
*** Including module: kernel-modules ***
*** Including module: mdraid ***
Skipping udev rule: 63-md-raid-arrays.rules
Skipping udev rule: 64-md-raid-assembly.rules
*** Including module: resume ***
*** Including module: rootfs-block ***
*** Including module: terminfo ***
*** Including module: udev-rules ***
Skipping udev rule: 91-permissions.rules
*** Including module: biosdevname ***
*** Including module: systemd ***
*** Including module: usrmount ***
*** Including module: base ***
*** Including module: fs-lib ***
*** Including module: shutdown ***
*** Including modules done ***
*** Installing kernel module dependencies and firmware ***
*** Installing kernel module dependencies and firmware done ***
*** Resolving executable dependencies ***
*** Resolving executable dependencies done***
*** Hardlinking files ***
*** Hardlinking files done ***
*** Stripping files ***
*** Stripping files done ***
*** Generating early-microcode cpio image ***
*** Creating image file ***
*** Creating image file done ***


[root@filemon1 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-123.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-123.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-7915074ded504cad8c1a20fe385f98a0
Found initrd image: /boot/initramfs-0-rescue-7915074ded504cad8c1a20fe385f98a0.img
Found Scientific Linux release 7.0 (Nitrogen) on /dev/md1
/usr/sbin/grub2-probe: error: cannot find a GRUB drive for /dev/block/9.  Check your device.map.
/usr/sbin/grub2-probe: error: cannot find a GRUB drive for /dev/block/9.  Check your device.map.
done

[root@filemon1 ~]# cat /boot/grub2/grub.cfg
#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub2-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
insmod load_video
insmod gzio
insmod linear
insmod raid
insmod mdraid
insmod part_msdos
insmod xfs
insmod raid0
insmod raid1
insmod mdraid09
insmod mdraid1x
set pager=1

if [ -s $prefix/grubenv ]; then
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="${saved_entry}"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}

function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

terminal_output console
if [ x$feature_timeout_style = xy ] ; then
  set timeout_style=menu
  set timeout=5
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
  set timeout=5
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/10_linux ###
menuentry 'Scientific Linux, with Linux 3.10.0-123.el7.x86_64' --class scientific --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option '
gnulinux-3.10.0-123.el7.x86_64-advanced-1d0048ef-11dc-48a7-8c03-91551b328724' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_msdos msdos
        insmod diskfilter mdraid09
        insmod xfs
        set root='mduuid/e8c6b4661c551378ea0a9c266e71ac0d'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/e8c6b4661c551378ea0a9c266e71ac0d'  0551df1f-991f-4b65-9cc9-69c7b17db57a
        else
          search --no-floppy --fs-uuid --set=root 0551df1f-991f-4b65-9cc9-69c7b17db57a
        fi
        linux16 /vmlinuz-3.10.0-123.el7.x86_64 root=UUID=1d0048ef-11dc-48a7-8c03-91551b328724 ro vconsole.keymap=us crashkernel=auto  vconsole.font=latarcyrheb-
sun16 rhgb quiet
        initrd16 /initramfs-3.10.0-123.el7.x86_64.img
}
menuentry 'Scientific Linux, with Linux 0-rescue-7915074ded504cad8c1a20fe385f98a0' --class scientific --class gnu-linux --class gnu --class os --unrestricted $m
enuentry_id_option 'gnulinux-0-rescue-7915074ded504cad8c1a20fe385f98a0-advanced-1d0048ef-11dc-48a7-8c03-91551b328724' {
        load_video
        insmod gzio
        insmod part_msdos msdos
        insmod diskfilter mdraid09
        insmod xfs
        set root='mduuid/e8c6b4661c551378ea0a9c266e71ac0d'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/e8c6b4661c551378ea0a9c266e71ac0d'  0551df1f-991f-4b65-9cc9-69c7b17db57a
        else
          search --no-floppy --fs-uuid --set=root 0551df1f-991f-4b65-9cc9-69c7b17db57a
        fi
        linux16 /vmlinuz-0-rescue-7915074ded504cad8c1a20fe385f98a0 root=UUID=1d0048ef-11dc-48a7-8c03-91551b328724 ro vconsole.keymap=us crashkernel=auto  vconso
le.font=latarcyrheb-sun16 rhgb quiet
        initrd16 /initramfs-0-rescue-7915074ded504cad8c1a20fe385f98a0.img
}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/20_ppc_terminfo ###
### END /etc/grub.d/20_ppc_terminfo ###

### BEGIN /etc/grub.d/30_os-prober ###
menuentry 'Scientific Linux release 7.0 (Nitrogen) (on /dev/md1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-simple-f0d225
55-199a-4278-8bc2-9218201da07e' {

        linux  /boot/initramfs-0-rescue-7915074ded504cad8c1a20fe385f98a0.img:root=/dev/md1
        initrd /vmlinuz-0-rescue-7915074ded504cad8c1a20fe385f98a0
}
submenu 'Advanced options for Scientific Linux release 7.0 (Nitrogen) (on /dev/md1)' $menuentry_id_option 'osprober-gnulinux-advanced-f0d22555-199a-4278-8bc2-92
18201da07e' {
        menuentry '0 (on /dev/md1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux---f0d22555-199a-4278-8bc2-9218201da07e' {

                linux  /boot/initramfs-0-rescue-7915074ded504cad8c1a20fe385f98a0.img:root=/dev/md1
                initrd /vmlinuz-0-rescue-7915074ded504cad8c1a20fe385f98a0
        }
        menuentry '0 (on /dev/md1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux---f0d22555-199a-4278-8bc2-9218201da07e' {

                linux  /boot/initramfs-3.10.0-123.el7.x86_64.img:root=/dev/md1
                initrd /vmlinuz-3.10.0-123.el7.x86_64
        }
}

### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg;
fi
### END /etc/grub.d/41_custom ###
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
Lelio Michele Lattari

ASKER
And I tried this:

[root@filemon1 ~]# dracut --add "mdraid dmraid" --add-drivers "linear raid0 raid1" --mdadmconf --fstab --force -v /boot/initramfs-$(uname -r).img $(uname -r)
Executing: /usr/sbin/dracut --add "mdraid dmraid" --add-drivers "linear raid0 raid1" --mdadmconf --fstab --force -v /boot/initramfs-3.10.0-123.el7.x86_64.img 3.
10.0-123.el7.x86_64
*** Including module: bash ***
*** Including module: i18n ***
*** Including module: network ***
*** Including module: ifcfg ***
*** Including module: drm ***
*** Including module: plymouth ***
*** Including module: dm ***
Skipping udev rule: 64-device-mapper.rules
Skipping udev rule: 60-persistent-storage-dm.rules
Skipping udev rule: 55-dm.rules
*** Including module: dmraid ***
*** Including module: kernel-modules ***
*** Including module: mdraid ***
Skipping udev rule: 63-md-raid-arrays.rules
Skipping udev rule: 64-md-raid-assembly.rules
*** Including module: resume ***
*** Including module: rootfs-block ***
*** Including module: terminfo ***
*** Including module: udev-rules ***
Skipping udev rule: 91-permissions.rules
*** Including module: biosdevname ***
*** Including module: systemd ***
*** Including module: usrmount ***
*** Including module: base ***
*** Including module: fs-lib ***
*** Including module: shutdown ***
*** Including modules done ***
*** Installing kernel module dependencies and firmware ***
*** Installing kernel module dependencies and firmware done ***
*** Resolving executable dependencies ***
*** Resolving executable dependencies done***
*** Hardlinking files ***
*** Hardlinking files done ***
*** Stripping files ***
*** Stripping files done ***
*** Generating early-microcode cpio image ***
*** Creating image file ***
*** Creating image file done ***
[root@filemon1 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-123.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-123.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-7915074ded504cad8c1a20fe385f98a0
Found initrd image: /boot/initramfs-0-rescue-7915074ded504cad8c1a20fe385f98a0.img
Found Scientific Linux release 7.0 (Nitrogen) on /dev/md1
done

And I have:

grub2-probe -t abstraction -d /dev/md0
diskfilter mdraid09
[root@filemon1 ~]# grub2-probe -t abstraction -d /dev/md1
diskfilter mdraid1x
[root@filemon1 ~]# grub2-probe -t abstraction -d /dev/md2
diskfilter mdraid1x
Lelio Michele Lattari

ASKER
But the problem persists at boot time and the device /dev/md1 for root filesystem cannot be found....
gheist

I would say that every time you mount one mirror member superblocks desynchronize and linux is right to deny mounting.
Experts Exchange is like having an extremely knowledgeable team sitting and waiting for your call. Couldn't do my job half as well as I do without it!
James Murphy
Lelio Michele Lattari

ASKER
But the /boot filesystem is mounted correctly from /dev/md0
Lelio Michele Lattari

ASKER
blkid

blkid
/dev/md1 does not exist!

/dev/md1 does not exist!
No md devices on system boot !

No md devices on system boot !
Lelio Michele Lattari

ASKER
Emergency mode - mdadm

Emergency mode mdadm
Emergency mode - devices in initramfs

Emergency mode disk devices present
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
Lelio Michele Lattari

ASKER
Why nobody can give me an advice? :-(
Lelio Michele Lattari

ASKER
Ok... I found that the array md1 is not being assembled at boot time but I can assemble it in emergency mode and mount root filesystem from it but then, when I exit dracut, mdmonitor.service fails to load.....and I am again in emergency mode
gheist

I just know how to create new md disk, not how to convert existing one....
You must have local_mdadmconf=yes in /etc/dracut.conf to add md configuration to initrd. Also you need to resynchronize mirror from boot disk, otherwise members are out of sync.
Your help has saved me hundreds of hours of internet surfing.
fblack61
Lelio Michele Lattari

ASKER
I added email to mdadm.conf and the service starts. I assembled raid for / in emergency mode but when I exit emergency mode then I cannot login
Lelio Michele Lattari

ASKER
Array md1 with the / filesystem is not being assembled at boot time. I can assemble it and run but then, when exit from emergency mode ,  my password for root is not accepted
gheist

I see no evidence mdadm.conf is packaged in initrd.
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
Lelio Michele Lattari

ASKER
Sorry but You dont read what I write :-)
gheist

did you change dracut.conf so that it really copies mdadm.conf to initrd?
Lelio Michele Lattari

ASKER
I did not change dracut.conf. I've added modules and drivers to to initrams manually:

dracut --mdadmconf --fstab --add="dm dmraid mdraid" --add-drivers="raid0 raid1" --force /boot/initramfs-$(uname -r).img $(uname -r) -M

I said earlier that I assembled the md array in emergency mode, that's using mdadm from initramfs.
But when I exit from emergency mode ( as said earlier ;-) ) I cannot login, althought the password is correct.

I can only boot with / mounted from sda3:

[root@filemon1 ~]# mount -t xfs
/dev/sda3 on / type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md2 on /home type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md0 on /boot type xfs (rw,relatime,attr2,inode64,noquota)
I started with Experts Exchange in 2004 and it's been a mainstay of my professional computing life since. It helped me launch a career as a programmer / Oracle data analyst
William Peck
Lelio Michele Lattari

ASKER
Now I have done some progress. I can assemble the array for root fs in emergency mode , exit and normally boot! Then the root fs is mounted from md1. But why shall I do it manually?

In emergency mode the device /dev/md1 does not exists. But I do the following: mdadm --assemble --run /dev/md1. The array is assembled and I exit from emergency  mode and continue normal boot process. Then is OK. But why shall I manually assemble the array for / ?
Last login: Sun Nov 16 13:42:10 2014
[root@filemon1 ~]# mount -t xfs
/dev/md1 on / type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md2 on /home type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md0 on /boot type xfs (rw,relatime,attr2,inode64,noquota)
[root@filemon1 ~]#
gheist

Once you booted with dm active you need to assemble initrd (dracut should find that md is active like in EL6), so it boots right way in future.
/ is mounted from kernel parameters, only rest is from fstab.
gheist

Try these:
check /etc/sysconfig/grub
check /etc/dracut.conf
run grub2-mkconfig and check if root=md
run dracut - check that dm is added
cross fingers and reboot.
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
Lelio Michele Lattari

ASKER
As I wrote earlier :-)  I included dm manually in initramfs

dracut --mdadmconf --fstab --add="dm dmraid mdraid" --add-drivers="raid0 raid1" --force /boot/initramfs-$(uname -r).img $(uname -r) -M

In grub2 menuentry root IS /dev/md1
gheist

I know :) I read it...
When root is /dev/md* or /dev/vol/* dracut and grub-mkconfig should catch up with that.
Lelio Michele Lattari

ASKER
All the tools that are required for assembling the array for / are present in initramfs but I have to do it manually in emergency mode at every boot. Why?
This is the best money I have ever spent. I cannot not tell you how many times these folks have saved my bacon. I learn so much from the contributors.
rwheeler23
gheist

Normally (EL6) once lvm.conf is included and root=/dev/lvm/root dracut somehow catches up and builds right LVM-capable initrd.
Lelio Michele Lattari

ASKER
I KNOW. I have never had problems with this using earlier version of grub or using lilo...
Lelio Michele Lattari

ASKER
The array for the / filesystem is present in initrafs but is not being assembled and started by the system. Why?

mdadm.conf from initrams:

CREATE owner=root group=disk mode=0660 auto=yes
DEVICE /dev/sd*[a-z][1-9]
ARRAY /dev/md0 UUID=d94999a4:4d706676:ea0a9c26:6e71ac0d
ARRAY /dev/md/1  metadata=1.2 UUID=a9189950:e3ce0155:ca5edd38:927ead3c name=filemon1:1
ARRAY /dev/md/2  metadata=1.2 UUID=a444c20b:a0673616:f94081dc:f85e56f0 name=filemon1:2
MAILADDR l.lattari@euroinfissi.eu
HOMEHOST <system>
AUTO +0.90 +1.x homehost -all
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
gheist

ouch
/dev/md/0 or /dev/md0 ?
hmmm...
mdadm documentation does not know either...
you can call them lvm style like /dev/md/root etc.
Lelio Michele Lattari

ASKER
I have tried this too... ;-) I have tried with UUID also... and the effect is the same: I must assemble the / array manually in emergency mode and then continue booting normally. The arrays for /boot (md0) and /home (md2) are assembled by the system...
Lelio Michele Lattari

ASKER
Tomorrow I will try the rootdelay kernel parameter and will install newest mdmadm from rpm... No other ideas!
All of life is about relationships, and EE has made a viirtual community a real community. It lifts everyone's boat
William Peck
Lelio Michele Lattari

ASKER
After long time searching on the net I have probably found the solution to my problem:

https://www.kernel.org/pub/linux/utils/boot/dracut/dracut.html#_md_raid

==============================================
" rd.auto rd.auto=1
    enable autoassembly of special devices like cryptoLUKS, dmraid, mdraid or lvm. Default is off as of dracut version >= 024. "
================================================
ASKER CERTIFIED SOLUTION
Lelio Michele Lattari

Log in or sign up to see answer
Become an EE member today7-DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform
Sign up - Free for 7 days
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
Not exactly the question you had in mind?
Sign up for an EE membership and get your own personalized solution. With an EE membership, you can ask unlimited troubleshooting, research, or opinion questions.
ask a question
gheist

Thanks a million for information.
Lelio Michele Lattari

ASKER
The reason is than HERE nobody could help me and I have found the answer researching the net! How it is possible that I pay for getting help from the EXPERTS but there is not valid help here!?
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.