Solved

Centos 7, grub2, problems putting root on raid1 md array

Posted on 2014-11-12
43
3,915 Views
Last Modified: 2014-11-22
Hi! I have just installed Centos 7 in text mode and I have created md raid1 arrays for /boot /home and / - respectively /dev/md0, /dev/md2 and /dev/md1. I have copied the filesystems to the arrays and the arrays work fine...
The system boots correctly from /dev/md0 (/boot) and the /dev/md3 array for /home also works, but after booting root is still on /dev/sda3 and I cannot find the way to put root on /dev/md1. When I try to boot with root on /dev/md1 the boot process stops and journalctl says that /dev/md1 does not exists - the same happens with UUID reference to the device...

After creating the md arrays I have done the following:

mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.old
dracut --mdadmconf --force /boot/initramfs-$(uname -r).img $(uname -r)

and

grub2-mkconfig -o /boot/grub2/grub.cfg


After booting with root on /dev/sda3 we have:

[root@filemon1 ~]# cat /proc/mdstat

Personalities : [raid1]
md1 : active raid1 sdb3[0]
      52395904 blocks super 1.2 [2/1] [U_]

md2 : active raid1 sda5[2] sdb5[0]
      911293248 blocks super 1.2 [2/1] [U_]
      [====>................]  recovery = 21.8% (199387776/911293248) finish=98.7min speed=120184K/sec

md0 : active raid1 sda1[1] sdb1[0]
      511936 blocks [2/2] [UU]

unused devices: <none>

============================================================================
[root@filemon1 ~]# blkid
/dev/sda1: UUID="164b8030-9374-544c-ea0a-9c266e71ac0d" TYPE="linux_raid_member"
/dev/sda2: UUID="5283dd61-f307-4713-aefc-5ae20be3d360" TYPE="swap"
/dev/sda3: UUID="ffd854fb-2873-42e2-a24f-418ce9403c06" TYPE="xfs"
/dev/sda5: UUID="18f7f4c8-f906-02a9-2b37-af7ae9376c77" UUID_SUB="0078a81e-59f6-82d8-d005-af55366d09a0" LABEL="filemon1:2" TYPE="linux_raid_member"
/dev/sdb1: UUID="164b8030-9374-544c-ea0a-9c266e71ac0d" TYPE="linux_raid_member"
/dev/sdb2: UUID="2b1f40b5-ba77-49df-aa5c-9a0a443a1797" TYPE="swap"
/dev/sdb3: UUID="966e1308-ae15-ed8b-1efd-7893b75eee13" UUID_SUB="1e967fb7-ab10-f1bb-353c-78954b74b312" LABEL="filemon1:1" TYPE="linux_raid_member"
/dev/sdb5: UUID="18f7f4c8-f906-02a9-2b37-af7ae9376c77" UUID_SUB="4c7a102a-1180-20f8-4d0f-cf542c91a315" LABEL="filemon1:2" TYPE="linux_raid_member"
/dev/md0: UUID="d3508712-dd77-4481-a2b8-614dc7e1df35" TYPE="xfs"
/dev/md2: UUID="f2f78b42-b663-4a29-8a71-2e3f44e71b46" TYPE="xfs"
/dev/md1: UUID="9c4cbd48-9319-45e4-8492-185551d3a955" TYPE="xfs"

============================================================================
[root@filemon1 ~]# cat /etc/fstab
UUID=9c4cbd48-9319-45e4-8492-185551d3a955 /                       xfs     defaults        1 1
UUID=d3508712-dd77-4481-a2b8-614dc7e1df35 /boot                   xfs     defaults        1 2
UUID=f2f78b42-b663-4a29-8a71-2e3f44e71b46 /home                   xfs     defaults        1 2
UUID=2b1f40b5-ba77-49df-aa5c-9a0a443a1797 swap                    swap    defaults        0 0
UUID=5283dd61-f307-4713-aefc-5ae20be3d360 swap                    swap    defaults        0 0

============================================================================

[root@filemon1 ~]# mount
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=12220856k,nr_inodes=3055214,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,seclabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpuacct,cpu)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/sda3 on / type xfs (rw,relatime,seclabel,attr2,inode64,noquota)        
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=34,pgrp=1,timeout=300,minproto=5,maxproto=5,direct)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
mqueue on /dev/mqueue type mqueue (rw,relatime,seclabel)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)
/dev/md2 on /home type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
/dev/md0 on /boot type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
[root@filemon1 ~]#

==================================================================================================

[root@filemon1 ~]# cat /boot/grub2/grub.cfg
#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub2-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
set pager=1

if [ -s $prefix/grubenv ]; then
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="${saved_entry}"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}

function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

terminal_output console
if [ x$feature_timeout_style = xy ] ; then
  set timeout_style=menu
  set timeout=5
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
  set timeout=5
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/09_swraid1_setup ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/09_swraid1_setup ###

### BEGIN /etc/grub.d/10_linux ###
menuentry 'CentOS Linux, with Linux 3.10.0-123.el7.x86_64' --class centos --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux
-3.10.0-123.el7.x86_64-advanced-ffd854fb-2873-42e2-a24f-418ce9403c06' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_msdos
        insmod diskfilter mdraid09
        insmod xfs
        set root='mduuid/164b80309374544cea0a9c266e71ac0d'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/164b80309374544cea0a9c266e71ac0d'  d3508712-dd77-4481-a2b8-614dc7e1df35
        else
          search --no-floppy --fs-uuid --set=root d3508712-dd77-4481-a2b8-614dc7e1df35
        fi
        linux16 /vmlinuz-3.10.0-123.el7.x86_64 root=UUID=ffd854fb-2873-42e2-a24f-418ce9403c06 ro vconsole.keymap=us crashkernel=auto  vconsole.font=latarcyrheb-
sun16 rhgb quiet
        initrd16 /initramfs-3.10.0-123.el7.x86_64.img
}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/20_ppc_terminfo ###
### END /etc/grub.d/20_ppc_terminfo ###

### BEGIN /etc/grub.d/30_os-prober ###
menuentry 'CentOS Linux release 7.0.1406 (Core)  (on /dev/md1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-simple-9c4cbd48
-9319-45e4-8492-185551d3a955' {
        insmod part_msdos
        insmod diskfilter mdraid09
        insmod xfs
        set root='mduuid/164b80309374544cea0a9c266e71ac0d'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/164b80309374544cea0a9c266e71ac0d'  d3508712-dd77-4481-a2b8-614dc7e1df35
        else
          search --no-floppy --fs-uuid --set=root d3508712-dd77-4481-a2b8-614dc7e1df35
        fi
        linux /vmlinuz-3.10.0-123.el7.x86_64 root=/dev/md1
        initrd /initramfs-3.10.0-123.el7.x86_64.img
}
submenu 'Advanced options for CentOS Linux release 7.0.1406 (Core)  (on /dev/md1)' $menuentry_id_option 'osprober-gnulinux-advanced-9c4cbd48-9319-45e4-8492-1855
51d3a955' {
        menuentry 'CentOS Linux release 7.0.1406 (Core)  (on /dev/md1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-/vmlinu
z-3.10.0-123.el7.x86_64--9c4cbd48-9319-45e4-8492-185551d3a955' {
                insmod part_msdos
                insmod diskfilter mdraid09
                insmod xfs
                set root='mduuid/164b80309374544cea0a9c266e71ac0d'
                if [ x$feature_platform_search_hint = xy ]; then
                  search --no-floppy --fs-uuid --set=root --hint='mduuid/164b80309374544cea0a9c266e71ac0d'  d3508712-dd77-4481-a2b8-614dc7e1df35
                else
                  search --no-floppy --fs-uuid --set=root d3508712-dd77-4481-a2b8-614dc7e1df35
                fi
                linux /vmlinuz-3.10.0-123.el7.x86_64 root=/dev/md1
                initrd /initramfs-3.10.0-123.el7.x86_64.img
        }
}

### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg;
fi
### END /etc/grub.d/41_custom ###

I would like to have / filesystem on /dev/md1 after booting.

Thank You for any suggestion!
0
Comment
Question by:Lelio Michele Lattari
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 28
  • 15
43 Comments
 
LVL 62

Expert Comment

by:gheist
ID: 40438596
why dont you mount using device names instead?
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40439553
Hi! How is it possible that nobody tries to help me !? :-((
0
 
LVL 62

Expert Comment

by:gheist
ID: 40439981
Evertthing mounts consistently:
mnttab
/dev/sda3 on / type xfs (rw,relatime,seclabel,attr2,inode64,noquota)  

blkid
/dev/sda3: UUID="ffd854fb-2873-42e2-a24f-418ce9403c06" TYPE="xfs"

grub:
root=UUID=ffd854fb-2873-42e2-a24f-418ce9403c06
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:Lelio Michele Lattari
ID: 40440132
Hi guys! As I said, I would like to have / filesystem on /dev/md1 after booting :-) Currently when I boot with filesystem / mounted on md1 ( menuentry 'CentOS Linux release 7.0.1406 (Core)  (on /dev/md1)' ) the system halts and, as I also said, journalctl says that /dev/md1 does not exists! Menuentry with / mounted on sda3 is the only that works, althought the /boot filesystem on md0 mounts correctly :-)
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40440614
The same happens with Scientific Linux 7, installed in text mode, after creating md arrays. I think grub2 misses some modules at the boot time. This happens only trying to mount / filesystem from md array but boot and home can be mounted from md arrays at boot time. Pls help
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40440748
Should I understand that nobody can help me? :-(((
0
 
LVL 62

Expert Comment

by:gheist
ID: 40440772
You can rebuild initrd with dracut -f
And add md modules in /etc/dracut.conf if they are still missing.
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40440993
As said earlier, after creating the md arrays I have done the following:

mv /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.old
dracut --mdadmconf --force /boot/initramfs-$(uname -r).img $(uname -r)

and

grub2-mkconfig -o /boot/grub2/grub.cfg
0
 
LVL 62

Expert Comment

by:gheist
ID: 40441292
dracut -v | less

and check if DM files are properly included.
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40442405
gheist: dracut -v | less produces nothing.... This does not solve my problem....

Executing: /usr/sbin/dracut -v
Will not override existing initramfs (/boot/initramfs-3.10.0-123.el7.x86_64.img) without --force

Message from syslogd@filemon1 at Nov 14 13:34:58 ...
 dracut:Will not override existing initramfs (/boot/initramfs-3.10.0-123.el7.x86_64.img) without --force
0
 
LVL 62

Expert Comment

by:gheist
ID: 40442471
dracut -v -f // should display what is packaged in initrd
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40442612
[r[root@filemon1 ~]# dracut --add "mdraid dmraid" --add-drivers "linear raid0 raid1" --add-device /dev/md1 --mdadmconf --force -v /boot/initramfs-$(uname -r).img
$(uname -r)
Executing: /usr/sbin/dracut --add "mdraid dmraid" --add-drivers "linear raid0 raid1" --add-device /dev/md1 --mdadmconf --force -v /boot/initramfs-3.10.0-123.el7
.x86_64.img 3.10.0-123.el7.x86_64
*** Including module: bash ***
*** Including module: i18n ***
*** Including module: network ***
*** Including module: ifcfg ***
*** Including module: drm ***
*** Including module: plymouth ***
*** Including module: dm ***
Skipping udev rule: 64-device-mapper.rules
Skipping udev rule: 60-persistent-storage-dm.rules
Skipping udev rule: 55-dm.rules
*** Including module: dmraid ***
*** Including module: kernel-modules ***
*** Including module: mdraid ***
Skipping udev rule: 63-md-raid-arrays.rules
Skipping udev rule: 64-md-raid-assembly.rules
*** Including module: resume ***
*** Including module: rootfs-block ***
*** Including module: terminfo ***
*** Including module: udev-rules ***
Skipping udev rule: 91-permissions.rules
*** Including module: biosdevname ***
*** Including module: systemd ***
*** Including module: usrmount ***
*** Including module: base ***
*** Including module: fs-lib ***
*** Including module: shutdown ***
*** Including modules done ***
*** Installing kernel module dependencies and firmware ***
*** Installing kernel module dependencies and firmware done ***
*** Resolving executable dependencies ***
*** Resolving executable dependencies done***
*** Hardlinking files ***
*** Hardlinking files done ***
*** Stripping files ***
*** Stripping files done ***
*** Generating early-microcode cpio image ***
*** Creating image file ***
*** Creating image file done ***


[root@filemon1 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-123.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-123.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-7915074ded504cad8c1a20fe385f98a0
Found initrd image: /boot/initramfs-0-rescue-7915074ded504cad8c1a20fe385f98a0.img
Found Scientific Linux release 7.0 (Nitrogen) on /dev/md1
/usr/sbin/grub2-probe: error: cannot find a GRUB drive for /dev/block/9.  Check your device.map.
/usr/sbin/grub2-probe: error: cannot find a GRUB drive for /dev/block/9.  Check your device.map.
done

[root@filemon1 ~]# cat /boot/grub2/grub.cfg
#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub2-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
insmod load_video
insmod gzio
insmod linear
insmod raid
insmod mdraid
insmod part_msdos
insmod xfs
insmod raid0
insmod raid1
insmod mdraid09
insmod mdraid1x
set pager=1

if [ -s $prefix/grubenv ]; then
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="${saved_entry}"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}

function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

terminal_output console
if [ x$feature_timeout_style = xy ] ; then
  set timeout_style=menu
  set timeout=5
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
  set timeout=5
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/10_linux ###
menuentry 'Scientific Linux, with Linux 3.10.0-123.el7.x86_64' --class scientific --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option '
gnulinux-3.10.0-123.el7.x86_64-advanced-1d0048ef-11dc-48a7-8c03-91551b328724' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_msdos msdos
        insmod diskfilter mdraid09
        insmod xfs
        set root='mduuid/e8c6b4661c551378ea0a9c266e71ac0d'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/e8c6b4661c551378ea0a9c266e71ac0d'  0551df1f-991f-4b65-9cc9-69c7b17db57a
        else
          search --no-floppy --fs-uuid --set=root 0551df1f-991f-4b65-9cc9-69c7b17db57a
        fi
        linux16 /vmlinuz-3.10.0-123.el7.x86_64 root=UUID=1d0048ef-11dc-48a7-8c03-91551b328724 ro vconsole.keymap=us crashkernel=auto  vconsole.font=latarcyrheb-
sun16 rhgb quiet
        initrd16 /initramfs-3.10.0-123.el7.x86_64.img
}
menuentry 'Scientific Linux, with Linux 0-rescue-7915074ded504cad8c1a20fe385f98a0' --class scientific --class gnu-linux --class gnu --class os --unrestricted $m
enuentry_id_option 'gnulinux-0-rescue-7915074ded504cad8c1a20fe385f98a0-advanced-1d0048ef-11dc-48a7-8c03-91551b328724' {
        load_video
        insmod gzio
        insmod part_msdos msdos
        insmod diskfilter mdraid09
        insmod xfs
        set root='mduuid/e8c6b4661c551378ea0a9c266e71ac0d'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/e8c6b4661c551378ea0a9c266e71ac0d'  0551df1f-991f-4b65-9cc9-69c7b17db57a
        else
          search --no-floppy --fs-uuid --set=root 0551df1f-991f-4b65-9cc9-69c7b17db57a
        fi
        linux16 /vmlinuz-0-rescue-7915074ded504cad8c1a20fe385f98a0 root=UUID=1d0048ef-11dc-48a7-8c03-91551b328724 ro vconsole.keymap=us crashkernel=auto  vconso
le.font=latarcyrheb-sun16 rhgb quiet
        initrd16 /initramfs-0-rescue-7915074ded504cad8c1a20fe385f98a0.img
}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/20_ppc_terminfo ###
### END /etc/grub.d/20_ppc_terminfo ###

### BEGIN /etc/grub.d/30_os-prober ###
menuentry 'Scientific Linux release 7.0 (Nitrogen) (on /dev/md1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux-simple-f0d225
55-199a-4278-8bc2-9218201da07e' {

        linux  /boot/initramfs-0-rescue-7915074ded504cad8c1a20fe385f98a0.img:root=/dev/md1
        initrd /vmlinuz-0-rescue-7915074ded504cad8c1a20fe385f98a0
}
submenu 'Advanced options for Scientific Linux release 7.0 (Nitrogen) (on /dev/md1)' $menuentry_id_option 'osprober-gnulinux-advanced-f0d22555-199a-4278-8bc2-92
18201da07e' {
        menuentry '0 (on /dev/md1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux---f0d22555-199a-4278-8bc2-9218201da07e' {

                linux  /boot/initramfs-0-rescue-7915074ded504cad8c1a20fe385f98a0.img:root=/dev/md1
                initrd /vmlinuz-0-rescue-7915074ded504cad8c1a20fe385f98a0
        }
        menuentry '0 (on /dev/md1)' --class gnu-linux --class gnu --class os $menuentry_id_option 'osprober-gnulinux---f0d22555-199a-4278-8bc2-9218201da07e' {

                linux  /boot/initramfs-3.10.0-123.el7.x86_64.img:root=/dev/md1
                initrd /vmlinuz-3.10.0-123.el7.x86_64
        }
}

### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg;
fi
### END /etc/grub.d/41_custom ###
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40442781
And I tried this:

[root@filemon1 ~]# dracut --add "mdraid dmraid" --add-drivers "linear raid0 raid1" --mdadmconf --fstab --force -v /boot/initramfs-$(uname -r).img $(uname -r)
Executing: /usr/sbin/dracut --add "mdraid dmraid" --add-drivers "linear raid0 raid1" --mdadmconf --fstab --force -v /boot/initramfs-3.10.0-123.el7.x86_64.img 3.
10.0-123.el7.x86_64
*** Including module: bash ***
*** Including module: i18n ***
*** Including module: network ***
*** Including module: ifcfg ***
*** Including module: drm ***
*** Including module: plymouth ***
*** Including module: dm ***
Skipping udev rule: 64-device-mapper.rules
Skipping udev rule: 60-persistent-storage-dm.rules
Skipping udev rule: 55-dm.rules
*** Including module: dmraid ***
*** Including module: kernel-modules ***
*** Including module: mdraid ***
Skipping udev rule: 63-md-raid-arrays.rules
Skipping udev rule: 64-md-raid-assembly.rules
*** Including module: resume ***
*** Including module: rootfs-block ***
*** Including module: terminfo ***
*** Including module: udev-rules ***
Skipping udev rule: 91-permissions.rules
*** Including module: biosdevname ***
*** Including module: systemd ***
*** Including module: usrmount ***
*** Including module: base ***
*** Including module: fs-lib ***
*** Including module: shutdown ***
*** Including modules done ***
*** Installing kernel module dependencies and firmware ***
*** Installing kernel module dependencies and firmware done ***
*** Resolving executable dependencies ***
*** Resolving executable dependencies done***
*** Hardlinking files ***
*** Hardlinking files done ***
*** Stripping files ***
*** Stripping files done ***
*** Generating early-microcode cpio image ***
*** Creating image file ***
*** Creating image file done ***
[root@filemon1 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-123.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-123.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-7915074ded504cad8c1a20fe385f98a0
Found initrd image: /boot/initramfs-0-rescue-7915074ded504cad8c1a20fe385f98a0.img
Found Scientific Linux release 7.0 (Nitrogen) on /dev/md1
done

And I have:

grub2-probe -t abstraction -d /dev/md0
diskfilter mdraid09
[root@filemon1 ~]# grub2-probe -t abstraction -d /dev/md1
diskfilter mdraid1x
[root@filemon1 ~]# grub2-probe -t abstraction -d /dev/md2
diskfilter mdraid1x
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40442851
But the problem persists at boot time and the device /dev/md1 for root filesystem cannot be found....
0
 
LVL 62

Expert Comment

by:gheist
ID: 40443102
I would say that every time you mount one mirror member superblocks desynchronize and linux is right to deny mounting.
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40443468
But the /boot filesystem is mounted correctly from /dev/md0
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40443521
blkid

blkid
/dev/md1 does not exist!

/dev/md1 does not exist!
No md devices on system boot !

No md devices on system boot !
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40443831
Emergency mode - mdadm

Emergency mode mdadm
Emergency mode - devices in initramfs

Emergency mode disk devices present
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40443836
Why nobody can give me an advice? :-(
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40444361
Ok... I found that the array md1 is not being assembled at boot time but I can assemble it in emergency mode and mount root filesystem from it but then, when I exit dracut, mdmonitor.service fails to load.....and I am again in emergency mode
0
 
LVL 62

Expert Comment

by:gheist
ID: 40444386
I just know how to create new md disk, not how to convert existing one....
You must have local_mdadmconf=yes in /etc/dracut.conf to add md configuration to initrd. Also you need to resynchronize mirror from boot disk, otherwise members are out of sync.
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40444393
I added email to mdadm.conf and the service starts. I assembled raid for / in emergency mode but when I exit emergency mode then I cannot login
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40444459
Array md1 with the / filesystem is not being assembled at boot time. I can assemble it and run but then, when exit from emergency mode ,  my password for root is not accepted
0
 
LVL 62

Expert Comment

by:gheist
ID: 40444827
I see no evidence mdadm.conf is packaged in initrd.
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40444922
Sorry but You dont read what I write :-)
0
 
LVL 62

Expert Comment

by:gheist
ID: 40444966
did you change dracut.conf so that it really copies mdadm.conf to initrd?
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40445120
I did not change dracut.conf. I've added modules and drivers to to initrams manually:

dracut --mdadmconf --fstab --add="dm dmraid mdraid" --add-drivers="raid0 raid1" --force /boot/initramfs-$(uname -r).img $(uname -r) -M

I said earlier that I assembled the md array in emergency mode, that's using mdadm from initramfs.
But when I exit from emergency mode ( as said earlier ;-) ) I cannot login, althought the password is correct.

I can only boot with / mounted from sda3:

[root@filemon1 ~]# mount -t xfs
/dev/sda3 on / type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md2 on /home type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md0 on /boot type xfs (rw,relatime,attr2,inode64,noquota)
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40445622
Now I have done some progress. I can assemble the array for root fs in emergency mode , exit and normally boot! Then the root fs is mounted from md1. But why shall I do it manually?

In emergency mode the device /dev/md1 does not exists. But I do the following: mdadm --assemble --run /dev/md1. The array is assembled and I exit from emergency  mode and continue normal boot process. Then is OK. But why shall I manually assemble the array for / ?
Last login: Sun Nov 16 13:42:10 2014
[root@filemon1 ~]# mount -t xfs
/dev/md1 on / type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md2 on /home type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md0 on /boot type xfs (rw,relatime,attr2,inode64,noquota)
[root@filemon1 ~]#
0
 
LVL 62

Expert Comment

by:gheist
ID: 40445645
Once you booted with dm active you need to assemble initrd (dracut should find that md is active like in EL6), so it boots right way in future.
/ is mounted from kernel parameters, only rest is from fstab.
0
 
LVL 62

Expert Comment

by:gheist
ID: 40445646
Try these:
check /etc/sysconfig/grub
check /etc/dracut.conf
run grub2-mkconfig and check if root=md
run dracut - check that dm is added
cross fingers and reboot.
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40445694
As I wrote earlier :-)  I included dm manually in initramfs

dracut --mdadmconf --fstab --add="dm dmraid mdraid" --add-drivers="raid0 raid1" --force /boot/initramfs-$(uname -r).img $(uname -r) -M

In grub2 menuentry root IS /dev/md1
0
 
LVL 62

Expert Comment

by:gheist
ID: 40445737
I know :) I read it...
When root is /dev/md* or /dev/vol/* dracut and grub-mkconfig should catch up with that.
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40445779
All the tools that are required for assembling the array for / are present in initramfs but I have to do it manually in emergency mode at every boot. Why?
0
 
LVL 62

Expert Comment

by:gheist
ID: 40445847
Normally (EL6) once lvm.conf is included and root=/dev/lvm/root dracut somehow catches up and builds right LVM-capable initrd.
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40445864
I KNOW. I have never had problems with this using earlier version of grub or using lilo...
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40445991
The array for the / filesystem is present in initrafs but is not being assembled and started by the system. Why?

mdadm.conf from initrams:

CREATE owner=root group=disk mode=0660 auto=yes
DEVICE /dev/sd*[a-z][1-9]
ARRAY /dev/md0 UUID=d94999a4:4d706676:ea0a9c26:6e71ac0d
ARRAY /dev/md/1  metadata=1.2 UUID=a9189950:e3ce0155:ca5edd38:927ead3c name=filemon1:1
ARRAY /dev/md/2  metadata=1.2 UUID=a444c20b:a0673616:f94081dc:f85e56f0 name=filemon1:2
MAILADDR l.lattari@euroinfissi.eu
HOMEHOST <system>
AUTO +0.90 +1.x homehost -all
0
 
LVL 62

Expert Comment

by:gheist
ID: 40446096
ouch
/dev/md/0 or /dev/md0 ?
hmmm...
mdadm documentation does not know either...
you can call them lvm style like /dev/md/root etc.
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40446113
I have tried this too... ;-) I have tried with UUID also... and the effect is the same: I must assemble the / array manually in emergency mode and then continue booting normally. The arrays for /boot (md0) and /home (md2) are assembled by the system...
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40446154
Tomorrow I will try the rootdelay kernel parameter and will install newest mdmadm from rpm... No other ideas!
0
 

Author Comment

by:Lelio Michele Lattari
ID: 40446236
After long time searching on the net I have probably found the solution to my problem:

https://www.kernel.org/pub/linux/utils/boot/dracut/dracut.html#_md_raid

==============================================
" rd.auto rd.auto=1
    enable autoassembly of special devices like cryptoLUKS, dmraid, mdraid or lvm. Default is off as of dracut version >= 024. "
================================================
0
 

Accepted Solution

by:
Lelio Michele Lattari earned 0 total points
ID: 40446982
This is the answer to my question.

In order to mount the / filesystem from an md array in my specific environment I had to add the kernel command rd.auto rd.auto=1. This command causes that the existent arrays are automatically ASSEMBLED at boot time. Early I had to manually assemble the / array in emergency mode, then exit the e mode and continue booting.

I am working with Centos 7 and Scientific Linux 7.

Now, after booting I have all the filesystems mounted correctly from RAID 1 disk arrays:

[root@filemon1 ~]# mount -t xfs
/dev/md1 on / type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md2 on /home type xfs (rw,relatime,attr2,inode64,noquota)
/dev/md0 on /boot type xfs (rw,relatime,attr2,inode64,noquota)

The arrays (degraded for the moment) look like this:

[root@filemon1 ~]# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdb1[0]
      511936 blocks [2/1] [U_]

md2 : active raid1 sdb5[0]
      911293248 blocks super 1.2 [2/1] [U_]

md1 : active raid1 sdb3[0]
      52395904 blocks super 1.2 [2/1] [U_]

unused devices: <none>

It was also necessary do add some modules and drivers for the creation of a correct initrams:

 dracut --mdadmconf --fstab --add="dm dmraid mdraid" --add-drivers="raid0 raid1" --force /boot/initramfs-$(uname -r).img $(uname -r) -M

I have also changed the /etc/default/grub file for preloading important modules:

GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT='Scientific Linux, with Linux 3.10.0-123.9.2.el7.x86_64'
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="rd.auto rd.auto=1 vconsole.keymap=us crashkernel=auto  vconsole.font=latarcyrheb-sun16 rhgb quiet"
GRUB_DISABLE_RECOVERY="true"
GRUB_PRELOAD_MODULES="part_gpt raid mdraid raid5rec raid6rec dm_nv mdraid09 mdraid1x xfs"

The menuentry that I use for booting:

menuentry 'Scientific Linux, with Linux 3.10.0-123.9.2.el7.x86_64' --class scientific --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_opti
on 'gnulinux-3.10.0-123.9.2.el7.x86_64-advanced-a0c1f14f-f88c-4a95-91f7-4894377fab2d' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_msdos
        insmod diskfilter mdraid09
        insmod xfs
        set root='mduuid/d94999a44d706676ea0a9c266e71ac0d'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='mduuid/d94999a44d706676ea0a9c266e71ac0d'  fa43cdb6-5bdd-40c3-be67-693b7155dcbf
        else
          search --no-floppy --fs-uuid --set=root fa43cdb6-5bdd-40c3-be67-693b7155dcbf
        fi
        linux16 /vmlinuz-3.10.0-123.9.2.el7.x86_64 root=UUID=a0c1f14f-f88c-4a95-91f7-4894377fab2d ro rd.auto rd.auto=1 vconsole.keymap=us crashkernel=auto  vcon
sole.font=latarcyrheb-sun16 rhgb quiet
        initrd16 /initramfs-3.10.0-123.9.2.el7.x86_64.img
}

Thank You friends, anyway, for any help!
0
 
LVL 62

Expert Comment

by:gheist
ID: 40447009
Thanks a million for information.
0
 

Author Closing Comment

by:Lelio Michele Lattari
ID: 40459265
The reason is than HERE nobody could help me and I have found the answer researching the net! How it is possible that I pay for getting help from the EXPERTS but there is not valid help here!?
0

Featured Post

Back Up Your Microsoft Windows Server®

Back up all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

In my business, I use the LTS (Long Term Support) versions of Linux. My workstations do real work, and so I rarely have the patience to deal with silly problems caused by an upgraded kernel that had experimental software on it to begin with from a r…
Google Drive is extremely cheap offsite storage, and it's even possible to get extra storage for free for two years.  You can use the free account 15GB, and if you have an Android device..when you install Google Drive for the first time it will give…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

739 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question