Solved

mdadm script to format drive

Posted on 2010-08-31
19
664 Views
Last Modified: 2013-12-26
Hi,

I need your assistance with a script, which main purpose would be to create a new partition table on drives and add type df on 4 drives. The next steps will be to format the drives with mdadm. While formatting the script will check with the percentage and it will grep for % sign, if that is not found then it will continue to format in ext3. After that finished it will check if mount point /media/usdisk exists other wise it will create it. Then it will copy the content from /var to the mount point. Last but not least  I need to modify the /etc/fstab in order to reflect the change.

Here is how my fstab looks like, and it is a bit of a challenge:

LABEL=/var                /var                    ext3    defaults        1 2
LABEL=/var/www          /var/www                ext3    defaults        1 2
LABEL=/var/log          /var/log                ext3    defaults        1 2

The challenge here would be to change only the line containing /var , from LABEL=/var to
/dev/md0.

I look forward to hearing from you.

Michael

#!/bin/bash



SDC="/dev/sdc"

SDD="/dev/sdd"

SDE="/dev/sde"

SDF="/dev/sdf"



FDISK=$(fdisk -l|grep autodetect|awk '{print $8}'|wc|awk '{print $1}')

MAX=4



if [ $FDISK lt $MAX ]

then

    echo "The drives /dev/sd[c,d,e,f]"

    echo "are already formated"

    exit 1

    else

    echo "There are 4 drives not formated continuing with the script"

fi





echo "n

p

1





t

df

w

" |fdisk $SDC



echo "n

p

1





t

df

w

" |fdisk $SDD



echo "n

p

1





t

df

w

" |fdisk $SDE



echo "n

p

1





t

df

w

" |fdisk $SDF



if [ $FDISK -lt $MAX ]

then

    echo "There should be 4 drives formated"

    echo "There are $FDISK disks formated"

    exit 1

    else

    echo "There are 4 drives formatted continuing with the script" 

fi



mdadm -v --create /dev/md0 --level=10 --raid-devices=4 /dev/sd[c,d,e,f]1



#Monitoring mdadm once it finishes it will format /dev/md0



MON=$(mdadm -D /dev/md0|grep %|wc|awk '{print $1}')



while [ $MON -gt "0" ];do

        sleep 300

        else

        echo "/dev/md0 has been created"

        echo "proceeding to format the disk"

done



sleep 3

mke2fs -j /dev/md0



if [ -d "/media/usbdisk" ] 

  then

      echo "Directory /media/usbdisk exists"

      echo "..Continuing"

  else

     mkdir -p /media/usbdisk

fi 





mount /dev/md0 /media/usbdisk



cp -rp /var/* /media/usbdisk



# I need help with the sed command to replace LABEL=/var

# with /dev/md0

#sed -i 'var/s//dev/md0 /etc/fstab

Open in new window

0
Comment
Question by:dpoper1
  • 10
  • 9
19 Comments
 
LVL 3

Expert Comment

by:egarciat
ID: 33569140
If I understood your question well, this may help, It's a bit of uncecesarly complex, but I asume you will not execute this script very often, if you want you may optimize it, if it works for you, consider increasing the expert points.

You can call this script this way:

# ./script.sh /etc/fstab /etc/newfstab "/var" "/dev/md0" your newly fstab will be in /etc/newfstab
 
#!/bin/bash

sfstab=$1
dfstab=$2
slabel=$3
dlabel=$4

slabel="LABEL=$slabel"
dlabel="LABEL=$dlabel"

[ -f "$dfstab" ] && rm "$dfstab"

cat $sfstab | while read line; do
        label=$(awk '{print $1}'<<<$line)
        if [ "$label" == "$slabel" ]; then
                c_slabel=$(sed -e 's/\//\\\//g'<<<$slabel)
                c_dlabel=$(sed -e 's/\//\\\//g'<<<$dlabel)
                sed s/$c_slabel/$c_dlabel/g<<<$line >> "$dfstab"
        else
                echo "$line" >> "$dfstab"
        fi
done

Open in new window

0
 

Author Comment

by:dpoper1
ID: 33569294
The script copies the exact file, and it does not do anything beyond that, it would be great if if would change LABEL=/var to /dev/md0 . I would appreciate your help.
0
 
LVL 3

Expert Comment

by:egarciat
ID: 33569801
mhmm.. ok, what are your expected results for a file like this:

LABEL=/var                /var                    ext3    defaults        1 2
LABEL=/var/www          /var/www                ext3    defaults        1 2
LABEL=/var/log          /var/log                ext3    defaults        1 2

CASE 1:

LABEL=/dev/md0                /var                    ext3    defaults        1 2
LABEL=/var/www          /var/www                ext3    defaults        1 2
LABEL=/var/log          /var/log                ext3    defaults        1 2

OR

CASE 2:

/dev/md0                /var                    ext3    defaults        1 2
LABEL=/var/www          /var/www                ext3    defaults        1 2
LABEL=/var/log          /var/log                ext3    defaults        1 2

???
0
 

Author Comment

by:dpoper1
ID: 33569865
case 2
0
 
LVL 3

Expert Comment

by:egarciat
ID: 33570078
Ok, please try this:


#!/bin/bash



sfstab=$1

dfstab=$2

slabel=$3

dlabel=$4



slabel="LABEL=$slabel"

#dlabel="LABEL=$dlabel"



[ -f "$dfstab" ] && rm "$dfstab"



cat $sfstab | while read line; do

        label=$(awk '{print $1}'<<<$line)

        if [ "$label" == "$slabel" ]; then

                c_slabel=$(sed -e 's/\//\\\//g'<<<$slabel)

                c_dlabel=$(sed -e 's/\//\\\//g'<<<$dlabel)

                sed s/$c_slabel/$c_dlabel/g<<<$line >> "$dfstab"

        else

                echo "$line" >> "$dfstab"

        fi

done

Open in new window

0
 
LVL 3

Expert Comment

by:egarciat
ID: 33570116
When I execute the script, I get the following results:

root@host:/tmp# more ./fstab
LABEL=/var      /var            ext3    defaults 1 2
LABEL=/var/www  /var/www        ext3    defaults 1 2
LABEL=/var/log  /var/log        ext3    defaults 1 2

root@host:/tmp# ./script.sh ./fstab ./newfstab "/var" "/dev/md0"

root@host:/tmp# more ./newfstab
/dev/md0 /var ext3 defaults 1 2
LABEL=/var/www  /var/www        ext3    defaults 1 2
LABEL=/var/log  /var/log        ext3    defaults 1 2
0
 

Author Comment

by:dpoper1
ID: 33570350
It works, but do you think would there be a simpler way of doing it. And the other question, if not doable , how would I be able to keep the the space in between.

/dev/md0 /var ext3 defaults 1 2
LABEL=/var/www          /var/www                ext3    defaults        1 2
LABEL=/var/log             /var/log                    ext3    defaults        1 2

Thank you.
0
 
LVL 3

Assisted Solution

by:egarciat
egarciat earned 500 total points
ID: 33570550
What happens is that fstab is a very important file and you should not want to get it messy..

Something simpler let's say..

sed -i'.bak' -e 's/LABEL=/var//dev/md0' /etc/fstab

(providing necesary quoting).. could work.

However, consider the case where

LABEL=/var/part/whatever
LABEL=/var
LABEL=/var/other

These all would be changed..

Also if the arguments are separated by spaces or tabs may make a difference...

I am just considering a generic script that will work in most situations, if your template fstab will ALWAYS be like one you are using as example, sed would do the trick in a more efficient way.

0
 

Author Comment

by:dpoper1
ID: 33570751
That solves my problem, now as far as my original script, I get an error after I start the mdadm command, what I want to do is to monitor the % sign when I run the command

mdadm -D /dev/md0


10% progress
......

 which would indicate that the actual array is being built. Once the % sign would not appear then will execute another command, in my case would be to format the newly created device.

Can you shed some light on the script please?




mdadm -v --create /dev/md0 --level=10 --raid-devices=4 /dev/sd[c,d,e,f]1



#Monitoring mdadm once it finishes it will format /dev/md0



MON=$(mdadm -D /dev/md0|grep %|wc|awk '{print $1}')



while [ $MON -gt "0" ];do

sleep 300

else

echo "/dev/md0 has been created"

echo "proceeding to format the disk"

done



mke2fs -j /dev/md0

Open in new window

0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 3

Expert Comment

by:egarciat
ID: 33578379
I have never used the mdadm command before, so I do not know it's behavior.

If you execute:

mdadm -v --create /dev/md0 --level=10 --raid-devices=4 /dev/sd[c,d,e,f]1

What will happen?..  will it run in the background?.. that is why you need to monitor it's progress?..

also , the following line:

MON=$(mdadm -D /dev/md0|grep %|wc|awk '{print $1}')

will only be executed once, so it the first call to MON=$(mdadm -D /dev/md0|grep %|wc|awk '{print $1}')
results in an exit code greater than 0, your while will loop forever.

What is the output of:

mdadm -D /dev/md0

Can it be called many many many times?



0
 

Author Comment

by:dpoper1
ID: 33579262
Hi,

Please find below the output of mdadm -D /dev/md0. The command itself can be queried as many times as needed without impacting anything.

Regards,

Michael

mdadm -D /dev/md0

/dev/md0:

        Version : 00.90.03

  Creation Time : Tue Mar 21 11:14:56 2006

     Raid Level : raid10

     Array Size : 2344252416 (2235.65 GiB 2400.51 GB)

    Device Size : 390708736 (372.61 GiB 400.09 GB)

   Raid Devices : 8

  Total Devices : 8

Preferred Minor : 0

    Persistence : Superblock is persistent



    Update Time : Wed Nov 29 11:03:51 2006

          State : recovering

 Active Devices : 4

Working Devices : 4

 Failed Devices : 0

  Spare Devices : 1



     Chunk Size : 256K



 Rebuild Status : 0% complete



           UUID : d57cea81:3be21b7d:183a67d9:782c3329

         Events : 0.854924



    Number   Major   Minor   RaidDevice State

       1       8       33        1      active sync   /dev/sdc1

       2       8       49        2      active sync   /dev/sdd1

       3       8       65        3      active sync   /dev/sde1

       4       8       81        4      active sync   /dev/sdf1

Open in new window

0
 
LVL 3

Expert Comment

by:egarciat
ID: 33579366
Ok thanks,

What's the output when the build is 100% complete?

 Rebuild Status : 100% complete ??

OR

 Rebuild Status : complete ??

OR

Rebuild Status does not appear any more in the output?..

Also, perhaps an exit code would be more accurate..

execute while building:

mdadm -D /dev/md0
echo $?

when finished

mdadm -D /dev/md0
echo $?


what is the exit code iin both situations?

You did not answered t my question, what happens when you execute:

mdadm -v --create /dev/md0 --level=10 --raid-devices=4 /dev/sd[c,d,e,f]

does this program goes to background or it just stays there until finished?

because I think we can optimize this using the "wait" shell command.

0
 

Author Comment

by:dpoper1
ID: 33579461
The line itself will dissapear, that is why I originally grep for %, since once it finishes rebuilding it will dissapear the line.
0
 

Author Comment

by:dpoper1
ID: 33579601
This is how the actual output looks like once it is finished.

/dev/md0:

        Version : 0.90

  Creation Time : Wed Sep  1 09:52:47 2010

     Raid Level : raid10

     Array Size : 586067072 (558.92 GiB 600.13 GB)

  Used Dev Size : 293033536 (279.46 GiB 300.07 GB)

   Raid Devices : 4

  Total Devices : 4

Preferred Minor : 0

    Persistence : Superblock is persistent



    Update Time : Wed Sep  1 10:46:59 2010

          State : active

 Active Devices : 4

Working Devices : 4

 Failed Devices : 0

  Spare Devices : 0



         Layout : near=2

     Chunk Size : 64K



           UUID : 310fab4d:e46ef04d:9189beea:c97480a4

         Events : 0.5



    Number   Major   Minor   RaidDevice State

       0       8       33        0      active sync   /dev/sdc1

       1       8       49        1      active sync   /dev/sdd1

       2       8       65        2      active sync   /dev/sde1

       3       8       81        3      active sync   /dev/sdf1

Open in new window

0
 
LVL 3

Accepted Solution

by:
egarciat earned 500 total points
ID: 33579786
I see..

Try this and let me know what happens

---------------------------------------------------

#!/bin/bash

SLEEP_TIME=300

mdadm -D /dev/md0 | grep "Rebuild Status" &>/dev/null
BUILDING=$?

while [ $BUILDING -eq 0 ];do
sleep $SLEEP_TIME
mdadm -D /dev/md0 | grep "Rebuild Status" &>/dev/null
BUILDING=$?
done

mdadm -D /dev/md0 | grep "State : active" &>/dev/null
if [ $? -eq 0 ]; then
echo "/dev/md0 has been created"
echo "proceeding to format the disk"
fi

---------------------------------------------------

Make sure mdadm send output to stdout some programs send output to stderr, as i told you before if this works for you, consider increasing the expert points, I need 650 :)


0
 
LVL 3

Assisted Solution

by:egarciat
egarciat earned 500 total points
ID: 33579802
A little modification..

while [ $BUILDING -eq 0 ];do
echo `/bin/date` building device...
sleep $SLEEP_TIME
mdadm -D /dev/md0 | grep "Rebuild Status" &>/dev/null
BUILDING=$?
done
0
 
LVL 3

Expert Comment

by:egarciat
ID: 33579839
It is very important that you use the BASH shell, if you dont the &>/dev/null will not work
if you want to use SH, replace all &>/dev/null for >/dev/null 2>&1

0
 

Author Comment

by:dpoper1
ID: 33580486
Here is the final script. Thank you very much for your assistance


#!/bin/bash



SDC="/dev/sdc"

SDD="/dev/sdd"

SDE="/dev/sde"

SDF="/dev/sdf"



FDISK=$(fdisk -l|grep autodetect|awk '{print $8}'|wc|awk '{print $1}')

MAX=4



if [ $FDISK lt $MAX ]

then

    echo "There are $FDISK disks, you would need at least 4 drives"

    exit 1

    else

    echo "There are 4 drives formatted continuing with the script"

fi





echo "n

p

1





t

fd

w

" |fdisk $SDC



echo "n

p

1





t

fd

w

" |fdisk $SDD



echo "n

p

1





t

fd

w

" |fdisk $SDE



echo "n

p

1





t

fd

w

" |fdisk $SDF



if [ $FDISK lt $MAX ]

then

    echo "There should be 4 drives formatted"

    echo "There are $FDISK"

    exit 1

    else

    echo "There are 4 drives formatted continuing with the script" 

fi



mdadm -v --create /dev/md0 --level=10 --raid-devices=4 /dev/sd[c,d,e,f]1



#Monitoring mdadm once it finishes it will format /dev/md0



SLEEP_TIME=300



mdadm -D /dev/md0 | grep "Rebuild Status" &>/dev/null

BUILDING=$?



while [ $BUILDING -eq 0 ];do

echo `/bin/date` building /dev/md0...

sleep $SLEEP_TIME

mdadm -D /dev/md0 | grep "Rebuild Status" &>/dev/null

BUILDING=$?

done



mdadm -D /dev/md0 | grep "State : active" &>/dev/null

if [ $? -eq 0 ]; then

echo "/dev/md0 has been created"

echo "proceeding to format the disk"

fi





mke2fs -j /dev/md0



mount /dev/md0 /media/usbdisk



cp -rp /var/* /media/usbdisk



clear



echo "Please do not forget to change in /etc/fstab"

echo ""

echo ""

echo "Change from LABEL=/var to /dev/md0" 

echo ""

Open in new window

0
 

Author Closing Comment

by:dpoper1
ID: 33580500
Above and beyond what I have expected.
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

Background Still having to process all these year-end "csv" files received from all these sources (including Government entities), sometimes we have the need to examine the contents due to data error, etc... As a "Unix" shop, our only readily …
Utilizing an array to gracefully append to a list of EmailAddresses
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now