mysql keeps external hard disk busy

I have a strange problem when making a backup of my mysql databases. In order to do so I have a 2TB external hard disk mounted when needed under /backup0.
Before making the backup I stop mysql (with /etc/rc.d/mysqld stop) and use rsync to copy all the data to the external hard disk.
As soon as the copy is done I start mysql again (/etc/rc.d/mysld start). The whole copy process takes about 20 minutes (all databases are ~60GB).

But... when I try to umount the backup unit I get

root@slackserver:/# umount /backup0
umount: /backup0: device is busy
umount: /backup0: device is busy

And when looking who is keeping the device busy (backup0 = /dev/sdf1):

root@slackserver:/# fuser -m /dev/sdf1
/dev/sdf1:           22420c
root@slackserver:/# ps auxw | grep 22420
root      5630  0.0  0.0   2056   644 pts/1    R+   17:03   0:00 grep 22420
root     22420  0.0  0.0   2760  1376 pts/0    S    11:13   0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --sql-mode=ansi --pid-file=/var/run/mysql/

The only way to umount the backup unit is to so before restarting mysqld.

But why is mysql keeping the device busy. All other hard disks are not "locked" by mysql. Does mysql see the data on the backup drive and recognize it as mysql databases?
As you can see, the datadir is set to /var/lib//mysql and that is (on my system): /dev/mapper/vg2-sql on /var/lib/mysql type xfs (rw) That is part of a software raid 5 with lvm 2 (device /sd[bcde]1).

What is going on here?
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Jan SpringerCommented:
Have you tried running 'lsof' to see what files are opened on the backup device?
collinuxAuthor Commented:
Yes, did try that. And it is making it stranger for me as it is already. lsof didn't show any open files on the backup unit.

All what I do is stop mysqld, do a rsync (rsync -a /var/lib/mysq/ /backup0/mysql/ --delete) and when that is done start mysqld again. (It runs from a bash script started through a cronjob at 3 in the morning).

There is a lot more that goes to the backup unit with this bash script. And I changed the script so the mysql backup goes last, umount the device and start mysld.

But is puzzles me why I can't umount the unit as mysqld is started before the umount command is given.
Jan SpringerCommented:
Were you 'cd' into the mount point of the backup directory while trying to umount it?

I agree that there should be no relationship between starting mysql and umounting the device.

Can you post a sanitized copy of the script?
Newly released Acronis True Image 2019

In announcing the release of the 15th Anniversary Edition of Acronis True Image 2019, the company revealed that its artificial intelligence-based anti-ransomware technology – stopped more than 200,000 ransomware attacks on 150,000 customers last year.

collinuxAuthor Commented:
Like I said, it runs from a cronjob and there is no need to cd into the backup unit first. No offense (I'm very glad you are trying to help), but I use Linux for over 10 years and have some idea how it works.
Anyway, the script used for making the backup:

mount /dev/sdf1 /backup0
sleep 5
if [ -d "/backup" ]; then
        rsync -a /var/www/ /backup0/www/ --delete
        rsync -a /data-local/ /backup0/data-local/ --delete
        rsync -a /home/ /backup0/home/ --delete
        rsync -a /samba/ /backup0/samba/ --delete
        rsync -a /var/spool/mail/ /backup0/mail/ --delete
        /etc/rc.d/rc.mysqld stop
        rsync -a /var/lib/mysql/ /backup0/mysql/ --delete
        umount /backup0
        /etc/rc.d/rc.mysqld start
        echo "$(date)  OK: daily backup created" >> /var/log/backuplog
        echo "$(date)  Error: '/backup0' does not exist!!" >> /var/log/backuplog
        exit 1
collinuxAuthor Commented:
A little to quick with copy/paste. The line if[-d ... should be read as if[-d "/backup0"]; then.
Did copy/paste from another similar script and overlooked this line.
Steve BinkCommented:
Does using --no-D affect this issue?
collinuxAuthor Commented:
Routinet, I have been searching for this option but where do you use it and what does it do?

Can't find "--no-D" by mount or mysql.
Steve BinkCommented: is used on rsync.  See

 -D                             same as --devices --specials
 --no-OPTION             turn off an implied OPTION (e.g. --no-D)

    This option causes rsync to transfer character and block device files to the remote system to recreate these devices. This option has no effect if the receiving rsync is not run as the super-user (see also the --super and --fake-super options).
    This option causes rsync to transfer special files such as named sockets and fifos.

Just a guess, but one of these could be the source of the lock.  

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux Distributions

From novice to tech pro — start learning today.