NFS client issues connecting to and unmounting NFS server

Hello,
Could you use lsof command to see if kickstart is in use ?

# lsof | grep kickstart
lsof       1104      root  cwd       DIR              253,1     4096    6422601 /opt/kickstart/mnt/scripts
grep       1105      root  cwd       DIR              253,1     4096    6422601 /opt/kickstart/mnt/scripts
lsof       1106      root  cwd       DIR              253,1     4096    6422601 /opt/kickstart/mnt/scripts
ntpd       1981       ntp   19u     IPv4              12671      0t0        UDP kickstart:ntp
ntpd       1981       ntp   20u     IPv4             886021      0t0        UDP kickstart13:ntp
ntpd       1981       ntp   23u     IPv4             886051      0t0        UDP kickstart16:ntp
sshd       4796      root    3r     IPv4             867621      0t0        TCP kickstart:ssh->10.10.14.22:62061 (ESTABLISHED)
bash       4798      root  cwd       DIR              253,1     4096    6422601 /opt/kickstart/mnt/scripts
sshd      18793      root    3r     IPv4             753509      0t0        TCP kickstart:ssh->10.10.14.9:51095 (ESTABLISHED)
sshd      18795   brendan    3u     IPv4             753509      0t0        TCP kickstart:ssh->10.10.14.9:51095 (ESTABLISHED)
bash      18796   brendan  cwd       DIR              253,1     4096    6423266 /opt/kickstart/mnt/5.7

# showmount -a
All mount points on kickstart:
10.10.12.16:/opt/kickstart/CENTOS7.0-1406-x86_64-ES
10.10.13.131:/opt/kickstart/RHEL5.8-x86_64-ES/Server
10.10.13.82:/opt/kickstart

I also ran scripts on two other servers (10.10.12.222 & 230) and they are still showing mounted client side but not shown here on the server.

could you logout/kill bash sessions inside /opt/kickstart maybe ?

And would umount force make a difference ?
umount -f /opt/kickstart

Or a lazy umount
umount -l

# umount -f /opt/kickstart
umount2: Device or resource busy
umount.nfs: /opt/kickstart: device is busy
umount2: Device or resource busy
umount.nfs: /opt/kickstart: device is busy

and -l doesn't exist.

umounting is less an issue than the freezing as it appears that connection falls of the planet and then cannot be unmounted on client with reboot.

that should read without a reboot.

Could you try to mount your NFS datastore with -o nobarrier option ?

Are you using hard mount or soft mount ?
When using hard mount if network is unavailable, application freezing is the expected behaviour but it should resume when the NFS server comes back. Hard mount is strongly suggested.

If mounts are not important like read only, you may use soft mount and see if it makes a difference.

-o nobarrier returns:
mount.nfs: an incorrect mount option was specified

This nobarrier flag is to use on the (supposed ext4) filesystem of your NFS datastore : the filesystem where you store files on the server.

/opt is a partition and /opt/kickstart is the NFS share.

from fstab:
/dev/mapper/optvg-lv_opt /opt                    ext4    defaults        1 2

How can it be set for /opt/kickstart?

You woud have to use it for all /opt
If you do have other things running under opt better mount directly your NFS under /opt/kickstart
or create another lv to perform the test.

I think the soft option may do the trick.  I tried one execution there and it succeeded without issue.  These are temporary mounts but many so I'll try a few dozen and see how it works.

next one has the same issue:

10.10.12.254:/opt/kickstart on /opt/kickstart type nfs (rw,nolock,udp,soft,vers=4,addr=10.10.12.254,clientaddr=10.10.12.222)

# umount /opt/kickstart
umount.nfs: /opt/kickstart: device is busy
umount.nfs: /opt/kickstart: device is busy

be sure not to allow writes on your NFS if using soft mount.
That could lead to corruption.

what OS/distrib are the client using ? Cause even my Fedora 8 has umount -l flag :

[root@ ~]# umount -V
umount (util-linux-ng 2.13.1)

[root@ ~]# cat /etc/fedora-release
Fedora release 8 (Werewolf)

[root@ ~]# man umount | grep "\-l"
       -l     Lazy unmount. Detach the filesystem from the filesystem hierarchy now, and cleanup all references to the filesystem as soon as it is not busy anymore.  (Requires kernel 2.4.11or later.)

added ro as option in addition to the soft option and had a couple more successful executions.  I'll see how it goes next week and get back.

# umount -V
umount (util-linux-ng 2.17.2)

I have many client versions but mostly versions of RHEL/CentOS.  The only thing that has changed is the server OS.

On my Centos 5 the -l flag exists :
[root@centos5 ~]# man umount | grep "\-l"
       -l     Lazy unmount. Detach the filesystem from the filesystem ...

I tried nobarrier option for /opt with no success.

At some point you got it working with the soft option, right ?
What is the difference between the 2 systems ?
Did you try a few dozen as you wished to ?

I tried a few but it failed quickly.  The only difference between old and new is the kickstart server OS.  I'm looking at NFS versions now.

Do you have the possibility to test upgrading the server to 6.6 and get latest updates ?

I can but I'd prefer not to, if possible.  May go to 7 if I upgrade.

Not possible only for a test like using a VM ?
You would not notice so much changes using 6.6 than moving to 7

NFS v4 is Inspired by the AFS protocol  and marks a full break with previous versions.
The entire protocol has been redesigned and the code rewritten.
Using v3 instead of v4 is kinda use a different program but as far as you get it work this way its ok i guess :)

I presume it is fully compatible with versions on all older OSes?

Yes better compatibility with version 3.
I also had to use version 3 on Raspbian to mount ReadyNAS NV+ on my Raspberry Pi.

Worked around the issue.