NFS STALE MOUNTS ON CLIENTS - CAN I DETECT THESE FROM THE SERVER

Hey all,

We have a Red Hat 5.1 cluster connecting to a GFS filesystem on a SAN.  We are experiencing difficulties with the cluster regularly and it tends to crash and not always failover properly.  We are attempting to deal with these machine panics, but in the meantime we are ending up finding lots of stale NFS mounts on client servers that are a pain to find - Generally only find them when the user comes to us and they ain't happy at that point.

What I was hoping to do was to have a utility or perhaps a script that can check any mounts from clients to see if they are considered stale or not after a crash occurs.  That way if I could generate a list of stale mounts then I can manually sort them out prior to the user seeing the problems.

If anyone knows of such a utility or has such a script I would be much in your debt.

Thanks in advance

HelenkinsellaAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

albuitraCommented:
showmount -a
this shows all mounts for a NFS share
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
HelenkinsellaAuthor Commented:
Thanks for the comment albuitra - Alas there can be as many as 600 servers connected to the NFS share at  any one time (and generally are) - It is true evil to have to check every server/client so what I was   hoping to do was get a list of which mounts have gone stale - It is possible that the server cannot check this so perhaps a script taking the list of servers from a mount -a command or from the /var/lib/nfs/rmtab file might do the trick -

Even better still if someone already had such a script or a utility existed to check the mounts.  Also to test 600 servers remotely may not help network traffic too much but sure you never know till you try........
0
HelenkinsellaAuthor Commented:
Here is something I found which may work although it would be as a crontab on all the clients so may take a while to implement - What would be great  is something that could be checked from the server side.

#! /bin/sh
#
# checkmounts
#
# Clear stale nfs mounts
#
# This script should be run periodically from crontab
#
# Unmount all the stale mounts
for m in `df -k 2>&1 | grep Stale | awk -F: '{print $1}' `
do
     echo Dismounting stale $m on `hostname`
     /usr/sbin/umount $m
done

# Remount those specified in fstab
/usr/sbin/mountall -F nfs

# End of script

0
Powerful Yet Easy-to-Use Network Monitoring

Identify excessive bandwidth utilization or unexpected application traffic with SolarWinds Bandwidth Analyzer Pack.

albuitraCommented:
In the NFS server you can't know the state of every client.
You would need an access from the server to every machine, like root, and that is a very bad practice in security.
So maybe you can think in migrate to AFS
http://www.openafs.org/ 
0
HelenkinsellaAuthor Commented:
Thanks albuitra - Way too big an environment to move over to AFS at this time - The real issue is the cluster crashing and not offering redundancy - If the cluster was working correctly then the mounts would not go stale in the first place - The script is a little simple as a df will not always show if the mount is stale and the umount sometimes needs the -f to force it - Not always available outside Solaris and we have multiple UX's
0
HelenkinsellaAuthor Commented:
Reckon there may not be such a utility - Thanks for taking the time to respond albuitra - I have begun looking into Nagios and trying to find a stale NFS plugin for it - Once rolled out this may be the best approach.

0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Networking

From novice to tech pro — start learning today.