Link to home
Start Free TrialLog in
Avatar of proteus-IV
proteus-IVFlag for Croatia

asked on

NFS mount error - "access denied by server"

I have installed and configured NFS on CentOS 6.5.
Here is my /etc/exports file:

/home/user/share 10.10.10.0/24(rw,sync,no_subtree_check)

I edited /etc/sysconfig/nfs and uncommented the following:

LOCKD_TCPPORT=32803
LOCKD_UDPPORT=32769
MOUNTD_PORT=892

After which I added the following iptables rules:

iptables -I INPUT -m state --state NEW -p tcp -m multiport --dport 111,892,2049,32803 -s 192.168.0.0/24 -j ACCEPT
 
iptables -I INPUT -m state --state NEW -p udp -m multiport --dport 111,892,2049,32769 -s 192.168.0.0/24 -j ACCEPT

When I issue the showmount command on the NFS client I get this:

#showmount -e 10.10.10.35
Export list for 10.10.10.35:
/home/user/share 10.10.10.0/24

When I try to mount the share from the NFS client, I get the following error:

#mount -t nfs 10.10.10.35:/home/user/share /mnt/nfs/share/
mount.nfs: access denied by server while mounting 10.10.10.35:/home/user/share

I tried setting the line in /etc/exports which reads 10.10.10.0/24 to 10.10.10.34, which is the IP of the NFS client, but to no avail.

What could be the issue here?
Avatar of Member_2_6582184
Member_2_6582184
Flag of Germany image

Hello,

from the things you supplied it is really hard to tell.
I think the firewall is configured correctly, otherwise you would not get the NFS server denial.
But I will take a wild guess and point out a common issue:
Are you trying to mount a NFS share belonging to root?
If so, this is not allowed by default.
Either change the owner on the exported directory or change the export itself to allow root mounts with no_root_squash:
# cat /etc/export
/home/user/share 10.10.10.0/24(rw,sync,no_subtree_check,no_root_squash)
# exortfs -a

Open in new window


Also check the contents of the exporting servers /etc/hosts.deny and /etc/hosts.allow files. (see man host_access for more info). For testing reasons you might want to clear them out.

Then try mounting the share again.

Does this help?
Avatar of arnold
Your exports files shares the data with 10.10.10.0/24 while your iptables rules allow access from 192.168.0.0/24

You allow attempts to NFS from 192.168.0.0/24, but only allow 10.10.10/0/24 systems NFS share access (excluding root)

The IPs/IP Range on the firewall from which you allow access need to match the Ip/IP range to which you grant NFS access.
Avatar of proteus-IV

ASKER

The owner and group of the exported directory are both set to nfsnobody.
The user was my regular user, but I found in an article to change the owner to nfsnobody.
It didn't help.
This is what the permissions on the export dir look like:

drwxr-xr-x. 2 nfsnobody nfsnobody 4096 Feb 15 06:43 share

I also already tried adding no_root_squash, but the error persists.
And I already checked /etc/hosts.deny and /etc/hosts.allow, they have no entries besides comments.

Sorry, I copy pasted those iptables rules from a website where I was following a tutorial.
My /etc/sysconfig/iptables actually states:

-A INPUT -s 10.10.10.0/24 -p udp -m state --state NEW -m multiport --dports 111,892,2049,32769 -j ACCEPT
-A INPUT -s 10.10.10.0/24 -p tcp -m state --state NEW -m multiport --dports 111,892,2049,32803 -j ACCEPT
I assume you have also tried turning off the firewall on the NSF server?
Please also post a 'chkconfig --list' from that maschine.
do you have nfsd, rpc.statd, lockd running on the system?

use
tcpdump -n src ip

See what is going on while at the same time try to capture what data is going from the system on which you want the NFS mount.

This way you can match what you are missing.

Simpler, drop the iptabels on the centos box and see if you can mount the NFS share.

run iptables -L INPUT
trying to make sure your rules are not below an explicit deny rule.
When i stopped iptables I got this message when trying to mount the export:

mount.nfs: Connection timed out

I then tried starting iptables again, but got the same "mount.nfs: Connection timed out" error.

Here is the output of "chkconfig --list":

abrt-ccpp            0:off      1:off      2:off      3:on      4:off      5:on      6:off
abrtd                0:off      1:off      2:off      3:on      4:off      5:on      6:off
acpid                0:off      1:off      2:on      3:on      4:on      5:on      6:off
atd                  0:off      1:off      2:off      3:on      4:on      5:on      6:off
auditd               0:off      1:off      2:on      3:on      4:on      5:on      6:off
autofs               0:off      1:off      2:off      3:on      4:on      5:on      6:off
blk-availability      0:off      1:on      2:on      3:on      4:on      5:on      6:off
certmonger           0:off      1:off      2:off      3:on      4:on      5:on      6:off
cgconfig             0:off      1:off      2:off      3:off      4:off      5:off      6:off
cgred                0:off      1:off      2:off      3:off      4:off      5:off      6:off
cpuspeed             0:off      1:on      2:on      3:on      4:on      5:on      6:off
crond                0:off      1:off      2:on      3:on      4:on      5:on      6:off
cups                 0:off      1:off      2:on      3:on      4:on      5:on      6:off
dovecot              0:off      1:off      2:on      3:on      4:on      5:on      6:off
haldaemon            0:off      1:off      2:off      3:on      4:on      5:on      6:off
ip6tables            0:off      1:off      2:on      3:on      4:on      5:on      6:off
iptables             0:off      1:off      2:on      3:on      4:on      5:on      6:off
irqbalance           0:off      1:off      2:off      3:on      4:on      5:on      6:off
kdump                0:off      1:off      2:off      3:on      4:on      5:on      6:off
lvm2-monitor         0:off      1:on      2:on      3:on      4:on      5:on      6:off
mdmonitor            0:off      1:off      2:on      3:on      4:on      5:on      6:off
messagebus           0:off      1:off      2:off      3:off      4:off      5:off      6:off
named                0:off      1:off      2:on      3:on      4:on      5:on      6:off
netconsole           0:off      1:off      2:off      3:off      4:off      5:off      6:off
netfs                0:off      1:off      2:off      3:on      4:on      5:on      6:off
network              0:off      1:off      2:on      3:on      4:on      5:on      6:off
nfs                  0:off      1:off      2:on      3:on      4:on      5:on      6:off
nfslock              0:off      1:off      2:off      3:on      4:on      5:on      6:off
ntpd                 0:off      1:off      2:off      3:off      4:off      5:off      6:off
ntpdate              0:off      1:off      2:off      3:off      4:off      5:off      6:off
numad                0:off      1:off      2:off      3:off      4:off      5:off      6:off
oddjobd              0:off      1:off      2:off      3:off      4:off      5:off      6:off
portreserve          0:off      1:off      2:on      3:on      4:on      5:on      6:off
postfix              0:off      1:off      2:on      3:on      4:on      5:on      6:off
psacct               0:off      1:off      2:off      3:off      4:off      5:off      6:off
quota_nld            0:off      1:off      2:off      3:off      4:off      5:off      6:off
rdisc                0:off      1:off      2:off      3:off      4:off      5:off      6:off
restorecond          0:off      1:off      2:off      3:off      4:off      5:off      6:off
rngd                 0:off      1:off      2:off      3:off      4:off      5:off      6:off
rpcbind              0:off      1:off      2:on      3:on      4:on      5:on      6:off
rpcgssd              0:off      1:off      2:off      3:on      4:on      5:on      6:off
rpcsvcgssd           0:off      1:off      2:off      3:off      4:off      5:off      6:off
rsyslog              0:off      1:off      2:on      3:on      4:on      5:on      6:off
saslauthd            0:off      1:off      2:off      3:off      4:off      5:off      6:off
slapd                0:off      1:off      2:on      3:on      4:on      5:on      6:off
smartd               0:off      1:off      2:off      3:off      4:off      5:off      6:off
squid                0:off      1:off      2:on      3:on      4:on      5:on      6:off
sshd                 0:off      1:off      2:on      3:on      4:on      5:on      6:off
sssd                 0:off      1:off      2:off      3:off      4:off      5:off      6:off
sysstat              0:off      1:on      2:on      3:on      4:on      5:on      6:off
udev-post            0:off      1:on      2:on      3:on      4:on      5:on      6:off
winbind              0:off      1:off      2:off      3:off      4:off      5:off      6:off
wpa_supplicant       0:off      1:off      2:off      3:off      4:off      5:off      6:off
ypbind               0:off      1:off      2:off      3:off      4:off      5:off      6:off

How do I check whether  nfsd, rpc.statd, lockd are running?

I ran "# tcpdump -i eth0 -n src 10.10.10.34 and port 32803" and tried mounting from the NFS client but no packets were appearing on tcpdump.

Here is the output of "iptables -L INPUT":

Chain INPUT (policy DROP)
target     prot opt source               destination        
ACCEPT     udp  --  anywhere             anywhere            state NEW udp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere            state NEW tcp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:smtp state NEW,ESTABLISHED
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:imap state NEW,ESTABLISHED
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:pop3 state NEW,ESTABLISHED
ACCEPT     tcp  --  10.10.10.0/24        vault13.wastelands.local tcp spts:1024:65535 dpt:squid state NEW,ESTABLISHED
ACCEPT     udp  --  10.10.10.0/24        anywhere            state NEW multiport dports sunrpc,892,nfs,filenet-rpc
ACCEPT     tcp  --  10.10.10.0/24        anywhere            state NEW multiport dports sunrpc,892,nfs,32803
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:ssh
From chkconfig, you are fine.
Did you make any changes in /etc/nfsmounts?

Check the services status with
service nfs status
service nfslock status
service rpcbind status

Open in new window

if the services are running. If not start them with a 'start' or 'restart' instead of status.

If you changed /etc/exports, you either need to restart the nfs service or run 'exportfs -a' as I mentioned above to apply the changes.

Keep iptables off for debugging.

After verifing all services are running, check with netstat for listening ports:
netstat -l|grep nfs

Open in new window

should look something like this:
tcp        0      0 0.0.0.0:nfs             0.0.0.0:*               LISTEN     
tcp6       0      0 [::]:nfs                [::]:*                  LISTEN     
udp        0      0 0.0.0.0:nfs             0.0.0.0:*                          
udp6       0      0 [::]:nfs                [::]:* 

Open in new window


Also try from the client machine, try telneting into one of the TCP ports, for instance:
telnet 10.10.10.35 2049
telnet 10.10.10.35 875

Open in new window

If this does not work or time out, you have a network connection issue.
I don't have the file /etc/nfsmounts, only /etc/nfsmount.conf and I didn't make any changes to it.

Here is the ouput of "service nfs status":

rpc.svcgssd is stopped
rpc.mountd (pid 1561) is running...
nfsd (pid 1576 1575 1574 1573 1572 1571 1570 1569) is running...
rpc.rquotad (pid 1557) is running...

"service nfslock status":
rpc.statd (pid  1322) is running...

and "service rpcbind status":

rpcbind (pid  1296) is running...

I ran "exportfs -a" everytime I changed something in /etc/exports.

Here is the ouput of "netstat -l|grep nfs":

tcp        0      0 *:nfs                       *:*                         LISTEN      
tcp        0      0 *:nfs                       *:*                         LISTEN      
udp        0      0 *:nfs                       *:*                                    
udp        0      0 *:nfs                       *:*            

I successfully managed to establish a telnet connection from the client to the server on port 2049, but not on port 875.
Sorry, this was my bad; I ment tcp/111; 875 is the quotas.

In brief, here is the NFS mount procedure from the client:
1. Contact server port 111, ask for mountd port
2. Contact server port 2049, ask for nfsd port

If everything is fine on the server, the nfs part from netstat should look like this:
netstat -l
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 0.0.0.0:nfs             0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:40584           0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:rquotad         0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:sunrpc          0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:mountd          0.0.0.0:*               LISTEN  

Open in new window

ps -ef | egrep '(nfs|rstat|lock)'

You need to have three processes:
nfsd (server)
rpc.statd
lockd

exportfs -a

Make sure you can mount NFS with iptables off.  Then work the iptables if there are issues after you enable them.
I can telnet to port 111 on the server.

Here is the outout of "ps -ef | egrep '(nfs|rstat|lock)'":

root        22     2  0 16:57 ?        00:00:00 [kblockd/0]
root        23     2  0 16:57 ?        00:00:00 [kblockd/1]
root      1586     2  0 16:58 ?        00:00:00 [lockd]
root      1587     2  0 16:58 ?        00:00:00 [nfsd4]
root      1588     2  0 16:58 ?        00:00:00 [nfsd4_callbacks]
root      1589     2  0 16:58 ?        00:00:00 [nfsd]
root      1590     2  0 16:58 ?        00:00:00 [nfsd]
root      1591     2  0 16:58 ?        00:00:00 [nfsd]
root      1592     2  0 16:58 ?        00:00:00 [nfsd]
root      1593     2  0 16:58 ?        00:00:00 [nfsd]
root      1594     2  0 16:58 ?        00:00:00 [nfsd]
root      1595     2  0 16:58 ?        00:00:00 [nfsd]
root      1596     2  0 16:58 ?        00:00:00 [nfsd]
root      2170  1814  0 17:20 pts/1    00:00:00 egrep (nfs|rstat|lock)

Seems that tpc.statd is missing.
No, most likely not. The process may as well been called rpc.statd:
$ ps -ef | egrep '(nfs|rpc|lock)'
root       128   121  0  2013 ?        00:00:13 [kblockd/0]
root       129   121  0  2013 ?        00:01:47 [kblockd/1]
root       130   121  0  2013 ?        00:00:24 [kblockd/2]
root       131   121  0  2013 ?        00:00:45 [kblockd/3]
root      3131     1  0 17:35 ?        00:00:00 rpc.rquotad
root      3155   121  0 17:35 ?        00:00:00 [nfsd4]
root      3156     1  0 17:35 ?        00:00:00 [lockd]
root      3157     1  0 17:35 ?        00:00:00 [nfsd]
root      3158     1  0 17:35 ?        00:00:00 [nfsd]
root      3159     1  0 17:35 ?        00:00:00 [nfsd]
root      3160     1  0 17:35 ?        00:00:00 [nfsd]
root      3161     1  0 17:35 ?        00:00:00 [nfsd]
root      3162     1  0 17:35 ?        00:00:00 [nfsd]
root      3163     1  0 17:35 ?        00:00:00 [nfsd]
root      3164     1  0 17:35 ?        00:00:00 [nfsd]
root      3167     1  0 17:35 ?        00:00:00 rpc.mountd
rpc       9269     1  0  2013 ?        00:00:00 portmap
root      9301   121  0  2013 ?        00:00:00 [rpciod/0]
root      9302   121  0  2013 ?        00:00:00 [rpciod/1]
root      9303   121  0  2013 ?        00:00:00 [rpciod/2]
root      9304   121  0  2013 ?        00:00:00 [rpciod/3]
rpcuser   9314     1  0  2013 ?        00:00:00 rpc.statd
root      9444     1  0  2013 ?        00:00:00 rpc.idmapd

Open in new window


What about the telnet test?
I am asking because NFS is really easy to configure on the server side. There may as well be something on your client blocking the connection.

Is it possible to try the mount from another client, even the server should be able to mount it's own share.
I tried mounting from the server, this is what I get:

# mount -t nfs 10.0.0.2:/home/user/share /mnt/nfs/share/

mount.nfs: access denied by server while mounting 10.0.0.2:/home/user/share
I thought the server was Ip .35; does it has multiple ip's? This can sometimes cause problems if they are in the same subnet.
Did you already run a verbose mount, eg.

mount -v -t nfs (...)

Please post the result.
ASKER CERTIFIED SOLUTION
Avatar of arnold
arnold
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
running into access issues there.
Arnold is making an excellent point: Is selinux interfering? Find out if it is enabled with 'getenforce' and set it to permissive with 'setenforce 0'
I changed the IP of the server because i connected it to a DMZ port on my router.
Firewal is off on the router while I am still testing.

I also tried removing and installing nfs-utils nfs-utils-libs again and now ehen I try to start
the nfs service I get the following error message:

# service nfs start
FATAL: Module nfsd not found.
FATAL: Error running install command for nfsd
Starting NFS services:                                     [  OK  ]
Starting NFS quotas:                                       [  OK  ]
Starting NFS mountd:                                       [  OK  ]
Starting NFS daemon: rpc.nfsd: Unable to access /proc/fs/nfsd errno 2 (No such file or directory).
Please try, as root, 'mount -t nfsd nfsd /proc/fs/nfsd' and then restart rpc.nfsd to correct the problem
                                                           [FAILED]
selinux is Disabled.
selinux is Disabled.
Was it off all the time?
Could it be there was an installation error all the time?
If you can try to reinstall the missing nfsd and with yum reinstall.

Maybe even setup a test VM and redo it entirely.
This is a good basic tutorial for centos/RHEL:
https://www.digitalocean.com/community/articles/how-to-set-up-an-nfs-mount-on-centos-6
I created the directories /export/share and ran "chmod 777" on share.
Now I can mount the folder and it is syncing without issues.

Thank you for your help.