Solved

ESXi 6 NFS Datastore APD

Posted on 2016-10-03
14
63 Views
Last Modified: 2016-10-04
We have a NFS share host on a Windows 2012 R2 server which host our LOG file and ISO images. Our Windows 2012 server was down for maintenance and now the datastore on all my host is listed as inactive and cannot unmount or anything else. Is there any other way to bring the datastore back online without a reboot which VMware support is stating is the only way to correct this. I am asking since this is impact 35 of my host.
0
Comment
Question by:compdigit44
  • 8
  • 6
14 Comments
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
We would not use Windows NFS to start with!

Once the NFS server is back online, normal NFS NAS it will automatically re-connect, as NFS polls.

NFS on Windows is available ? you have checked by connecting to it, with another server/device/workstation/client ?

can you unmount it , and re-mount it from an ESXi host ?

if you issue a refresh/re-scan, what does the log state ?
0
 
LVL 19

Author Comment

by:compdigit44
Comment Utility
I cannot unmount and already tried a rescan not luck... Vmware says my only option is a reboot which I think is  B.S answser
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
host logs ?

what is the error ?
0
 
LVL 19

Author Comment

by:compdigit44
Comment Utility
Yes it is the ESXi host logs.. When I right click on the datastore everything is grayed out and I do not have the option to unmounted ....

And yes the NFS share is online and accessible...
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
do you get any errors in the logs ? why it cannot mount ?
0
 
LVL 19

Author Comment

by:compdigit44
Comment Utility
OK I think I am on to something.. I have comfirmed that a reboot does bring the NFS share back online.

I was able to unmounts the NFS datastore then set it up again via Add Storage and it worked..

How can I script the unmounts of this datastore on all host then remount it.

HOw when I tried to manually unmounts the datastore from other host it states it cannont because the device is in use

Some VM are set to boot to specific ISO also I am not able to unmount the host log file since this is listed on each VM...

So it looks Like I will need to removed the mapping from all host and VM then try this... When 40 host and 120 Vm this is going to take a while
0
 
LVL 19

Author Comment

by:compdigit44
Comment Utility
Here is what I am seeing in the VM kernel log for the NFS share the host the ESXi log files..

2016-10-03T23:11:49.337Z cpu29:9043435 opID=91b4f7c0)WARNING: NFS: 2208: Failed to get attributes (No connection)
2016-10-03T23:11:49.337Z cpu29:9043435 opID=91b4f7c0)NFS: 2264: Failed to get object 40 28b45cbd fa14778d 2a 10000 2a 10000 34d4aa4c 320b50 c0432038c5c48b5f 0 0 0 :No connection
0
Get up to 2TB FREE CLOUD per backup license!

An exclusive Black Friday offer just for Expert Exchange audience! Buy any of our top-rated backup solutions & get up to 2TB free cloud per system! Perform local & cloud backup in the same step, and restore instantly—anytime, anywhere. Grab this deal now before it disappears!

 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
ah, well datastore's in use, is anothe issue, isos, or Heartbeat datastores, VMware HA, but you also stated that this is your LOG mount.

and that cannot be changed without a reboot!

Don't think you've got much choice, because this is your LOG datastore.

But NFS should re-connect, but then this is Windoze!
0
 
LVL 19

Author Comment

by:compdigit44
Comment Utility
If a VM has a datastore mounted it will not be able to be removed I presume. I tried change the log location on three host restarting the management services and still no go...

I feel like I am so close here
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
Restarting management agents is not going to cure the log location.

It only takes effect after a restart.

If a VM has a datatore mounted it's  busy!

and to be honest with you if you have to start kill (ing) all the services on a host which has production VMs in place, I would no longer be confident in it's production status!
0
 
LVL 19

Author Comment

by:compdigit44
Comment Utility
Ugh... I was able to address my ISO mount location but the log location is my last hurdle. I do not recall have to reboot the host to setup the remote log location so why now???
0
 
LVL 117

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE) earned 500 total points
Comment Utility
Log location only take affects after a reboot.

So if this is already set, trying to write to a location which does not exist, the host will think it's busy, and okay, you can change in advanced settings, but it will not change until a reboot.

or you kill all the services and restart them, and do you really want to kill your server with VMs running.

Migrate VMs, using vMotion and reboot host.

re-think log location in the future.... or address it if you need to restart a Windoze box! (which you'll need to do every month wit patching!)
0
 
LVL 19

Author Comment

by:compdigit44
Comment Utility
on a test host running our dev VM's I try to stop and start the syslog service and now not able to.. how would I force kill it an restart it
0
 
LVL 19

Author Comment

by:compdigit44
Comment Utility
Well I called VMware support and they did find one way to get around this issue for me without having to reboot all of my host. It was to kill the PID of the syslog service.

This whole issue have been an eye opener 1) I showed we how weak I am in Powercli since I had to connect to each host kill this process the reconnect the NFS share. 2) I have rebooted my NFS server before but it has never been down for as long as it was before so I need to research the NFS heartbeat keep alive settings..

In short I still have so much for to learn.... :o)
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

This is an issue that we can get adding / removing permissions in the vCSA 6.0. We can also have issues searching for users / groups in the AD (using your identify sources). This is how one of the ways to handle this issues and fix it.
Veeam Backup & Replication has added a new integration – Veeam Backup for Microsoft Office 365.  In this blog, we will discuss how you can benefit from Office 365 email backup with the Veeam’s new product and try to shed some light on the needs and …
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …
Teach the user how to join ESXi hosts to Active Directory domains Open vSphere Client: Join ESXi host to AD domain: Verify ESXi computer account in AD: Configure permissions for domain user in ESXi: Test domain user login to ESXi host:

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now