Solved

ESXi partition in need of recovery

Posted on 2016-07-29
29
22 Views
Last Modified: 2016-08-03
I recently had an issue where I lost some network connectivity with my iSCSI NAS where my ESXi datastores are.  After reconnecting ESXi shows it mounts the target however the datastore does not appear.  When I go to add new storage it presents the LUN and shows there is a partition there (see attached pic) however I can't seem to figure out how to use that partition without wiping it out.  I am on ESXi 6.  Any ideas?
0
Comment
Question by:huntson
  • 16
  • 12
29 Comments
 
LVL 119
ID: 41734676
you only use the ADD Storage to ADD NEW Storage, this is not new storage!

Okay, can you get to the console, or remote via SSH ?
0
 

Author Comment

by:huntson
ID: 41734694
I just did it to verify that it can see the partition. Yes I can.
0
 
LVL 119

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE^2)
Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 41734699
okay, I need you to type the following...

esxcfg-volume -l 

Open in new window


at the bash prompt.

this should show something like below

~ # esxcfg-volume -l
VMFS3 UUID/label: 4cc89503-f054d0d4-2865-00221958e471/nas3-vmfs
Can mount: Yes
Can resignature: No (the volume is being actively used)
Extent name: naa.600140588e9bab7d2b7ed4b53dadc1de:1     range: 0 - 1048319 (MB)

and then do the following

esxcfg-volume -M 4cc89503-f054d0d4-2865-00221958e471

Open in new window


or

esxcfg-volume -M nas3-vmfs

Open in new window


(obvsiously your GUID or friendly datastore name)
0
What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

 

Author Comment

by:huntson
ID: 41734703
That first command just gives me a carriage return.  I am at the SSH prompt if that makes a snip-5.PNGdifference.
0
 
LVL 119
ID: 41734710
that's very unusual, but if you click Add Storage it finds the LUNs ?

and wants to overwrite them ?

can you check the LUNs, are present, but datastores are not ?

have you tried re-scanning the storage adaptor ?

failing this, you will have to check the logs /var/logs/vmkernel.log

check why they are not being mounted
1
 

Author Comment

by:huntson
ID: 41734735
What's the best way to access those logs as that's the next step?
0
 
LVL 11

Expert Comment

by:Mr Tortur
ID: 41734758
Hi,
view them at console or ssh session
or
get them using Winscp
0
 

Author Comment

by:huntson
ID: 41734759
Here we go.  I know the device I'm looking for is naa.6001405136dec50da155d4eb5d9574df

Any clue?
vmkernel.log
0
 
LVL 119
ID: 41734787
can you confirm, if the LUNS exists ?
0
 

Author Comment

by:huntson
ID: 41734797
It does indeed.
snip-6.PNG
0
 
LVL 119
ID: 41734870
whats the dead or error ?
0
 

Author Comment

by:huntson
ID: 41734880
I rebooted and that disappeared.  It was a previously mounted volume that no longer exists.  Any idea what to do next?  I don't have any support contract or anything with vmware - is there a way I can get a per-incident case?  Anywhere else I can turn?
0
 
LVL 119
ID: 41734912
see my other post about weirdness with Synology NAS and volumes and LUNs.

hey stay tuned, we will try and get this resolved.

I'm here for you man!
0
 

Author Comment

by:huntson
ID: 41734961
Can yo post a link please?

Thanks for your help!!
0
 

Author Comment

by:huntson
ID: 41734995
I see it's on my other question.  Thanks for checking into it.
0
 
LVL 119
ID: 41734997
0
 

Author Comment

by:huntson
ID: 41735009
I see although that's not necessarily the same here.  I was not close to running out of space before and certainly am not now (even though I understand I provisioned the entirety of the volume).  

You do bring up a good point in your discussion with that other user - is there any way to access the volume via iSCSI from Windows?  Off the top of my head I'd say that Windows/Mac doesn't understand VMFS.
0
 
LVL 119
ID: 41735040
Windows/Mac cannot read and write VMFS (well, unless you use a third party driver!) which is read only.

Okay, I can reproduced the issue here, using Synlogy NAS.

I created a 100GB iSCSI Thin LUN on a 200GB free Volume.

Presented to VMware ESXi, added a datastore, added a virtual machine.

Increase the size of the LUN on Synology 10GB at a time, increase space on datastore. all worked correctly.

As soon, as I increased the size of the LUN, so the volume space was 0.

datastore and LUN, on ESXi became unavailable.

I think you need to refer to Synology support.
0
 

Author Comment

by:huntson
ID: 41735045
Synology support?  They're a joke.

Thank you though for investigating.
0
 

Author Comment

by:huntson
ID: 41735046
This read-only concept sounds good though.  At least I can get all my machines off the volume assuming I am able to get it to mount.
0
 
LVL 119
ID: 41735090
We only use NFS for Backups.
0
 

Author Comment

by:huntson
ID: 41735188
What version of DSM are you using?  I'm going to call Synology and say I and someone else with x version had this issue.
0
 

Accepted Solution

by:
huntson earned 0 total points
ID: 41735190
FYI I'm on the phone with VMware (I purchased one of their per-indicdent support packages) and they have to do an experimental procedure to try to get my data back.  They say the 90meg VMFS header is damaged.  They want to try and copy the header from another volume with the same size onto this one and see what happens.
0
 
LVL 119
ID: 41735205
that would be why it's not being mounted and recognised.

DSM 5.x the latest edition , we've not upgraded to 6.x yet.

No backups ?
0
 

Author Comment

by:huntson
ID: 41735213
It was in the works to get some backups of our machines.  That'll be another question I post - best backup solution for budget vmware users.
0
 
LVL 119
ID: 41735215
Oh very dear......
0
 

Author Comment

by:huntson
ID: 41735255
They did it.  You should have seen the stuff they did.  Quite impressive and for a measly $300 or so.  Thanks for all your help here.

Since the solution was found externally what's the best way to mark a solution here?  I want to ensure you get points.
0
 
LVL 119
ID: 41735265
I would be more concerned as to why it got corrupted!

select your own Answer.

https:#a41735190
0
 

Author Closing Comment

by:huntson
ID: 41740324
I wound up calling support as this was quite a complicated corruption issue.
0

Featured Post

Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

In this article, I show you step by step with screenshots to assist you - HOW TO: Deploy and Install the VMware vCenter Server Appliance 6.5 (VCSA 6.5), with some helpful tips along the way.
In this article, I am going to show you how to simulate a multi-site Lab environment on a single Hyper-V host. I use this method successfully in my own lab to simulate three fully routed global AD Sites on a Windows 10 Hyper-V host.
Teach the user how to rename, unmount, delete and upgrade VMFS datastores. Open vSphere Web Client: Rename VMFS and NFS datastores: Upgrade VMFS-3 volume to VMFS-5: Unmount VMFS datastore: Delete a VMFS datastore:
How to install and configure Citrix XenApp 6.5 - Part 1. In this video tutorial we have explained step by step installation of Citrix XenApp 6.5 Server on Windows Server 2008 R2 is explained in this video. We have explained the difference between…

777 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question