Solved

ESXi partition in need of recovery

Posted on 2016-07-29
29
16 Views
Last Modified: 2016-08-03
I recently had an issue where I lost some network connectivity with my iSCSI NAS where my ESXi datastores are.  After reconnecting ESXi shows it mounts the target however the datastore does not appear.  When I go to add new storage it presents the LUN and shows there is a partition there (see attached pic) however I can't seem to figure out how to use that partition without wiping it out.  I am on ESXi 6.  Any ideas?
0
Comment
Question by:huntson
  • 16
  • 12
29 Comments
 
LVL 117
ID: 41734676
you only use the ADD Storage to ADD NEW Storage, this is not new storage!

Okay, can you get to the console, or remote via SSH ?
0
 

Author Comment

by:huntson
ID: 41734694
I just did it to verify that it can see the partition. Yes I can.
0
 
LVL 117

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE)
Andrew Hancock (VMware vExpert / EE MVE) earned 500 total points
ID: 41734699
okay, I need you to type the following...

esxcfg-volume -l 

Open in new window


at the bash prompt.

this should show something like below

~ # esxcfg-volume -l
VMFS3 UUID/label: 4cc89503-f054d0d4-2865-00221958e471/nas3-vmfs
Can mount: Yes
Can resignature: No (the volume is being actively used)
Extent name: naa.600140588e9bab7d2b7ed4b53dadc1de:1     range: 0 - 1048319 (MB)

and then do the following

esxcfg-volume -M 4cc89503-f054d0d4-2865-00221958e471

Open in new window


or

esxcfg-volume -M nas3-vmfs

Open in new window


(obvsiously your GUID or friendly datastore name)
0
 

Author Comment

by:huntson
ID: 41734703
That first command just gives me a carriage return.  I am at the SSH prompt if that makes a snip-5.PNGdifference.
0
 
LVL 117
ID: 41734710
that's very unusual, but if you click Add Storage it finds the LUNs ?

and wants to overwrite them ?

can you check the LUNs, are present, but datastores are not ?

have you tried re-scanning the storage adaptor ?

failing this, you will have to check the logs /var/logs/vmkernel.log

check why they are not being mounted
1
 

Author Comment

by:huntson
ID: 41734735
What's the best way to access those logs as that's the next step?
0
 
LVL 11

Expert Comment

by:Mr Tortur
ID: 41734758
Hi,
view them at console or ssh session
or
get them using Winscp
0
 

Author Comment

by:huntson
ID: 41734759
Here we go.  I know the device I'm looking for is naa.6001405136dec50da155d4eb5d9574df

Any clue?
vmkernel.log
0
 
LVL 117
ID: 41734787
can you confirm, if the LUNS exists ?
0
 

Author Comment

by:huntson
ID: 41734797
It does indeed.
snip-6.PNG
0
 
LVL 117
ID: 41734870
whats the dead or error ?
0
 

Author Comment

by:huntson
ID: 41734880
I rebooted and that disappeared.  It was a previously mounted volume that no longer exists.  Any idea what to do next?  I don't have any support contract or anything with vmware - is there a way I can get a per-incident case?  Anywhere else I can turn?
0
 
LVL 117
ID: 41734912
see my other post about weirdness with Synology NAS and volumes and LUNs.

hey stay tuned, we will try and get this resolved.

I'm here for you man!
0
 

Author Comment

by:huntson
ID: 41734961
Can yo post a link please?

Thanks for your help!!
0
Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

 

Author Comment

by:huntson
ID: 41734995
I see it's on my other question.  Thanks for checking into it.
0
 
LVL 117
ID: 41734997
0
 

Author Comment

by:huntson
ID: 41735009
I see although that's not necessarily the same here.  I was not close to running out of space before and certainly am not now (even though I understand I provisioned the entirety of the volume).  

You do bring up a good point in your discussion with that other user - is there any way to access the volume via iSCSI from Windows?  Off the top of my head I'd say that Windows/Mac doesn't understand VMFS.
0
 
LVL 117
ID: 41735040
Windows/Mac cannot read and write VMFS (well, unless you use a third party driver!) which is read only.

Okay, I can reproduced the issue here, using Synlogy NAS.

I created a 100GB iSCSI Thin LUN on a 200GB free Volume.

Presented to VMware ESXi, added a datastore, added a virtual machine.

Increase the size of the LUN on Synology 10GB at a time, increase space on datastore. all worked correctly.

As soon, as I increased the size of the LUN, so the volume space was 0.

datastore and LUN, on ESXi became unavailable.

I think you need to refer to Synology support.
0
 

Author Comment

by:huntson
ID: 41735045
Synology support?  They're a joke.

Thank you though for investigating.
0
 

Author Comment

by:huntson
ID: 41735046
This read-only concept sounds good though.  At least I can get all my machines off the volume assuming I am able to get it to mount.
0
 
LVL 117
ID: 41735090
We only use NFS for Backups.
0
 

Author Comment

by:huntson
ID: 41735188
What version of DSM are you using?  I'm going to call Synology and say I and someone else with x version had this issue.
0
 

Accepted Solution

by:
huntson earned 0 total points
ID: 41735190
FYI I'm on the phone with VMware (I purchased one of their per-indicdent support packages) and they have to do an experimental procedure to try to get my data back.  They say the 90meg VMFS header is damaged.  They want to try and copy the header from another volume with the same size onto this one and see what happens.
0
 
LVL 117
ID: 41735205
that would be why it's not being mounted and recognised.

DSM 5.x the latest edition , we've not upgraded to 6.x yet.

No backups ?
0
 

Author Comment

by:huntson
ID: 41735213
It was in the works to get some backups of our machines.  That'll be another question I post - best backup solution for budget vmware users.
0
 
LVL 117
ID: 41735215
Oh very dear......
0
 

Author Comment

by:huntson
ID: 41735255
They did it.  You should have seen the stuff they did.  Quite impressive and for a measly $300 or so.  Thanks for all your help here.

Since the solution was found externally what's the best way to mark a solution here?  I want to ensure you get points.
0
 
LVL 117
ID: 41735265
I would be more concerned as to why it got corrupted!

select your own Answer.

https:#a41735190
0
 

Author Closing Comment

by:huntson
ID: 41740324
I wound up calling support as this was quite a complicated corruption issue.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

#Citrix #XenApp #Citrix Scout #Citrix Insight Services #Microsoft VMMAP #Microsoft ADEXPLORE #Microsoft RAMMAP #Microsoft TCPVIEW #Microsoft AUTORUNS #Microsoft PROCESS EXPLORER #Microsoft PROCESS MONITOR
David Varnum recently wrote up his impressions of PRTG, based on a presentation by my colleague Christian at Tech Field Day at VMworld in Barcelona. Thanks David, for your detailed and honest evaluation!
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …
This video shows you how easy it is to boot from ISO images for virtual machines with the ISO images stored on a local datastore on the ESXi host.

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now