ESXi partition in need of recovery

I recently had an issue where I lost some network connectivity with my iSCSI NAS where my ESXi datastores are.  After reconnecting ESXi shows it mounts the target however the datastore does not appear.  When I go to add new storage it presents the LUN and shows there is a partition there (see attached pic) however I can't seem to figure out how to use that partition without wiping it out.  I am on ESXi 6.  Any ideas?
huntsonAsked:
Who is Participating?
 
huntsonConnect With a Mentor Author Commented:
FYI I'm on the phone with VMware (I purchased one of their per-indicdent support packages) and they have to do an experimental procedure to try to get my data back.  They say the 90meg VMFS header is damaged.  They want to try and copy the header from another volume with the same size onto this one and see what happens.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
you only use the ADD Storage to ADD NEW Storage, this is not new storage!

Okay, can you get to the console, or remote via SSH ?
0
 
huntsonAuthor Commented:
I just did it to verify that it can see the partition. Yes I can.
0
Improve Your Query Performance Tuning

In this FREE six-day email course, you'll learn from Janis Griffin, Database Performance Evangelist. She'll teach 12 steps that you can use to optimize your queries as much as possible and see measurable results in your work. Get started today!

 
Andrew Hancock (VMware vExpert / EE MVE^2)Connect With a Mentor VMware and Virtualization ConsultantCommented:
okay, I need you to type the following...

esxcfg-volume -l 

Open in new window


at the bash prompt.

this should show something like below

~ # esxcfg-volume -l
VMFS3 UUID/label: 4cc89503-f054d0d4-2865-00221958e471/nas3-vmfs
Can mount: Yes
Can resignature: No (the volume is being actively used)
Extent name: naa.600140588e9bab7d2b7ed4b53dadc1de:1     range: 0 - 1048319 (MB)

and then do the following

esxcfg-volume -M 4cc89503-f054d0d4-2865-00221958e471

Open in new window


or

esxcfg-volume -M nas3-vmfs

Open in new window


(obvsiously your GUID or friendly datastore name)
0
 
huntsonAuthor Commented:
That first command just gives me a carriage return.  I am at the SSH prompt if that makes a snip-5.PNGdifference.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
that's very unusual, but if you click Add Storage it finds the LUNs ?

and wants to overwrite them ?

can you check the LUNs, are present, but datastores are not ?

have you tried re-scanning the storage adaptor ?

failing this, you will have to check the logs /var/logs/vmkernel.log

check why they are not being mounted
1
 
huntsonAuthor Commented:
What's the best way to access those logs as that's the next step?
0
 
Mr TorturSystem EngineerCommented:
Hi,
view them at console or ssh session
or
get them using Winscp
0
 
huntsonAuthor Commented:
Here we go.  I know the device I'm looking for is naa.6001405136dec50da155d4eb5d9574df

Any clue?
vmkernel.log
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
can you confirm, if the LUNS exists ?
0
 
huntsonAuthor Commented:
It does indeed.
snip-6.PNG
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
whats the dead or error ?
0
 
huntsonAuthor Commented:
I rebooted and that disappeared.  It was a previously mounted volume that no longer exists.  Any idea what to do next?  I don't have any support contract or anything with vmware - is there a way I can get a per-incident case?  Anywhere else I can turn?
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
see my other post about weirdness with Synology NAS and volumes and LUNs.

hey stay tuned, we will try and get this resolved.

I'm here for you man!
0
 
huntsonAuthor Commented:
Can yo post a link please?

Thanks for your help!!
0
 
huntsonAuthor Commented:
I see it's on my other question.  Thanks for checking into it.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
0
 
huntsonAuthor Commented:
I see although that's not necessarily the same here.  I was not close to running out of space before and certainly am not now (even though I understand I provisioned the entirety of the volume).  

You do bring up a good point in your discussion with that other user - is there any way to access the volume via iSCSI from Windows?  Off the top of my head I'd say that Windows/Mac doesn't understand VMFS.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Windows/Mac cannot read and write VMFS (well, unless you use a third party driver!) which is read only.

Okay, I can reproduced the issue here, using Synlogy NAS.

I created a 100GB iSCSI Thin LUN on a 200GB free Volume.

Presented to VMware ESXi, added a datastore, added a virtual machine.

Increase the size of the LUN on Synology 10GB at a time, increase space on datastore. all worked correctly.

As soon, as I increased the size of the LUN, so the volume space was 0.

datastore and LUN, on ESXi became unavailable.

I think you need to refer to Synology support.
0
 
huntsonAuthor Commented:
Synology support?  They're a joke.

Thank you though for investigating.
0
 
huntsonAuthor Commented:
This read-only concept sounds good though.  At least I can get all my machines off the volume assuming I am able to get it to mount.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
We only use NFS for Backups.
0
 
huntsonAuthor Commented:
What version of DSM are you using?  I'm going to call Synology and say I and someone else with x version had this issue.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
that would be why it's not being mounted and recognised.

DSM 5.x the latest edition , we've not upgraded to 6.x yet.

No backups ?
0
 
huntsonAuthor Commented:
It was in the works to get some backups of our machines.  That'll be another question I post - best backup solution for budget vmware users.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Oh very dear......
0
 
huntsonAuthor Commented:
They did it.  You should have seen the stuff they did.  Quite impressive and for a measly $300 or so.  Thanks for all your help here.

Since the solution was found externally what's the best way to mark a solution here?  I want to ensure you get points.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
I would be more concerned as to why it got corrupted!

select your own Answer.

https:#a41735190
0
 
huntsonAuthor Commented:
I wound up calling support as this was quite a complicated corruption issue.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.