Avatar of hongedit
hongeditFlag for United Kingdom of Great Britain and Northern Ireland

asked on 

Unable to start VM - file locked?

Hi

I am unable to start one of my machines after powering it off. Error:

Unable to access file [Datastore] SQL/SQL.vmdk since it is locked.

See the error stack for details on the cause of this problem.

Reason: Failed to lock the file.
Cannot open the disk '/vmfs/volumes/UUID/SQL.vmdk or one of the snapshot disks it depends on.

I tried to clone the VM, but the clone also fails to start with error that the file is not a valid virtual machine.

The Datastore is an iSCSI SAN.

Any ideas?
VMware

Avatar of undefined
Last Comment
coolsport00
Avatar of Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)
Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)
Flag of United Kingdom of Great Britain and Northern Ireland image

is there any *.LCK file or folder present in the folder with the VM files?
Avatar of hongedit
hongedit
Flag of United Kingdom of Great Britain and Northern Ireland image

ASKER

Not that I can see, no
Avatar of hongedit
hongedit
Flag of United Kingdom of Great Britain and Northern Ireland image

ASKER

I ran vmkfstools -D /vmfs/volumes/path/to/file via SSH and the output I got was:

login as: root
root@192.168.23.4's password:
You have activated Tech Support Mode.
The time and date of this activation have been sent to the system logs.

VMware offers supported, powerful system administration tools.  Please
see www.vmware.com/go/sysadmintools for details.

Tech Support Mode may be disabled by an administrative user.
Please consult the ESXi Configuration Guide for additional
important information.

~ # vmkfstools -D /vmfs/volumes/4dc914f7-43e9978a-1c40-00259033dd4c/Saints-SQL/Saints-SQL.vmdk
Lock [type 10c00001 offset 58759168 v 118, hb offset 3182592
gen 87, mode 0, owner 00000000-00000000-0000-000000000000 mtime 11611]
Addr <4, 125, 11>, gen 12, links 1, type reg, flags 0, uid 0, gid 0, mode 600
len 548, nb 1 tbz 0, cow 0, zla 2, bs 65536
try restarting management agents on all hosts that connect to that datastore to make sure that there are no other processes locking it like a failed vmotion.
also try vmkfstools -D /vmfs/volumes/4dc914f7-43e9978a-1c40-00259033dd4c/Saints-SQL/Saints-SQL[b][i]-flat[/b][/i].vmdk
Avatar of hongedit
hongedit
Flag of United Kingdom of Great Britain and Northern Ireland image

ASKER

I have rebooted both the hosts, is that enough? This VM has never been vmotioned anywhere, its only a few days old.

Avatar of hongedit
hongedit
Flag of United Kingdom of Great Britain and Northern Ireland image

ASKER

Results:

~ # vmkfstools -D /vmfs/volumes/4dc914f7-43e9978a-1c40-00259033dd4c/Saints-SQL/Saints-SQL-flat.vmdk
Lock [type 10c00001 offset 58757120 v 61, hb offset 3440640
gen 87, mode 0, owner 00000000-00000000-0000-000000000000 mtime 1651]
Addr <4, 125, 10>, gen 11, links 1, type reg, flags 0, uid 0, gid 0, mode 600
len 107374182400, nb 16135 tbz 0, cow 0, zla 3, bs 1048576

I dont even know what any of it means.
Avatar of coolsport00
coolsport00
Flag of United States of America image

Do you have a VM backup solution? If so, has it failed on the VM in question?

~coolsport00
Avatar of hongedit
hongedit
Flag of United Kingdom of Great Britain and Northern Ireland image

ASKER

No VM Backup as of yet.

This is a new virtual infrastructure, just waiting on some parts to arrive to test out VDR.
Avatar of polaquita7
polaquita7

The file that corresponds to the disk is locked. There must be a lck file with the VMWare image files that is preventing you to use it.
Avatar of coolsport00
coolsport00
Flag of United States of America image

Look at this VMware KB on VM's not being able to be powered on due to being locked:
http://kb.vmware.com/kb/10051

(I think the KB site is down at the moment, but look when it's up) :)

~coolsport00
Have you used vDR to backup this VM?

Check the VMWare vDR appliance does not have this disk attached to it?

Check the properties of the vDR appliance, edit the Virtual Machine properties, and check how many disks are currently attached, if this VMs disk is attached to vDR, it's either currenty backing up the VM, or Backup has failed.

Detact disk, by Delete the Disk from the VM, but not from the disk.
that was Detach the disk, by Deleting the Disk from the VM, but not from the disk!
Avatar of hongedit
hongedit
Flag of United Kingdom of Great Britain and Northern Ireland image

ASKER

There is definately no lck file :/

I have not used VDR yet, so no chance of that locking anything. Literally just been deployed.
yeah, reboots would take care of any process locks.

The vmkfstools -D command will return the UUID of the system that has the file lock, so when it comes back 0000's it can mean that the server the command was ran from has the lock or there's some other kind of locking going on.

Has the SAN or host(s) crashed or been powered off unexpectedly?
Avatar of coolsport00
coolsport00
Flag of United States of America image

Look at the SCSI controller...is it IDE or LSI Logic?
.lck's are used by NFS, so you shouldn't see them.  they are seen with 'ls -al' command
Avatar of hongedit
hongedit
Flag of United Kingdom of Great Britain and Northern Ireland image

ASKER

The SAN has not been powered off "unexpectedly" - only when all the VM's were shut down cleanly and hosts turned off.

SCSI Controller is LSI Logic
Avatar of hongedit
hongedit
Flag of United Kingdom of Great Britain and Northern Ireland image

ASKER

Ah. Here is the log:

/vmfs/volumes/4dc914f7-43e9978a-1c40-00259033dd4c/Saints-SQL # ls -al
drwxr-xr-x    1 root     root               2240 May 13 21:59 .
drwxr-xr-t    1 root     root               1120 May 13 21:59 ..
-rw-r--r--    1 root     root                  1 May 10 22:27 Saints-SQL-08744d6         f.hlog
-rw-------    1 root     root       107374182400 May 13 16:23 Saints-SQL-flat.vm         dk
-rw-------    1 root     root               8684 May 13 16:24 Saints-SQL.nvram
-rw-------    1 root     root                548 May 10 10:38 Saints-SQL.vmdk
-rw-r--r--    1 root     root                  0 May 10 11:35 Saints-SQL.vmsd
-rwxr-xr-x    1 root     root               3709 May 13 22:11 Saints-SQL.vmx
-rw-r--r--    1 root     root                265 May 13 22:11 Saints-SQL.vmxf
-rw-r--r--    1 root     root              38506 May 13 17:36 vmware-14.log
-rw-r--r--    1 root     root              38511 May 13 17:38 vmware-15.log
-rw-r--r--    1 root     root              38511 May 13 17:39 vmware-16.log
-rw-r--r--    1 root     root              38492 May 13 17:57 vmware-17.log
-rw-r--r--    1 root     root              38509 May 13 19:46 vmware-18.log
-rw-r--r--    1 root     root              38516 May 13 21:45 vmware-19.log
-rw-r--r--    1 root     root              38515 May 13 21:59 vmware.log
/vmfs/volumes/4dc914f7-43e9978a-1c40-00259033dd4c/Saints-SQL #
ASKER CERTIFIED SOLUTION
Avatar of coolsport00
coolsport00
Flag of United States of America image

Blurred text
THIS SOLUTION IS ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
is that "Saints-SQL-flat.vm         dk" entry a typo or just the way it posted?
Avatar of coolsport00
coolsport00
Flag of United States of America image

BTW...when you remove the disk from the current VM, do *NOT* select to delete from disk; ONLY DELETE/REMOVE from the VM.
Avatar of hongedit
hongedit
Flag of United Kingdom of Great Britain and Northern Ireland image

ASKER

Oo - that worked. It's powweing on.

Can I just use this as the VM from now on, and delete the other one?
Avatar of coolsport00
coolsport00
Flag of United States of America image

Or, maybe another option is to rt-click on the VM and select 'Remove from Inventory'. Once removed, open the Datastore Browser the VM is on. Go into the VM's folder, rt-click the .vmx file and select 'Add to Inventory'.
I'd start looking at logs or calling support...

grep Saints-SQL.vmdk /var/log/vmware/hostd.log
grep Saints-SQL.vmdk /var/log/vmkernel
grep Saints-SQL.vmdk /vmfs/volumes/4dc914f7-43e9978a-1c40-00259033dd4c/Saints-SQL/vmware.log
Avatar of coolsport00
coolsport00
Flag of United States of America image

YES!..because the other VM doesn't have a disk associated with it. Awesome...glad you're back up :)
Avatar of coolsport00
coolsport00
Flag of United States of America image

What probably happened was, somehow, your .vmx file got corrupted...not sure why or how. Creating a new VM creates a new .vmx file. We coulda spent time looking in the current .vmx file for what went wrong, but why when we can create a whole new VM in seconds :)
nice catch, coolsport00.  I was thinking it might have been a .vmdk corruption...
Avatar of coolsport00
coolsport00
Flag of United States of America image

well, as you know...a VM not powering on due to lock could be several different things. next up would certainly have been the vmdk :)
Avatar of hongedit
hongedit
Flag of United Kingdom of Great Britain and Northern Ireland image

ASKER

Um I think I misclicked for points, can a mod help please!
I'll object for coolsport00  :)
Avatar of coolsport00
coolsport00
Flag of United States of America image

Yeah...just click Request Attention :)
Avatar of hongedit
hongedit
Flag of United Kingdom of Great Britain and Northern Ireland image

ASKER

Thank you all
Avatar of coolsport00
coolsport00
Flag of United States of America image

Ah...your the man "danm66" ha :)
Avatar of hongedit
hongedit
Flag of United Kingdom of Great Britain and Northern Ireland image

ASKER

Well just for a laugh I shut down the newly created VM, re-added the disk to the old VM and it booted up, lol.

Go figure.
Avatar of coolsport00
coolsport00
Flag of United States of America image

:)
VMware
VMware

VMware, a software company founded in 1998, was one of the first commercially successful companies to offer x86 virtualization. The storage company EMC purchased VMware in 1994. Dell Technologies acquired EMC in 2016. VMware’s parent company is now Dell Technologies. VMware has many software products that run on desktops, Microsoft Windows, Linux, and macOS, which allows the virtualizing of the x86 architecture. Its enterprise software hypervisor for servers, VMware vSphere Hypervisor (ESXi), is a bare-metal hypervisor that runs directly on the server hardware and does not require an additional underlying operating system.

39K
Questions
--
Followers
--
Top Experts
Get a personalized solution from industry experts
Ask the experts
Read over 600 more reviews

TRUSTED BY

IBM logoIntel logoMicrosoft logoUbisoft logoSAP logo
Qualcomm logoCitrix Systems logoWorkday logoErnst & Young logo
High performer badgeUsers love us badge
LinkedIn logoFacebook logoX logoInstagram logoTikTok logoYouTube logo