Link to home
Start Free TrialLog in
Avatar of hongedit
hongeditFlag for United Kingdom of Great Britain and Northern Ireland

asked on

Unable to start VM - file locked?

Hi

I am unable to start one of my machines after powering it off. Error:

Unable to access file [Datastore] SQL/SQL.vmdk since it is locked.

See the error stack for details on the cause of this problem.

Reason: Failed to lock the file.
Cannot open the disk '/vmfs/volumes/UUID/SQL.vmdk or one of the snapshot disks it depends on.

I tried to clone the VM, but the clone also fails to start with error that the file is not a valid virtual machine.

The Datastore is an iSCSI SAN.

Any ideas?
Avatar of Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)
Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)
Flag of United Kingdom of Great Britain and Northern Ireland image

is there any *.LCK file or folder present in the folder with the VM files?
Avatar of hongedit

ASKER

Not that I can see, no
I ran vmkfstools -D /vmfs/volumes/path/to/file via SSH and the output I got was:

login as: root
root@192.168.23.4's password:
You have activated Tech Support Mode.
The time and date of this activation have been sent to the system logs.

VMware offers supported, powerful system administration tools.  Please
see www.vmware.com/go/sysadmintools for details.

Tech Support Mode may be disabled by an administrative user.
Please consult the ESXi Configuration Guide for additional
important information.

~ # vmkfstools -D /vmfs/volumes/4dc914f7-43e9978a-1c40-00259033dd4c/Saints-SQL/Saints-SQL.vmdk
Lock [type 10c00001 offset 58759168 v 118, hb offset 3182592
gen 87, mode 0, owner 00000000-00000000-0000-000000000000 mtime 11611]
Addr <4, 125, 11>, gen 12, links 1, type reg, flags 0, uid 0, gid 0, mode 600
len 548, nb 1 tbz 0, cow 0, zla 2, bs 65536
try restarting management agents on all hosts that connect to that datastore to make sure that there are no other processes locking it like a failed vmotion.
also try vmkfstools -D /vmfs/volumes/4dc914f7-43e9978a-1c40-00259033dd4c/Saints-SQL/Saints-SQL[b][i]-flat[/b][/i].vmdk
I have rebooted both the hosts, is that enough? This VM has never been vmotioned anywhere, its only a few days old.

Results:

~ # vmkfstools -D /vmfs/volumes/4dc914f7-43e9978a-1c40-00259033dd4c/Saints-SQL/Saints-SQL-flat.vmdk
Lock [type 10c00001 offset 58757120 v 61, hb offset 3440640
gen 87, mode 0, owner 00000000-00000000-0000-000000000000 mtime 1651]
Addr <4, 125, 10>, gen 11, links 1, type reg, flags 0, uid 0, gid 0, mode 600
len 107374182400, nb 16135 tbz 0, cow 0, zla 3, bs 1048576

I dont even know what any of it means.
Do you have a VM backup solution? If so, has it failed on the VM in question?

~coolsport00
No VM Backup as of yet.

This is a new virtual infrastructure, just waiting on some parts to arrive to test out VDR.
Avatar of polaquita7
polaquita7

The file that corresponds to the disk is locked. There must be a lck file with the VMWare image files that is preventing you to use it.
Look at this VMware KB on VM's not being able to be powered on due to being locked:
http://kb.vmware.com/kb/10051

(I think the KB site is down at the moment, but look when it's up) :)

~coolsport00
Have you used vDR to backup this VM?

Check the VMWare vDR appliance does not have this disk attached to it?

Check the properties of the vDR appliance, edit the Virtual Machine properties, and check how many disks are currently attached, if this VMs disk is attached to vDR, it's either currenty backing up the VM, or Backup has failed.

Detact disk, by Delete the Disk from the VM, but not from the disk.
that was Detach the disk, by Deleting the Disk from the VM, but not from the disk!
There is definately no lck file :/

I have not used VDR yet, so no chance of that locking anything. Literally just been deployed.
yeah, reboots would take care of any process locks.

The vmkfstools -D command will return the UUID of the system that has the file lock, so when it comes back 0000's it can mean that the server the command was ran from has the lock or there's some other kind of locking going on.

Has the SAN or host(s) crashed or been powered off unexpectedly?
Look at the SCSI controller...is it IDE or LSI Logic?
.lck's are used by NFS, so you shouldn't see them.  they are seen with 'ls -al' command
The SAN has not been powered off "unexpectedly" - only when all the VM's were shut down cleanly and hosts turned off.

SCSI Controller is LSI Logic
Ah. Here is the log:

/vmfs/volumes/4dc914f7-43e9978a-1c40-00259033dd4c/Saints-SQL # ls -al
drwxr-xr-x    1 root     root               2240 May 13 21:59 .
drwxr-xr-t    1 root     root               1120 May 13 21:59 ..
-rw-r--r--    1 root     root                  1 May 10 22:27 Saints-SQL-08744d6         f.hlog
-rw-------    1 root     root       107374182400 May 13 16:23 Saints-SQL-flat.vm         dk
-rw-------    1 root     root               8684 May 13 16:24 Saints-SQL.nvram
-rw-------    1 root     root                548 May 10 10:38 Saints-SQL.vmdk
-rw-r--r--    1 root     root                  0 May 10 11:35 Saints-SQL.vmsd
-rwxr-xr-x    1 root     root               3709 May 13 22:11 Saints-SQL.vmx
-rw-r--r--    1 root     root                265 May 13 22:11 Saints-SQL.vmxf
-rw-r--r--    1 root     root              38506 May 13 17:36 vmware-14.log
-rw-r--r--    1 root     root              38511 May 13 17:38 vmware-15.log
-rw-r--r--    1 root     root              38511 May 13 17:39 vmware-16.log
-rw-r--r--    1 root     root              38492 May 13 17:57 vmware-17.log
-rw-r--r--    1 root     root              38509 May 13 19:46 vmware-18.log
-rw-r--r--    1 root     root              38516 May 13 21:45 vmware-19.log
-rw-r--r--    1 root     root              38515 May 13 21:59 vmware.log
/vmfs/volumes/4dc914f7-43e9978a-1c40-00259033dd4c/Saints-SQL #
ASKER CERTIFIED SOLUTION
Avatar of coolsport00
coolsport00
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
is that "Saints-SQL-flat.vm         dk" entry a typo or just the way it posted?
BTW...when you remove the disk from the current VM, do *NOT* select to delete from disk; ONLY DELETE/REMOVE from the VM.
Oo - that worked. It's powweing on.

Can I just use this as the VM from now on, and delete the other one?
Or, maybe another option is to rt-click on the VM and select 'Remove from Inventory'. Once removed, open the Datastore Browser the VM is on. Go into the VM's folder, rt-click the .vmx file and select 'Add to Inventory'.
I'd start looking at logs or calling support...

grep Saints-SQL.vmdk /var/log/vmware/hostd.log
grep Saints-SQL.vmdk /var/log/vmkernel
grep Saints-SQL.vmdk /vmfs/volumes/4dc914f7-43e9978a-1c40-00259033dd4c/Saints-SQL/vmware.log
YES!..because the other VM doesn't have a disk associated with it. Awesome...glad you're back up :)
What probably happened was, somehow, your .vmx file got corrupted...not sure why or how. Creating a new VM creates a new .vmx file. We coulda spent time looking in the current .vmx file for what went wrong, but why when we can create a whole new VM in seconds :)
nice catch, coolsport00.  I was thinking it might have been a .vmdk corruption...
well, as you know...a VM not powering on due to lock could be several different things. next up would certainly have been the vmdk :)
Um I think I misclicked for points, can a mod help please!
I'll object for coolsport00  :)
Yeah...just click Request Attention :)
Thank you all
Ah...your the man "danm66" ha :)
Well just for a laugh I shut down the newly created VM, re-added the disk to the old VM and it booted up, lol.

Go figure.