URGENT - Recovering data from orphaned .VMDK files in ESX Server 3.5

Posted on 2010-01-08
Last Modified: 2012-05-08
have a really messed up scenario with a Windows 2003 Server Virtual Machine running on VMWare ESX Server 3.5

The Story:
1)Overnight, the Datastore that 2 of my VMs run from filled.
2) I moved all of the files for one of the VMs (VM1 - our main Database server) to another datastore, using VeemFastSCP, removed it from the inventory in Infrastructure Client and registered it from the new location.
3) On startup - Poweron failed giving me a file Not Found Error.
4) The files present were as follows:
VM1.VMDK (165GB)
VM1000001.VMDK  (5GB)
VM1000002.VMDK (17MB)
VM1000003.VMDK (50GB)
VM1000004.VMDK (50 GB)
VM1000005-delta.VMDK (31 GB)
VM1000006.VMDK (32 GB)
VM1000007.VMDK (23GB)
VM1000008.VMDK (500MB)
VM1000009.VMDK (50 GB)
5) I took a look at the Hard Disk of the VM and it's Disk File was set to VM1000009.VMDK
6) In a panic - I changed the disk file location to VM1.VMK and powered on. The VM powered up, and the guest OS started, but on investigation, all the files very as of the 16Dec09!
7)since this VM is a SQL server, and I have seperate, hourly file backups of all my databases that run on it, i decided to just run a quick restore on all my databases - with the previous night's data. That worked fine and I was able to get all my users back on-line.
8) However, one of our apps dumps raw data text files onto a fileshare on the VM. I have now lost all the files that were created since the 16th Dec.
9) To compound issues, I had overwritten my filebased backup of this server before I noticed the missing files!

My Question:
Does anyone know if it's possible to recoved the data from these now orphaned .VMDK files?

any help would be greatly appreciated.
Question by:lanson
    LVL 21

    Expert Comment

    If you have no backups of the VMDKs and their snapshot files ... the only way to get some of the data out of the snapshot vmdks (and I don't know how you would do this) would be use a disk recovery tool or something that could mount VMDK files. I came across this post so hope it can help you ... but you may be out of luck ...
    LVL 24

    Expert Comment

    Well dont delete them!
    You are not supposed to point it the master VM1.VMDK, the behavior is normal when you do snapshots, original vmdk will not be touched and change to read only the moment you take snapshot, that is why you can normally copy/download the main vmdk file as the file is no longer locked by vmfs, any changes will be written to delta file instead
    so instead of something like this scsi:0.filename = "VM1.VMDK" it is now like this scsi:0.filename = "VM1000009.VMDK"

    For your case, try fixing the CID chains
    Refer to this

    Read it slowly I'm sure there are still hope, i had fixed the similar issue for few customers, it worked great! Hopefully it can solve your problem
    LVL 1

    Author Comment

    ryder0707 - the problem is that vm1.vmdk has been running live now as the main vmdk, for 2 days.
    I presume fixing the cid chain won't work since the vm1.vmdk had been updated?
    LVL 24

    Accepted Solution

    aaah yeah that does make a difference...hmm no clue what will happen after that
    If i were in your situation i will still try though, copy all files to a test directory and fix the chain, register as different VM, disconnect the adapter from network and start it just to see what happen, hopefully if commit the latest snapshot it can magically merge somehow but this is only my imagination, or do a V2V after fixing the chain and hopefully vmware converter can consolidate the snapshot
    sorry mate i have no other idea, if you think the data is too important you may want to pay vmware for per incident support
    LVL 2

    Expert Comment

    You may want to duplicate your remaining files to another folder or datastore, then create a new vm using the original disk as the boot image. Leave off a network adapter so you have no conflicts with the existing vm. I would then attempt to fix any cid issues and see if you can get that to boot.
    LVL 1

    Author Comment

    out of luck i'm afraid. the CID fix didn't work and I raise a call with VMWare.
    It's impossible to recover once the parent .vmdk is modified.

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    6 Surprising Benefits of Threat Intelligence

    All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

    This article is an update and follow-up of my previous article:   Storage 101: common concepts in the IT enterprise storage This time, I expand on more frequently used storage concepts.
    VM backup deduplication is a method of reducing the amount of storage space needed to save VM backups. In most organizations, VMs contain many duplicate copies of data, such as VMs deployed from the same template, VMs with the same OS, or VMs that h…
    Advanced tutorial on how to run the esxtop command to capture a batch file in csv format in order to export the file and use it for performance analysis. He demonstrates how to download the file using a vSphere web client (or vSphere client) and exp…
    In this video tutorial I show you the main steps to install and configure  a VMware ESXi6.0 server. The video has my comments as text on the screen and you can pause anytime when needed. Hope this will be helpful. Verify that your hardware and BIO…

    761 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    8 Experts available now in Live!

    Get 1:1 Help Now