Link to home
Start Free TrialLog in
Avatar of Jim Nicolis
Jim NicolisFlag for Australia

asked on

Virtual Machine has gone back 5 days after removing snapshot

I urgently need some advice. I have a client running ESXI 5 and has 3 virtual machines
running on the server. The Host is a IBM X3650 running in raid 5. The hardware is perfectly o.k but I had a situation where the main virtual machine running Windows Server 2003 bombed out as the virtual machine ran out of space. Upon checking I discovered 3 Snapshots taken that were taking up space that I think were taken by the backup VMX software.
I deleted each snapshot in order, this cleared space and the virtual machine was able to run but it has gone back 5 days. Is there anyway to get back to the present.

I have attached a picture of the virtual machines directory

Is there any hope or do I just have to wear it.
server.jpg
SOLUTION
Avatar of RickEpnet
RickEpnet
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Avatar of asavener
asavener
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of Jim Nicolis

ASKER

No I have no snapshots in the snapshot manager, I deleted them but I think the consolidation failed as there are what appears to be snapshots in the directory but missing the delta files.
I am trying to setup another ESXI server so I can copy the virtual machine to it so I can
attempt consolidation without the risk.
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Hi yes, I have discovered there are orphaned delta files, 1 is approx 3gb and the other is about 30gb. The snapshot manager must have not reconsolidated files due to lack of space.

I believe / hope this contains the data that  I need.

I have copied the virtual machine to another host and tried performing another snapshot
and then delete all in the hope that it would reconsolidate but no luck.

Just wondering how I can reconsolidate these orphaned Delta Files, will cloning as you suggest do this?

I would appreciate your advice.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Thanks for response.

I have checked VM is off

VM is not running on a snapshot

I have not used remote via ssh but want to learn.

I am not on site so Console at the moment is not a option

I have updated with latest screen shot of VM directory listing

You can see the orphaned delta files.

For some reason when I copied the files to a local share the files actually say Delta
but in the vm directory I dont see the Delta wording in the name.

I copied these files to a windows share as for some reason the vm copy stopped after
the vm files where copied. no other files where copied so I had to do it remotely as I could
not access the machine which ran the copy job. I used veeam.
VMDirectory.jpg
vmcontents.JPG
Just for you reference, I know why the problem happened.

about a week ago I use VMX backup software to do a backup. I cant remember the
exact reason but cancelled the backup. I was unaware that it had not deleted or cleaned up
its snapshot.

I had a call from client about 5 days later that machine was running very badly. upon checking
vm could not start due to lack of space. I discovered the snapshot and deleted them 1 by one
everything went ok to my knowledge but about 30 minutes later up getting the machine going I got a call from client saying there work was not there and had reverted to the 05/12/2013

I assume the snapshots did not reconsolidate properly thus leaving the orphaned snapshots in the directory. am I correct to assume this happend ?
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Yes correct, and i have setup remote ssh using putty.
Did you click on "revert to current snapshot?"

The option appears disabled so it must think there is no snapshot available.

I have checked and VM is not running on any snapshot
you may be able to clone your existing disk, and merge the remaing snapshot, to create a new disk. If you want to try this let me know, I can work with you to show you how.

Do you think I should try this?
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
I have time, you were going to show me how to do it through the console or remotely via ssh.

I have not restored or converted the virtual machine. I have made a copy of it to another host
just as a backup. The client is down at the moment and as mentioned 5 days of data missing due to the failed consolidation of snapshot.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Got putty and enabled SSH, I am doing this remotely so I assume we are doing Remote Console.
Hi I have read the articl and followed the instructions, I am able to remote in now via ssh. not sure what your next recommend step is?

Thanks
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Sorry for my inexperience with command line arguements. I have cd to the directory where
the files are but I need to copy to another host as I dont have the space on this host.

the host is ip 10.0.0.238 and the datastore where the file vm is to to in is located in /vmfs/volumes/datastore3/temp

can you give me a example of how to structure the command.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Okay, just so I get this right I need to copy the 2 Delta files (only) to the host and then execute
the command in the temp directory. Do I need to copy any other files or just the 2 snapshot files.

Sorry for all the questions.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
When trying to do the clone procedure I keep getting the followin error.


Failed to open 'server.workgroup.local-000004.vmdk': The parent of this virtual disk could not be opened (23).
/vmfs/volumes/519872be-01d542e0-1758-001a64c1311c/server.workgroup.local # Failed to open 'server.workgroup.local-0000
04.vmdk': The parent of this virtual disk could not be opened (23).
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
This is what I have

# Disk DescriptorFile
version=1
encoding="UTF-8"
CID=a2212306
parentCID=6b0ed997
isNativeSnapshot="no"
createType="vmfsSparse"
parentFileNameHint="server.workgroup.local-000003.vmdk"
# Extent description
RW 146806784 VMFSSPARSE "server.workgroup.local-000004-delta.vmdk"

# The Disk Data Base
#DDB

ddb.longContentID = "7b0f21f2d7547fdf4201094ea2212306"
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
That's correct, it is no longer there.
Is it still possible to consolidate the
 snapshot ?
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Willing to try anything at this point, I am up for it if you are.

Thanks

Jim
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
I have not been able to find the descriptor file, but I have had some help from VWARE paid support. who managed to clone the file as hanccocka sugggested and was on the right track. They then attached the cloned drive to the running VM and I was able to view the directory structure and extract the file that  I needed. The snapshot was definately orphaned and they had to do some editing to bring the snapshot in line with the original VM. I am sure this was the road we were heading on with the advice from hanccocka and I would have loved to have seen the instructions he would have offered for future reference. Thank you all for your advice and especially to  hanccocka for his efforts, your advice was invaluable and a learning experience for me.

Jim
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial