Problem with VM backups using Veeam
Hello all, I am having some problems backing up a few of my VMs (vsphere 5.0) with Veeam. I have tried looking at the logs and they are completely cryptic and I have no idea what they mean. I tried some google research and came up with nothing.
A few of my servers are coming up with this message.
Backing up object "[Admin-DS-11] Admin-Ajax-64Bit/QA-Dev Ajax-64Bit.vmdk"
BackupDiskFull failed
Client error: Failed to open VDDK disk [[Admin-DS-11] Admin-Ajax-64Bit/QA-Dev Ajax-64Bit-000002.vmdk] ( is read-only mode - [true] )
Failed to open VMDK.
Logon attempt with parameters [VC/ESX: [admin-vmh-11];Port: 902;Login: [root];VMX Spec: [moref=193];Snapshot mor: [193-snapshot-208];Transpo
Server error: End of file
Now.. the kicker is my VBK files are being created, so it looks like it's being backed up, but I have not done a restore to make sure everything is working correctly. One of them is a critical server, so I am a little worried to do so with this error that's plaguing the backups for about 5 of my VMs. All the rest of my VMs (about 30 of them) are having no issues using the same job.
I have also created a brand new job with no luck. Any help is appreciated please!!
A few of my servers are coming up with this message.
Backing up object "[Admin-DS-11] Admin-Ajax-64Bit/QA-Dev Ajax-64Bit.vmdk"
BackupDiskFull failed
Client error: Failed to open VDDK disk [[Admin-DS-11] Admin-Ajax-64Bit/QA-Dev Ajax-64Bit-000002.vmdk] ( is read-only mode - [true] )
Failed to open VMDK.
Logon attempt with parameters [VC/ESX: [admin-vmh-11];Port: 902;Login: [root];VMX Spec: [moref=193];Snapshot mor: [193-snapshot-208];Transpo
Server error: End of file
Now.. the kicker is my VBK files are being created, so it looks like it's being backed up, but I have not done a restore to make sure everything is working correctly. One of them is a critical server, so I am a little worried to do so with this error that's plaguing the backups for about 5 of my VMs. All the rest of my VMs (about 30 of them) are having no issues using the same job.
I have also created a brand new job with no luck. Any help is appreciated please!!
ASKER
Nope, no snapshots at all. I made sure of it.
how did you check?
this file
QA-Dev Ajax-64Bit-000002.vmdk
is a snapshot, the number 2 indicates there are two!
the destination location is not out of storage space?
the datastore where the VMs are located are not out of storage space?
what is the size of disk?
what is the max size of disk for the datastore? (blocksize)
this file
QA-Dev Ajax-64Bit-000002.vmdk
is a snapshot, the number 2 indicates there are two!
the destination location is not out of storage space?
the datastore where the VMs are located are not out of storage space?
what is the size of disk?
what is the max size of disk for the datastore? (blocksize)
ASKER
I was checking in Vsphere on the VM itself under the snapshot manager. It's not showing in there at all.. is there another place I can check. The VMs are created on the local host server.
I am new to this company so I am not familiar with their infrastructure. I am still a little green with Vsphere.
I am new to this company so I am not familiar with their infrastructure. I am still a little green with Vsphere.
ASKER
The Datastore its being backed up to is not out of space, but I cannot find the VMDK file it's creating. It seems to be a different name than the VM itself. I browsed the datastore and cannot seem to find it :(
ASKER
I also cannot find any file named QA-Dev Ajax-64Bit-000002.vmdk in the data store when I browse it. If it's not in that VM folder, where would it be located?
can you screenshot the datastore?
ASKER
Absolutely. Here is what I'm seeing when I browse.
Also, for my other VMs that have the 000001.VMDK file in their data store folders, is it ok to just delete the file?
Thanks again for your help.
datastore-snapshot.jpg
Also, for my other VMs that have the 000001.VMDK file in their data store folders, is it ok to just delete the file?
Thanks again for your help.
datastore-snapshot.jpg
No you cannot just manually delete a snapshot you must remove via Snapshot Manager and be patient.
Very odd no snapshots if you create a manual job does it always fail?
Very odd no snapshots if you create a manual job does it always fail?
ASKER
There are no snapshots showing in the manager, which is what's confusing me.
Snapshot.jpg
Snapshot.jpg
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I started a new backup of one of the servers in question and it worked. No errors. I have no idea why it did not work as part of the initial job that was already there. However, the other server I have is still not working when I create a new job for it. This one does have the 00001.vmdk file when browsing the datastore. No snapshots exist in the snapshot manager.
This server is throwing up a different error.
Disk "Hard disk 2" has incorrect changed block tracking configuration.
Backing up object "[Admin-DS-12] DocMallSBS11/DocMallSBS11_
One or more VM disks have incorrect changed block tracking configuration. To resolve this, open VMware vSphere Client, right-click the VM, choose Edit Settings, Options tab, select General, click Configuration Parameters, and set all entries with ‘ctkEnabled’ substring to false. Veeam Backup will then automatically re-enable changed block tracking with the correct settings during the next job run.
BackupDiskFull failed
Client error: Failed to open VDDK disk [[Admin-DS-12] DocMallSBS11/DocMallSBS11_
Failed to open VMDK.
Logon attempt with parameters [VC/ESX: [admin-vmh-12];Port: 902;Login: [root];VMX Spec: [moref=288];Snapshot mor: [288-snapshot-252];Transpo
Server error: End of file
I have done what the error suggested to do and it still has not corrected the problem. I researched other articles on this error, also followed it's suggestions and still not working. Quite frustrating.
This server is throwing up a different error.
Disk "Hard disk 2" has incorrect changed block tracking configuration.
Backing up object "[Admin-DS-12] DocMallSBS11/DocMallSBS11_
One or more VM disks have incorrect changed block tracking configuration. To resolve this, open VMware vSphere Client, right-click the VM, choose Edit Settings, Options tab, select General, click Configuration Parameters, and set all entries with ‘ctkEnabled’ substring to false. Veeam Backup will then automatically re-enable changed block tracking with the correct settings during the next job run.
BackupDiskFull failed
Client error: Failed to open VDDK disk [[Admin-DS-12] DocMallSBS11/DocMallSBS11_
Failed to open VMDK.
Logon attempt with parameters [VC/ESX: [admin-vmh-12];Port: 902;Login: [root];VMX Spec: [moref=288];Snapshot mor: [288-snapshot-252];Transpo
Server error: End of file
I have done what the error suggested to do and it still has not corrected the problem. I researched other articles on this error, also followed it's suggestions and still not working. Quite frustrating.
Okay ensure backup is stopped.
Create a snapshot in Snapshot manager
Wait 60 seconds
Delete all snapshots
Hopefully this will get rid of all snapshots
Then remove all ctk files
Repeat test backup
Create a snapshot in Snapshot manager
Wait 60 seconds
Delete all snapshots
Hopefully this will get rid of all snapshots
Then remove all ctk files
Repeat test backup
ASKER
Ok, will do.
Can the CTK files be removed manually when browsing the datastore? Or is there a process for this separately?
Can the CTK files be removed manually when browsing the datastore? Or is there a process for this separately?
ASKER
Turns out making a brand new job in veeam with my VMs in question worked. The other server problem regarding a problem with changed block tracking had to do with hard disk 2 on that specific VM being in independent mode.
Once I turned off independent mode, all was working.
Once I turned off independent mode, all was working.
ASKER
The new backup job suggestion ended up fixing the problem for some of the VMs, but the main one in question needed independent mode turned off on that disk.
Can you confirm the snapshots exist, and size?
screenshot will do.
Please consult my EE Article
HOW TO: VMware Snapshots :- Be Patient
and Ask advice before Delting a Snapshot!