• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 3885
  • Last Modified:

VSS Error with VM Snapshots

VMware 5, VDR 2.

1 VM is unable to be backed up because of error:

Failed to create snapshot for DC, error -3960 ( cannot quiesce virtual machine)

This was the only machine being backed up at the time and the host is otherwise under no load, so I dont think this is an I/O problem. Destination is a NAS drive (NFS) on a 1GB link LAN,

Had a search about, and I think it is to do with VSS on the Guest OS.

As per this article: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1018194

Eventvwr on the OS has a single error 11: The driver detected a controller error on \Device\Harddisk2\DR2.

And cmd > vssadmin list writers gives lots of errors:

C:\Users\administrator.EVO>vssadmin list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

Waiting for responses.
These may be delayed if a shadow copy is being prepared.

Writer name: 'Task Scheduler Writer'
   Writer Id: {d61d61c8-d73a-4eee-8cdd-f6f9786b7124}
   Writer Instance Id: {1bddd48e-5052-49db-9b07-b96f96727e6b}
   State: [1] Stable
   Last error: No error

Writer name: 'VSS Metadata Store Writer'
   Writer Id: {75dfb225-e2e4-4d39-9ac9-ffaff65ddf06}
   Writer Instance Id: {088e7a7d-09a8-4cc6-a609-ad90e75ddc93}
   State: [1] Stable
   Last error: No error

Writer name: 'Performance Counters Writer'
   Writer Id: {0bada1de-01a9-4625-8278-69e735f39dd2}
   Writer Instance Id: {f0086dda-9efc-47c5-8eb6-a944c3d09381}
   State: [1] Stable
   Last error: No error

Writer name: 'System Writer'
   Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
   Writer Instance Id: {d6e54a9f-297b-4e7d-9a63-39f445907f33}
   State: [10] Failed
   Last error: Timed out

Writer name: 'FRS Writer'
   Writer Id: {d76f5a28-3092-4589-ba48-2958fb88ce29}
   Writer Instance Id: {dadc4e12-412a-4613-8e55-0e935a1d43ea}
   State: [10] Failed
   Last error: Timed out

Writer name: 'ASR Writer'
   Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
   Writer Instance Id: {b29b4aad-d175-4906-afb0-1da1e9fdfd7b}
   State: [1] Stable
   Last error: No error

Writer name: 'SqlServerWriter'
   Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
   Writer Instance Id: {4af7caa2-af9f-4f87-9d59-8304e50016c7}
   State: [10] Failed
   Last error: Timed out

Writer name: 'NPS VSS Writer'
   Writer Id: {35e81631-13e1-48db-97fc-d5bc721bb18a}
   Writer Instance Id: {9a0cd13c-ccd5-4e2e-94cc-d5afe8d951f2}
   State: [10] Failed
   Last error: Timed out

Writer name: 'Dhcp Jet Writer'
   Writer Id: {be9ac81e-3619-421f-920f-4c6fea9e93ad}
   Writer Instance Id: {f0d61536-8d0d-4ca4-8565-c349d17f3b3b}
   State: [10] Failed
   Last error: Non-retryable error

Writer name: 'Backup Exec Deduplication Storage Writer'
   Writer Id: {55f6d76d-8617-4fc8-81e9-df0de529b555}
   Writer Instance Id: {02b3b2f7-249a-4142-996a-5a4e7be94ae2}
   State: [1] Stable
   Last error: No error

Writer name: 'WMI Writer'
   Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
   Writer Instance Id: {3c24d2c2-7f9a-4f33-aa4f-034ff71962b5}
   State: [10] Failed
   Last error: Timed out

Writer name: 'NTDS'
   Writer Id: {b2014c9e-8711-4c5c-a5a9-3cf384484757}
   Writer Instance Id: {49100f8b-c054-41f3-83c4-d982ddca84de}
   State: [10] Failed
   Last error: Non-retryable error

Writer name: 'COM+ REGDB Writer'
   Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
   Writer Instance Id: {fad48f52-b286-467a-ac64-702b12f625b6}
   State: [1] Stable
   Last error: No error

Writer name: 'Registry Writer'
   Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
   Writer Instance Id: {ed91fa86-31e5-456e-a1e0-c0e1fe9cbc8c}
   State: [1] Stable
   Last error: No error

Writer name: 'Shadow Copy Optimization Writer'
   Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
   Writer Instance Id: {dd54ed26-18e9-4451-9580-1e70873c70f3}
   State: [1] Stable
   Last error: No error

How should I proceed?
0
hongedit
Asked:
hongedit
  • 8
  • 4
  • 3
  • +1
2 Solutions
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Yes, this can happen, try uninstalling VMware Tools, and Re-installing VMware Tools. To registration of VSS again.

You can confirm if it's a Snapshot issue, but Snapshotting with Quiece Guest and Untick Memory in Snapshot Manager, it's liklely to give you the same issue.
0
 
hongeditAuthor Commented:
I get the same error by doing a manual snapshot.

Will try reinstalling VMTools again now
0
 
hongeditAuthor Commented:
Done, same error?
0
Microsoft Certification Exam 74-409

Veeam® is happy to provide the Microsoft community with a study guide prepared by MVP and MCT, Orin Thomas. This guide will take you through each of the exam objectives, helping you to prepare for and pass the examination.

 
Luciano PatrãoICT Senior Infraestructure Engineer Commented:
Hi

Uninstall and install the VMware tools, fix the problem many times.

Is VSS service enable on the VM?

Jail
0
 
hongeditAuthor Commented:
I have already uninstalled VMWare Tools, rebooted, reinstalled and rebooted again.

All services look correct as per VMWare website article.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Okay, the error -3960 ( cannot quiesce virtual machine) is a common error, that can be resolved sometimes by uninstalling and re-installing VMware Tools.

The fault is with the VM, and not vDR, in it's inability to Quieces the VM.
0
 
hongeditAuthor Commented:
But I have already uninstalled and reinstalled VMWare Tools!!
0
 
Luciano PatrãoICT Senior Infraestructure Engineer Commented:
Hi

Well this is the solution that fixes the problem in v4.

Do you have any other snapshots on that VM?

You cannot create a snapshot manually?

Jail
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
The registration process of the COM+ Object on installing VMware Tools is the issue. (it's happened in all versions of ESX/ESXi).
0
 
hongeditAuthor Commented:
BestWay: I am unable to create a snapshot if the Quiece option is ticked, even manually. No snapshots running on any VM's.

Han, how do I fix that?
0
 
Luciano PatrãoICT Senior Infraestructure Engineer Commented:
Hi

Has we said before, most problems can be fix by uninstall and install VMware tools again.
This is a new version of vSphere but the same issue.

Again try this:

Uninstall VMware Tools.
Allow the system to reboot.
Reinstall VMware Tools. Ensure to click Custom Install.
Deselect VSS.

After the installation, you are able to take a snapshot where the quiescing operation is not performed even if specifically requested


Do a custom installation of VMware tools and exclude the VSS drivers. Use the VSS drivers provided by windows(this service need to be enable on Windows) to quiesce the snapshot

Some KBs about this issue:

http://kb.vmware.com/kb/1018194

http://kb.vmware.com/kb/1009073

Take a good attention on this one:
http://kb.vmware.com/kb/1007696

Hope this can help

Jail
0
 
Danny McDanielClinical Systems AnalystCommented:
This is a frequent problem with domain controllers.  http://kb.vmware.com/kb/1000171 (which is listed in Jail's last link)  This kb specifies 2003 in the title but you'll see it references 2008, also.

Doing a restore of a single DC in a multi-DC environment is just going to cause issues with replication and you'll end up doing a lot of work  just to get it to work.  If you want a backup of AD, you're better off using another tool to do it like http://www.petri.co.il/backup-windows-server-2003-active-directory.htm shows.  If you want to be able to bring up a DC quickly should this one crash, you should just keep an updated server template that will be ready to be promoted should you need it.
0
 
hongeditAuthor Commented:
Sorry, I dont mean to be rude but I feel the replies have not been read properly.

As categorically stated in replies 2, 3, 5 and 7 (4 times) I have ALREADY UNINSTALLED AND REINSTALLED VMWARE TOOLS.

I chose "COMPLETE" install.

I still cannot create a manual snapshot if the Quiesce option is ticked.

Danm - I have a single DC environment at the moment. My end goal is to be able to store backups of my VM's which are NOT on the datastores, in case of server failure or VMWare Hypervisor issues like I had recenrtly with a failed ESXi upgrade.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Uninstalling/Reinstalling VMware Tools *CAN* fix the COM+ Object Registration correctly, and this is the easiest method, one our clients needed to uninstall and re-install 26 times!

Otherwise it requires manual intervention to correctly setup the VM. (I have tried on many occasions to take EE users through the process, without success, due to the nature of this postal forum).

If you search EE, you may find several error -3960 that were fixed by uninstalling and reinstalling VMware Tools, however, those that have not been solved, Questions will have been deleted.

I would recommend that you escalate to VMware, because they have the advantage over the EE forum, of being able to WebEx a session, and apply the manual process of registering the COM+ Object which has failed.

Taken from the VDR 2.0 Manual

Backups fails with error -3960 (cannot quiesce virtual machine)

VMware Tools is installed and up to date. If current tools are not installed, uninstall any existing versions of VMware Tools, and then install the correct version of VMware Tools. This may resolve this issue.

If backups continue to fail, try manually creating snapshot of the virtual machine with Snapshot virtual machine's memory unchecked and Quiesce Guest File system checked.

For Windows 2003 and later virtual machines, check if system and application event logs for VSS and application writers related messages. Check if ntbackup or Windows Server Backup can be used in the virtual machine to perform backup using VSS in the guest.

Source
http://pubs.vmware.com/vsphere-50/index.jsp?topic=/com.vmware.datarecovery.admin.doc_20/GUID-748CAB0C-934D-49C9-BE97-875CEF3367C6.html
0
 
hongeditAuthor Commented:
Thanks Han, very informative post.

Jesus 26 times! I've dont it 3 times so far!

Well, not getting any further with this here, I think I might have to bite the bullet and speak to vmware TAC.

Thanks again.
0
 
hongeditAuthor Commented:
Thank you
0

Featured Post

Important Lessons on Recovering from Petya

In their most recent webinar, Skyport Systems explores ways to isolate and protect critical databases to keep the core of your company safe from harm.

  • 8
  • 4
  • 3
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now