Solved

VSS Error with VM Snapshots

Posted on 2011-09-03
16
3,688 Views
Last Modified: 2012-05-12
VMware 5, VDR 2.

1 VM is unable to be backed up because of error:

Failed to create snapshot for DC, error -3960 ( cannot quiesce virtual machine)

This was the only machine being backed up at the time and the host is otherwise under no load, so I dont think this is an I/O problem. Destination is a NAS drive (NFS) on a 1GB link LAN,

Had a search about, and I think it is to do with VSS on the Guest OS.

As per this article: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1018194

Eventvwr on the OS has a single error 11: The driver detected a controller error on \Device\Harddisk2\DR2.

And cmd > vssadmin list writers gives lots of errors:

C:\Users\administrator.EVO>vssadmin list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

Waiting for responses.
These may be delayed if a shadow copy is being prepared.

Writer name: 'Task Scheduler Writer'
   Writer Id: {d61d61c8-d73a-4eee-8cdd-f6f9786b7124}
   Writer Instance Id: {1bddd48e-5052-49db-9b07-b96f96727e6b}
   State: [1] Stable
   Last error: No error

Writer name: 'VSS Metadata Store Writer'
   Writer Id: {75dfb225-e2e4-4d39-9ac9-ffaff65ddf06}
   Writer Instance Id: {088e7a7d-09a8-4cc6-a609-ad90e75ddc93}
   State: [1] Stable
   Last error: No error

Writer name: 'Performance Counters Writer'
   Writer Id: {0bada1de-01a9-4625-8278-69e735f39dd2}
   Writer Instance Id: {f0086dda-9efc-47c5-8eb6-a944c3d09381}
   State: [1] Stable
   Last error: No error

Writer name: 'System Writer'
   Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
   Writer Instance Id: {d6e54a9f-297b-4e7d-9a63-39f445907f33}
   State: [10] Failed
   Last error: Timed out

Writer name: 'FRS Writer'
   Writer Id: {d76f5a28-3092-4589-ba48-2958fb88ce29}
   Writer Instance Id: {dadc4e12-412a-4613-8e55-0e935a1d43ea}
   State: [10] Failed
   Last error: Timed out

Writer name: 'ASR Writer'
   Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
   Writer Instance Id: {b29b4aad-d175-4906-afb0-1da1e9fdfd7b}
   State: [1] Stable
   Last error: No error

Writer name: 'SqlServerWriter'
   Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
   Writer Instance Id: {4af7caa2-af9f-4f87-9d59-8304e50016c7}
   State: [10] Failed
   Last error: Timed out

Writer name: 'NPS VSS Writer'
   Writer Id: {35e81631-13e1-48db-97fc-d5bc721bb18a}
   Writer Instance Id: {9a0cd13c-ccd5-4e2e-94cc-d5afe8d951f2}
   State: [10] Failed
   Last error: Timed out

Writer name: 'Dhcp Jet Writer'
   Writer Id: {be9ac81e-3619-421f-920f-4c6fea9e93ad}
   Writer Instance Id: {f0d61536-8d0d-4ca4-8565-c349d17f3b3b}
   State: [10] Failed
   Last error: Non-retryable error

Writer name: 'Backup Exec Deduplication Storage Writer'
   Writer Id: {55f6d76d-8617-4fc8-81e9-df0de529b555}
   Writer Instance Id: {02b3b2f7-249a-4142-996a-5a4e7be94ae2}
   State: [1] Stable
   Last error: No error

Writer name: 'WMI Writer'
   Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
   Writer Instance Id: {3c24d2c2-7f9a-4f33-aa4f-034ff71962b5}
   State: [10] Failed
   Last error: Timed out

Writer name: 'NTDS'
   Writer Id: {b2014c9e-8711-4c5c-a5a9-3cf384484757}
   Writer Instance Id: {49100f8b-c054-41f3-83c4-d982ddca84de}
   State: [10] Failed
   Last error: Non-retryable error

Writer name: 'COM+ REGDB Writer'
   Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
   Writer Instance Id: {fad48f52-b286-467a-ac64-702b12f625b6}
   State: [1] Stable
   Last error: No error

Writer name: 'Registry Writer'
   Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
   Writer Instance Id: {ed91fa86-31e5-456e-a1e0-c0e1fe9cbc8c}
   State: [1] Stable
   Last error: No error

Writer name: 'Shadow Copy Optimization Writer'
   Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
   Writer Instance Id: {dd54ed26-18e9-4451-9580-1e70873c70f3}
   State: [1] Stable
   Last error: No error

How should I proceed?
0
Comment
Question by:hongedit
  • 8
  • 4
  • 3
  • +1
16 Comments
 
LVL 117
ID: 36477977
Yes, this can happen, try uninstalling VMware Tools, and Re-installing VMware Tools. To registration of VSS again.

You can confirm if it's a Snapshot issue, but Snapshotting with Quiece Guest and Untick Memory in Snapshot Manager, it's liklely to give you the same issue.
0
 
LVL 1

Author Comment

by:hongedit
ID: 36478019
I get the same error by doing a manual snapshot.

Will try reinstalling VMTools again now
0
 
LVL 1

Author Comment

by:hongedit
ID: 36478069
Done, same error?
0
 
LVL 22

Expert Comment

by:Luciano Patrão
ID: 36478258
Hi

Uninstall and install the VMware tools, fix the problem many times.

Is VSS service enable on the VM?

Jail
0
 
LVL 1

Author Comment

by:hongedit
ID: 36478388
I have already uninstalled VMWare Tools, rebooted, reinstalled and rebooted again.

All services look correct as per VMWare website article.
0
 
LVL 117
ID: 36478444
Okay, the error -3960 ( cannot quiesce virtual machine) is a common error, that can be resolved sometimes by uninstalling and re-installing VMware Tools.

The fault is with the VM, and not vDR, in it's inability to Quieces the VM.
0
 
LVL 1

Author Comment

by:hongedit
ID: 36478697
But I have already uninstalled and reinstalled VMWare Tools!!
0
 
LVL 22

Expert Comment

by:Luciano Patrão
ID: 36478749
Hi

Well this is the solution that fixes the problem in v4.

Do you have any other snapshots on that VM?

You cannot create a snapshot manually?

Jail
0
Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

 
LVL 117
ID: 36478843
The registration process of the COM+ Object on installing VMware Tools is the issue. (it's happened in all versions of ESX/ESXi).
0
 
LVL 1

Author Comment

by:hongedit
ID: 36479123
BestWay: I am unable to create a snapshot if the Quiece option is ticked, even manually. No snapshots running on any VM's.

Han, how do I fix that?
0
 
LVL 22

Expert Comment

by:Luciano Patrão
ID: 36479197
Hi

Has we said before, most problems can be fix by uninstall and install VMware tools again.
This is a new version of vSphere but the same issue.

Again try this:

Uninstall VMware Tools.
Allow the system to reboot.
Reinstall VMware Tools. Ensure to click Custom Install.
Deselect VSS.

After the installation, you are able to take a snapshot where the quiescing operation is not performed even if specifically requested


Do a custom installation of VMware tools and exclude the VSS drivers. Use the VSS drivers provided by windows(this service need to be enable on Windows) to quiesce the snapshot

Some KBs about this issue:

http://kb.vmware.com/kb/1018194

http://kb.vmware.com/kb/1009073

Take a good attention on this one:
http://kb.vmware.com/kb/1007696

Hope this can help

Jail
0
 
LVL 16

Assisted Solution

by:danm66
danm66 earned 250 total points
ID: 36479607
This is a frequent problem with domain controllers.  http://kb.vmware.com/kb/1000171 (which is listed in Jail's last link)  This kb specifies 2003 in the title but you'll see it references 2008, also.

Doing a restore of a single DC in a multi-DC environment is just going to cause issues with replication and you'll end up doing a lot of work  just to get it to work.  If you want a backup of AD, you're better off using another tool to do it like http://www.petri.co.il/backup-windows-server-2003-active-directory.htm shows.  If you want to be able to bring up a DC quickly should this one crash, you should just keep an updated server template that will be ready to be promoted should you need it.
0
 
LVL 1

Author Comment

by:hongedit
ID: 36480132
Sorry, I dont mean to be rude but I feel the replies have not been read properly.

As categorically stated in replies 2, 3, 5 and 7 (4 times) I have ALREADY UNINSTALLED AND REINSTALLED VMWARE TOOLS.

I chose "COMPLETE" install.

I still cannot create a manual snapshot if the Quiesce option is ticked.

Danm - I have a single DC environment at the moment. My end goal is to be able to store backups of my VM's which are NOT on the datastores, in case of server failure or VMWare Hypervisor issues like I had recenrtly with a failed ESXi upgrade.
0
 
LVL 117

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE) earned 250 total points
ID: 36480287
Uninstalling/Reinstalling VMware Tools *CAN* fix the COM+ Object Registration correctly, and this is the easiest method, one our clients needed to uninstall and re-install 26 times!

Otherwise it requires manual intervention to correctly setup the VM. (I have tried on many occasions to take EE users through the process, without success, due to the nature of this postal forum).

If you search EE, you may find several error -3960 that were fixed by uninstalling and reinstalling VMware Tools, however, those that have not been solved, Questions will have been deleted.

I would recommend that you escalate to VMware, because they have the advantage over the EE forum, of being able to WebEx a session, and apply the manual process of registering the COM+ Object which has failed.

Taken from the VDR 2.0 Manual

Backups fails with error -3960 (cannot quiesce virtual machine)

VMware Tools is installed and up to date. If current tools are not installed, uninstall any existing versions of VMware Tools, and then install the correct version of VMware Tools. This may resolve this issue.

If backups continue to fail, try manually creating snapshot of the virtual machine with Snapshot virtual machine's memory unchecked and Quiesce Guest File system checked.

For Windows 2003 and later virtual machines, check if system and application event logs for VSS and application writers related messages. Check if ntbackup or Windows Server Backup can be used in the virtual machine to perform backup using VSS in the guest.

Source
http://pubs.vmware.com/vsphere-50/index.jsp?topic=/com.vmware.datarecovery.admin.doc_20/GUID-748CAB0C-934D-49C9-BE97-875CEF3367C6.html
0
 
LVL 1

Author Comment

by:hongedit
ID: 36480326
Thanks Han, very informative post.

Jesus 26 times! I've dont it 3 times so far!

Well, not getting any further with this here, I think I might have to bite the bullet and speak to vmware TAC.

Thanks again.
0
 
LVL 1

Author Closing Comment

by:hongedit
ID: 36504351
Thank you
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

One of the new features of a version 7.0 or later virtual machine, supported in VMware vSphere 4.1, 5.0 or the VMware vSphere Hypervisor ESXi 4.1, ESXi 5.0 often overlooked by VMware Administrators is the ability to add and connect USB devices conne…
This article is an update and follow-up of my previous article:   Storage 101: common concepts in the IT enterprise storage This time, I expand on more frequently used storage concepts.
Teach the user how to edit .vmx files to add advanced configuration options Open vSphere Web Client: Edit Settings for a VM: Choose VM Options -> Advanced: Add Configuration Parameters:
Teach the user how to use configure the vCenter Server storage filters Open vSphere Web Client:  Navigate to vCenter Server Advanced Settings: Add the four vCenter Server storage filters: Review the advanced settings: Modify the values of the four v…

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now