Solved

VSS Error with VM Snapshots

Posted on 2011-09-03
16
3,751 Views
Last Modified: 2012-05-12
VMware 5, VDR 2.

1 VM is unable to be backed up because of error:

Failed to create snapshot for DC, error -3960 ( cannot quiesce virtual machine)

This was the only machine being backed up at the time and the host is otherwise under no load, so I dont think this is an I/O problem. Destination is a NAS drive (NFS) on a 1GB link LAN,

Had a search about, and I think it is to do with VSS on the Guest OS.

As per this article: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1018194

Eventvwr on the OS has a single error 11: The driver detected a controller error on \Device\Harddisk2\DR2.

And cmd > vssadmin list writers gives lots of errors:

C:\Users\administrator.EVO>vssadmin list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.

Waiting for responses.
These may be delayed if a shadow copy is being prepared.

Writer name: 'Task Scheduler Writer'
   Writer Id: {d61d61c8-d73a-4eee-8cdd-f6f9786b7124}
   Writer Instance Id: {1bddd48e-5052-49db-9b07-b96f96727e6b}
   State: [1] Stable
   Last error: No error

Writer name: 'VSS Metadata Store Writer'
   Writer Id: {75dfb225-e2e4-4d39-9ac9-ffaff65ddf06}
   Writer Instance Id: {088e7a7d-09a8-4cc6-a609-ad90e75ddc93}
   State: [1] Stable
   Last error: No error

Writer name: 'Performance Counters Writer'
   Writer Id: {0bada1de-01a9-4625-8278-69e735f39dd2}
   Writer Instance Id: {f0086dda-9efc-47c5-8eb6-a944c3d09381}
   State: [1] Stable
   Last error: No error

Writer name: 'System Writer'
   Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
   Writer Instance Id: {d6e54a9f-297b-4e7d-9a63-39f445907f33}
   State: [10] Failed
   Last error: Timed out

Writer name: 'FRS Writer'
   Writer Id: {d76f5a28-3092-4589-ba48-2958fb88ce29}
   Writer Instance Id: {dadc4e12-412a-4613-8e55-0e935a1d43ea}
   State: [10] Failed
   Last error: Timed out

Writer name: 'ASR Writer'
   Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
   Writer Instance Id: {b29b4aad-d175-4906-afb0-1da1e9fdfd7b}
   State: [1] Stable
   Last error: No error

Writer name: 'SqlServerWriter'
   Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
   Writer Instance Id: {4af7caa2-af9f-4f87-9d59-8304e50016c7}
   State: [10] Failed
   Last error: Timed out

Writer name: 'NPS VSS Writer'
   Writer Id: {35e81631-13e1-48db-97fc-d5bc721bb18a}
   Writer Instance Id: {9a0cd13c-ccd5-4e2e-94cc-d5afe8d951f2}
   State: [10] Failed
   Last error: Timed out

Writer name: 'Dhcp Jet Writer'
   Writer Id: {be9ac81e-3619-421f-920f-4c6fea9e93ad}
   Writer Instance Id: {f0d61536-8d0d-4ca4-8565-c349d17f3b3b}
   State: [10] Failed
   Last error: Non-retryable error

Writer name: 'Backup Exec Deduplication Storage Writer'
   Writer Id: {55f6d76d-8617-4fc8-81e9-df0de529b555}
   Writer Instance Id: {02b3b2f7-249a-4142-996a-5a4e7be94ae2}
   State: [1] Stable
   Last error: No error

Writer name: 'WMI Writer'
   Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
   Writer Instance Id: {3c24d2c2-7f9a-4f33-aa4f-034ff71962b5}
   State: [10] Failed
   Last error: Timed out

Writer name: 'NTDS'
   Writer Id: {b2014c9e-8711-4c5c-a5a9-3cf384484757}
   Writer Instance Id: {49100f8b-c054-41f3-83c4-d982ddca84de}
   State: [10] Failed
   Last error: Non-retryable error

Writer name: 'COM+ REGDB Writer'
   Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
   Writer Instance Id: {fad48f52-b286-467a-ac64-702b12f625b6}
   State: [1] Stable
   Last error: No error

Writer name: 'Registry Writer'
   Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
   Writer Instance Id: {ed91fa86-31e5-456e-a1e0-c0e1fe9cbc8c}
   State: [1] Stable
   Last error: No error

Writer name: 'Shadow Copy Optimization Writer'
   Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
   Writer Instance Id: {dd54ed26-18e9-4451-9580-1e70873c70f3}
   State: [1] Stable
   Last error: No error

How should I proceed?
0
Comment
Question by:hongedit
  • 8
  • 4
  • 3
  • +1
16 Comments
 
LVL 120
ID: 36477977
Yes, this can happen, try uninstalling VMware Tools, and Re-installing VMware Tools. To registration of VSS again.

You can confirm if it's a Snapshot issue, but Snapshotting with Quiece Guest and Untick Memory in Snapshot Manager, it's liklely to give you the same issue.
0
 
LVL 1

Author Comment

by:hongedit
ID: 36478019
I get the same error by doing a manual snapshot.

Will try reinstalling VMTools again now
0
 
LVL 1

Author Comment

by:hongedit
ID: 36478069
Done, same error?
0
What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

 
LVL 23

Expert Comment

by:Luciano Patrão
ID: 36478258
Hi

Uninstall and install the VMware tools, fix the problem many times.

Is VSS service enable on the VM?

Jail
0
 
LVL 1

Author Comment

by:hongedit
ID: 36478388
I have already uninstalled VMWare Tools, rebooted, reinstalled and rebooted again.

All services look correct as per VMWare website article.
0
 
LVL 120
ID: 36478444
Okay, the error -3960 ( cannot quiesce virtual machine) is a common error, that can be resolved sometimes by uninstalling and re-installing VMware Tools.

The fault is with the VM, and not vDR, in it's inability to Quieces the VM.
0
 
LVL 1

Author Comment

by:hongedit
ID: 36478697
But I have already uninstalled and reinstalled VMWare Tools!!
0
 
LVL 23

Expert Comment

by:Luciano Patrão
ID: 36478749
Hi

Well this is the solution that fixes the problem in v4.

Do you have any other snapshots on that VM?

You cannot create a snapshot manually?

Jail
0
 
LVL 120
ID: 36478843
The registration process of the COM+ Object on installing VMware Tools is the issue. (it's happened in all versions of ESX/ESXi).
0
 
LVL 1

Author Comment

by:hongedit
ID: 36479123
BestWay: I am unable to create a snapshot if the Quiece option is ticked, even manually. No snapshots running on any VM's.

Han, how do I fix that?
0
 
LVL 23

Expert Comment

by:Luciano Patrão
ID: 36479197
Hi

Has we said before, most problems can be fix by uninstall and install VMware tools again.
This is a new version of vSphere but the same issue.

Again try this:

Uninstall VMware Tools.
Allow the system to reboot.
Reinstall VMware Tools. Ensure to click Custom Install.
Deselect VSS.

After the installation, you are able to take a snapshot where the quiescing operation is not performed even if specifically requested


Do a custom installation of VMware tools and exclude the VSS drivers. Use the VSS drivers provided by windows(this service need to be enable on Windows) to quiesce the snapshot

Some KBs about this issue:

http://kb.vmware.com/kb/1018194

http://kb.vmware.com/kb/1009073

Take a good attention on this one:
http://kb.vmware.com/kb/1007696

Hope this can help

Jail
0
 
LVL 16

Assisted Solution

by:danm66
danm66 earned 250 total points
ID: 36479607
This is a frequent problem with domain controllers.  http://kb.vmware.com/kb/1000171 (which is listed in Jail's last link)  This kb specifies 2003 in the title but you'll see it references 2008, also.

Doing a restore of a single DC in a multi-DC environment is just going to cause issues with replication and you'll end up doing a lot of work  just to get it to work.  If you want a backup of AD, you're better off using another tool to do it like http://www.petri.co.il/backup-windows-server-2003-active-directory.htm shows.  If you want to be able to bring up a DC quickly should this one crash, you should just keep an updated server template that will be ready to be promoted should you need it.
0
 
LVL 1

Author Comment

by:hongedit
ID: 36480132
Sorry, I dont mean to be rude but I feel the replies have not been read properly.

As categorically stated in replies 2, 3, 5 and 7 (4 times) I have ALREADY UNINSTALLED AND REINSTALLED VMWARE TOOLS.

I chose "COMPLETE" install.

I still cannot create a manual snapshot if the Quiesce option is ticked.

Danm - I have a single DC environment at the moment. My end goal is to be able to store backups of my VM's which are NOT on the datastores, in case of server failure or VMWare Hypervisor issues like I had recenrtly with a failed ESXi upgrade.
0
 
LVL 120

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 250 total points
ID: 36480287
Uninstalling/Reinstalling VMware Tools *CAN* fix the COM+ Object Registration correctly, and this is the easiest method, one our clients needed to uninstall and re-install 26 times!

Otherwise it requires manual intervention to correctly setup the VM. (I have tried on many occasions to take EE users through the process, without success, due to the nature of this postal forum).

If you search EE, you may find several error -3960 that were fixed by uninstalling and reinstalling VMware Tools, however, those that have not been solved, Questions will have been deleted.

I would recommend that you escalate to VMware, because they have the advantage over the EE forum, of being able to WebEx a session, and apply the manual process of registering the COM+ Object which has failed.

Taken from the VDR 2.0 Manual

Backups fails with error -3960 (cannot quiesce virtual machine)

VMware Tools is installed and up to date. If current tools are not installed, uninstall any existing versions of VMware Tools, and then install the correct version of VMware Tools. This may resolve this issue.

If backups continue to fail, try manually creating snapshot of the virtual machine with Snapshot virtual machine's memory unchecked and Quiesce Guest File system checked.

For Windows 2003 and later virtual machines, check if system and application event logs for VSS and application writers related messages. Check if ntbackup or Windows Server Backup can be used in the virtual machine to perform backup using VSS in the guest.

Source
http://pubs.vmware.com/vsphere-50/index.jsp?topic=/com.vmware.datarecovery.admin.doc_20/GUID-748CAB0C-934D-49C9-BE97-875CEF3367C6.html
0
 
LVL 1

Author Comment

by:hongedit
ID: 36480326
Thanks Han, very informative post.

Jesus 26 times! I've dont it 3 times so far!

Well, not getting any further with this here, I think I might have to bite the bullet and speak to vmware TAC.

Thanks again.
0
 
LVL 1

Author Closing Comment

by:hongedit
ID: 36504351
Thank you
0

Featured Post

NEW Veeam Agent for Microsoft Windows

Backup and recover physical and cloud-based servers and workstations, as well as endpoint devices that belong to remote users. Avoid downtime and data loss quickly and easily for Windows-based physical or public cloud-based workloads!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Veeam Backup & Replication has added a new integration – Veeam Backup for Microsoft Office 365.  In this blog, we will discuss how you can benefit from Office 365 email backup with the Veeam’s new product and try to shed some light on the needs and …
This article outlines why you need to choose a backup solution that protects your entire environment – including your VMware ESXi and Microsoft Hyper-V virtualization hosts – not just your virtual machines.
Teach the user how to delpoy the vCenter Server Appliance and how to configure its network settings Deploy OVF: Open VM console and configure networking:
Teach the user how to rename, unmount, delete and upgrade VMFS datastores. Open vSphere Web Client: Rename VMFS and NFS datastores: Upgrade VMFS-3 volume to VMFS-5: Unmount VMFS datastore: Delete a VMFS datastore:

696 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question