Hyper-V backup of SQL Server fails

Our backup of a Hyper-V virtual machine running SQL is failing every time.  We're using Windows Server Backup on the Hyper-V host and that's logging the following events every time:

ID: 18012 Source: Hyper-V-VMMS Text: Checkpoint operation for (vmname) failed.
ID: 10150 Source: Hyper-V-VMMS Text: Could not create backup checkpoint for virtual machine (vmname). The shadow copy provider had an unexpected error while trying to process the specified operation.
ID: 10172 Source: Hyper-V-VMMS Text: VSS writers inside virtual machine (vmname) failed to perform BackupComplete to its shadow copy (VSS snapshot) set: A function call was made when the object was in an incorrect state for that function (0x80042301).
ID: 16010 Source: Hyper-V-VMMS Text: The operation failed.


Within the VM itself, the following Application events are logged:

ID: 19 Source: vmicvss Text: Not all the shadow volumes arrived in the guest operating system.
ID: 12293 Source: VSS Text: Volume Shadow Copy Service error: Error calling a routine on a Shadow Copy Provider {74600e39-7dc5-4567-a03b-f091d6c7b092}. Routine details PreFinalCommitSnapshots({22f41230-dd98-4114-a925-58f7fa93b96f}, 3) [hr = 0x800705b4, This operation returned because the timeout period expired.
].
ID: 1 Source: SQLVDI Text: SQLVDI: Loc=SignalAbort. Desc=Client initiates abort. ErrorCode=(0). Process=4200. Thread=9200. Client. Instance=ACT7. VD=Global\{3F4CE388-6942-4F5C-B3EC-545A0FAAEC32}1_SQLVDIMemoryName_0.
ID: 24583 Source: SQLWRITER Text: Sqllib error: OLEDB Error encountered calling ICommandText::Execute. hr = 0x80040e14. SQLSTATE: 42000, Native Error: 3013
Error state: 1, Severity: 16
Source: Microsoft SQL Server Native Client 10.0
Error message: BACKUP DATABASE is terminating abnormally.
SQLSTATE: 42000, Native Error: 3271
Error state: 1, Severity: 16
Source: Microsoft SQL Server Native Client 10.0
Error message: A nonrecoverable I/O error occurred on file "{3F4CE388-6942-4F5C-B3EC-545A0FAAEC32}2:" 995(failed to retrieve text for this error. Reason: 15105).
SQLSTATE: 01000, Native Error: 4035
Error state: 1, Severity: 0
Source: Microsoft SQL Server Native Client 10.0
Error message: Processed 0 pages for database 'msdb', file 'MSDBData' on file 1.

The host is Server 2012 R2, the guest is Server 2008 R2 running SQL 2008.
All other VMs are backing up fine from this host.  We've installed the KB2996928v2 hotfix on the SQL VM and KB3013769 on the host.  I've checked the SQLVDI.dll in use (both 32-bit and 64-bit) and it's the correct version for SQL 2008.

Thanks in advance.

KP146958
mtxitAsked:
Who is Participating?

Improve company productivity with a Business Account.Sign Up

x
 
Philip ElderConnect With a Mentor Technical Architect - HA/Compute/StorageCommented:
Your VSS services are conflicting with each other.

We do _not_ use Windows Server Backup on a Hyper-V host in any setting whether standalone or clustered.

If you must use WSB to back up the VM then do so _within_ the guest.
0
 
mtxitAuthor Commented:
We backup many Hyper-V hosts in this manner including others with guests running SQL servers - none of them have an issue like this, and we don't have to backup from within the guest anywhere else.
0
 
Philip ElderConnect With a Mentor Technical Architect - HA/Compute/StorageCommented:
Our experience has been different on standalone and clustered hosts. Note that we've been building on Hyper-V for both standalone and clustered virtualization solutions since 2008 RTM.

VSS can be very fragile and finicky. We've had hosts go full-stop, databases get toasted, and mail server database problems. Because of the problems experienced we won't back up the host unless using a third party product such as Veeam. That's it.
0
Problems using Powershell and Active Directory?

Managing Active Directory does not always have to be complicated.  If you are spending more time trying instead of doing, then it's time to look at something else. For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why

 
mtxitAuthor Commented:
This got listed as abandoned as I didn't understand the process and was waiting for a usable response from another user.  I don't accept the responses from Philip Elder as I'm requesting assistance with an issue with a using a Microsoft product for which it is designed, therefore 'use a different tool' is not a helpful response.
0
 
Philip ElderConnect With a Mentor Technical Architect - HA/Compute/StorageCommented:
We've been working with Hyper-V since Longhorn. We've had standalone and clustered Hyper-V in production since 2008 RTM.

We've run the gamut as far as backups are concerned.

Take our experience for what it's worth. The fact that some are working, and have been for a while, does not negate the fact that WSB was not designed to back up full guests from get-go. The VSS writer install on 2008 RTM was a pain. Getting it to work in 2008 R2 was a pain as well. Hyper-V VSS Writer in 2012 RTM/R2 is not a lot better.

VSS is a finicky service. As mentioned above, they are in collision with each other. Straightening out the VSS structures to allow the proper backup of guest VHD/VHDXs from the host requires either Veeam or DPM (apparently from my discussions with the PM last MVP Summit in Redmond).

VSS Big Picture:

Host Hyper-V VSS --> In-Host HV VSS --> VSC --> App VSS (of which there are a plethora)

Being a bearer of bad news, or an answer that is not acceptable to the OP, does not negate what has been posted.
0
 
Anthony PerkinsCommented:
I'm requesting assistance with an issue with a using a Microsoft product for which it is designed, therefore 'use a different tool' is not a helpful response.
Then you may want to read up on The experts told me “you can not do that”. What do I do now?
0
 
Anthony PerkinsCommented:
I have requested an EE Moderator to re-open the question as you appear to have selected the wrong solution.  

This got listed as abandoned as I didn't understand the process and was waiting for a usable response from another user.
I realize that closing questions appropriately, may be a daunting task for you so feel free to ask how this is accomplished correctly.
0
 
South ModModeratorCommented:
mtxit,

I've reawarded points in this question. As was pointed out, an answer of "You can't do that" is an acceptable solution, and does not warrant a low grade.

Please see this link for more information: http://support.experts-exchange.com/customer/portal/articles/755964-the-experts-told-me-%E2%80%9Cyou-can-not-do-that%E2%80%9D-what-do-i-do-now-

SouthMod
Community Support Moderator
0
 
mrichardsintCommented:
While I don't want to discount experiences Philip Elder may have had with Windows Server Backup, the logic he used is flawed.  If his argument is that one shouldn't use Windows Server Backup because it uses Microsoft VSS and VSS is finicky, then he can't well recommend you use a third-party software such as Veeam since they use Microsoft VSS as well!

I agree VSS can be troublesome, but backing up full guest VMs from a Hyper-V host with Windows Server Backup is an explicit functionality of that program and is fully supported by Microsoft.  It should be possible and the answer of "no, it's not feasible" simply is not correct.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.