VM Backups failing due to VSS writers failing

We have a 2012 R2 Host Server that had been backing up 100% Using Altaro Software. Now we are getting failed backups. We have found that the "Microsoft Hyper-V VSS Writer" keeps failing. During our investigations we have found that we appear to have a corrupt file (Catsrvut.dll) being part of the COM process we have tried to fix the corruption. The error message is below. If we just run "SFC /SCANNOW" the results say everything is fine but it clearly is not if you runf SCANFILE. Running this command also comes back with a clean report "DISM.exe /Online /Cleanup-image /Restorehealth"

It seems that this file is not easy to fix so we are asking for some assistance to resolve this corruption. Thanks in advance guys.

"C:\Windows\system32>sfc /scanfile=c:\windows\system32\catsrvut.dll

Windows Resource Protection found corrupt files but was unable to fix some
of them. Details are included in the CBS.Log windir\Logs\CBS\CBS.log. For
example C:\Windows\Logs\CBS\CBS.log. Note that logging is currently not
supported in offline servicing scenarios."
dilesAsked:
Who is Participating?
 
dilesAuthor Commented:
Phillip,

I have found the answer to my question. Copy and paste was not the solution as you suggested which is why I asked the question. I followed the instruction in an excellent article found in this link.  https://social.technet.microsoft.com/Forums/en-US/c203334f-7ae5-4716-aeb5-0fa3d60da216/kb2919355-issues-being-investigatedtracked?forum=winserver8gen

Backups are now working as expected. Many thanks for your input.

Regards
Dave
0
 
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
Make sure the host based backup is not running at the same time or within ~10 minutes of any in-guest VSS (Volume Shadow Copy snapshots being the main culprit here) snapshots.

Make sure all of the guest partitions have at least 30% space free to allow for the VSS snapshot to run correctly,

Make sure the VSS snapshot cache is not full. Delete them all and start fresh if they are.

Contact Altaro. This is a common issue.
0
 
dilesAuthor Commented:
We are Gold Partners with Altaro, as this is a System issue its up to us to Fix. We have already checked all your suggestions and more. My question is really the way to resolve the corrupt file. As it is part of the COM process we suspect this is why the Hyper-V VSS writer fails, as we do not find this very many times. Yes it can fail but after a reset its normally fine. This one keeps failing after an attempted backup.

Thanks,
Dave
0
Improve Your Query Performance Tuning

In this FREE six-day email course, you'll learn from Janis Griffin, Database Performance Evangelist. She'll teach 12 steps that you can use to optimize your queries as much as possible and see measurable results in your work. Get started today!

 
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
Suggestion: Remove Altaro and install ShadowProtect by StorageCraft. The trial is 30 days. Run backups to see if the error occurs. If it does not, chase it up with Altaro.

I don't believe this is a system issue. It's an issue with the way Altaro handles VSS snapshots to create its backups. We run with Veeam and ShadowProtect and do not experience these kinds of problems ... as long as we either keep the partition 30% free or set up a dedicated VHDX just for the VSS snapshots that the backups use.
0
 
dilesAuthor Commented:
Philip,

Thanks for this but it is a system error. We have used Altaro for a few years now and have had great success with it. We have never had the "Microsoft Hyper-V VSS Writer" writer fail continually like this one is doing. I ran the "Scanfile" over another server and you will see the results below. The first is not the troubled Server.

Expected Correct Result

C:\Windows\system32>sfc /scanfile=c:\windows\system32\catsrvut.dll

Windows Resource Protection did not find any integrity violations.

Troubled Server
C:\Windows\system32>sfc /scanfile=c:\windows\system32\catsrvut.dll

 Windows Resource Protection found corrupt files but was unable to fix some
 of them. Details are included in the CBS.Log windir\Logs\CBS\CBS.log. For
 example C:\Windows\Logs\CBS\CBS.log. Note that logging is currently not
 supported in offline servicing scenarios.

My question was really how to resolve this issue with the Catsrvut.dll. I only mentioned the failing backups to give it perspective.

I have tried various fixes myself including updating the server.

Regards
Dave
0
 
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
Copy good file over bad file. Verify in Properties for each that they are the same version and size.
0
 
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
Great find! Which point was the fix/workaround please?
0
 
dilesAuthor Commented:
Found the answer in the link provided in my last post.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.