VEEAM backup jobs periodically fail with VSS error
I am running VEEAM Backup & Replication 9.5 on a Windows Server 2012 R2. VEEAM is configured to backup VM's from 2 Hyper-V Servers that are running Windows Server Data Center 2012 R2. While all VM's backup successfully 'on occasion', often, I get the following error:
Failed to create snapshot (Microsoft Software Shadow Copy provider 1.0) (mode: Veeam application-aware processing) Details: Writer 'Microsoft Hyper-V VSS Writer' is failed at 'VSS_WS_FAILED_AT_PREPARE_
I go to the VM Server identified, run the command "vssadmin list writers", and see that a number of VSS entries says 'timed out. I stop/start the Volume Shadow Copy service then restart the server. The backup jog then completes successfully. This issue will occur again after the 2nd or 3rd successful backup and again, I have to restart the server to get successful backups.
Obviously I cannot continue restarting these affected servers (all of which has databases including Exchange Server 2013). Is there anything that would likely cause the VSS services to periodically time out and fail a backup? Thanks
Failed to create snapshot (Microsoft Software Shadow Copy provider 1.0) (mode: Veeam application-aware processing) Details: Writer 'Microsoft Hyper-V VSS Writer' is failed at 'VSS_WS_FAILED_AT_PREPARE_
I go to the VM Server identified, run the command "vssadmin list writers", and see that a number of VSS entries says 'timed out. I stop/start the Volume Shadow Copy service then restart the server. The backup jog then completes successfully. This issue will occur again after the 2nd or 3rd successful backup and again, I have to restart the server to get successful backups.
Obviously I cannot continue restarting these affected servers (all of which has databases including Exchange Server 2013). Is there anything that would likely cause the VSS services to periodically time out and fail a backup? Thanks
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Blue Street Tech
i updated my last post...please refresh to see the changes.
ASKER
Because we are not running SQL Server 2012, only instances for various programs and I do not currently have a test environment, I will run the registry check and look for .bak entries. Thanks
ASKER
Nope, no .bak entries.
ASKER
Yes, we have 'Application Aware' checked.
This may apply then, have you checked a known issue documented in this Microsoft KB: http://support2.microsoft.com/kb/2987610
If so, the solution is to run this query to see if the issue is present:
SELECT 1 FROM SYS.AVAILABILITY_REPLICAS AS AR INNER JOIN SYS.DATABASES AS D ON DB_ID(N'xxxx') = D.DATABASE_ID AND AR.REPLICA_ID = D.REPLICA_ID INNER JOIN SYS.DM_HADR_AVAILABILITY_R
If you receive the following, this confirms the issue. Invalid object name 'SYS.AVAILABILITY_REPLICAS
This issue is resolved by the following Microsoft KB: http://support2.microsoft.com/kb/2983175
If so, the solution is to run this query to see if the issue is present:
SELECT 1 FROM SYS.AVAILABILITY_REPLICAS AS AR INNER JOIN SYS.DATABASES AS D ON DB_ID(N'xxxx') = D.DATABASE_ID AND AR.REPLICA_ID = D.REPLICA_ID INNER JOIN SYS.DM_HADR_AVAILABILITY_R
If you receive the following, this confirms the issue. Invalid object name 'SYS.AVAILABILITY_REPLICAS
This issue is resolved by the following Microsoft KB: http://support2.microsoft.com/kb/2983175
ASKER
Yes, I ran the query and received the following error: Error:208, Severity:16, State:1
Invalid object name 'SYS.AVAILABILITY_REPLICAS
This VM Server is running on Windows 2008 R2. Is this kb suggesting that I install SQL Server 2012?
Invalid object name 'SYS.AVAILABILITY_REPLICAS
This VM Server is running on Windows 2008 R2. Is this kb suggesting that I install SQL Server 2012?
No, SQL runs on 2012. It is not suggesting you install it...it is a patch for it. So if you don't have it installed don't worry about it. I don't know your infrastructure so please disregard if you don't have SQL installed. I mentioned it because I've seen this issue with SQL server and it was a Microsoft issue.
ASKER
Got it.... Thanks
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Veeam ahave a support team so it is better to lock a call (regeister a ticket)
here are some issue may you need to check first:
1 it's too much backing up to drives at once we'd get that error "forced closed"
2 Replication of local backup and size.
3 Communication between your site and beckup point.
here are some issue may you need to check first:
1 it's too much backing up to drives at once we'd get that error "forced closed"
2 Replication of local backup and size.
3 Communication between your site and beckup point.
Good job