Link to home
Start Free TrialLog in
Avatar of joukiejouk
joukiejouk

asked on

Veeam backup keep failing on a particular server.

We have one particular Windows server (an AD sync server that relates to Office 365) that has been failing for days. Three other servers in the same backup group is fine, and can back up with no problem. I removed open snapshots on this particular server but still to no avail. Attached is a screenshot of the error I continue to get. Please advise on what I can do to resolve this issue. There is more than enough space on the C: and storage provided to the VM.

User generated image
Avatar of Seth Simmons
Seth Simmons
Flag of United States of America image

not a veeam issue...it's the vss writer for sql that isn't working causing the veeam backup to fail

is the server low on disk space?  have you tried to reboot?
Avatar of joukiejouk
joukiejouk

ASKER

15 GB free of C: drive space on the server. This is an email server so it is challenging to schedule a reboot at this time. Anything else that can be looked into first before we attempt that?
If you Take a Manual Snapshot of the VM with the quicese option ticked does it  create a snapshot without error ?

SQL and Exchange Servers have high I/O and activity, this could be the cause!
I do understand that Exchange and SQL have high I/O and activity, but we've been backing up this server for a year now without issues up until now.
Can you perform this test ? as requested?

If you Take a Manual Snapshot of the VM with the quicese option ticked does it  create a snapshot without error ?

This is becoming more and more of an issue and reported here on EE!
Unfortunately, I do not have the leisure of creating a snapshot at the moment as this is PROD. And sometimes creating a snapshot could slow down the performance during production time. Anything else I can try?
Well you need to confirm if the above test fails. - this will either confirm if VSS is broken in the VM.

However, VSS also fails because datastores are too slow, or VM is too busy, e.g. High I/O.

So only run one VM backup job at a time, and run the backup job at the quietest time of activity for the VM.

turn off VSS quicese of the VM, remove the sync driver and create crash consistent backups only!

It is a common problem with the SNAPSHOT SYSTEM (VSS).

Not the only question on EE

see here

Veeam Backup and Replication VSS errors on Windows systems
Veeam backup uses the VSS architecture in the background to fetch the application data and create a VSS snapshot. So in order to isolate the issue please run the following command from an elevated command prompt inside the VM and post the output.

VSSADMIN LIST WRITERS
VSSADMIN LIST PROVIDERS

We would be interested in knowing if only the SQL writer is failing or are there multiple writers that are in failed state.
Also try to run a system state backup using Windows Backup from inside the VM and see if it goes through. If not please post the error message.

Regards,
Utshab Mukherjee
Utshab Mukherjee,

Here are the results for the command you mention to execute.

User generated imageUser generated image
Does this help with what is need to troubleshoot this?
change the registry key "NT SERVICE\Sql Writer" from 1 to 0
Restart the SQL VSS writer service
vssadmin list writers  to check if error is resolved
if resolved retry backup
David Johnson,,

Where can i find the "NT SERVICE\Sql Writer" in regedit?
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VSS\VssAccessControl]
Hi joukiejouk,

Thanks for posting the VSSADMIN LIST Writers output.

As per the output it looks like a lot of writers are stuck in waiting for completion state.

Please reboot the server and check if all the writers are stable.

Then re-run the LIST WRITERS command and check the output.

If the issue with SQL writers persists then please try regsirtering these dlls

- SQLRESLD.dll
- SQLVDI.dll
- SQLUNIRL.dll
- SQLSVC.dll

use

regsvr32  sqlvdi.dll and so on for all the other dlls

Regards,
Utshab Mukherjee
App4rent.com
Still having back up issues on this server. I noticed that SqlServerWriter periodically fail (see screenshot)) even after restarting server,  and doing the following:
change the registry key "NT SERVICE\Sql Writer" from 1 to 0
Restart the SQL VSS writer service

Please  advise.

User generated image
I was able to create a manual snapshot with no issue.
and the quicese option was ticked, and snapshot deleted with no error ?

if that is correct, what does that tell you, and what do you deduce from that test ?
So any solution here?
Did snapshot work with the quiesce option ticked?
I never saw an option for that when taking the snapshot.

User generated image
no need to snapshot the virtual machines memory ...

interesting it would seem the HTML5 does not have that option.

please try with the Web Client
Keep getting shock wave errors when i accessing the web client, that is the reason why i am using the html 5 version. But what is it that we are trying to prove here?
Keep getting shock wave errors when i accessing the web client

new question needed for that issue...you could use the older vSphere Client if support for your host.

Snapshot works!

because this is the same function that Backup API uses!

If you can snapshot the VM with no issues or error, the issue is not the VM and VSS!
Keep getting shock wave errors when i accessing the web client

that is a known issue...the latest version of flash (27.0.0.170, also included with chrome 62) breaks the web clients
a lot of complaints about that the last few days

interesting it would seem the HTML5 does not have that option

i think we both know html5 client is feature incomplete; disappointing even more so now that flash breaks the fully functioning web client
Easily fixed, but that's a discussion for a new question! if people had been paying attention, they would have seen the fix I posted!

and that's not Adobe's fix, create a new VM, and don't upgrade Chrome for use with the Web Client, and use that VM for Management with VMware vSphere Web Client!

It was broken 7 days ago, week ago today!
The back up issue was resolved by disabling this option in Veeam.

User generated image
ASKER CERTIFIED SOLUTION
Avatar of Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)
Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)
Flag of United Kingdom of Great Britain and Northern Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial