Link to home
Create AccountLog in
Avatar of Daniel Booker
Daniel BookerFlag for United States of America

asked on

SBS 2011 server backup failing.

I have a SBS 2011 server that recently the server backups stopped working. I believe the owner powered off the server during an image backup.
Most things pointed to running "PSConfig.exe -cmd upgrade -inplace b2b -force -cmd applicationcontent -install -cmd installfeatures" which has already been ran... I have even reran it just in case.
The error from windows event viewer
The backup operation that started at '‎2017‎-‎04‎-‎13T07:00:45.445000000Z' has failed because the Volume Shadow Copy Service operation to create a shadow copy of the volumes being backed up failed with following error code '2155348129'. Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.

I looked at the vss writers and noted that only 1 has a problem...
Writer name: 'SqlServerWriter'

Writer ID: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
Writer Instance ID: {57af718e-2f96-4aa0-a04f-0d060bf2aafa}
State: [8] Failed
Last error: Inconsistent shadow copy
Avatar of Larry Struckmeyer MVP
Larry Struckmeyer MVP
Flag of United States of America image

VSS writers will do this from time to time, and hard shutdown could very well be a cause.  Here is a Tech Net article on how to fix:

https://social.technet.microsoft.com/wiki/contents/articles/1261.troubleshooting-vss-writer-failure.aspx

Including this link near the bottom on registering the VSS writers:

https://blogs.technet.microsoft.com/asksbs/2008/04/23/troubleshooting-vss-and-backup/
ASKER CERTIFIED SOLUTION
Avatar of pgm554
pgm554
Flag of United States of America image

Link to home
membership
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
Avatar of Daniel Booker

ASKER

I have not tried to uninstall server backup and reinstall... I will add that on my list of things to try.

I tried as it suggest to backup some data that didnt include the "system volume". So i choose 1 folder on my DataDrive D:\ and it failed. Not sure why Error1 is popping up on a simple folder backup.

Looks like i got 3 errors.
Error1 - Source: VSS EventID:8230
Volume Shadow Copy Service error: Failed resolving account spsearch with status 1376. Check connection to domain controller and VssAccessControl registry key.

Operation:
   Gather writers' status
   Executing Asynchronous Operation

Context:
   Current State: GatherWriterStatus

Error-specific details:
   Error: NetLocalGroupGetMemebers(spsearch), 0x80070560, The specified local group does not exist.

Error2 - Source: ESE EventID: 2007
Information Store (2648) Shadow copy instance 3 aborted.

For more information, click http://www.microsoft.com/contentredirect.asp.

Error3 - Source: Backup EventID:521
The backup operation that started at '‎2017‎-‎04‎-‎13T15:26:44.745000000Z' has failed because the Volume Shadow Copy Service operation to create a shadow copy of the volumes being backed up failed with following error code '2155348129'. Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.
Just noticed and it was probably like this on the last time i looked and just did not notice but half of the say...
State: [1] Stable
and the other half say
State: [5] Waiting for Completition

While only the SqlServerWriter says
State: [8] Failed
You may have more than one issue.  Follow the links given and review the Event Logs in the Event Viewer for clues.  If not clear you can get additional info a www.eventid.net
pgm554
Does it require a reboot after I run the batch file?
I ran the bat file and got the following error towards the begining.

The system event notifaction service service is stopping.
A system error has occured.
system error 997 has occured.
overlapped I/O operation is in progress.
the system event tofication service service was stopped successfully.

Then at the end I got a message about "Windows resource protection could not start the repair service."
... I did vssadmin list writers I only have like half the writers now that I had before but they all says stable and no error.

I tried to run a backup of the a single data folder and it COMPLETED!

Looks like my software backup assist is running Exchange Bacukp.
Will not know until later if a full server image is done.

LOOKING GOOD!
I took an inventory of writers i have now...

On server that i just ran bat file to re-register writers.
Task Scheduler writer
vss metadata store writer
performance counter writer
sqlserverwriter
asr writer
shadow copy optimization writer
frs writer
dhcp jet writer
com+ regdb writer
registry writer

Compared to another SBS 2011 server has the following writers.
Task Scheduler writer
vss metadata store writer
performance counter writer
System Writer
sqlserverwriter
asr writer
Sharepoint Services writer
frs writer
FSRM Writer
TS Gateway Writer
IIS Config Writer
registry writer
Certificate authority
WMI writer
IIS metabase writer
NPS vss writer
NTDS
shadow copy optimization writer
Microsoft Exchange Writer
com+ regdb writer
SPSearch4 vss writer
mssearch service writer
dhcp jet writer

So the following writers are missing...
System Writer
Sharepoint Services writer
FSRM Writer
TS Gateway Writer
IIS Config Writer
Certificate authority
WMI writer
IIS metabase writer
NPS vss writer
NTDS
Microsoft Exchange Writer
SPSearch4 vss writer
mssearch service writer

Is all I need to do is go to the registry entry HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS\ParametersSystem\ and create a dword "Disable Exchange Writer" with a value of 0?
And repeat for each missing vss writer?

I am scheduling a server reboot tonight... I guess i should wait until after it reboots before doing anything further.
Reboot and check.

Here's my writers
writers.txt
Thank you pgm554 for your help at least you got me on track to getting everything fixed.
I can at least now run backup jobs, which i could not before... but after running the batch file i missing several writers and will need to go back and find a way to re-add them.

System Writer
Sharepoint Services writer
FSRM Writer
TS Gateway Writer
IIS Config Writer
Certificate authority
WMI writer
IIS metabase writer
NPS vss writer
NTDS
Microsoft Exchange Writer
SPSearch4 vss writer
mssearch service writer
After the reboot if they still aren't there ,rerun the bat.

From the original error it looks almost as if you had two different backup jobs conflicting.
Ok, will do. Thank you again for your help.
After rebooting all my missing VSS writers are now back!!!!    :)
Sad, when trying to do a full image of the server i'm right back were i started :(

Oh well, i'll open up another question after i can verify some more facts about what is going on.
Full image using what backup program?
Full image using Windows Server Backup.

I use Windows Server Backup and Backup assist.
Windows Server Image runs at 1am and takes normally like 30mins-1hr to finish. (starts and stops within less than a minute of running)
Backup Assist backups Exchange at 3am. (from what I know backup assist uses windows server backup but adds features)
What's the make and model of the usb drive you are using for the backup?

You are aware of the 4K sector limitation in the built in server 2008 back up program ,right?
There is a 4TB internal SATA drive we use for server images. (so far used < 1TB for image backups going back to June 2016).

There is a 750GB USB drive for exchange backups.
Try disabling one of the jobs and see what happens.
Preferably the 4tb drive backup.
Yeah, that is what i planned on doing tonight. To make sure at least the exchange VSS writer was working properly.
The built in back up program does funny things with very large >2tb drives.

I know it doesn't work on usb ,but the large internal sata I'm not quite sure of.

The reason I don't even look at BA is it's just a fancy front end for windows backup which is limited to 2tb drives and 512 sectors.

M$  fixed it in Server 2012 ,but 2008 is very iffy.
Found out disabling the SQL Server VSS writer service makes my backups succeed. Will look into reinstalling the "Microsoft VSS Writer for SQL Sever 20xx" later tonight.
I found the instance of SQL causing the problem by going through each instance disabling it one at a time and then trying the backup and found the instance related to a 3rd party software that the company uses to run their company. I'll contact their Software development team and figure out whats going on with this.