Avatar of easyworks
easyworks
Flag for United States of America asked on

SBS 2011 server backup failing.

I have a SBS 2011 server that recently the server backups stopped working. I believe the owner powered off the server during an image backup.
Most things pointed to running "PSConfig.exe -cmd upgrade -inplace b2b -force -cmd applicationcontent -install -cmd installfeatures" which has already been ran... I have even reran it just in case.
The error from windows event viewer
The backup operation that started at '‎2017‎-‎04‎-‎13T07:00:45.445000000Z' has failed because the Volume Shadow Copy Service operation to create a shadow copy of the volumes being backed up failed with following error code '2155348129'. Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.

I looked at the vss writers and noted that only 1 has a problem...
Writer name: 'SqlServerWriter'

Writer ID: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
Writer Instance ID: {57af718e-2f96-4aa0-a04f-0d060bf2aafa}
State: [8] Failed
Last error: Inconsistent shadow copy
SBSWindows OS* Volume Snapshot Service (VSS)

Avatar of undefined
Last Comment
easyworks

8/22/2022 - Mon
Larry Struckmeyer MVP

VSS writers will do this from time to time, and hard shutdown could very well be a cause.  Here is a Tech Net article on how to fix:

https://social.technet.microsoft.com/wiki/contents/articles/1261.troubleshooting-vss-writer-failure.aspx

Including this link near the bottom on registering the VSS writers:

https://blogs.technet.microsoft.com/asksbs/2008/04/23/troubleshooting-vss-and-backup/
ASKER CERTIFIED SOLUTION
pgm554

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
GET A PERSONALIZED SOLUTION
Ask your own question & get feedback from real experts
Find out why thousands trust the EE community with their toughest problems.
easyworks

ASKER
I have not tried to uninstall server backup and reinstall... I will add that on my list of things to try.

I tried as it suggest to backup some data that didnt include the "system volume". So i choose 1 folder on my DataDrive D:\ and it failed. Not sure why Error1 is popping up on a simple folder backup.

Looks like i got 3 errors.
Error1 - Source: VSS EventID:8230
Volume Shadow Copy Service error: Failed resolving account spsearch with status 1376. Check connection to domain controller and VssAccessControl registry key.

Operation:
   Gather writers' status
   Executing Asynchronous Operation

Context:
   Current State: GatherWriterStatus

Error-specific details:
   Error: NetLocalGroupGetMemebers(spsearch), 0x80070560, The specified local group does not exist.

Error2 - Source: ESE EventID: 2007
Information Store (2648) Shadow copy instance 3 aborted.

For more information, click http://www.microsoft.com/contentredirect.asp.

Error3 - Source: Backup EventID:521
The backup operation that started at '‎2017‎-‎04‎-‎13T15:26:44.745000000Z' has failed because the Volume Shadow Copy Service operation to create a shadow copy of the volumes being backed up failed with following error code '2155348129'. Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.
easyworks

ASKER
Just noticed and it was probably like this on the last time i looked and just did not notice but half of the say...
State: [1] Stable
and the other half say
State: [5] Waiting for Completition

While only the SqlServerWriter says
State: [8] Failed
I started with Experts Exchange in 2004 and it's been a mainstay of my professional computing life since. It helped me launch a career as a programmer / Oracle data analyst
William Peck
Larry Struckmeyer MVP

You may have more than one issue.  Follow the links given and review the Event Logs in the Event Viewer for clues.  If not clear you can get additional info a www.eventid.net
easyworks

ASKER
pgm554
Does it require a reboot after I run the batch file?
easyworks

ASKER
I ran the bat file and got the following error towards the begining.

The system event notifaction service service is stopping.
A system error has occured.
system error 997 has occured.
overlapped I/O operation is in progress.
the system event tofication service service was stopped successfully.

Then at the end I got a message about "Windows resource protection could not start the repair service."
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
easyworks

ASKER
... I did vssadmin list writers I only have like half the writers now that I had before but they all says stable and no error.

I tried to run a backup of the a single data folder and it COMPLETED!

Looks like my software backup assist is running Exchange Bacukp.
Will not know until later if a full server image is done.

LOOKING GOOD!
easyworks

ASKER
I took an inventory of writers i have now...

On server that i just ran bat file to re-register writers.
Task Scheduler writer
vss metadata store writer
performance counter writer
sqlserverwriter
asr writer
shadow copy optimization writer
frs writer
dhcp jet writer
com+ regdb writer
registry writer

Compared to another SBS 2011 server has the following writers.
Task Scheduler writer
vss metadata store writer
performance counter writer
System Writer
sqlserverwriter
asr writer
Sharepoint Services writer
frs writer
FSRM Writer
TS Gateway Writer
IIS Config Writer
registry writer
Certificate authority
WMI writer
IIS metabase writer
NPS vss writer
NTDS
shadow copy optimization writer
Microsoft Exchange Writer
com+ regdb writer
SPSearch4 vss writer
mssearch service writer
dhcp jet writer

So the following writers are missing...
System Writer
Sharepoint Services writer
FSRM Writer
TS Gateway Writer
IIS Config Writer
Certificate authority
WMI writer
IIS metabase writer
NPS vss writer
NTDS
Microsoft Exchange Writer
SPSearch4 vss writer
mssearch service writer

Is all I need to do is go to the registry entry HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS\ParametersSystem\ and create a dword "Disable Exchange Writer" with a value of 0?
And repeat for each missing vss writer?

I am scheduling a server reboot tonight... I guess i should wait until after it reboots before doing anything further.
pgm554

Reboot and check.

Here's my writers
writers.txt
Your help has saved me hundreds of hours of internet surfing.
fblack61
easyworks

ASKER
Thank you pgm554 for your help at least you got me on track to getting everything fixed.
easyworks

ASKER
I can at least now run backup jobs, which i could not before... but after running the batch file i missing several writers and will need to go back and find a way to re-add them.

System Writer
Sharepoint Services writer
FSRM Writer
TS Gateway Writer
IIS Config Writer
Certificate authority
WMI writer
IIS metabase writer
NPS vss writer
NTDS
Microsoft Exchange Writer
SPSearch4 vss writer
mssearch service writer
pgm554

After the reboot if they still aren't there ,rerun the bat.

From the original error it looks almost as if you had two different backup jobs conflicting.
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
easyworks

ASKER
Ok, will do. Thank you again for your help.
easyworks

ASKER
After rebooting all my missing VSS writers are now back!!!!    :)
easyworks

ASKER
Sad, when trying to do a full image of the server i'm right back were i started :(

Oh well, i'll open up another question after i can verify some more facts about what is going on.
Experts Exchange is like having an extremely knowledgeable team sitting and waiting for your call. Couldn't do my job half as well as I do without it!
James Murphy
pgm554

Full image using what backup program?
easyworks

ASKER
Full image using Windows Server Backup.

I use Windows Server Backup and Backup assist.
Windows Server Image runs at 1am and takes normally like 30mins-1hr to finish. (starts and stops within less than a minute of running)
Backup Assist backups Exchange at 3am. (from what I know backup assist uses windows server backup but adds features)
pgm554

What's the make and model of the usb drive you are using for the backup?

You are aware of the 4K sector limitation in the built in server 2008 back up program ,right?
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
easyworks

ASKER
There is a 4TB internal SATA drive we use for server images. (so far used < 1TB for image backups going back to June 2016).

There is a 750GB USB drive for exchange backups.
pgm554

Try disabling one of the jobs and see what happens.
Preferably the 4tb drive backup.
easyworks

ASKER
Yeah, that is what i planned on doing tonight. To make sure at least the exchange VSS writer was working properly.
All of life is about relationships, and EE has made a viirtual community a real community. It lifts everyone's boat
William Peck
pgm554

The built in back up program does funny things with very large >2tb drives.

I know it doesn't work on usb ,but the large internal sata I'm not quite sure of.

The reason I don't even look at BA is it's just a fancy front end for windows backup which is limited to 2tb drives and 512 sectors.

M$  fixed it in Server 2012 ,but 2008 is very iffy.
easyworks

ASKER
pgm554

⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
easyworks

ASKER
Found out disabling the SQL Server VSS writer service makes my backups succeed. Will look into reinstalling the "Microsoft VSS Writer for SQL Sever 20xx" later tonight.
easyworks

ASKER
I found the instance of SQL causing the problem by going through each instance disabling it one at a time and then trying the backup and found the instance related to a 3rd party software that the company uses to run their company. I'll contact their Software development team and figure out whats going on with this.