Volume Shadow Copy Service - Exchange Writer Failure

Hello everyone. I got a bit of a nasty one here. We have Symantec Backup Exec v12.0 for Windows Servers SP3 with AOFO and Exchange Agent as our backup software. Both the backup server and the exchange server are Windows Server 2003 standard edition, service pack 2.  On my exchange server during backup operations I am getting the following errors:

Event Type:      Error
Event Source:      COM+
Event Category:      Unknown
Event ID:      4786
Date:            5/23/2009
Time:            11:00:55 PM
User:            N/A
Computer:      XXXX
Description:
The system has called a custom component and that component has failed and generated an exception. This indicates a problem with the custom component. Notify the developer of this component that a failure has occurred and provide them with the information below.
Component Prog ID:
Server Application ID: {E5D57B82-667A-40E2-8A40-7E34D2924EA1}
Server Application Instance ID:
{CCB969C0-C01A-467A-BFCB-2BCB15FB7E37}
Server Application Name: Symantec SymSnap VSS Provider
The serious nature of this error has caused the process to terminate.
Exception: E06D7363
Address: 0x77E4BEF7
Call Stack:
kernel32!RaiseException + 0x3c
SymSnapProvider!DllUnregisterServer + 0x8e0b8
SymSnapProvider!DllUnregisterServer + 0x16c0

-----------------------------------------------------------

Event Type:      Error
Event Source:      VSS
Event Category:      None
Event ID:      12293
Date:            5/23/2009
Time:            11:00:56 PM
User:            N/A
Computer:      XXXX
Description:
Volume Shadow Copy Service error: Error calling a routine on the Shadow Copy Provider {262b716e-bb23-41b5-aaef-e2c15e767167}. Routine details Cannot ask provider {262b716e-bb23-41b5-aaef-e2c15e767167} if volume is supported. [0x800706ba] [hr = 0x800706ba].

-----------------------------------------------------------

Event Type:      Error
Event Source:      VSS
Event Category:      None
Event ID:      12298
Date:            5/20/2009
Time:            11:02:14 PM
User:            N/A
Computer:      XXXX
Description:
Volume Shadow Copy Service error: The I/O writes cannot be held during the shadow copy creation period on volume \\?\Volume{a362cf46-acaa-11dd-aca0-806e6f6e6963}\. The volume index in the shadow copy set is 0. Error details: Open[0x00000000], Flush[0x00000000], Release[0x80042314], OnRun[0x00000000].

-----------------------------------------------------------

Event Type:      Error
Event Source:      VSS
Event Category:      None
Event ID:      12310
Date:            5/20/2009
Time:            11:02:14 PM
User:            N/A
Computer:      XXXX
Description:
Volume Shadow Copy Service error: The shadow copy could not be committed - operation timed out. Error context: DeviceIoControl(\\?\Volume{a362cf46-acaa-11dd-aca0-806e6f6e6963} - 00000170,0x0053c010,00B10048,0,00B11050,4096,[0]).


When doing "vssadmin list writers" I get the following entry:


D:\>vssadmin list writers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001 Microsoft Corp.

Writer name: 'System Writer'
   Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
   Writer Instance Id: {6041cb48-7004-4be9-b824-1b13b1b327c6}
   State: [1] Stable
   Last error: No error

Writer name: 'Microsoft Exchange Writer'
   Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
   Writer Instance Id: {be22d970-cc72-4c54-a4cb-229170ad6f04}
   State: [7] Failed
   Last error: Retryable error

Writer name: 'IIS Metabase Writer'
   Writer Id: {59b1f0cf-90ef-465f-9609-6ca8b2938366}
   Writer Instance Id: {3e4c9c3d-c543-4ef4-8165-5cc24d325ac9}
   State: [1] Stable
   Last error: No error

Writer name: 'Registry Writer'
   Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
   Writer Instance Id: {1aebfc6a-f620-4452-9a66-228b5a802170}
   State: [1] Stable
   Last error: No error

Writer name: 'MSDEWriter'
   Writer Id: {f8544ac1-0611-4fa5-b04b-f7ee00b03277}
   Writer Instance Id: {de6cc5ec-febb-484c-9560-aba1c0bb4147}
   State: [1] Stable
   Last error: No error

Writer name: 'BITS Writer'
   Writer Id: {4969d978-be47-48b0-b100-f328f07ac1e0}
   Writer Instance Id: {bffb7422-2639-4b88-98cf-7856d888c336}
   State: [1] Stable
   Last error: No error

Writer name: 'COM+ REGDB Writer'
   Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
   Writer Instance Id: {17ea2ad6-a04d-405b-aa0b-889d8a1ac346}
   State: [1] Stable
   Last error: No error

Writer name: 'Event Log Writer'
   Writer Id: {eee8c692-67ed-4250-8d86-390603070d00}
   Writer Instance Id: {22001e31-e309-47be-b979-bad4963b10d5}
   State: [1] Stable
   Last error: No error

Writer name: 'WMI Writer'
   Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
   Writer Instance Id: {26d1cc45-37fc-42d8-ac8c-3e1ed987e8c8}
   State: [1] Stable
   Last error: No error

When listing shadow copies, it shows one copy on the disk. When I try to delete it, it says that there are no copies with that GUID on the disk. When I try "vssadmin delete shadows /all" it skips the part where I am supposed to confirm and automatically enters a 'NO'.

I'm guessing that the error on the VSS exchange writer are related to the 'ghost' shadow copy that cannot be deleted, but still shows, but can't really find anything from Microsoft on how to troubleshoot this.

lithargoelAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

itprotocallCommented:
there problem seems to be more on the Symantec side, rather than Microsoft.
from the first error, the cause is 'Symantec SymSnap VSS Provider'

For the state of the exchange writer,
State: [7] Failed
Last error: Retryable error

its indicating the last attempt failed, but it can be reused. So something wrong in the Symantec VSS provider which is leading to this failure.

you could try a repair or reinstall of Symantec products if possible.
0
lithargoelAuthor Commented:
Problem appears to have been solved. It was definitely related to the Microsoft VSS Exchange Writer. After rebooting a couple of times we finally managed to delete the defect shadow copy using the VSS admin tool. After deleting it, we could run backup jobs as normal.

Having gone over the configuration of the Backup Exec jobs, It seems that two jobs were set to run at exactly the same time, on exactly the same disk. My bet is that this is what caused the shadow copy of that disk to fail in the first place.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Storage Software

From novice to tech pro — start learning today.