V16Pro
asked on
Exchange 2007 Backup Failure (Event ID 565), When Running VSS Backup of Exchange Mailbox Database
Hello!
I am experiencing issues backing up an MS Exchange 2007 SP 3 instance running on Windows Server 2008 R2, using the Windows Server Backup solution.
The backup solution is experiencing consistency check issues when backing up the database, causing the backup to complete with the warning that the Exchange application will be non-recoverable.
The mailbox database on which the check is failing was previously the source database for an SCR seed. I have read that this type of consistency check backup error is not uncommon when SCR is used. However, SCR replication for that storage group has since been disabled, and even so, the Exchange portion of the backup still fails.
I have already tried creating a DWORD entry for HKLM\SOFTWARE\Microsoft\Ex
I'm pasting below a listing of all the messages logged to the Application logs while the backup was running, by order of appearance. Any assistance is greatly needed and appreciated.
I am experiencing issues backing up an MS Exchange 2007 SP 3 instance running on Windows Server 2008 R2, using the Windows Server Backup solution.
The backup solution is experiencing consistency check issues when backing up the database, causing the backup to complete with the warning that the Exchange application will be non-recoverable.
The mailbox database on which the check is failing was previously the source database for an SCR seed. I have read that this type of consistency check backup error is not uncommon when SCR is used. However, SCR replication for that storage group has since been disabled, and even so, the Exchange portion of the backup still fails.
I have already tried creating a DWORD entry for HKLM\SOFTWARE\Microsoft\Ex
I'm pasting below a listing of all the messages logged to the Application logs while the backup was running, by order of appearance. Any assistance is greatly needed and appreciated.
Log Name: Application
Source: Microsoft-Windows-Backup
Date: 9/21/2013 7:30:00 PM
Event ID: 753
Task Category: None
Level: Information
Keywords:
User: SYSTEM
Computer: server.domain.local
Description:
The Block Level Backup Engine service has successfully started.
====================
Log Name: Application
Source: MSExchangeIS
Date: 9/21/2013 7:30:09 PM
Event ID: 9606
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: server.domain.local
Description:
Exchange VSS Writer (instance 50ae3333-8bc9-41eb-a211-b461434272ed) has prepared for backup successfully.
====================
Log Name: Application
Source: ESE
Date: 9/21/2013 7:30:11 PM
Event ID: 2005
Task Category: ShadowCopy
Level: Information
Keywords: Classic
User: N/A
Computer: server.domain.local
Description:
Information Store (4700) Shadow copy instance 3 starting. This will be a Full shadow copy.
For more information, click http://www.microsoft.com/contentredirect.asp.
====================
Log Name: Application
Source: MSExchangeIS
Date: 9/21/2013 7:30:11 PM
Event ID: 9811
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: server.domain.local
Description:
Exchange VSS Writer (instance 3) has successfully prepared the database engine for a full or copy backup of storage group 'PHP Database'. The following 1 databases are mounted and will be backed up:
Mailbox Database 1
====================
Log Name: Application
Source: MSExchangeIS
Date: 9/21/2013 7:30:11 PM
Event ID: 9811
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: server.domain.local
Description:
Exchange VSS Writer (instance 3) has successfully prepared the database engine for a full or copy backup of storage group 'PHP Public Folders 1'. The following 1 databases are mounted and will be backed up:
Public Folder Database 1
====================
Log Name: Application
Source: MSExchangeIS
Date: 9/21/2013 7:30:11 PM
Event ID: 9608
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: server.domain.local
Description:
Exchange VSS Writer (instance 50ae3333-8bc9-41eb-a211-b461434272ed:3) has prepared for Snapshot successfully.
====================
Log Name: Application
Source: ESENT
Date: 9/21/2013 7:30:11 PM
Event ID: 103
Task Category: General
Level: Information
Keywords: Classic
User: N/A
Computer: server.domain.local
Description:
DFSRs (1240) \\.\H:\System Volume Information\DFSR\database_C4C2_1120_C211_17EE\dfsr.db: The database engine stopped the instance (0).
====================
Log Name: Application
Source: MSSQL$MICROSOFT##SSEE
Date: 9/21/2013 7:30:12 PM
Event ID: 3197
Task Category: (2)
Level: Information
Keywords: Classic
User: SYSTEM
Computer: server.domain.local
Description:
The description for Event ID 3197 from source MSSQL$MICROSOFT##SSEE cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
model
The specified resource type cannot be found in the image file
====================
Log Name: Application
Source: ESE
Date: 9/21/2013 7:30:12 PM
Event ID: 2001
Task Category: ShadowCopy
Level: Information
Keywords: Classic
User: N/A
Computer: server.domain.local
Description:
Information Store (4700) Shadow copy instance 3 freeze started.
For more information, click http://www.microsoft.com/contentredirect.asp.
====================
Log Name: Application
Source: ESE
Date: 9/21/2013 7:30:12 PM
Event ID: 2001
Task Category: ShadowCopy
Level: Information
Keywords: Classic
User: N/A
Computer: server.domain.local
Description:
MSExchangeIS (4700) PHP Database: Shadow copy instance 3 freeze started.
For more information, click http://www.microsoft.com/contentredirect.asp.
====================
Log Name: Application
Source: MSSQL$MICROSOFT##SSEE
Date: 9/21/2013 7:30:12 PM
Event ID: 3197
Task Category: (2)
Level: Information
Keywords: Classic
User: SYSTEM
Computer: server.domain.local
Description:
The description for Event ID 3197 from source MSSQL$MICROSOFT##SSEE cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
msdb
The specified resource type cannot be found in the image file
====================
Log Name: Application
Source: ESE
Date: 9/21/2013 7:30:12 PM
Event ID: 2001
Task Category: ShadowCopy
Level: Information
Keywords: Classic
User: N/A
Computer: server.domain.local
Description:
MSExchangeIS (4700) PHP Public Folders 1: Shadow copy instance 3 freeze started.
For more information, click http://www.microsoft.com/contentredirect.asp.
====================
Log Name: Application
Source: MSSQL$MICROSOFT##SSEE
Date: 9/21/2013 7:30:12 PM
Event ID: 3197
Task Category: (2)
Level: Information
Keywords: Classic
User: SYSTEM
Computer: server.domain.local
Description:
The description for Event ID 3197 from source MSSQL$MICROSOFT##SSEE cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
SUSDB
The specified resource type cannot be found in the image file
====================
Log Name: Application
Source: MSSQL$MICROSOFT##SSEE
Date: 9/21/2013 7:30:12 PM
Event ID: 3197
Task Category: (2)
Level: Information
Keywords: Classic
User: SYSTEM
Computer: server.domain.local
Description:
The description for Event ID 3197 from source MSSQL$MICROSOFT##SSEE cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
master
The specified resource type cannot be found in the image file
====================
Log Name: Application
Source: MSExchangeIS
Date: 9/21/2013 7:30:12 PM
Event ID: 9610
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: server.domain.local
Description:
Exchange VSS Writer (instance 50ae3333-8bc9-41eb-a211-b461434272ed:3) has frozen the storage group(s) successfully.
====================
Log Name: Application
Source: MSSQL$MICROSOFT##SSEE
Date: 9/21/2013 7:30:12 PM
Event ID: 3198
Task Category: (2)
Level: Information
Keywords: Classic
User: SYSTEM
Computer: server.domain.local
Description:
The description for Event ID 3198 from source MSSQL$MICROSOFT##SSEE cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
SUSDB
The specified resource type cannot be found in the image file
====================
Log Name: Application
Source: MSSQL$MICROSOFT##SSEE
Date: 9/21/2013 7:30:12 PM
Event ID: 3198
Task Category: (2)
Level: Information
Keywords: Classic
User: SYSTEM
Computer: server.domain.local
Description:
The description for Event ID 3198 from source MSSQL$MICROSOFT##SSEE cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
model
The specified resource type cannot be found in the image file
====================
Log Name: Application
Source: MSSQL$MICROSOFT##SSEE
Date: 9/21/2013 7:30:12 PM
Event ID: 3198
Task Category: (2)
Level: Information
Keywords: Classic
User: SYSTEM
Computer: server.domain.local
Description:
The description for Event ID 3198 from source MSSQL$MICROSOFT##SSEE cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
master
The specified resource type cannot be found in the image file
====================
Log Name: Application
Source: MSSQL$MICROSOFT##SSEE
Date: 9/21/2013 7:30:12 PM
Event ID: 3198
Task Category: (2)
Level: Information
Keywords: Classic
User: SYSTEM
Computer: server.domain.local
Description:
The description for Event ID 3198 from source MSSQL$MICROSOFT##SSEE cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
msdb
The specified resource type cannot be found in the image file
====================
Log Name: Application
Source: ESE
Date: 9/21/2013 7:30:12 PM
Event ID: 2003
Task Category: ShadowCopy
Level: Information
Keywords: Classic
User: N/A
Computer: server.domain.local
Description:
Information Store (4700) Shadow copy instance 3 freeze ended.
For more information, click http://www.microsoft.com/contentredirect.asp.
====================
Log Name: Application
Source: MSExchangeIS
Date: 9/21/2013 7:30:12 PM
Event ID: 9612
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: server.domain.local
Description:
Exchange VSS Writer (instance 50ae3333-8bc9-41eb-a211-b461434272ed:3) has thawed the storage group(s) successfully.
====================
Log Name: Application
Source: ESENT
Date: 9/21/2013 7:30:15 PM
Event ID: 102
Task Category: General
Level: Information
Keywords: Classic
User: N/A
Computer: server.domain.local
Description:
DFSRs (1240) \\.\H:\System Volume Information\DFSR\database_C4C2_1120_C211_17EE\dfsr.db: The database engine (6.01.7601.0000) started a new instance (0).
====================
Log Name: Application
Source: MSExchangeIS
Date: 9/21/2013 7:30:16 PM
Event ID: 9622
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: server.domain.local
Description:
Exchange VSS Writer (instance 50ae3333-8bc9-41eb-a211-b461434272ed:3) has processed the post-snapshot event successfully.
====================
Log Name: Application
Source: MSSQL$MICROSOFT##SSEE
Date: 9/21/2013 7:30:16 PM
Event ID: 18264
Task Category: (6)
Level: Information
Keywords: Classic
User: SYSTEM
Computer: server.domain.local
Description:
The description for Event ID 18264 from source MSSQL$MICROSOFT##SSEE cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
master
2013/09/21
15:12:35
1
398:368:37
398:392:1
1
FILE=1, TYPE=VIRTUAL_DEVICE: {'{FF0B7DF7-6405-403D-8DD3-697BC5F41916}4'}
The specified resource type cannot be found in the image file
====================
Log Name: Application
Source: MSSQL$MICROSOFT##SSEE
Date: 9/21/2013 7:30:16 PM
Event ID: 18264
Task Category: (6)
Level: Information
Keywords: Classic
User: SYSTEM
Computer: server.domain.local
Description:
The description for Event ID 18264 from source MSSQL$MICROSOFT##SSEE cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
model
2003/04/08
09:13:36
1
29:464:37
29:488:1
1
FILE=1, TYPE=VIRTUAL_DEVICE: {'{FF0B7DF7-6405-403D-8DD3-697BC5F41916}1'}
The specified resource type cannot be found in the image file
====================
Log Name: Application
Source: MSSQL$MICROSOFT##SSEE
Date: 9/21/2013 7:30:16 PM
Event ID: 18264
Task Category: (6)
Level: Information
Keywords: Classic
User: SYSTEM
Computer: server.domain.local
Description:
The description for Event ID 18264 from source MSSQL$MICROSOFT##SSEE cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
SUSDB
2013/01/21
12:06:42
1
64165:93:51
64165:114:1
1
FILE=1, TYPE=VIRTUAL_DEVICE: {'{FF0B7DF7-6405-403D-8DD3-697BC5F41916}3'}
The specified resource type cannot be found in the image file
====================
Log Name: Application
Source: MSSQL$MICROSOFT##SSEE
Date: 9/21/2013 7:30:16 PM
Event ID: 18264
Task Category: (6)
Level: Information
Keywords: Classic
User: SYSTEM
Computer: server.domain.local
Description:
The description for Event ID 18264 from source MSSQL$MICROSOFT##SSEE cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
msdb
2005/10/14
01:54:05
1
118:576:172
118:656:1
1
FILE=1, TYPE=VIRTUAL_DEVICE: {'{FF0B7DF7-6405-403D-8DD3-697BC5F41916}2'}
The specified resource type cannot be found in the image file
====================
Log Name: Application
Source: VSS
Date: 9/21/2013 7:30:34 PM
Event ID: 8220
Task Category: None
Level: Information
Keywords: Classic
User: N/A
Computer: server.domain.local
Description:
Ran out of time while deleting files.
Operation:
OnPostSnapshot event
PostSnapshot Event
Context:
Execution Context: Shadow Copy Optimization Writer
Execution Context: Writer
Writer Class Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
Writer Name: Shadow Copy Optimization Writer
Writer Instance ID: {79842fdc-b2d0-4ef1-8323-54894580dc16}
====================
Log Name: Application
Source: Microsoft-Windows-Backup
Date: 9/21/2013 7:30:46 PM
Event ID: 565
Task Category: None
Level: Error
Keywords:
User: SYSTEM
Computer: server.domain.local
Description:
The consistency check for the component '188a696e-0ee3-4637-9108-6f859dcab338'\'Microsoft Exchange Server\Microsoft Information Store\EXCHANGESERVER1' has failed. The application 'Exchange' will not be available in the backup created at '¿2013¿-¿09¿-¿21T19:30:06.145448400Z'. Review the event details for information about consistency check issues.
On the disk where the databases are stored, open properties of the disk and make sure Shadow Copies are enabled (keep the max space small e.g. 10GB) to allow VSS to work.
ASKER
Thanks, cmsxpjh. I've just enabled Shadow Copies for that disk as per your suggestion, and re-ran the backup. I again received the "consistency check" warning for the Exchange application several seconds after the backup began.
I would run ESEUTIL /K and maybe ESEUTIL /P to check and repair the databases before trying another backup.
http://support.microsoft.com/kb/2616952
http://support.microsoft.com/kb/2616952
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Folks --
Thanks for those suggestions! I think I'll use the 'eseutil /k' command to discover which database(s) (mailbox or public folder) have integrity issues, and then, should I choose to move forward from there, investigate moving contents to the new database as per Simon's suggestion.
I am in the midst of upgrading our Exchange environment to Exchange 2010, so I don't foresee needing the 2007 databases much longer. However, while I have the second node in my two-node Exchange environment running Exchange 2010 in coexistence (and thus no SCR configuration or public folder replication in place), I'd like to have some confidence that the bulk of my data, while it remains in the 2007 environment, has some change at being recoverable "if the worst should happen."
I'm familiar with the process for moving mailboxes to a new mailbox server. However, if it should happen that the public folders database is not consistent, what would be the procedure for moving data to a different public folders database on the same node?
Thanks!
Thanks for those suggestions! I think I'll use the 'eseutil /k' command to discover which database(s) (mailbox or public folder) have integrity issues, and then, should I choose to move forward from there, investigate moving contents to the new database as per Simon's suggestion.
I am in the midst of upgrading our Exchange environment to Exchange 2010, so I don't foresee needing the 2007 databases much longer. However, while I have the second node in my two-node Exchange environment running Exchange 2010 in coexistence (and thus no SCR configuration or public folder replication in place), I'd like to have some confidence that the bulk of my data, while it remains in the 2007 environment, has some change at being recoverable "if the worst should happen."
I'm familiar with the process for moving mailboxes to a new mailbox server. However, if it should happen that the public folders database is not consistent, what would be the procedure for moving data to a different public folders database on the same node?
Thanks!
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
I would consider it unlikely that SCR would disable the service.
Yes, if the service is not required, then setting it to manual is okay.
Yes, if the service is not required, then setting it to manual is okay.
ASKER
The fix was to adjust the "Startup Type" for the "Microsoft Exchange Server Extension for Windows Server Backup" service from "Disabled" to "Manual." Not sure how it became disabled if SCR didn't do it. We have only two administrators supervising our Exchange environment, and neither of us changed it to "Disabled."
I'll accept solutions from Sembee2 and cmsxpjh as a token of my thanks for your assistance with this issue. Had our databases turned out to have been corrupted, I have no doubt that the solutions you each offered would have proven invaluable.
Thanks!
I'll accept solutions from Sembee2 and cmsxpjh as a token of my thanks for your assistance with this issue. Had our databases turned out to have been corrupted, I have no doubt that the solutions you each offered would have proven invaluable.
Thanks!
ASKER
The fix was to adjust the "Startup Type" for the "Microsoft Exchange Server Extension for Windows Server Backup" service from "Disabled" to "Manual." Not sure how it became disabled if SCR didn't do it. We have only two administrators supervising our Exchange environment, and neither of us changed it to "Disabled."
I'll accept solutions from Sembee2 and cmsxpjh as a token of my thanks for your assistance with this issue. Had our databases turned out to have been corrupted, I have no doubt that the solutions you each offered would have proven invaluable.
Thanks!
I'll accept solutions from Sembee2 and cmsxpjh as a token of my thanks for your assistance with this issue. Had our databases turned out to have been corrupted, I have no doubt that the solutions you each offered would have proven invaluable.
Thanks!
ASKER
1) enabling circular logging on both the mailbox and public folder databases,
2) dismounting and remounting each database to truncate log files,
3) returning circular logging setting to 'disabled' for the mailbox and public folder databases,
4) dismounting and remounting each database to apply the setting.
Subsequently, upon attempting a backup, I again received the customary consistency check error message several seconds into the backup job. The job proceeded to complete "Successfully" with the "Warning" that Exchange would not be recoverable.
I would appreciate any additional insight you might offer.