asked on

Exchnage Server Database Dismounts With Many Constant Errors Nightly

Hello experts,

we have an exchange server at a client site whose mailbox database keeps dismounting and then exchange starts throwing large amounts of errors in regards to the database, i have place them below. restarting the information store seems to do the trick for about a day, but it goes down again each night. The client is getting quite upset and i don't have many ideas on this one. I have tried restarting the server, and i also tried moving the database to a different location but that process returned a bunch of errors. I saw a command in a microsoft that someone else did "eseutil /r e01 /l:"d:\Exchange Server\V14\Mailbox\Archived" /s:"d:\Exchange Server\V14" but i'm not sure what that command does (obviously their issue was with a mailbox called archived. I'll look into it on EMS and try to figure it out but any insight ya'll can provide would be helpful. the error messages are below. Any help is greatly appreciated, thank you.

_____________

Log Name: Application
Source: ExchangeStoreDB
Date: 12/20/2014 8:17:48 AM
Event ID: 231
Task Category: Database recovery
Level: Error
Keywords: Classic
User: N/A
Computer: UP-MS1.uptrucking.local
Description:
At '12/20/2014 8:17:47 AM', the copy of database 'Mailbox Database' on this server encountered an error during the mount operation. For more information, consult the Event log on the server for "ExchangeStoreDb" or "MSExchangeRepl" events. The mount operation will be tried again automatically.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="ExchangeStoreDB" />
<EventID Qualifiers="49156">231</EventID>
<Level>2</Level>
<Task>1</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2014-12-20T14:17:48.000000000Z" />
<EventRecordID>230157</EventRecordID>
<Channel>Application</Channel>
<Computer>UP-MS1.uptrucking.local</Computer>
<Security />
</System>
<EventData>
<Data>Mailbox Database</Data>
<Data>e2c413e4-89be-4914-9e13-0ca59c7d17b9</Data>
<Data>Store</Data>
<Data>GenericMountFailure</Data>
<Data>Mailbox Database</Data>
<Data>Store</Data>
<Data>There is only one copy of this mailbox database (Mailbox Database). Automatic recovery is not available.</Data>
<Data>12/20/2014 8:17:47 AM</Data>
</EventData>
</Event>

_______________________

Log Name: Application
Source: ExchangeStoreDB
Date: 12/20/2014 8:17:48 AM
Event ID: 215
Task Category: Database recovery
Level: Error
Keywords: Classic
User: N/A
Computer: UP-MS1.uptrucking.local
Description:
At '12/20/2014 8:17:47 AM' the Microsoft Exchange Information Store Database 'Mailbox Database' copy on this server experienced a serious error which caused it to terminate its functional activity. The error returned by the remount attempt was "There is only one copy of this mailbox database (Mailbox Database). Automatic recovery is not available.". Consult the event log on the server for other storage and "ExchangeStoreDb" events for more specific information about the failures.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="ExchangeStoreDB" />
<EventID Qualifiers="49156">215</EventID>
<Level>2</Level>
<Task>1</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2014-12-20T14:17:48.000000000Z" />
<EventRecordID>230155</EventRecordID>
<Channel>Application</Channel>
<Computer>UP-MS1.uptrucking.local</Computer>
<Security />
</System>
<EventData>
<Data>Mailbox Database</Data>
<Data>e2c413e4-89be-4914-9e13-0ca59c7d17b9</Data>
<Data>Ese</Data>
<Data>Remount</Data>
<Data>Mailbox Database</Data>
<Data>
</Data>
<Data>There is only one copy of this mailbox database (Mailbox Database). Automatic recovery is not available.</Data>
<Data>12/20/2014 8:17:47 AM</Data>
<Data>Meta</Data>
<Data>C:\Program Files\Microsoft\Exchange Server\V14\Mailbox\Mailbox Database\Mailbox Database.edb</Data>
<Data>0</Data>
<Data>0</Data>
</EventData>
</Event>

__________________________________

Log Name: Application
Source: MSExchangeRepl
Date: 12/20/2014 8:17:47 AM
Event ID: 3154
Task Category: Service
Level: Error
Keywords: Classic
User: N/A
Computer: UP-MS1.uptrucking.local
Description:
Active Manager failed to mount database Mailbox Database on server UP-MS1.uptrucking.local. Error: An Active Manager operation failed. Error: The database action failed. Error: Operation failed with message: MapiExceptionCallFailed: Unable to mount database. (hr=0x80004005, ec=-1032)
.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="MSExchangeRepl" />
<EventID Qualifiers="49156">3154</EventID>
<Level>2</Level>
<Task>1</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2014-12-20T14:17:47.000000000Z" />
<EventRecordID>230153</EventRecordID>
<Channel>Application</Channel>
<Computer>UP-MS1.uptrucking.local</Computer>
<Security />
</System>
<EventData>
<Data>Mailbox Database</Data>
<Data>UP-MS1.uptrucking.local</Data>
<Data>An Active Manager operation failed. Error: The database action failed. Error: Operation failed with message: MapiExceptionCallFailed: Unable to mount database. (hr=0x80004005, ec=-1032)
.</Data>
</EventData>
</Event>

_____________________________________
Log Name: Application
Source: MSExchangeIS
Date: 12/20/2014 8:17:47 AM
Event ID: 9519
Task Category: General
Level: Error
Keywords: Classic
User: N/A
Computer: UP-MS1.uptrucking.local
Description:
While starting database Mailbox Database, the following error occurred: 0xfffffbf8.
Start DB failed..
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="MSExchangeIS" />
<EventID Qualifiers="49158">9519</EventID>
<Level>2</Level>
<Task>6</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2014-12-20T14:17:47.000000000Z" />
<EventRecordID>230152</EventRecordID>
<Channel>Application</Channel>
<Computer>UP-MS1.uptrucking.local</Computer>
<Security />
</System>
<EventData>
<Data>0xfffffbf8</Data>
<Data>Mailbox Database</Data>
<Data>Start DB failed.</Data>
<Binary>5B444941475F4354585D00009E000000FF602A0000000000000290000000B7AC0010FE00000091EA40100F010480B7AC00100F01048091EA40100F010480B7AC00100F01048091EA40100F010480B7AC00100F01048091EA40100F010480B7AC00100F01048091EA40100F010480BD5F20100000000022274010F8FBFFFFF1704010F8FBFFFF130A4010F8FBFFFFF31C4010F8FBFFFF730D4010F8FBFFFF3D890010760400003DD14010F8FBFFFF</Binary>
</EventData>
</Event>

_________________________________________

Log Name: Application
Source: MSExchangeIS
Date: 12/20/2014 8:17:47 AM
Event ID: 9519
Task Category: General
Level: Error
Keywords: Classic
User: N/A
Computer: UP-MS1.uptrucking.local
Description:
While starting database Mailbox Database, the following error occurred: 0xfffffbf8.
Failed to configure MDB.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="MSExchangeIS" />
<EventID Qualifiers="49158">9519</EventID>
<Level>2</Level>
<Task>6</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2014-12-20T14:17:47.000000000Z" />
<EventRecordID>230151</EventRecordID>
<Channel>Application</Channel>
<Computer>UP-MS1.uptrucking.local</Computer>
<Security />
</System>
<EventData>
<Data>0xfffffbf8</Data>
<Data>Mailbox Database</Data>
<Data>Failed to configure MDB</Data>
<Binary>5B444941475F4354585D00008E000000FF602A0000000000000280000000B7AC0010FE00000091EA40100F010480B7AC00100F01048091EA40100F010480B7AC00100F01048091EA40100F010480B7AC00100F01048091EA40100F010480B7AC00100F01048091EA40100F010480BD5F20100000000022274010F8FBFFFFF1704010F8FBFFFF130A4010F8FBFFFFF31C4010F8FBFFFF730D4010F8FBFFFF</Binary>
</EventData>
</Event>

_________________________________

Log Name: Application
Source: ESE
Date: 12/20/2014 8:17:47 AM
Event ID: 489
Task Category: General
Level: Error
Keywords: Classic
User: N/A
Computer: UP-MS1.uptrucking.local
Description:
Information Store (22452) Mailbox Database: An attempt to open the file "C:\Program Files\Microsoft\Exchange Server\V14\Mailbox\Mailbox Database\Mailbox Database.edb" for read only access failed with system error 32 (0x00000020): "The process cannot access the file because it is being used by another process. ". The open file operation will fail with error -1032 (0xfffffbf8).

For more information, click http://www.microsoft.com/contentredirect.asp.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="ESE" />
<EventID Qualifiers="0">489</EventID>
<Level>2</Level>
<Task>1</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2014-12-20T14:17:47.000000000Z" />
<EventRecordID>230150</EventRecordID>
<Channel>Application</Channel>
<Computer>UP-MS1.uptrucking.local</Computer>
<Security />
</System>
<EventData>
<Data>Information Store</Data>
<Data>22452</Data>
<Data>Mailbox Database: </Data>
<Data>C:\Program Files\Microsoft\Exchange Server\V14\Mailbox\Mailbox Database\Mailbox Database.edb</Data>
<Data>-1032 (0xfffffbf8)</Data>
<Data>32 (0x00000020)</Data>
<Data>The process cannot access the file because it is being used by another process. </Data>
</EventData>
</Event>

____________________________________________

ASKER CERTIFIED SOLUTION

Simon Butler (Sembee)

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

ctagle

ASKER

Thanks for the answer, How do you do that? Is that a tool in exchange similar to the move database path wizard?

ctagle

ASKER

Actually I see how to move them, what do you mean by Arbitration though?

Simon Butler (Sembee)

I am not referring to moving the database, but moving the mailboxes. You use the move mailbox wizard to do that.
Arbitration mailboxes are special mailboxes that are used by the Exchange system. You will be unable to drop the old database until you have removed them.

get-mailbox -Database “Mailbox Database 1″ -Arbitration | New-MoveRequest –TargetDatabase “Mailbox Database 2″

Simon.

ctagle

ASKER

yea i did some research and got everything moved over, i've moved all the mailboxes including the arbitration ones and have dismounted the old database, i'll check it out tomorrow morning to see it the new one i created dismounts itself and update here. thanks.

ctagle

ASKER

Logged in this morning and the database had dismounted itself again. Any ideas? Is it possible there is a mailbox that is causing this issue? You had also mentioned size could be a factor, right now the database is 35 gigs, would that make a difference?

ctagle

ASKER

Not sure if I was clear, the new one is doing the same thing

ctagle

ASKER

Here is another error that I found this morning, many people suggest that its AV causing that issue, but there is no AV on that server. This error still appears to be a result of the original problem as this is part of a chain of errors generated every minute or two. I haven't been able to find the original error log because it completely fills up the error logs with messages when it happens. I noticed today that it had over 200 errors from active sync generated in the space of a second.

_________________________________

Log Name: Application
Source: ESE
Date: 12/20/2014 10:41:53 PM
Event ID: 489
Task Category: General
Level: Error
Keywords: Classic
User: N/A
Computer:
Description:
Information Store (22100) Database_new: An attempt to open the file "E:\Exchange\Database_new.edb" for read only access failed with system error 32 (0x00000020): "The process cannot access the file because it is being used by another process. ". The open file operation will fail with error -1032 (0xfffffbf8).

For more information, click http://www.microsoft.com/contentredirect.asp.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="ESE" />
<EventID Qualifiers="0">489</EventID>
<Level>2</Level>
<Task>1</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2014-12-21T04:41:53.000000000Z" />
<EventRecordID>234662</EventRecordID>
<Channel>Application</Channel>
<Computer></Computer>
<Security />
</System>
<EventData>
<Data>Information Store</Data>
<Data>22100</Data>
<Data>Database_new: </Data>
<Data>E:\Exchange\Database_new.edb</Data>
<Data>-1032 (0xfffffbf8)</Data>
<Data>32 (0x00000020)</Data>
<Data>The process cannot access the file because it is being used by another process. </Data>
</EventData>
</Event>

ctagle

ASKER

Update: I suspected a backup program may be the culprit so I ran a manual back using the same settings and the scheduled backup and I noticed that the database dismounted itself right after the program finished copying the Exchange database. the backup program uses VSS to do the backups. I'm not sure if its a coincidence or if that is the issue or not though because after I uninstalled the program the database remounted itself. It stayed unmounted the entire time that the program was installed after that manual backup finished. Immediately after uninstalling it the database remounted itself, my only guess would be that the backup program is not releasing the database properly and it causes conflicts in Exchange. I'll check it early tomorrow and give an update if it fixes the issue.

Simon Butler (Sembee)

The other thing to check is AV software on the machine is not scanning the Exchange directories.
The correct exclusions are listed on TechNet.

Simon.

Viral Rathod

Error 0xfffffbf8 - This issue can occur if you do not have the appropriate permissions on the exchange folder to mount the mailbox store ,please check permission on exchange folder where database is stored.

ctagle

ASKER

This wasn't the answer but it did help to eliminate the database as the problem, turns out the program I had installed on the server to run nightly backups was killing the database and it wasn't remounting, uninstalled it and it worked just fine.