?
Solved

Corrupt User Mailbox Exchange 2010

Posted on 2014-08-29
10
Medium Priority
?
1,762 Views
Last Modified: 2014-09-10
VERY long story short, we had a failure. Had to restore exchange to VM. That all went fine.

The event log was throwing off I/O or read errors for the database. The DB is up, and will dismount clean and remount fine, but the event errors worried me. Since I wanted to free up whitespace anyway, and it is a good practice after something goes wrong. I created a new DB and began moving mailboxes to it. I'd say 80% moved fine, but I have a handful that have failed some at 0% all the way to 75%.

My first step was to run this to try and repair the user, and get him moved.
New-MailboxRepairRequest -Mailbox User -CorruptionType ProvisionedFolder,SearchFolder,AggregateCounts,Folderview

Open in new window


This created 2 events, one a 10047 that it started. Then a few mins later it fails with the below.
Log Name:      Application
Source:        MSExchangeIS Mailbox Store
Date:          8/29/2014 8:39:17 AM
Event ID:      10049
Task Category: General
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      myserver.abc.loc
Description:
Online integrity check for request da339ff0-1356-4a49-a2c6-e3b47284aefa failed with error -1018.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="MSExchangeIS Mailbox Store" />
    <EventID Qualifiers="49158">10049</EventID>
    <Level>2</Level>
    <Task>6</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2014-08-29T13:39:17.000000000Z" />
    <EventRecordID>2151157</EventRecordID>
    <Channel>Application</Channel>
    <Computer>myserver.abc.loc</Computer>
    <Security />
  </System>
  <EventData>
    <Data>da339ff0-1356-4a49-a2c6-e3b47284aefa</Data>
    <Data>-1018</Data>
    <Binary>5B444941475F4354585D000006020000FF281500000000000003F801000029E340100F010480D25560200F0104800201FE670201FE6729E340100F010480D25560200F0104800201FE670F01048029E340100F010480D25560200F0104800201FE670F01048029E340100F010480D25560200F0104800201FE670201FE6729E340100F010480D25560200F0104800201FE670F01048029E340100F010480D25560200F0104800201FE671701048029E340100F010480D25560200F0104800201FE670201FE6729E340100F010480D25560200F0104800201FE671701048029E340100F010480D25560200F0104800201FE670F01048029E340100F010480D25560200F0104800201FE670201FE6729E340100F010480D25560200F0104800B00F46717010480D25560200F01048003006E670F010480D25560200F0104804000F6670F010480D25560200F0104800B00F4670201FE67D25560200F01048003006E670F010480D25560200F0104804000F6670F010480B434401006FCFFFF7FF200100F0104807FBA401006FCFFFF7FF5401006FCFFFF9FCA401006FCFFFF9FB210105FB4CA001FBD00100F0104801FFD401006FCFFFF9FFE101050B4CA009F9E401006FCFFFF9FE0101050B4CA009FA0401006FCFFFF1FBD00100F0104801FFD401006FCFFFF97B500100F01048097F5401006FCFFFF17C600100F01048037B9401006FCFFFF37F900100F01048017E6401006FCFFFF</Binary>
  </EventData>
</Event>

Open in new window


What are my options for getting this user moved?
0
Comment
Question by:bhieb
  • 6
  • 2
  • 2
10 Comments
 

Author Comment

by:bhieb
ID: 40292567
If it helps here is the event log that caused me to move the users to another DB and let this one die.

Log Name:      Application
Source:        ESE
Date:          8/29/2014 8:53:13 AM
Event ID:      474
Task Category: Database Page Cache
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      myserver.abc.loc
Description:
Information Store (2060) Mailbox_2: The database page read from the file "F:\ExchangeMailbox\Mailbox_2\Mailbox_2.edb" at offset 596017152 (0x0000000023868000) (database page 18188 (0x470C)) for 32768 (0x00008000) bytes failed verification due to a page checksum mismatch.  The expected checksum was [f8ba07458bdd6052:2239ddc6c056fa0e:801c7fe3ebddfab3:f1c0f1c0165e8920] and the actual checksum was [f8bc07432cea7335:2239ddc6c056fa0e:801c7fe3ebddfab3:e136e136a19f8ab1].  The read operation will fail with error -1018 (0xfffffc06).  If this condition persists then please restore the database from a previous backup.  This problem is likely due to faulty hardware. Please contact your hardware vendor for further assistance diagnosing the problem. 

For more information, click http://www.microsoft.com/contentredirect.asp.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="ESE" />
    <EventID Qualifiers="0">474</EventID>
    <Level>2</Level>
    <Task>2</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2014-08-29T13:53:13.000000000Z" />
    <EventRecordID>2151359</EventRecordID>
    <Channel>Application</Channel>
    <Computer>ARMAIL.alanritchey.loc</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Information Store</Data>
    <Data>2060</Data>
    <Data>Mailbox_2: </Data>
    <Data>F:\ExchangeMailbox\Mailbox_2\Mailbox_2.edb</Data>
    <Data>596017152 (0x0000000023868000)</Data>
    <Data>32768 (0x00008000)</Data>
    <Data>-1018 (0xfffffc06)</Data>
    <Data>[f8ba07458bdd6052:2239ddc6c056fa0e:801c7fe3ebddfab3:f1c0f1c0165e8920]</Data>
    <Data>[f8bc07432cea7335:2239ddc6c056fa0e:801c7fe3ebddfab3:e136e136a19f8ab1]</Data>
    <Data>18188 (0x470C)</Data>
  </EventData>
</Event>

Open in new window

0
 

Author Comment

by:bhieb
ID: 40292568
I have a feeling my only option here is to export the users stuff to a PST kill the old mailbox, and setup a new one then import. Luckily I guess, mine is one of them so I have someone to test on.
0
 
LVL 9

Accepted Solution

by:
dexIT earned 2000 total points
ID: 40292617
You could restore and mount a copy of the mailbox db and make an attempt to pull his mailbox from there:
http://exchangeserverpro.com/restore-individual-mailbox-exchange-2010/

This, of course, is a lengthy and tedious task, so I will have to agree that if you can in fact export his PST successfully with minimal corruption/errors, that creating a new mailbox maybe the most effective way to do this,

If the user still has a cached copy of his mailbox in Outlook, while offline, you could export everything from there, NK2, tasks, contacts and calendar included to a PST file. Create a new mailbox and import it.
0
Creating Active Directory Users from a Text File

If your organization has a need to mass-create AD user accounts, watch this video to see how its done without the need for scripting or other unnecessary complexities.

 

Author Comment

by:bhieb
ID: 40292704
Yah the problem with a restore is that since this disaster happened last week and it took several days to restore all of the data, I'd actually lose almost a week of emails. PST sounds like the option.

One other thought I had was to try a eseutil /p then /d on the database since there are so few users still in it then move them afterward. That entails a good deal of downtime. So far mine is the only one that is completely inaccessible.
0
 
LVL 9

Expert Comment

by:dexIT
ID: 40292721
ESEUTIL is great if you have the open window but its not always effective. I would recommend exporting any cached copies you may have in Outlook and use that PST to import to your new mailbox.
0
 

Author Comment

by:bhieb
ID: 40292799
All great ideas. It is 80% through a backup so once that is done I'll give some of it a try. I'll hold this open for a day in case someone else comes up with a magic bullet I'm just not seeing.
0
 
LVL 38

Expert Comment

by:Hypercat (Deb)
ID: 40292964
Can you open the mailbox at this point to export it to PST?  You might want to look at the move mailbox log first. If the error is caused by a specific item or items that are corrupt, the log will tell you what they are and you may be able to open the mailbox, delete those items from the mailbox and then move it successfully.
0
 

Author Comment

by:bhieb
ID: 40293099
Good catch. Here is the log. Doesn't look like it is any corrupt items, and is probably deeper seems to be a read failure right from the get go.

8/29/2014 12:26:58 AM [myserver] 'abc.loc/Users/Administrator' created move request.
8/29/2014 12:42:22 AM [myserver] The Microsoft Exchange Mailbox Replication service 'myserver.abc.loc' (14.3.178.0 caps:07) is examining the request.
8/29/2014 12:42:22 AM [myserver] Connected to target mailbox 'Primary (8f697241-dc48-4298-ac96-01357a40ab71)', database 'Mbx1', Mailbox server 'myserver.abc.loc' Version 14.3 (Build 123.0).
8/29/2014 12:42:22 AM [myserver] Connected to source mailbox 'Primary (8f697241-dc48-4298-ac96-01357a40ab71)', database 'Mailbox_1', Mailbox server 'myserver.abc.loc' Version 14.3 (Build 195.0).
8/29/2014 12:42:22 AM [myserver] Request processing started.
8/29/2014 12:42:22 AM [myserver] Fatal error MapiExceptionJetErrorReadVerifyFailure has occurred.
Error details: MapiExceptionJetErrorReadVerifyFailure: Unable to read mailbox signature basic info. (hr=0x80004005, ec=-1018)
Diagnostic context:
    Lid: 46143  
    Lid: 62527   StoreEc: 0xFFFFFC06
    Lid: 1494    ---- Remote Context Beg ----
    Lid: 1238    Remote Context Overflow
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 13236   StoreEc: 0xFFFFFC06
    Lid: 43096  
    Lid: 61528   StoreEc: 0xFFFFFC06
    Lid: 60504  
    Lid: 35928   StoreEc: 0xFFFFFC06
    Lid: 36952  
    Lid: 56383   StoreEc: 0xFFFFFC06
    Lid: 33855   StoreEc: 0xFFFFFC06
    Lid: 37119   StoreEc: 0xFFFFFC06
    Lid: 1750    ---- Remote Context End ----
    Lid: 42047  
    Lid: 58431   StoreEc: 0xFFFFFC06
    Lid: 45887  
    Lid: 62271   StoreEc: 0xFFFFFC06
   at Microsoft.Mapi.MapiExceptionHelper.ThrowIfError(String message, Int32 hresult, SafeExInterfaceHandle iUnknown, Exception innerException)
   at Microsoft.Mapi.ExRpcAdmin.GetMailboxSignatureBasicInfo(Guid guidMdb, Guid guidMailbox, MailboxSignatureFlags flags)
   at Microsoft.Mapi.ExRpcAdmin.GetMailboxBasicInfo(Guid guidMdb, Guid guidMailbox, MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.LocalSourceMailbox.Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo(MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.SourceMailboxWrapper.<>c__DisplayClass1.<Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo>b__0()
   at Microsoft.Exchange.MailboxReplicationService.ExecutionContext.Execute(GenericCallDelegate operation)
   at Microsoft.Exchange.MailboxReplicationService.SourceMailboxWrapper.Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo(MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.MoveBaseJob.<>c__DisplayClass2b.<CreateDestinationMailbox>b__25()
   at Microsoft.Exchange.MailboxReplicationService.ExecutionContext.Execute(GenericCallDelegate operation)
   at Microsoft.Exchange.MailboxReplicationService.MoveBaseJob.CreateDestinationMailbox(Object[] wiParams)
   at Microsoft.Exchange.MailboxReplicationService.CommonUtils.CatchKnownExceptions(GenericCallDelegate del, FailureDelegate failureDelegate)
Error context: --------
Operation: ISourceMailbox.GetMailboxBasicInfo
OperationSide: Source
Primary (8f697241-dc48-4298-ac96-01357a40ab71)
Flags: GetMailboxSignature
--------
Mailbox: Primary (8f697241-dc48-4298-ac96-01357a40ab71)
8/29/2014 12:42:22 AM [myserver] Relinquishing job.
8/29/2014 7:51:32 AM [myserver] 'abc.loc/Users/Administrator' resumed move request.
8/29/2014 7:51:38 AM [myserver] Job resumed with status 'Queued'.
8/29/2014 7:51:38 AM [myserver] Relinquishing job.
8/29/2014 7:51:44 AM [myserver] The Microsoft Exchange Mailbox Replication service 'myserver.abc.loc' (14.3.178.0 caps:07) is examining the request.
8/29/2014 7:51:45 AM [myserver] Connected to target mailbox 'Primary (8f697241-dc48-4298-ac96-01357a40ab71)', database 'Mbx1', Mailbox server 'myserver.abc.loc' Version 14.3 (Build 123.0).
8/29/2014 7:51:45 AM [myserver] Connected to source mailbox 'Primary (8f697241-dc48-4298-ac96-01357a40ab71)', database 'Mailbox_1', Mailbox server 'myserver.abc.loc' Version 14.3 (Build 195.0).
8/29/2014 7:51:45 AM [myserver] Request processing started.
8/29/2014 7:51:45 AM [myserver] Fatal error MapiExceptionJetErrorReadVerifyFailure has occurred.
Error details: MapiExceptionJetErrorReadVerifyFailure: Unable to read mailbox signature basic info. (hr=0x80004005, ec=-1018)
Diagnostic context:
    Lid: 46143  
    Lid: 62527   StoreEc: 0xFFFFFC06
    Lid: 1494    ---- Remote Context Beg ----
    Lid: 1238    Remote Context Overflow
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 13236   StoreEc: 0xFFFFFC06
    Lid: 43096  
    Lid: 61528   StoreEc: 0xFFFFFC06
    Lid: 60504  
    Lid: 35928   StoreEc: 0xFFFFFC06
    Lid: 36952  
    Lid: 56383   StoreEc: 0xFFFFFC06
    Lid: 33855   StoreEc: 0xFFFFFC06
    Lid: 37119   StoreEc: 0xFFFFFC06
    Lid: 1750    ---- Remote Context End ----
    Lid: 42047  
    Lid: 58431   StoreEc: 0xFFFFFC06
    Lid: 45887  
    Lid: 62271   StoreEc: 0xFFFFFC06
   at Microsoft.Mapi.MapiExceptionHelper.ThrowIfError(String message, Int32 hresult, SafeExInterfaceHandle iUnknown, Exception innerException)
   at Microsoft.Mapi.ExRpcAdmin.GetMailboxSignatureBasicInfo(Guid guidMdb, Guid guidMailbox, MailboxSignatureFlags flags)
   at Microsoft.Mapi.ExRpcAdmin.GetMailboxBasicInfo(Guid guidMdb, Guid guidMailbox, MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.LocalSourceMailbox.Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo(MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.SourceMailboxWrapper.<>c__DisplayClass1.<Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo>b__0()
   at Microsoft.Exchange.MailboxReplicationService.ExecutionContext.Execute(GenericCallDelegate operation)
   at Microsoft.Exchange.MailboxReplicationService.SourceMailboxWrapper.Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo(MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.MoveBaseJob.<>c__DisplayClass2b.<CreateDestinationMailbox>b__25()
   at Microsoft.Exchange.MailboxReplicationService.ExecutionContext.Execute(GenericCallDelegate operation)
   at Microsoft.Exchange.MailboxReplicationService.MoveBaseJob.CreateDestinationMailbox(Object[] wiParams)
   at Microsoft.Exchange.MailboxReplicationService.CommonUtils.CatchKnownExceptions(GenericCallDelegate del, FailureDelegate failureDelegate)
Error context: --------
Operation: ISourceMailbox.GetMailboxBasicInfo
OperationSide: Source
Primary (8f697241-dc48-4298-ac96-01357a40ab71)
Flags: GetMailboxSignature
--------
Mailbox: Primary (8f697241-dc48-4298-ac96-01357a40ab71)
8/29/2014 7:51:45 AM [myserver] Relinquishing job.

Open in new window

0
 
LVL 38

Expert Comment

by:Hypercat (Deb)
ID: 40293367
This appears to be a pretty catastrophic failure.  I would be surprised if you can open the mailbox at all.  Have you tried?  If you can open it, you just might be able to export it to a PST, and/or use the Inbox Repair tool to repair it.

A Microsoft tech article I found about mailbox repair request errors indicated that if you keep getting this error you should run eseutil to repair the database.  That 474 error also points in the same direction.

If you can't open these mailboxes, and really need to try to recover them, I would go with dexIT's suggestion to try to restore a copy of the DB and repair it.  Or if you have a pretty recent database backup before the distaster occurred, you might be able to restore and mount that and at least recover most of the data from those mailboxes.
0
 

Author Comment

by:bhieb
ID: 40293521
I do have a fairly recent backup, and mine is really the only one fully inaccessible. I'm in the process of moving every stable mailbox to a new store, after that is complete I'll come back and see what can be done about the few that won't move. But even then most of them are usable just too corrupt to move. I'm hopeful between a eseutil /p to repair it and possible a restore from recovery just to be sure some old ones don't go missing, it will work out.
0

Featured Post

Problems using Powershell and Active Directory?

Managing Active Directory does not always have to be complicated.  If you are spending more time trying instead of doing, then it's time to look at something else. For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Want to know how to use Exchange Server Eseutil command? Go through this article as it gives you the know-how.
Are you looking for the options available for exporting EDB files to PST? You may be confused as they are different in different Exchange versions. Here, I will discuss some options available.
This tutorial will show how to configure a single USB drive with a separate folder for each day of the week. This will allow each of the backups to be kept separate preventing the previous day’s backup from being overwritten. The USB drive must be s…
Exchange organizations may use the Journaling Agent of the Transport Service to archive messages going through Exchange. However, if the Transport Service is integrated with some email content management application (such as an anti-spam), the admin…
Suggested Courses

829 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question