Corrupt User Mailbox Exchange 2010

VERY long story short, we had a failure. Had to restore exchange to VM. That all went fine.

The event log was throwing off I/O or read errors for the database. The DB is up, and will dismount clean and remount fine, but the event errors worried me. Since I wanted to free up whitespace anyway, and it is a good practice after something goes wrong. I created a new DB and began moving mailboxes to it. I'd say 80% moved fine, but I have a handful that have failed some at 0% all the way to 75%.

My first step was to run this to try and repair the user, and get him moved.
New-MailboxRepairRequest -Mailbox User -CorruptionType ProvisionedFolder,SearchFolder,AggregateCounts,Folderview

Open in new window


This created 2 events, one a 10047 that it started. Then a few mins later it fails with the below.
Log Name:      Application
Source:        MSExchangeIS Mailbox Store
Date:          8/29/2014 8:39:17 AM
Event ID:      10049
Task Category: General
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      myserver.abc.loc
Description:
Online integrity check for request da339ff0-1356-4a49-a2c6-e3b47284aefa failed with error -1018.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="MSExchangeIS Mailbox Store" />
    <EventID Qualifiers="49158">10049</EventID>
    <Level>2</Level>
    <Task>6</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2014-08-29T13:39:17.000000000Z" />
    <EventRecordID>2151157</EventRecordID>
    <Channel>Application</Channel>
    <Computer>myserver.abc.loc</Computer>
    <Security />
  </System>
  <EventData>
    <Data>da339ff0-1356-4a49-a2c6-e3b47284aefa</Data>
    <Data>-1018</Data>
    <Binary>5B444941475F4354585D000006020000FF281500000000000003F801000029E340100F010480D25560200F0104800201FE670201FE6729E340100F010480D25560200F0104800201FE670F01048029E340100F010480D25560200F0104800201FE670F01048029E340100F010480D25560200F0104800201FE670201FE6729E340100F010480D25560200F0104800201FE670F01048029E340100F010480D25560200F0104800201FE671701048029E340100F010480D25560200F0104800201FE670201FE6729E340100F010480D25560200F0104800201FE671701048029E340100F010480D25560200F0104800201FE670F01048029E340100F010480D25560200F0104800201FE670201FE6729E340100F010480D25560200F0104800B00F46717010480D25560200F01048003006E670F010480D25560200F0104804000F6670F010480D25560200F0104800B00F4670201FE67D25560200F01048003006E670F010480D25560200F0104804000F6670F010480B434401006FCFFFF7FF200100F0104807FBA401006FCFFFF7FF5401006FCFFFF9FCA401006FCFFFF9FB210105FB4CA001FBD00100F0104801FFD401006FCFFFF9FFE101050B4CA009F9E401006FCFFFF9FE0101050B4CA009FA0401006FCFFFF1FBD00100F0104801FFD401006FCFFFF97B500100F01048097F5401006FCFFFF17C600100F01048037B9401006FCFFFF37F900100F01048017E6401006FCFFFF</Binary>
  </EventData>
</Event>

Open in new window


What are my options for getting this user moved?
bhiebAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

bhiebAuthor Commented:
If it helps here is the event log that caused me to move the users to another DB and let this one die.

Log Name:      Application
Source:        ESE
Date:          8/29/2014 8:53:13 AM
Event ID:      474
Task Category: Database Page Cache
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      myserver.abc.loc
Description:
Information Store (2060) Mailbox_2: The database page read from the file "F:\ExchangeMailbox\Mailbox_2\Mailbox_2.edb" at offset 596017152 (0x0000000023868000) (database page 18188 (0x470C)) for 32768 (0x00008000) bytes failed verification due to a page checksum mismatch.  The expected checksum was [f8ba07458bdd6052:2239ddc6c056fa0e:801c7fe3ebddfab3:f1c0f1c0165e8920] and the actual checksum was [f8bc07432cea7335:2239ddc6c056fa0e:801c7fe3ebddfab3:e136e136a19f8ab1].  The read operation will fail with error -1018 (0xfffffc06).  If this condition persists then please restore the database from a previous backup.  This problem is likely due to faulty hardware. Please contact your hardware vendor for further assistance diagnosing the problem. 

For more information, click http://www.microsoft.com/contentredirect.asp.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="ESE" />
    <EventID Qualifiers="0">474</EventID>
    <Level>2</Level>
    <Task>2</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2014-08-29T13:53:13.000000000Z" />
    <EventRecordID>2151359</EventRecordID>
    <Channel>Application</Channel>
    <Computer>ARMAIL.alanritchey.loc</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Information Store</Data>
    <Data>2060</Data>
    <Data>Mailbox_2: </Data>
    <Data>F:\ExchangeMailbox\Mailbox_2\Mailbox_2.edb</Data>
    <Data>596017152 (0x0000000023868000)</Data>
    <Data>32768 (0x00008000)</Data>
    <Data>-1018 (0xfffffc06)</Data>
    <Data>[f8ba07458bdd6052:2239ddc6c056fa0e:801c7fe3ebddfab3:f1c0f1c0165e8920]</Data>
    <Data>[f8bc07432cea7335:2239ddc6c056fa0e:801c7fe3ebddfab3:e136e136a19f8ab1]</Data>
    <Data>18188 (0x470C)</Data>
  </EventData>
</Event>

Open in new window

0
bhiebAuthor Commented:
I have a feeling my only option here is to export the users stuff to a PST kill the old mailbox, and setup a new one then import. Luckily I guess, mine is one of them so I have someone to test on.
0
dexITCommented:
You could restore and mount a copy of the mailbox db and make an attempt to pull his mailbox from there:
http://exchangeserverpro.com/restore-individual-mailbox-exchange-2010/

This, of course, is a lengthy and tedious task, so I will have to agree that if you can in fact export his PST successfully with minimal corruption/errors, that creating a new mailbox maybe the most effective way to do this,

If the user still has a cached copy of his mailbox in Outlook, while offline, you could export everything from there, NK2, tasks, contacts and calendar included to a PST file. Create a new mailbox and import it.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

bhiebAuthor Commented:
Yah the problem with a restore is that since this disaster happened last week and it took several days to restore all of the data, I'd actually lose almost a week of emails. PST sounds like the option.

One other thought I had was to try a eseutil /p then /d on the database since there are so few users still in it then move them afterward. That entails a good deal of downtime. So far mine is the only one that is completely inaccessible.
0
dexITCommented:
ESEUTIL is great if you have the open window but its not always effective. I would recommend exporting any cached copies you may have in Outlook and use that PST to import to your new mailbox.
0
bhiebAuthor Commented:
All great ideas. It is 80% through a backup so once that is done I'll give some of it a try. I'll hold this open for a day in case someone else comes up with a magic bullet I'm just not seeing.
0
Hypercat (Deb)Commented:
Can you open the mailbox at this point to export it to PST?  You might want to look at the move mailbox log first. If the error is caused by a specific item or items that are corrupt, the log will tell you what they are and you may be able to open the mailbox, delete those items from the mailbox and then move it successfully.
0
bhiebAuthor Commented:
Good catch. Here is the log. Doesn't look like it is any corrupt items, and is probably deeper seems to be a read failure right from the get go.

8/29/2014 12:26:58 AM [myserver] 'abc.loc/Users/Administrator' created move request.
8/29/2014 12:42:22 AM [myserver] The Microsoft Exchange Mailbox Replication service 'myserver.abc.loc' (14.3.178.0 caps:07) is examining the request.
8/29/2014 12:42:22 AM [myserver] Connected to target mailbox 'Primary (8f697241-dc48-4298-ac96-01357a40ab71)', database 'Mbx1', Mailbox server 'myserver.abc.loc' Version 14.3 (Build 123.0).
8/29/2014 12:42:22 AM [myserver] Connected to source mailbox 'Primary (8f697241-dc48-4298-ac96-01357a40ab71)', database 'Mailbox_1', Mailbox server 'myserver.abc.loc' Version 14.3 (Build 195.0).
8/29/2014 12:42:22 AM [myserver] Request processing started.
8/29/2014 12:42:22 AM [myserver] Fatal error MapiExceptionJetErrorReadVerifyFailure has occurred.
Error details: MapiExceptionJetErrorReadVerifyFailure: Unable to read mailbox signature basic info. (hr=0x80004005, ec=-1018)
Diagnostic context:
    Lid: 46143  
    Lid: 62527   StoreEc: 0xFFFFFC06
    Lid: 1494    ---- Remote Context Beg ----
    Lid: 1238    Remote Context Overflow
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 13236   StoreEc: 0xFFFFFC06
    Lid: 43096  
    Lid: 61528   StoreEc: 0xFFFFFC06
    Lid: 60504  
    Lid: 35928   StoreEc: 0xFFFFFC06
    Lid: 36952  
    Lid: 56383   StoreEc: 0xFFFFFC06
    Lid: 33855   StoreEc: 0xFFFFFC06
    Lid: 37119   StoreEc: 0xFFFFFC06
    Lid: 1750    ---- Remote Context End ----
    Lid: 42047  
    Lid: 58431   StoreEc: 0xFFFFFC06
    Lid: 45887  
    Lid: 62271   StoreEc: 0xFFFFFC06
   at Microsoft.Mapi.MapiExceptionHelper.ThrowIfError(String message, Int32 hresult, SafeExInterfaceHandle iUnknown, Exception innerException)
   at Microsoft.Mapi.ExRpcAdmin.GetMailboxSignatureBasicInfo(Guid guidMdb, Guid guidMailbox, MailboxSignatureFlags flags)
   at Microsoft.Mapi.ExRpcAdmin.GetMailboxBasicInfo(Guid guidMdb, Guid guidMailbox, MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.LocalSourceMailbox.Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo(MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.SourceMailboxWrapper.<>c__DisplayClass1.<Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo>b__0()
   at Microsoft.Exchange.MailboxReplicationService.ExecutionContext.Execute(GenericCallDelegate operation)
   at Microsoft.Exchange.MailboxReplicationService.SourceMailboxWrapper.Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo(MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.MoveBaseJob.<>c__DisplayClass2b.<CreateDestinationMailbox>b__25()
   at Microsoft.Exchange.MailboxReplicationService.ExecutionContext.Execute(GenericCallDelegate operation)
   at Microsoft.Exchange.MailboxReplicationService.MoveBaseJob.CreateDestinationMailbox(Object[] wiParams)
   at Microsoft.Exchange.MailboxReplicationService.CommonUtils.CatchKnownExceptions(GenericCallDelegate del, FailureDelegate failureDelegate)
Error context: --------
Operation: ISourceMailbox.GetMailboxBasicInfo
OperationSide: Source
Primary (8f697241-dc48-4298-ac96-01357a40ab71)
Flags: GetMailboxSignature
--------
Mailbox: Primary (8f697241-dc48-4298-ac96-01357a40ab71)
8/29/2014 12:42:22 AM [myserver] Relinquishing job.
8/29/2014 7:51:32 AM [myserver] 'abc.loc/Users/Administrator' resumed move request.
8/29/2014 7:51:38 AM [myserver] Job resumed with status 'Queued'.
8/29/2014 7:51:38 AM [myserver] Relinquishing job.
8/29/2014 7:51:44 AM [myserver] The Microsoft Exchange Mailbox Replication service 'myserver.abc.loc' (14.3.178.0 caps:07) is examining the request.
8/29/2014 7:51:45 AM [myserver] Connected to target mailbox 'Primary (8f697241-dc48-4298-ac96-01357a40ab71)', database 'Mbx1', Mailbox server 'myserver.abc.loc' Version 14.3 (Build 123.0).
8/29/2014 7:51:45 AM [myserver] Connected to source mailbox 'Primary (8f697241-dc48-4298-ac96-01357a40ab71)', database 'Mailbox_1', Mailbox server 'myserver.abc.loc' Version 14.3 (Build 195.0).
8/29/2014 7:51:45 AM [myserver] Request processing started.
8/29/2014 7:51:45 AM [myserver] Fatal error MapiExceptionJetErrorReadVerifyFailure has occurred.
Error details: MapiExceptionJetErrorReadVerifyFailure: Unable to read mailbox signature basic info. (hr=0x80004005, ec=-1018)
Diagnostic context:
    Lid: 46143  
    Lid: 62527   StoreEc: 0xFFFFFC06
    Lid: 1494    ---- Remote Context Beg ----
    Lid: 1238    Remote Context Overflow
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 13236   StoreEc: 0xFFFFFC06
    Lid: 43096  
    Lid: 61528   StoreEc: 0xFFFFFC06
    Lid: 60504  
    Lid: 35928   StoreEc: 0xFFFFFC06
    Lid: 36952  
    Lid: 56383   StoreEc: 0xFFFFFC06
    Lid: 33855   StoreEc: 0xFFFFFC06
    Lid: 37119   StoreEc: 0xFFFFFC06
    Lid: 1750    ---- Remote Context End ----
    Lid: 42047  
    Lid: 58431   StoreEc: 0xFFFFFC06
    Lid: 45887  
    Lid: 62271   StoreEc: 0xFFFFFC06
   at Microsoft.Mapi.MapiExceptionHelper.ThrowIfError(String message, Int32 hresult, SafeExInterfaceHandle iUnknown, Exception innerException)
   at Microsoft.Mapi.ExRpcAdmin.GetMailboxSignatureBasicInfo(Guid guidMdb, Guid guidMailbox, MailboxSignatureFlags flags)
   at Microsoft.Mapi.ExRpcAdmin.GetMailboxBasicInfo(Guid guidMdb, Guid guidMailbox, MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.LocalSourceMailbox.Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo(MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.SourceMailboxWrapper.<>c__DisplayClass1.<Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo>b__0()
   at Microsoft.Exchange.MailboxReplicationService.ExecutionContext.Execute(GenericCallDelegate operation)
   at Microsoft.Exchange.MailboxReplicationService.SourceMailboxWrapper.Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo(MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.MoveBaseJob.<>c__DisplayClass2b.<CreateDestinationMailbox>b__25()
   at Microsoft.Exchange.MailboxReplicationService.ExecutionContext.Execute(GenericCallDelegate operation)
   at Microsoft.Exchange.MailboxReplicationService.MoveBaseJob.CreateDestinationMailbox(Object[] wiParams)
   at Microsoft.Exchange.MailboxReplicationService.CommonUtils.CatchKnownExceptions(GenericCallDelegate del, FailureDelegate failureDelegate)
Error context: --------
Operation: ISourceMailbox.GetMailboxBasicInfo
OperationSide: Source
Primary (8f697241-dc48-4298-ac96-01357a40ab71)
Flags: GetMailboxSignature
--------
Mailbox: Primary (8f697241-dc48-4298-ac96-01357a40ab71)
8/29/2014 7:51:45 AM [myserver] Relinquishing job.

Open in new window

0
Hypercat (Deb)Commented:
This appears to be a pretty catastrophic failure.  I would be surprised if you can open the mailbox at all.  Have you tried?  If you can open it, you just might be able to export it to a PST, and/or use the Inbox Repair tool to repair it.

A Microsoft tech article I found about mailbox repair request errors indicated that if you keep getting this error you should run eseutil to repair the database.  That 474 error also points in the same direction.

If you can't open these mailboxes, and really need to try to recover them, I would go with dexIT's suggestion to try to restore a copy of the DB and repair it.  Or if you have a pretty recent database backup before the distaster occurred, you might be able to restore and mount that and at least recover most of the data from those mailboxes.
0
bhiebAuthor Commented:
I do have a fairly recent backup, and mine is really the only one fully inaccessible. I'm in the process of moving every stable mailbox to a new store, after that is complete I'll come back and see what can be done about the few that won't move. But even then most of them are usable just too corrupt to move. I'm hopeful between a eseutil /p to repair it and possible a restore from recovery just to be sure some old ones don't go missing, it will work out.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Exchange

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.