We help IT Professionals succeed at work.

Corrupt User Mailbox Exchange 2010

bhieb
bhieb asked
on
2,892 Views
Last Modified: 2014-09-10
VERY long story short, we had a failure. Had to restore exchange to VM. That all went fine.

The event log was throwing off I/O or read errors for the database. The DB is up, and will dismount clean and remount fine, but the event errors worried me. Since I wanted to free up whitespace anyway, and it is a good practice after something goes wrong. I created a new DB and began moving mailboxes to it. I'd say 80% moved fine, but I have a handful that have failed some at 0% all the way to 75%.

My first step was to run this to try and repair the user, and get him moved.
New-MailboxRepairRequest -Mailbox User -CorruptionType ProvisionedFolder,SearchFolder,AggregateCounts,Folderview

Open in new window


This created 2 events, one a 10047 that it started. Then a few mins later it fails with the below.
Log Name:      Application
Source:        MSExchangeIS Mailbox Store
Date:          8/29/2014 8:39:17 AM
Event ID:      10049
Task Category: General
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      myserver.abc.loc
Description:
Online integrity check for request da339ff0-1356-4a49-a2c6-e3b47284aefa failed with error -1018.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="MSExchangeIS Mailbox Store" />
    <EventID Qualifiers="49158">10049</EventID>
    <Level>2</Level>
    <Task>6</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2014-08-29T13:39:17.000000000Z" />
    <EventRecordID>2151157</EventRecordID>
    <Channel>Application</Channel>
    <Computer>myserver.abc.loc</Computer>
    <Security />
  </System>
  <EventData>
    <Data>da339ff0-1356-4a49-a2c6-e3b47284aefa</Data>
    <Data>-1018</Data>
    <Binary>5B444941475F4354585D000006020000FF281500000000000003F801000029E340100F010480D25560200F0104800201FE670201FE6729E340100F010480D25560200F0104800201FE670F01048029E340100F010480D25560200F0104800201FE670F01048029E340100F010480D25560200F0104800201FE670201FE6729E340100F010480D25560200F0104800201FE670F01048029E340100F010480D25560200F0104800201FE671701048029E340100F010480D25560200F0104800201FE670201FE6729E340100F010480D25560200F0104800201FE671701048029E340100F010480D25560200F0104800201FE670F01048029E340100F010480D25560200F0104800201FE670201FE6729E340100F010480D25560200F0104800B00F46717010480D25560200F01048003006E670F010480D25560200F0104804000F6670F010480D25560200F0104800B00F4670201FE67D25560200F01048003006E670F010480D25560200F0104804000F6670F010480B434401006FCFFFF7FF200100F0104807FBA401006FCFFFF7FF5401006FCFFFF9FCA401006FCFFFF9FB210105FB4CA001FBD00100F0104801FFD401006FCFFFF9FFE101050B4CA009F9E401006FCFFFF9FE0101050B4CA009FA0401006FCFFFF1FBD00100F0104801FFD401006FCFFFF97B500100F01048097F5401006FCFFFF17C600100F01048037B9401006FCFFFF37F900100F01048017E6401006FCFFFF</Binary>
  </EventData>
</Event>

Open in new window


What are my options for getting this user moved?
Comment
Watch Question

Author

Commented:
If it helps here is the event log that caused me to move the users to another DB and let this one die.

Log Name:      Application
Source:        ESE
Date:          8/29/2014 8:53:13 AM
Event ID:      474
Task Category: Database Page Cache
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      myserver.abc.loc
Description:
Information Store (2060) Mailbox_2: The database page read from the file "F:\ExchangeMailbox\Mailbox_2\Mailbox_2.edb" at offset 596017152 (0x0000000023868000) (database page 18188 (0x470C)) for 32768 (0x00008000) bytes failed verification due to a page checksum mismatch.  The expected checksum was [f8ba07458bdd6052:2239ddc6c056fa0e:801c7fe3ebddfab3:f1c0f1c0165e8920] and the actual checksum was [f8bc07432cea7335:2239ddc6c056fa0e:801c7fe3ebddfab3:e136e136a19f8ab1].  The read operation will fail with error -1018 (0xfffffc06).  If this condition persists then please restore the database from a previous backup.  This problem is likely due to faulty hardware. Please contact your hardware vendor for further assistance diagnosing the problem. 

For more information, click http://www.microsoft.com/contentredirect.asp.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="ESE" />
    <EventID Qualifiers="0">474</EventID>
    <Level>2</Level>
    <Task>2</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2014-08-29T13:53:13.000000000Z" />
    <EventRecordID>2151359</EventRecordID>
    <Channel>Application</Channel>
    <Computer>ARMAIL.alanritchey.loc</Computer>
    <Security />
  </System>
  <EventData>
    <Data>Information Store</Data>
    <Data>2060</Data>
    <Data>Mailbox_2: </Data>
    <Data>F:\ExchangeMailbox\Mailbox_2\Mailbox_2.edb</Data>
    <Data>596017152 (0x0000000023868000)</Data>
    <Data>32768 (0x00008000)</Data>
    <Data>-1018 (0xfffffc06)</Data>
    <Data>[f8ba07458bdd6052:2239ddc6c056fa0e:801c7fe3ebddfab3:f1c0f1c0165e8920]</Data>
    <Data>[f8bc07432cea7335:2239ddc6c056fa0e:801c7fe3ebddfab3:e136e136a19f8ab1]</Data>
    <Data>18188 (0x470C)</Data>
  </EventData>
</Event>

Open in new window

Author

Commented:
I have a feeling my only option here is to export the users stuff to a PST kill the old mailbox, and setup a new one then import. Luckily I guess, mine is one of them so I have someone to test on.
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION

Author

Commented:
Yah the problem with a restore is that since this disaster happened last week and it took several days to restore all of the data, I'd actually lose almost a week of emails. PST sounds like the option.

One other thought I had was to try a eseutil /p then /d on the database since there are so few users still in it then move them afterward. That entails a good deal of downtime. So far mine is the only one that is completely inaccessible.

Commented:
ESEUTIL is great if you have the open window but its not always effective. I would recommend exporting any cached copies you may have in Outlook and use that PST to import to your new mailbox.

Author

Commented:
All great ideas. It is 80% through a backup so once that is done I'll give some of it a try. I'll hold this open for a day in case someone else comes up with a magic bullet I'm just not seeing.
Hypercat (Deb)President
CERTIFIED EXPERT

Commented:
Can you open the mailbox at this point to export it to PST?  You might want to look at the move mailbox log first. If the error is caused by a specific item or items that are corrupt, the log will tell you what they are and you may be able to open the mailbox, delete those items from the mailbox and then move it successfully.

Author

Commented:
Good catch. Here is the log. Doesn't look like it is any corrupt items, and is probably deeper seems to be a read failure right from the get go.

8/29/2014 12:26:58 AM [myserver] 'abc.loc/Users/Administrator' created move request.
8/29/2014 12:42:22 AM [myserver] The Microsoft Exchange Mailbox Replication service 'myserver.abc.loc' (14.3.178.0 caps:07) is examining the request.
8/29/2014 12:42:22 AM [myserver] Connected to target mailbox 'Primary (8f697241-dc48-4298-ac96-01357a40ab71)', database 'Mbx1', Mailbox server 'myserver.abc.loc' Version 14.3 (Build 123.0).
8/29/2014 12:42:22 AM [myserver] Connected to source mailbox 'Primary (8f697241-dc48-4298-ac96-01357a40ab71)', database 'Mailbox_1', Mailbox server 'myserver.abc.loc' Version 14.3 (Build 195.0).
8/29/2014 12:42:22 AM [myserver] Request processing started.
8/29/2014 12:42:22 AM [myserver] Fatal error MapiExceptionJetErrorReadVerifyFailure has occurred.
Error details: MapiExceptionJetErrorReadVerifyFailure: Unable to read mailbox signature basic info. (hr=0x80004005, ec=-1018)
Diagnostic context:
    Lid: 46143  
    Lid: 62527   StoreEc: 0xFFFFFC06
    Lid: 1494    ---- Remote Context Beg ----
    Lid: 1238    Remote Context Overflow
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 13236   StoreEc: 0xFFFFFC06
    Lid: 43096  
    Lid: 61528   StoreEc: 0xFFFFFC06
    Lid: 60504  
    Lid: 35928   StoreEc: 0xFFFFFC06
    Lid: 36952  
    Lid: 56383   StoreEc: 0xFFFFFC06
    Lid: 33855   StoreEc: 0xFFFFFC06
    Lid: 37119   StoreEc: 0xFFFFFC06
    Lid: 1750    ---- Remote Context End ----
    Lid: 42047  
    Lid: 58431   StoreEc: 0xFFFFFC06
    Lid: 45887  
    Lid: 62271   StoreEc: 0xFFFFFC06
   at Microsoft.Mapi.MapiExceptionHelper.ThrowIfError(String message, Int32 hresult, SafeExInterfaceHandle iUnknown, Exception innerException)
   at Microsoft.Mapi.ExRpcAdmin.GetMailboxSignatureBasicInfo(Guid guidMdb, Guid guidMailbox, MailboxSignatureFlags flags)
   at Microsoft.Mapi.ExRpcAdmin.GetMailboxBasicInfo(Guid guidMdb, Guid guidMailbox, MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.LocalSourceMailbox.Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo(MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.SourceMailboxWrapper.<>c__DisplayClass1.<Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo>b__0()
   at Microsoft.Exchange.MailboxReplicationService.ExecutionContext.Execute(GenericCallDelegate operation)
   at Microsoft.Exchange.MailboxReplicationService.SourceMailboxWrapper.Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo(MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.MoveBaseJob.<>c__DisplayClass2b.<CreateDestinationMailbox>b__25()
   at Microsoft.Exchange.MailboxReplicationService.ExecutionContext.Execute(GenericCallDelegate operation)
   at Microsoft.Exchange.MailboxReplicationService.MoveBaseJob.CreateDestinationMailbox(Object[] wiParams)
   at Microsoft.Exchange.MailboxReplicationService.CommonUtils.CatchKnownExceptions(GenericCallDelegate del, FailureDelegate failureDelegate)
Error context: --------
Operation: ISourceMailbox.GetMailboxBasicInfo
OperationSide: Source
Primary (8f697241-dc48-4298-ac96-01357a40ab71)
Flags: GetMailboxSignature
--------
Mailbox: Primary (8f697241-dc48-4298-ac96-01357a40ab71)
8/29/2014 12:42:22 AM [myserver] Relinquishing job.
8/29/2014 7:51:32 AM [myserver] 'abc.loc/Users/Administrator' resumed move request.
8/29/2014 7:51:38 AM [myserver] Job resumed with status 'Queued'.
8/29/2014 7:51:38 AM [myserver] Relinquishing job.
8/29/2014 7:51:44 AM [myserver] The Microsoft Exchange Mailbox Replication service 'myserver.abc.loc' (14.3.178.0 caps:07) is examining the request.
8/29/2014 7:51:45 AM [myserver] Connected to target mailbox 'Primary (8f697241-dc48-4298-ac96-01357a40ab71)', database 'Mbx1', Mailbox server 'myserver.abc.loc' Version 14.3 (Build 123.0).
8/29/2014 7:51:45 AM [myserver] Connected to source mailbox 'Primary (8f697241-dc48-4298-ac96-01357a40ab71)', database 'Mailbox_1', Mailbox server 'myserver.abc.loc' Version 14.3 (Build 195.0).
8/29/2014 7:51:45 AM [myserver] Request processing started.
8/29/2014 7:51:45 AM [myserver] Fatal error MapiExceptionJetErrorReadVerifyFailure has occurred.
Error details: MapiExceptionJetErrorReadVerifyFailure: Unable to read mailbox signature basic info. (hr=0x80004005, ec=-1018)
Diagnostic context:
    Lid: 46143  
    Lid: 62527   StoreEc: 0xFFFFFC06
    Lid: 1494    ---- Remote Context Beg ----
    Lid: 1238    Remote Context Overflow
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67EF0003
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 23921   StoreEc: 0x3EC     
    Lid: 21970   StoreEc: 0x8004010F PropTag: 0x67910014
    Lid: 13236   StoreEc: 0xFFFFFC06
    Lid: 43096  
    Lid: 61528   StoreEc: 0xFFFFFC06
    Lid: 60504  
    Lid: 35928   StoreEc: 0xFFFFFC06
    Lid: 36952  
    Lid: 56383   StoreEc: 0xFFFFFC06
    Lid: 33855   StoreEc: 0xFFFFFC06
    Lid: 37119   StoreEc: 0xFFFFFC06
    Lid: 1750    ---- Remote Context End ----
    Lid: 42047  
    Lid: 58431   StoreEc: 0xFFFFFC06
    Lid: 45887  
    Lid: 62271   StoreEc: 0xFFFFFC06
   at Microsoft.Mapi.MapiExceptionHelper.ThrowIfError(String message, Int32 hresult, SafeExInterfaceHandle iUnknown, Exception innerException)
   at Microsoft.Mapi.ExRpcAdmin.GetMailboxSignatureBasicInfo(Guid guidMdb, Guid guidMailbox, MailboxSignatureFlags flags)
   at Microsoft.Mapi.ExRpcAdmin.GetMailboxBasicInfo(Guid guidMdb, Guid guidMailbox, MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.LocalSourceMailbox.Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo(MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.SourceMailboxWrapper.<>c__DisplayClass1.<Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo>b__0()
   at Microsoft.Exchange.MailboxReplicationService.ExecutionContext.Execute(GenericCallDelegate operation)
   at Microsoft.Exchange.MailboxReplicationService.SourceMailboxWrapper.Microsoft.Exchange.MailboxReplicationService.ISourceMailbox.GetMailboxBasicInfo(MailboxSignatureFlags flags)
   at Microsoft.Exchange.MailboxReplicationService.MoveBaseJob.<>c__DisplayClass2b.<CreateDestinationMailbox>b__25()
   at Microsoft.Exchange.MailboxReplicationService.ExecutionContext.Execute(GenericCallDelegate operation)
   at Microsoft.Exchange.MailboxReplicationService.MoveBaseJob.CreateDestinationMailbox(Object[] wiParams)
   at Microsoft.Exchange.MailboxReplicationService.CommonUtils.CatchKnownExceptions(GenericCallDelegate del, FailureDelegate failureDelegate)
Error context: --------
Operation: ISourceMailbox.GetMailboxBasicInfo
OperationSide: Source
Primary (8f697241-dc48-4298-ac96-01357a40ab71)
Flags: GetMailboxSignature
--------
Mailbox: Primary (8f697241-dc48-4298-ac96-01357a40ab71)
8/29/2014 7:51:45 AM [myserver] Relinquishing job.

Open in new window

Hypercat (Deb)President
CERTIFIED EXPERT

Commented:
This appears to be a pretty catastrophic failure.  I would be surprised if you can open the mailbox at all.  Have you tried?  If you can open it, you just might be able to export it to a PST, and/or use the Inbox Repair tool to repair it.

A Microsoft tech article I found about mailbox repair request errors indicated that if you keep getting this error you should run eseutil to repair the database.  That 474 error also points in the same direction.

If you can't open these mailboxes, and really need to try to recover them, I would go with dexIT's suggestion to try to restore a copy of the DB and repair it.  Or if you have a pretty recent database backup before the distaster occurred, you might be able to restore and mount that and at least recover most of the data from those mailboxes.

Author

Commented:
I do have a fairly recent backup, and mine is really the only one fully inaccessible. I'm in the process of moving every stable mailbox to a new store, after that is complete I'll come back and see what can be done about the few that won't move. But even then most of them are usable just too corrupt to move. I'm hopeful between a eseutil /p to repair it and possible a restore from recovery just to be sure some old ones don't go missing, it will work out.

Gain unlimited access to on-demand training courses with an Experts Exchange subscription.

Get Access
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Empower Your Career
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE

Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Unlock the solution to this question.
Join our community and discover your potential

Experts Exchange is the only place where you can interact directly with leading experts in the technology field. Become a member today and access the collective knowledge of thousands of technology experts.

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.