Link to home
Start Free TrialLog in
Avatar of BMConsult
BMConsult

asked on

Accidentally removed Mailbox Store connector using ADSI Edit, manually added it back, problems since

Here's the background: I was using ADSI Edit to manually remove a mail gateway from the ".../CN=First Routing Group/CN=Connections" and accidentally removed the connection for the Mailbox Store from ".../CN=Organization/CN=Connections" instead. So I manually reconstructed the connection using the exact same string for the distinguished name (which I obtained from elsewhere) and did a best guess on the other object parameters. In hindsight, I think I should have immediately done an Active Directory restore as I had done a system state backup and full Exchange backup prior to doing any changes.  Subsequent to that, I restarted the server and mail started flowing back into the Mailbox Store and I thought it was all resolved.

However, since then (now a week ago) there has been a problem where the .log files consumed all disk space after the application trying to connect to the mail gateway continually errored out with "Exchange Gateway logon failed - OpenMsgStore failed - MAPI_E_CORRUPT_DATA".  I stopped the application from connecting and the logs have stabilized but are still sitting around -- all 12,000+ of them.  If I re-enable it, I get the message "Exchange Gateway logon failed - OpenMsgStore failed - MAPI_E_FAILONEPROVIDER".  A full backup is now twice the size of what it was before the change (100GB vs 50GB).  I could restore the Active Directory and restore the Exchange backup but I don't want to lose a week's worth of emails.  What's my best approach for both restoring the connection to the way it was before and also not losing the new emails that have arrived since?

I tried adding a second mailbox store and moving accounts between them, but Active Directory reports a permissions error doing that.

My guess is that when I manually reconstructed the connection for the Mailbox Store, I got a parameter wrong and now it's in the database file under a different folder or something, or maybe it's storing it in two places since the size looks to be double. So if I simply restore Active Directory, it might not recognize that folder and I could lose everything since... or maybe it will be fine. I'm presently running eseutil /g on the file and it's taking forever (630MB at about 3%) but will report more as I find out.

Outlook periodically reports synchronization errors, such as:

11:19:53 Error synchronizing folder
11:19:53              [8004010F-501-8004010F-0]
11:19:53              The client operation failed.
11:19:53              Microsoft Exchange Server Information Store
11:19:53              For more information on this failure, click the URL below:
11:19:53              http://www.microsoft.com/support/prodredirect/outlook2000_us.asp?err=8004010f-501-8004010f-0

The folder listed before this message varies... sometimes Sent Items, sometimes Deleted Items, etc.  But the error number is always consistent.

As you can tell, I may not be an Active Directory guru... but I generally don't make boneheaded mistakes like deleting an entry from the wrong connections folder. If you can provide a solution to my dilemma with steps to take, please outline the detailed steps involved so that I don't assume something incorrectly.

Many thanks in advance!
Avatar of abhi_ak
abhi_ak
Flag of India image

Hello ,
Did you try re-creating the users OST file
Avatar of BMConsult
BMConsult

ASKER

There are hundreds of users, I want something that will do all of them without having to go through each one individually.
Update on integrity check:

C:\Program Files\Exchsrvr\MDBDATA>..\bin\eseutil /g priv1.edb

Microsoft(R) Exchange Server Database Utilities
Version 6.5
Copyright (C) Microsoft Corporation. All Rights Reserved.

Initiating INTEGRITY mode...
        Database: priv1.edb
  Streaming File: priv1.STM
  Temp. Database: TEMPINTEG7968.EDB

Checking database integrity.

                     Scanning Status (% complete)

          0    10   20   30   40   50   60   70   80   90  100
          |----|----|----|----|----|----|----|----|----|----|
          ...................................................


Integrity check successful.

Operation completed successfully in 14743.31 seconds.

--

So this didn't appear to report any errors in integrity.
any errors in the application or the system logs
please do let me know on < removed by sembee> and possibly send me the application and the system logs,
Checksum results:

Microsoft(R) Exchange Server Database Utilities
Version 6.5
Copyright (C) Microsoft Corporation. All Rights Reserved.

Initiating CHECKSUM mode...
        Database: priv1.edb
  Streaming File: priv1.STM
  Temp. Database: TEMPCHKSUM7524.EDB


File: priv1.edb

                     Checksum Status (% complete)

          0    10   20   30   40   50   60   70   80   90  100
          |----|----|----|----|----|----|----|----|----|----|
          ...................................................


22207890 pages seen
0 bad checksums
5259 uninitialized pages
0 wrong page numbers
0x1330f64f highest dbtime (pgno 0x15171b9)

1387994 reads performed
86749 MB read
2107 seconds taken
41 MB/second
-2138372612 milliseconds used
1553 milliseconds per read
4844 milliseconds for the slowest read
0 milliseconds for the fastest read



File: priv1.STM

                     Checksum Status (% complete)

          0    10   20   30   40   50   60   70   80   90  100
          |----|----|----|----|----|----|----|----|----|----|
          ...................................................


4594688 pages seen
0 bad checksums
412730 uninitialized pages


Operation completed successfully in 2716.94 seconds.
File dump:

Microsoft(R) Exchange Server Database Utilities
Version 6.5
Copyright (C) Microsoft Corporation. All Rights Reserved.

Initiating FILE DUMP mode...
         Database: priv1.edb

        File Type: Database
   Format ulMagic: 0x89abcdef
   Engine ulMagic: 0x89abcdef
 Format ulVersion: 0x620,11
 Engine ulVersion: 0x620,11
Created ulVersion: 0x620,9
     DB Signature: Create time:02/28/2005 17:22:38 Rand:24734849 Computer:
         cbDbPage: 4096
           dbtime: 321975887 (0x1330f64f)
            State: Clean Shutdown
     Log Required: 0-0 (0x0-0x0)
   Streaming File: Yes
         Shadowed: Yes
       Last Objid: 106087
     Scrub Dbtime: 0 (0x0)
       Scrub Date: 00/00/1900 00:00:00
     Repair Count: 0
      Repair Date: 00/00/1900 00:00:00
  Last Consistent: (0x92A1,1794,3C)  05/23/2006 20:48:08
      Last Attach: (0x92A0,140F,19F)  05/23/2006 19:57:43
      Last Detach: (0x92A1,1794,3C)  05/23/2006 20:48:08
             Dbid: 1
    Log Signature: Create time:02/28/2005 17:22:35 Rand:24685950 Computer:
       OS Version: (5.2.3790 SP 1)

Previous Full Backup:
        Log Gen: 37444-37449 (0x9244-0x9249)
           Mark: (0x9244,1F00,4E)
           Mark: 05/20/2006 18:32:25

Previous Incremental Backup:
        Log Gen: 25069-37534 (0x61ed-0x929e)
           Mark: (0x929F,8,16)
           Mark: 05/23/2006 17:35:58

Current Full Backup:
        Log Gen: 0-0 (0x0-0x0)
           Mark: (0x0,0,0)
           Mark: 00/00/1900 00:00:00

Current Shadow copy backup:
        Log Gen: 0-0 (0x0-0x0)
           Mark: (0x0,0,0)
           Mark: 00/00/1900 00:00:00

     cpgUpgrade55Format: 0
    cpgUpgradeFreePages: 0
cpgUpgradeSpaceMapPages: 0

Operation completed successfully in 3.0 seconds.
Interesting part here:

C:\Program Files\Exchsrvr\MDBDATA>..\bin\isinteg -s myserver -verbose -test alltests
Databases for server myserver:
Only databases marked as Offline can be checked

Index  Status       Database-Name
Storage Group Name: First Storage Group
  1    Offline      Mailbox Store (MYSERVER)
  2    Online       Public Folder Store (MYSERVER)
Storage Group Name: Recovery Storage Group
  3    Offline      Mailbox Store (MYSERVER)
Enter a number to select a database or press Return to exit.

--

There should only be one Mailbox Store present here and a Public Folder Store. However, the isinteg utility thinks there are two Mailbox Stores.
when and where do you get this error
 "Exchange Gateway logon failed - OpenMsgStore failed - MAPI_E_CORRUPT_DATA".  
The Exchange Gateway Login error is in a third-party application that integrates with Exchange. Never been a problem before, only started those errors once the ADSI info got messed up.
Hello ,
PLease also check with this KB http://support.microsoft.com/kb/839531/en-us

and let me know weather we have these permissions on the ORG level.

Regaards,
Abhijeet
can you meet me on < removed by sembee> is my id
Re: http://support.microsoft.com/kb/839531/en-us

Not an issue. Everything appears on the surface to behave fine from Outlook's perspective, save the very infrequent sync issues.

It's extremely late here. I'll pick this thread up again in the morning.
ok then send me a email on the above mail address.... as i wont be on yahoo till then i would switch to msn and i cannot give u my msn id out here send me a mail and will provide u my msn id my mail addy is <removed by sembee>
More information: if I create a new mailbox store called Second Store and attempt to move a mailbox there, I get this:

  <?xml version="1.0" encoding="unicode" ?>
- <taskWizardRun taskName="Move Mailbox" dcName="MYSERVER" buildNumber="7226" runningAs="Administrator@MYDOMAIN.COM">
  <timespan startTime="2006-05-24 16:10:14.733" milliseconds="219" />
- <moveMailbox mixedMode="true" maxBadItems="0">
- <destination>
  <database>/dc=com/dc=mydomain/cn=Configuration/cn=Services/cn=Microsoft Exchange/cn=MyOrganization/cn=Administrative Groups/cn=First Administrative Group/cn=Servers/cn=MYSERVER/cn=InformationStore/cn=First Storage Group/cn=Second Store</database>
  </destination>
  </moveMailbox>
  <taskSummary errorCount="1" completedCount="0" warningCount="0" errorCode="0x00000000" />
- <items>
- <item adsPath="John Smith" class="user">
  <progress code="0" milliseconds="219">Opening source mailbox.</progress>
- <summary isWarning="false" errorCode="0xc1050000">
  The information store could not be opened. The MAPI provider failed. MAPI 1.0 ID no: 8004011d-0289-00000000
- <details>
- <source>
  <database>/dc=com/dc=mydomain/cn=Configuration/cn=Services/cn=Microsoft Exchange/cn=MyOrganization/cn=Administrative Groups/cn=First Administrative Group/cn=Servers/cn=MYSERVER/cn=InformationStore/cn=First Storage Group/cn=Mailbox Store (MYSERVER)</database>
  </source>
  </details>
  </summary>
  </item>
  </items>
  </taskWizardRun>
ASKER CERTIFIED SOLUTION
Avatar of abhi_ak
abhi_ak
Flag of India image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Thank you for the suggestions.  Application event info/warnings/errors from the time of move, in order:


MSExchangeIS, Event ID 1008:
MYDOMAIN\Administrator connected as /o=MyOrganization/ou=First Administrative Group/cn=Configuration/cn=Servers/cn=MYSERVER/cn=Microsoft System Attendant using administrator privileges.


MSExchangeIS, Event ID 9548:
Disabled user /o=MyOrganization/ou=First Administrative Group/cn=Recipients/cn=jsmith does not have a master account SID. Please use Active Directory MMC to set an active account as this user's master account.


MSExchangeIS Mailbox, Event ID 1022:
Logon Failure on database "First Storage Group\Mailbox Store (MYSERVER)" - Windows 2000 account MYDOMAIN\Administrator; mailbox /o=MyOrganization/ou=First Administrative Group/cn=Recipients/cn=jsmith.
Error: -2147221231


MSExchangeAdmin, Event ID 9175:
The MAPI call 'OpenMsgStore' failed with the following error:
The information store could not be opened.
The MAPI provider failed.
MAPI 1.0
ID no: 8004011d-0289-00000000


MSExchangeAdmin, Event ID 9168:
Failed to open mailbox '/o=MyOrganization/ou=First Administrative Group/cn=Recipients/cn=jsmith' in mailbox store '/o=MyOrganization/ou=First Administrative Group/cn=Configuration/cn=Servers/cn=MYSERVER/cn=Microsoft Private MDB' on server 'MYSERVER'.
Error: The information store could not be opened.
The MAPI provider failed.
MAPI 1.0
ID no: 8004011d-0289-00000000


MSExchangeAdmin, Event ID 1008:
Unable to move mailbox 'John Smith'.
Error: The information store could not be opened.
The MAPI provider failed.
MAPI 1.0
ID no: 8004011d-0289-00000000
Ran through various utilities... after countless backups and running eseutil and isinteg to do tests and then fixes, and finally running a defrag on the Exchange store seems to have resolved this.  I still haven't had to do an Active Directory restore as it apparently is working fine with the new connector.

Awarding points to abhi_ak as he's the only one to respond and his suggestions to check the event log after encountering those errors were helpful in resolving the issue.