BMConsult
asked on
Accidentally removed Mailbox Store connector using ADSI Edit, manually added it back, problems since
Here's the background: I was using ADSI Edit to manually remove a mail gateway from the ".../CN=First Routing Group/CN=Connections" and accidentally removed the connection for the Mailbox Store from ".../CN=Organization/CN=Co nnections" instead. So I manually reconstructed the connection using the exact same string for the distinguished name (which I obtained from elsewhere) and did a best guess on the other object parameters. In hindsight, I think I should have immediately done an Active Directory restore as I had done a system state backup and full Exchange backup prior to doing any changes. Subsequent to that, I restarted the server and mail started flowing back into the Mailbox Store and I thought it was all resolved.
However, since then (now a week ago) there has been a problem where the .log files consumed all disk space after the application trying to connect to the mail gateway continually errored out with "Exchange Gateway logon failed - OpenMsgStore failed - MAPI_E_CORRUPT_DATA". I stopped the application from connecting and the logs have stabilized but are still sitting around -- all 12,000+ of them. If I re-enable it, I get the message "Exchange Gateway logon failed - OpenMsgStore failed - MAPI_E_FAILONEPROVIDER". A full backup is now twice the size of what it was before the change (100GB vs 50GB). I could restore the Active Directory and restore the Exchange backup but I don't want to lose a week's worth of emails. What's my best approach for both restoring the connection to the way it was before and also not losing the new emails that have arrived since?
I tried adding a second mailbox store and moving accounts between them, but Active Directory reports a permissions error doing that.
My guess is that when I manually reconstructed the connection for the Mailbox Store, I got a parameter wrong and now it's in the database file under a different folder or something, or maybe it's storing it in two places since the size looks to be double. So if I simply restore Active Directory, it might not recognize that folder and I could lose everything since... or maybe it will be fine. I'm presently running eseutil /g on the file and it's taking forever (630MB at about 3%) but will report more as I find out.
Outlook periodically reports synchronization errors, such as:
11:19:53 Error synchronizing folder
11:19:53 [8004010F-501-8004010F-0]
11:19:53 The client operation failed.
11:19:53 Microsoft Exchange Server Information Store
11:19:53 For more information on this failure, click the URL below:
11:19:53 http://www.microsoft.com/support/prodredirect/outlook2000_us.asp?err=8004010f-501-8004010f-0
The folder listed before this message varies... sometimes Sent Items, sometimes Deleted Items, etc. But the error number is always consistent.
As you can tell, I may not be an Active Directory guru... but I generally don't make boneheaded mistakes like deleting an entry from the wrong connections folder. If you can provide a solution to my dilemma with steps to take, please outline the detailed steps involved so that I don't assume something incorrectly.
Many thanks in advance!
However, since then (now a week ago) there has been a problem where the .log files consumed all disk space after the application trying to connect to the mail gateway continually errored out with "Exchange Gateway logon failed - OpenMsgStore failed - MAPI_E_CORRUPT_DATA". I stopped the application from connecting and the logs have stabilized but are still sitting around -- all 12,000+ of them. If I re-enable it, I get the message "Exchange Gateway logon failed - OpenMsgStore failed - MAPI_E_FAILONEPROVIDER". A full backup is now twice the size of what it was before the change (100GB vs 50GB). I could restore the Active Directory and restore the Exchange backup but I don't want to lose a week's worth of emails. What's my best approach for both restoring the connection to the way it was before and also not losing the new emails that have arrived since?
I tried adding a second mailbox store and moving accounts between them, but Active Directory reports a permissions error doing that.
My guess is that when I manually reconstructed the connection for the Mailbox Store, I got a parameter wrong and now it's in the database file under a different folder or something, or maybe it's storing it in two places since the size looks to be double. So if I simply restore Active Directory, it might not recognize that folder and I could lose everything since... or maybe it will be fine. I'm presently running eseutil /g on the file and it's taking forever (630MB at about 3%) but will report more as I find out.
Outlook periodically reports synchronization errors, such as:
11:19:53 Error synchronizing folder
11:19:53 [8004010F-501-8004010F-0]
11:19:53 The client operation failed.
11:19:53 Microsoft Exchange Server Information Store
11:19:53 For more information on this failure, click the URL below:
11:19:53 http://www.microsoft.com/support/prodredirect/outlook2000_us.asp?err=8004010f-501-8004010f-0
The folder listed before this message varies... sometimes Sent Items, sometimes Deleted Items, etc. But the error number is always consistent.
As you can tell, I may not be an Active Directory guru... but I generally don't make boneheaded mistakes like deleting an entry from the wrong connections folder. If you can provide a solution to my dilemma with steps to take, please outline the detailed steps involved so that I don't assume something incorrectly.
Many thanks in advance!
ASKER
There are hundreds of users, I want something that will do all of them without having to go through each one individually.
ASKER
Update on integrity check:
C:\Program Files\Exchsrvr\MDBDATA>..\ bin\eseuti l /g priv1.edb
Microsoft(R) Exchange Server Database Utilities
Version 6.5
Copyright (C) Microsoft Corporation. All Rights Reserved.
Initiating INTEGRITY mode...
Database: priv1.edb
Streaming File: priv1.STM
Temp. Database: TEMPINTEG7968.EDB
Checking database integrity.
Scanning Status (% complete)
0 10 20 30 40 50 60 70 80 90 100
|----|----|----|----|----| ----|----| ----|----| ----|
.......................... .......... .......... .....
Integrity check successful.
Operation completed successfully in 14743.31 seconds.
--
So this didn't appear to report any errors in integrity.
C:\Program Files\Exchsrvr\MDBDATA>..\
Microsoft(R) Exchange Server Database Utilities
Version 6.5
Copyright (C) Microsoft Corporation. All Rights Reserved.
Initiating INTEGRITY mode...
Database: priv1.edb
Streaming File: priv1.STM
Temp. Database: TEMPINTEG7968.EDB
Checking database integrity.
Scanning Status (% complete)
0 10 20 30 40 50 60 70 80 90 100
|----|----|----|----|----|
..........................
Integrity check successful.
Operation completed successfully in 14743.31 seconds.
--
So this didn't appear to report any errors in integrity.
any errors in the application or the system logs
please do let me know on < removed by sembee> and possibly send me the application and the system logs,
ASKER
Checksum results:
Microsoft(R) Exchange Server Database Utilities
Version 6.5
Copyright (C) Microsoft Corporation. All Rights Reserved.
Initiating CHECKSUM mode...
Database: priv1.edb
Streaming File: priv1.STM
Temp. Database: TEMPCHKSUM7524.EDB
File: priv1.edb
Checksum Status (% complete)
0 10 20 30 40 50 60 70 80 90 100
|----|----|----|----|----| ----|----| ----|----| ----|
.......................... .......... .......... .....
22207890 pages seen
0 bad checksums
5259 uninitialized pages
0 wrong page numbers
0x1330f64f highest dbtime (pgno 0x15171b9)
1387994 reads performed
86749 MB read
2107 seconds taken
41 MB/second
-2138372612 milliseconds used
1553 milliseconds per read
4844 milliseconds for the slowest read
0 milliseconds for the fastest read
File: priv1.STM
Checksum Status (% complete)
0 10 20 30 40 50 60 70 80 90 100
|----|----|----|----|----| ----|----| ----|----| ----|
.......................... .......... .......... .....
4594688 pages seen
0 bad checksums
412730 uninitialized pages
Operation completed successfully in 2716.94 seconds.
Microsoft(R) Exchange Server Database Utilities
Version 6.5
Copyright (C) Microsoft Corporation. All Rights Reserved.
Initiating CHECKSUM mode...
Database: priv1.edb
Streaming File: priv1.STM
Temp. Database: TEMPCHKSUM7524.EDB
File: priv1.edb
Checksum Status (% complete)
0 10 20 30 40 50 60 70 80 90 100
|----|----|----|----|----|
..........................
22207890 pages seen
0 bad checksums
5259 uninitialized pages
0 wrong page numbers
0x1330f64f highest dbtime (pgno 0x15171b9)
1387994 reads performed
86749 MB read
2107 seconds taken
41 MB/second
-2138372612 milliseconds used
1553 milliseconds per read
4844 milliseconds for the slowest read
0 milliseconds for the fastest read
File: priv1.STM
Checksum Status (% complete)
0 10 20 30 40 50 60 70 80 90 100
|----|----|----|----|----|
..........................
4594688 pages seen
0 bad checksums
412730 uninitialized pages
Operation completed successfully in 2716.94 seconds.
ASKER
File dump:
Microsoft(R) Exchange Server Database Utilities
Version 6.5
Copyright (C) Microsoft Corporation. All Rights Reserved.
Initiating FILE DUMP mode...
Database: priv1.edb
File Type: Database
Format ulMagic: 0x89abcdef
Engine ulMagic: 0x89abcdef
Format ulVersion: 0x620,11
Engine ulVersion: 0x620,11
Created ulVersion: 0x620,9
DB Signature: Create time:02/28/2005 17:22:38 Rand:24734849 Computer:
cbDbPage: 4096
dbtime: 321975887 (0x1330f64f)
State: Clean Shutdown
Log Required: 0-0 (0x0-0x0)
Streaming File: Yes
Shadowed: Yes
Last Objid: 106087
Scrub Dbtime: 0 (0x0)
Scrub Date: 00/00/1900 00:00:00
Repair Count: 0
Repair Date: 00/00/1900 00:00:00
Last Consistent: (0x92A1,1794,3C) 05/23/2006 20:48:08
Last Attach: (0x92A0,140F,19F) 05/23/2006 19:57:43
Last Detach: (0x92A1,1794,3C) 05/23/2006 20:48:08
Dbid: 1
Log Signature: Create time:02/28/2005 17:22:35 Rand:24685950 Computer:
OS Version: (5.2.3790 SP 1)
Previous Full Backup:
Log Gen: 37444-37449 (0x9244-0x9249)
Mark: (0x9244,1F00,4E)
Mark: 05/20/2006 18:32:25
Previous Incremental Backup:
Log Gen: 25069-37534 (0x61ed-0x929e)
Mark: (0x929F,8,16)
Mark: 05/23/2006 17:35:58
Current Full Backup:
Log Gen: 0-0 (0x0-0x0)
Mark: (0x0,0,0)
Mark: 00/00/1900 00:00:00
Current Shadow copy backup:
Log Gen: 0-0 (0x0-0x0)
Mark: (0x0,0,0)
Mark: 00/00/1900 00:00:00
cpgUpgrade55Format: 0
cpgUpgradeFreePages: 0
cpgUpgradeSpaceMapPages: 0
Operation completed successfully in 3.0 seconds.
Microsoft(R) Exchange Server Database Utilities
Version 6.5
Copyright (C) Microsoft Corporation. All Rights Reserved.
Initiating FILE DUMP mode...
Database: priv1.edb
File Type: Database
Format ulMagic: 0x89abcdef
Engine ulMagic: 0x89abcdef
Format ulVersion: 0x620,11
Engine ulVersion: 0x620,11
Created ulVersion: 0x620,9
DB Signature: Create time:02/28/2005 17:22:38 Rand:24734849 Computer:
cbDbPage: 4096
dbtime: 321975887 (0x1330f64f)
State: Clean Shutdown
Log Required: 0-0 (0x0-0x0)
Streaming File: Yes
Shadowed: Yes
Last Objid: 106087
Scrub Dbtime: 0 (0x0)
Scrub Date: 00/00/1900 00:00:00
Repair Count: 0
Repair Date: 00/00/1900 00:00:00
Last Consistent: (0x92A1,1794,3C) 05/23/2006 20:48:08
Last Attach: (0x92A0,140F,19F) 05/23/2006 19:57:43
Last Detach: (0x92A1,1794,3C) 05/23/2006 20:48:08
Dbid: 1
Log Signature: Create time:02/28/2005 17:22:35 Rand:24685950 Computer:
OS Version: (5.2.3790 SP 1)
Previous Full Backup:
Log Gen: 37444-37449 (0x9244-0x9249)
Mark: (0x9244,1F00,4E)
Mark: 05/20/2006 18:32:25
Previous Incremental Backup:
Log Gen: 25069-37534 (0x61ed-0x929e)
Mark: (0x929F,8,16)
Mark: 05/23/2006 17:35:58
Current Full Backup:
Log Gen: 0-0 (0x0-0x0)
Mark: (0x0,0,0)
Mark: 00/00/1900 00:00:00
Current Shadow copy backup:
Log Gen: 0-0 (0x0-0x0)
Mark: (0x0,0,0)
Mark: 00/00/1900 00:00:00
cpgUpgrade55Format: 0
cpgUpgradeFreePages: 0
cpgUpgradeSpaceMapPages: 0
Operation completed successfully in 3.0 seconds.
ASKER
Interesting part here:
C:\Program Files\Exchsrvr\MDBDATA>..\ bin\isinte g -s myserver -verbose -test alltests
Databases for server myserver:
Only databases marked as Offline can be checked
Index Status Database-Name
Storage Group Name: First Storage Group
1 Offline Mailbox Store (MYSERVER)
2 Online Public Folder Store (MYSERVER)
Storage Group Name: Recovery Storage Group
3 Offline Mailbox Store (MYSERVER)
Enter a number to select a database or press Return to exit.
--
There should only be one Mailbox Store present here and a Public Folder Store. However, the isinteg utility thinks there are two Mailbox Stores.
C:\Program Files\Exchsrvr\MDBDATA>..\
Databases for server myserver:
Only databases marked as Offline can be checked
Index Status Database-Name
Storage Group Name: First Storage Group
1 Offline Mailbox Store (MYSERVER)
2 Online Public Folder Store (MYSERVER)
Storage Group Name: Recovery Storage Group
3 Offline Mailbox Store (MYSERVER)
Enter a number to select a database or press Return to exit.
--
There should only be one Mailbox Store present here and a Public Folder Store. However, the isinteg utility thinks there are two Mailbox Stores.
when and where do you get this error
"Exchange Gateway logon failed - OpenMsgStore failed - MAPI_E_CORRUPT_DATA".
"Exchange Gateway logon failed - OpenMsgStore failed - MAPI_E_CORRUPT_DATA".
ASKER
The Exchange Gateway Login error is in a third-party application that integrates with Exchange. Never been a problem before, only started those errors once the ADSI info got messed up.
Hello ,
PLease also check with this KB http://support.microsoft.com/kb/839531/en-us
and let me know weather we have these permissions on the ORG level.
Regaards,
Abhijeet
PLease also check with this KB http://support.microsoft.com/kb/839531/en-us
and let me know weather we have these permissions on the ORG level.
Regaards,
Abhijeet
can you meet me on < removed by sembee> is my id
ASKER
Re: http://support.microsoft.com/kb/839531/en-us
Not an issue. Everything appears on the surface to behave fine from Outlook's perspective, save the very infrequent sync issues.
It's extremely late here. I'll pick this thread up again in the morning.
Not an issue. Everything appears on the surface to behave fine from Outlook's perspective, save the very infrequent sync issues.
It's extremely late here. I'll pick this thread up again in the morning.
ok then send me a email on the above mail address.... as i wont be on yahoo till then i would switch to msn and i cannot give u my msn id out here send me a mail and will provide u my msn id my mail addy is <removed by sembee>
ASKER
More information: if I create a new mailbox store called Second Store and attempt to move a mailbox there, I get this:
<?xml version="1.0" encoding="unicode" ?>
- <taskWizardRun taskName="Move Mailbox" dcName="MYSERVER" buildNumber="7226" runningAs="Administrator@M YDOMAIN.CO M">
<timespan startTime="2006-05-24 16:10:14.733" milliseconds="219" />
- <moveMailbox mixedMode="true" maxBadItems="0">
- <destination>
<database>/dc=com/dc=mydom ain/cn=Con figuration /cn=Servic es/cn=Micr osoft Exchange/cn=MyOrganization /cn=Admini strative Groups/cn=First Administrative Group/cn=Servers/cn=MYSERV ER/cn=Info rmationSto re/cn=Firs t Storage Group/cn=Second Store</database>
</destination>
</moveMailbox>
<taskSummary errorCount="1" completedCount="0" warningCount="0" errorCode="0x00000000" />
- <items>
- <item adsPath="John Smith" class="user">
<progress code="0" milliseconds="219">Opening source mailbox.</progress>
- <summary isWarning="false" errorCode="0xc1050000">
The information store could not be opened. The MAPI provider failed. MAPI 1.0 ID no: 8004011d-0289-00000000
- <details>
- <source>
<database>/dc=com/dc=mydom ain/cn=Con figuration /cn=Servic es/cn=Micr osoft Exchange/cn=MyOrganization /cn=Admini strative Groups/cn=First Administrative Group/cn=Servers/cn=MYSERV ER/cn=Info rmationSto re/cn=Firs t Storage Group/cn=Mailbox Store (MYSERVER)</database>
</source>
</details>
</summary>
</item>
</items>
</taskWizardRun>
<?xml version="1.0" encoding="unicode" ?>
- <taskWizardRun taskName="Move Mailbox" dcName="MYSERVER" buildNumber="7226" runningAs="Administrator@M
<timespan startTime="2006-05-24 16:10:14.733" milliseconds="219" />
- <moveMailbox mixedMode="true" maxBadItems="0">
- <destination>
<database>/dc=com/dc=mydom
</destination>
</moveMailbox>
<taskSummary errorCount="1" completedCount="0" warningCount="0" errorCode="0x00000000" />
- <items>
- <item adsPath="John Smith" class="user">
<progress code="0" milliseconds="219">Opening
- <summary isWarning="false" errorCode="0xc1050000">
The information store could not be opened. The MAPI provider failed. MAPI 1.0 ID no: 8004011d-0289-00000000
- <details>
- <source>
<database>/dc=com/dc=mydom
</source>
</details>
</summary>
</item>
</items>
</taskWizardRun>
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thank you for the suggestions. Application event info/warnings/errors from the time of move, in order:
MSExchangeIS, Event ID 1008:
MYDOMAIN\Administrator connected as /o=MyOrganization/ou=First Administrative Group/cn=Configuration/cn= Servers/cn =MYSERVER/ cn=Microso ft System Attendant using administrator privileges.
MSExchangeIS, Event ID 9548:
Disabled user /o=MyOrganization/ou=First Administrative Group/cn=Recipients/cn=jsm ith does not have a master account SID. Please use Active Directory MMC to set an active account as this user's master account.
MSExchangeIS Mailbox, Event ID 1022:
Logon Failure on database "First Storage Group\Mailbox Store (MYSERVER)" - Windows 2000 account MYDOMAIN\Administrator; mailbox /o=MyOrganization/ou=First Administrative Group/cn=Recipients/cn=jsm ith.
Error: -2147221231
MSExchangeAdmin, Event ID 9175:
The MAPI call 'OpenMsgStore' failed with the following error:
The information store could not be opened.
The MAPI provider failed.
MAPI 1.0
ID no: 8004011d-0289-00000000
MSExchangeAdmin, Event ID 9168:
Failed to open mailbox '/o=MyOrganization/ou=Firs t Administrative Group/cn=Recipients/cn=jsm ith' in mailbox store '/o=MyOrganization/ou=Firs t Administrative Group/cn=Configuration/cn= Servers/cn =MYSERVER/ cn=Microso ft Private MDB' on server 'MYSERVER'.
Error: The information store could not be opened.
The MAPI provider failed.
MAPI 1.0
ID no: 8004011d-0289-00000000
MSExchangeAdmin, Event ID 1008:
Unable to move mailbox 'John Smith'.
Error: The information store could not be opened.
The MAPI provider failed.
MAPI 1.0
ID no: 8004011d-0289-00000000
MSExchangeIS, Event ID 1008:
MYDOMAIN\Administrator connected as /o=MyOrganization/ou=First
MSExchangeIS, Event ID 9548:
Disabled user /o=MyOrganization/ou=First
MSExchangeIS Mailbox, Event ID 1022:
Logon Failure on database "First Storage Group\Mailbox Store (MYSERVER)" - Windows 2000 account MYDOMAIN\Administrator; mailbox /o=MyOrganization/ou=First
Error: -2147221231
MSExchangeAdmin, Event ID 9175:
The MAPI call 'OpenMsgStore' failed with the following error:
The information store could not be opened.
The MAPI provider failed.
MAPI 1.0
ID no: 8004011d-0289-00000000
MSExchangeAdmin, Event ID 9168:
Failed to open mailbox '/o=MyOrganization/ou=Firs
Error: The information store could not be opened.
The MAPI provider failed.
MAPI 1.0
ID no: 8004011d-0289-00000000
MSExchangeAdmin, Event ID 1008:
Unable to move mailbox 'John Smith'.
Error: The information store could not be opened.
The MAPI provider failed.
MAPI 1.0
ID no: 8004011d-0289-00000000
ASKER
Ran through various utilities... after countless backups and running eseutil and isinteg to do tests and then fixes, and finally running a defrag on the Exchange store seems to have resolved this. I still haven't had to do an Active Directory restore as it apparently is working fine with the new connector.
Awarding points to abhi_ak as he's the only one to respond and his suggestions to check the event log after encountering those errors were helpful in resolving the issue.
Awarding points to abhi_ak as he's the only one to respond and his suggestions to check the event log after encountering those errors were helpful in resolving the issue.
Did you try re-creating the users OST file