awarenessit
asked on
Unexplained Large Mailbox in Exchange 2003 Enterprise
I am running Exchange 2003 Enterprise SP2 on a Windows 2003 R2 server. I have about 15 users on this server who use Outlook 2003 SP3 in cached Exchange mode as an email client. One user has a rather large mailbox according to Exchange System Manager but this does not appear to be the case from the user's interface. ESM has been saying that this user's mailbox is more than 2.3 GB with more than 32,000 items for months now. Since the system limits are set to issue a warning at 1,048,576KB, this user gets an email daily indicating that his mailbox is too large.
The user claims he has archived data appropriately and should no longer receive the email. When I look at the user's mailbox via Outlook Web Access and via his Outlook client I agree that it does not appear that this user has 32,000 emails, notes, calendar items, etc. In my opinion this mailbox's true size should be much smaller than what ESM reports.
Running the Mailbox Manager in report mode shows this user having 3029 items which is far more believable. I would like to reduce the size of this users mailbox according to ESM. If the items really are there I would like to find a way to see all of them so we can clean them up. If the items are not there, I would like to make ESM report the correct size of this mailbox.
I have verified that the nightly backup processes are running and completing normally (Full backups on Monday nights; Differential backups Tuesday Sunday; System State backups nightly). I have also verified that Event Codes 1206 and 1207 (the mailbox store cleanup start and complete messages in the Event Viewer) are running and completing normally. Item retention is set to 7 days, the option Do not permanently delete mailboxes and items until the store has been backed up is not enabled.
Yesterday I ran an offline defrag of the mailbox store. While it shrunk the database by a bit more than 1.5GB, it did not help the size of this users mailbox.
Some history: Prior to implementing Exchange Server all users had a POP3 account that ran on a different email host software (SmarterMail). The .pst files from those accounts were imported to the respective user's Exchange account. Following import, an archive.pst file was set up for each user on their respective PCs.
Some specifics to this user: This user's PC has been replaced and his archived mail .pst file was imported to Outlook personal folders on his local machine. Presently the archive.pst file in question is less than 1 GB (827MB) and the current .ost file is about 159MB. I have checked the data file location to be sure that there are no other data files lurking out there, his archive.pst file is the only data file showing.
The user claims he has archived data appropriately and should no longer receive the email. When I look at the user's mailbox via Outlook Web Access and via his Outlook client I agree that it does not appear that this user has 32,000 emails, notes, calendar items, etc. In my opinion this mailbox's true size should be much smaller than what ESM reports.
Running the Mailbox Manager in report mode shows this user having 3029 items which is far more believable. I would like to reduce the size of this users mailbox according to ESM. If the items really are there I would like to find a way to see all of them so we can clean them up. If the items are not there, I would like to make ESM report the correct size of this mailbox.
I have verified that the nightly backup processes are running and completing normally (Full backups on Monday nights; Differential backups Tuesday Sunday; System State backups nightly). I have also verified that Event Codes 1206 and 1207 (the mailbox store cleanup start and complete messages in the Event Viewer) are running and completing normally. Item retention is set to 7 days, the option Do not permanently delete mailboxes and items until the store has been backed up is not enabled.
Yesterday I ran an offline defrag of the mailbox store. While it shrunk the database by a bit more than 1.5GB, it did not help the size of this users mailbox.
Some history: Prior to implementing Exchange Server all users had a POP3 account that ran on a different email host software (SmarterMail). The .pst files from those accounts were imported to the respective user's Exchange account. Following import, an archive.pst file was set up for each user on their respective PCs.
Some specifics to this user: This user's PC has been replaced and his archived mail .pst file was imported to Outlook personal folders on his local machine. Presently the archive.pst file in question is less than 1 GB (827MB) and the current .ost file is about 159MB. I have checked the data file location to be sure that there are no other data files lurking out there, his archive.pst file is the only data file showing.
ASKER
I did run ESEUTIL with the /d yesterday to do the offline defrag however I did not run it with the g switch. I will run that this evening and let you know the results.
ASKER
I've run ESEUTIL /G, there were now messages regarding a corrupt store. Only "Integrity check successful. Operation completed successfully in 192.15 seconds."
Ah - well that's not what I was hoping for!
Can you create a second mailstore (just for now), move this user from the existing store to the new store and then see what size the store shows for their mailbox.
If all is well - you can move them back again.
Can you create a second mailstore (just for now), move this user from the existing store to the new store and then see what size the store shows for their mailbox.
If all is well - you can move them back again.
ASKER
I think we are on the right track. I've created a secondary store, then I used the Exchange Task Wizard to move the mailbox to the new store. The newly created mailbox in the second store is approximately 136 MB with 2,256 items. That is much closer to what I would expect to see. The original mailbox still exists in the first mailbox store, and it still shows that it is 2.3 GB with 32,003 items. I suppose we will need to do something with that before I can move the mailbox back to the original store.
If you moved the mailbox, then the original mailbox should disappear!
Can you leave the server overnight to do it's tidy up and check in the morning to see if the mailbox has gone from the original store?
If it has, then you should be able to move the mailbox back and all should be well.
If not - then I'll come up with Plan B!
Can you leave the server overnight to do it's tidy up and check in the morning to see if the mailbox has gone from the original store?
If it has, then you should be able to move the mailbox back and all should be well.
If not - then I'll come up with Plan B!
ASKER
Sorry, I closed Exchange System Manager and reopenned it, the original mailbox is now gone.
Hurrah - then you should be able to move it back and then check the mailbox size is as you were expecting.
Fingers crossed.
Alan
Fingers crossed.
Alan
ASKER
I've tried moving the mailbox back but the system seems offended by that. The error log indicates that I need to wait for the cleanup process to complete before I can move the mailbox again. So I suspect that I will need to let the overnight process run tonight and then I should be able to move the mailbox back to the original store in the morning. Does that sound correct?
Wait overnight then.
Try moving again tomorrow.
Try moving again tomorrow.
ASKER
ESM will still not allow me to move this mailbox back to the original Mailbox Store. I now have two mailbox stores, one called "Mailbox Store" the other cleverly called "Secondary Mailbox Store". To correct the mailbox size issue I created the "Secondary Mailbox Store" and moved the user's (Tim Boyles) mailbox to that store. Now when I attempt to move the mailbox back I continue to get these error messages from the event viewer:
Source: MSExchangeAdmin
Event ID: 1006
Description: Started to move mailbox 'Tim Boyles'.
Source Database: /o=Mydomain/ou=First Administrative Group/cn=Configuration/cn=
Destination Database: /o=Mydomain/ou=First Administrative Group/cn=Configuration/cn=
Exchange DN: /o=Mydomain/ou=First Administrative Group/cn=Recipients/cn=tim
Source: MSExchangeIS Mailbox Store
Event ID: 1205
Description: Failed to create the mailbox of /o=Mydomain/ou=First Administrative Group/cn=Recipients/cn=tim
Source: MSExchangeAdmin
Event ID: 9169
Description: Failed to copy basic mailbox information.
Error: After moving a mailbox, you must wait for cleanup operations to complete before you can move it again.
I think this would seem to indicate that the cleanup process has not run, however, the logs indicate that they are running and completing. My retention limits for deleted items and mailboxes on the Secondary Mailbox Store are set to 7 and 30 days respectively. My retention limits on the first mailbox store were set to 7 and 7, but yesterday I set them to 1 and 1 and manually ran a complete backup. I hoped the cleanup process last night would have allowed me to move the mailbox this morning but it has not seemed to help. Here are the Event Viewer entries for the secondary mailbox store cleanup:
Source: MSExchangeIS Mailbox Store
EventID: 1206
Description: Starting cleanup of items past retention date for Item Recovery on database "First Storage Group\Secondary Mailbox Store".
Source: MSExchangeIS Mailbox Store
EventID: 1207
Description: Cleanup of items past retention date for Item Recovery is complete for database "First Storage Group\Secondary Mailbox Store".
Start: 167 items; 348 Kbytes
End: 167 items; 348 Kbytes
Source: MSExchangeIS Mailbox Store
EventID: 9531
Description: Starting cleanup of deleted mailboxes that are past the retention date on database "First Storage Group\Secondary Mailbox Store".
Source: MSExchangeIS Mailbox Store
EventID: 9535
Description: Cleanup of deleted mailboxes that are past the retention date is finished on database "First Storage Group\Secondary Mailbox Store".
0 deleted mailboxes (0 KB) have been removed.
0 deleted mailboxes (0 KB) have been retained.
And here are the entries for the first mailbox store cleanup:
Source: MSExchangeIS Mailbox Store
EventID: 1206
Description: Starting cleanup of items past retention date for Item Recovery on database "First Storage Group\Mailbox Store (RADMAIL)".
Source: MSExchangeIS Mailbox Store
EventID: 1207
Description: Cleanup of items past retention date for Item Recovery is complete for database "First Storage Group\Mailbox Store (RADMAIL)".
Start: 3372 items; 88806 Kbytes
End: 3354 items; 88593 Kbytes
Source: MSExchangeIS Mailbox Store
EventID: 9531
Description: Starting cleanup of deleted mailboxes that are past the retention date on database "First Storage Group\Mailbox Store (RADMAIL)".
Source: MSExchangeIS Mailbox Store
EventID: 9532
Description: The user account for 'Katie Tipton' does not exist in the directory or is not enabled for Exchange mail. This mailbox has been removed from mailbox store 'First Storage Group/Mailbox Store (RADMAIL)'.
Source: MSExchangeIS Mailbox Store
EventID: 9535
Description: Cleanup of deleted mailboxes that are past the retention date is finished on database "First Storage Group\Mailbox Store (RADMAIL)".
1 deleted mailboxes (178634 KB) have been removed.
0 deleted mailboxes (0 KB) have been retained.
Any ideas?
Source: MSExchangeAdmin
Event ID: 1006
Description: Started to move mailbox 'Tim Boyles'.
Source Database: /o=Mydomain/ou=First Administrative Group/cn=Configuration/cn=
Destination Database: /o=Mydomain/ou=First Administrative Group/cn=Configuration/cn=
Exchange DN: /o=Mydomain/ou=First Administrative Group/cn=Recipients/cn=tim
Source: MSExchangeIS Mailbox Store
Event ID: 1205
Description: Failed to create the mailbox of /o=Mydomain/ou=First Administrative Group/cn=Recipients/cn=tim
Source: MSExchangeAdmin
Event ID: 9169
Description: Failed to copy basic mailbox information.
Error: After moving a mailbox, you must wait for cleanup operations to complete before you can move it again.
I think this would seem to indicate that the cleanup process has not run, however, the logs indicate that they are running and completing. My retention limits for deleted items and mailboxes on the Secondary Mailbox Store are set to 7 and 30 days respectively. My retention limits on the first mailbox store were set to 7 and 7, but yesterday I set them to 1 and 1 and manually ran a complete backup. I hoped the cleanup process last night would have allowed me to move the mailbox this morning but it has not seemed to help. Here are the Event Viewer entries for the secondary mailbox store cleanup:
Source: MSExchangeIS Mailbox Store
EventID: 1206
Description: Starting cleanup of items past retention date for Item Recovery on database "First Storage Group\Secondary Mailbox Store".
Source: MSExchangeIS Mailbox Store
EventID: 1207
Description: Cleanup of items past retention date for Item Recovery is complete for database "First Storage Group\Secondary Mailbox Store".
Start: 167 items; 348 Kbytes
End: 167 items; 348 Kbytes
Source: MSExchangeIS Mailbox Store
EventID: 9531
Description: Starting cleanup of deleted mailboxes that are past the retention date on database "First Storage Group\Secondary Mailbox Store".
Source: MSExchangeIS Mailbox Store
EventID: 9535
Description: Cleanup of deleted mailboxes that are past the retention date is finished on database "First Storage Group\Secondary Mailbox Store".
0 deleted mailboxes (0 KB) have been removed.
0 deleted mailboxes (0 KB) have been retained.
And here are the entries for the first mailbox store cleanup:
Source: MSExchangeIS Mailbox Store
EventID: 1206
Description: Starting cleanup of items past retention date for Item Recovery on database "First Storage Group\Mailbox Store (RADMAIL)".
Source: MSExchangeIS Mailbox Store
EventID: 1207
Description: Cleanup of items past retention date for Item Recovery is complete for database "First Storage Group\Mailbox Store (RADMAIL)".
Start: 3372 items; 88806 Kbytes
End: 3354 items; 88593 Kbytes
Source: MSExchangeIS Mailbox Store
EventID: 9531
Description: Starting cleanup of deleted mailboxes that are past the retention date on database "First Storage Group\Mailbox Store (RADMAIL)".
Source: MSExchangeIS Mailbox Store
EventID: 9532
Description: The user account for 'Katie Tipton' does not exist in the directory or is not enabled for Exchange mail. This mailbox has been removed from mailbox store 'First Storage Group/Mailbox Store (RADMAIL)'.
Source: MSExchangeIS Mailbox Store
EventID: 9535
Description: Cleanup of deleted mailboxes that are past the retention date is finished on database "First Storage Group\Mailbox Store (RADMAIL)".
1 deleted mailboxes (178634 KB) have been removed.
0 deleted mailboxes (0 KB) have been retained.
Any ideas?
Change the mailbox store retention time from whatever it is to 0. (right-click the store and choose properties. On the Limits tab there are two settings. Change the Keep Deleted Mailboxes setting to 0).
Then either wait for the cleanup process to process overnight, or encourage it manually.
Then you should be able to move the mailbox back and remove the secondary store.
Once done - set the Deleted Mailboxes retention back to what it was.
Then either wait for the cleanup process to process overnight, or encourage it manually.
Then you should be able to move the mailbox back and remove the secondary store.
Once done - set the Deleted Mailboxes retention back to what it was.
ASKER
I have set the limits to 0 and 0 on both mailbox stores and have manually run the cleanup agent on both stores. I still get exactly the same error messages in the event viewer indicating that the cleanup process must run before I can move the mailbox a second time.
Do you see any events in the event log suggesting that the Tim Boyles mailbox has been deleted by the cleanup, or is it ignoring him?
ASKER
There is no indication in the event log that the mailbox for Tim Boyles has been deleted. Of course I would not expect to see it deleted since I am only trying to move it. Remember that I initially moved the mailbox to the secondary store, and it disappeared from the primary store (as we expected it to). Now I would like to move it back. I will try performing a complete backup again and then try running the cleanup agent in case the system will not delete until a backup is performed.
Sure - but I would like to see that the mailbox has been cleaned up from the first store otherwise I expect you won't be able to move it back, as it already exists!
Have you tried an eseutil /p on the first store now? Might be a corrupt database.
Have you tried an eseutil /p on the first store now? Might be a corrupt database.
ASKER
I've looked through all of the logs and can not find anything to indicate that the mailbox for Tim Boyles was deleted from the primary store. Even after the backup and running the cleanup agent I continue to get the same error. I will run eseutil /p tonight and see if that will correct the issue. I will let you know tomorrow how it went. We'll get this sorted out yet.
Come hell or high water !
ASKER
OK, I've run eseutil /p on the first mailbox store. The message returned was: "Operation completed successfully with 205 (Unknown error, Unknown Error) after 180.6 seconds." That did not say anything about corruption per se, so I ran eseutil /G and that completed normally with the message: "Integrity Check Successful. Operation completed in 178.31 seconds." Then I mounted the store and ran the cleanup agent. Finally I tried to move that mailbox back from the secodary store to the primary store. I received the same errors as listed above saying that I had to wait for the cleanup process to run before I could move the mailbox back. Just for general principals I ran eseutil /p on the secondary store (no errors) but still not able to move the mailbox.
I'm wondering what the "205 (Unknown Error, Unknown Error)" issue is when I ran eseutil /p on the primary store. Could this be the fly in the ointment?
I'm wondering what the "205 (Unknown Error, Unknown Error)" issue is when I ran eseutil /p on the primary store. Could this be the fly in the ointment?
ASKER
Here is a bit more information. In the Event Viewer, I see an interesting entry I did not see before. It is:
Source: MSExchangeIS
EventID: 1240
Description: The expected registry value 'Background Cleanup' under registry key 'SYSTEM\CurrentControlSet\
This registry entry does exist with a value of 60000 at 'SYSTEM\CurrentControlSet\
It is interesting that I have 3 private entries under my server since I only have two private mailbox stores.
Source: MSExchangeIS
EventID: 1240
Description: The expected registry value 'Background Cleanup' under registry key 'SYSTEM\CurrentControlSet\
This registry entry does exist with a value of 60000 at 'SYSTEM\CurrentControlSet\
It is interesting that I have 3 private entries under my server since I only have two private mailbox stores.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Good news, we have success. I reviewed the hotfix which was essentially to install SP2. My system is already on SP2. So I added the missing registry D-Word entries, restarted the MS Exchange Information Store service, and ran the cleanup agent again on both mailbox stores before attempting to move the mailbox again. When I did attempt to move the mailbox I was successful. So now this mailbox is back in the correct store and its size is appropriate to the content. I call this issue resolved.
In hind site, I suppose that we could have simply added those registry entries earlier and avoided having to move the mailbox, but hindsight is always 20/20 as my mother used to say. I appreciate your help and thank you for your efforts.
In hind site, I suppose that we could have simply added those registry entries earlier and avoided having to move the mailbox, but hindsight is always 20/20 as my mother used to say. I appreciate your help and thank you for your efforts.
Hurrah! Great news. Glad we got there in the end. Well done.
Alan
Alan
It could be that you have a slightly messed up Exchange Database.
Have you / can you run at a suitable moment ESEUTIL /G against the store - it needs to be dismounted first. This will check the integrity of the store and may throw up some messages about a corrupt store.
Reference info for ESEUTIL below
http://www.msexchange.org/
If you get messages about corruption, you will have to run ESEUTIL /p tp repair, followed by
ISINTEG -s (yourservername) -fix -test alltests
Run these commands from a DOS prompt in the x:zprogram files\exchsrvr\bin directory.
ESEUTIL command should be similar to this:
c:\program files\exchsrvr\bin> eseutil /g "c:\program files\exchsrvr\mdbdata\pri
Alan