Exchange 2003 size limit problem

Our mail server keeps going offline because it 'thinks" its at the 75G limit

xchange store 'First Storage Group\Mailbox Store (MAILSERVER)': The current physical size of this database (the .edb file and the .stm file) is 82 GB. This database has exceeded the size limit of 75 GB. However, the logical free space in this database has not yet been evaluated. Therefore, it is possible that this database contains enough free space to bring its logical size below the maximum size limit.

If the logical database size exceeds the maximum size limit, it will be dismounted on a regular basis.

The problem is I added up all the users email it comes out to about 45G. The physical file sizes are
priv1.edb 56G
priv1.stm 29G
pub1.edb 1.4G
pub.stm 1.1G

I've run every sort of offline defrag with all the options I could find. Nothing seems to reduce these file sizes. If it was a question of limiting users email I'd just do it but we're not even close to the limit. Can anyone tell me how to purge whatever the server thinks is taking up this remaining space?
LVL 2
Axis52401Security AnalystAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Nick RhodeIT DirectorCommented:
Probably log files not being purged after doing a backup of the exchange datastore.  Are you running full backups?  You can download and install treesize to see where the space is being used.

http://www.jam-software.com/treesize_free/

Run a backup and make sure it purges the logs

http://www.msexchange.org/articles-tutorials/exchange-server-2003/high-availability-recovery/Exchange-2003-Backup-Restore-NTBACKUP.html
0
Axis52401Security AnalystAuthor Commented:
The physical disk isn't running out of free space we have over 120G free on that disk its the database size.

I thought the backup only purged those 5M log files that build up and it does delete those after the backup but I don't think thats the problem.
0
piattndCommented:
When you look in the exchange system management console at this particular mailbox database, what do all of the combined mailboxes equal in terms of size?
0
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

Axis52401Security AnalystAuthor Commented:
about 45G I just did a rough est. Sorted them by mail box size and did a rough guess its not where near 75G when they are all added up.
0
piattndCommented:
The problem won't be the log files.  Those don't contribute to the size of the actual database.  If you were running out of physical disk space, rather than exceeding the 72GB limit, then I'd say you need to do a full backup so those transaction logs are cleared out.

I've done offline defrags many times to compress the database.  The only way it won't compress space is if it's not empty, meaning exchange is still using it.  Are you sure there's no mailboxes inside that database that are sitting as disconnected and waiting to be purged (or exceed the retention time)?
0
piattndCommented:
What commands did you run for the offline defrag?  Have you tried redirecting it to a completely different file name so you know a new file is created?
0
Axis52401Security AnalystAuthor Commented:
How would I check mailboxes inside that database that are sitting as disconnected and waiting to be purged (or exceed the retention time)?
0
piattndCommented:
They'd show up as a red X when you're looking in exchange system management at the actual mailbox database and the mailboxes in that database.
0
Axis52401Security AnalystAuthor Commented:
I don't have that with me right now I'm not onsite, I think I just googled it when I was doing it I know it was a ESEUTIL command. I don't recall it creating a new DB or if it does it reconnects its new one and works.
0
piattndCommented:
0
Axis52401Security AnalystAuthor Commented:
No none with a Red X on them
0
Axis52401Security AnalystAuthor Commented:
It was Eseutil in /D that I had done before.
What does this mean by it might take 2-3 days to complete. I can't take their only mail server down for 3 days
0
Nick RhodeIT DirectorCommented:
Thats just a microsoft CYA warning.  I dont think it would seriously take that long.  It all depends on the hardware and the size but typically it is about 5-10Gb per hour.
0
Axis52401Security AnalystAuthor Commented:
I;m not too clear on the steps it lists here.
It says to run Eseutil in /P first I don't know if I need anything repaired I need free space in the DB back but either way what does that do?

Then it says to run the one I already have Eseutil in /D

Then it says something about moving/backup transaction logs for my DB or it won't mount. Move what logs to and from where?

then it says to run Run Isinteg in -fix -test -alltests mode.
What does that do?
0
piattndCommented:
isinteg checks the integrity and fixes the database files.  As for how long it will take, the estimation is approximately 9GB per hour.  At 72 GB, you're looking at an 8 hour window.  That can vary depending on fragmentation inside of the file, hard drive speed, and other procedures running on that server at the same time.
0
Axis52401Security AnalystAuthor Commented:
Which one of these do you think might stop it from dismounting thinking its out of space? When it happens I can reboot and it works for a while till it thinks its out of space again
0
piattndCommented:
You'll probably need to run a few of them.  If the database isn't being compacted, then it thinks space is being used that's not being used.  That could be because the file is dirty or corrupt.  I'd try the isinteg first and repair after that if it doesn't result in any difference.
0
Axis52401Security AnalystAuthor Commented:
OK, looks like I'll have to schedule some down time for this
0
Simon Butler (Sembee)ConsultantCommented:
Before you waste your time doing a defrag, be aware that the list of users will NEVER equal the database.
This is because ESM only shows you what is on one of the databases.
The number you have got (45gb) is about what I would expect as most Exchange databases are around 50/50 between the two files.

Do you get event ID 1221 overnight to indicate the white space?
If so then the white space is being accurately counted. Time to start planning an upgrade.

Simon.
0
Axis52401Security AnalystAuthor Commented:
Right, I know the physical size of that file might not go down and thats fine. I just need the system to work and it not to dismount and take down the company by thinking its out of space.

There are some entries for event ID 1221 some say
The database "First Storage Group\Public Folder Store (Server)" has 40 megabytes of free space after online defragmentation has terminated.

some say The database "First Storage Group\Mailbox Store (server)" has 30801 megabytes of free space after online defragmentation has terminated.
0
piattndCommented:
One other thing you can look at adjusting is the deleted items retention period.  I dealt with the same issue trying to identify why ESM showed 25GB of mailbox data, but the database showed 40gb in size with relatively low whitespace.  The remaining data was deleted messages that's still within retention period, thus not considered whitespace until it passes the period.

I think default is 7 days.  Try shrinking that down to 3 days, especially if you have a backup method that allows for quick retrieval of messages.
0
piattndCommented:
http://msdn.microsoft.com/en-us/library/exchange/aa143732(v=exchg.65).aspx

That link describes the WMI Class for exchange.  Quoted below is the section about looking at the space taken by the mailbox deleted items awaiting the retention period that will not be shown in ESM:

DeletedMessageSizeExtended Property  


[read,
 Units("Bytes")] uint64 DeletedMessageSizeExtended;


The DeletedMessageSizeExtended property indicates the cumulative size of all deleted messages that are still being retained according to retention policy settings.
 
0
Axis52401Security AnalystAuthor Commented:
WMI class and VB scripting is a little over my head
0
piattndCommented:
Hah ok, well look at your deleted items retention policy.  If you're set at 7 days (or greater) check into lowering it.  This may gain you some pretty valuable space in your database.  I'll see if I can't find the script I used to generate my report of mailbox sizes, including the space used by items awaiting retention period to expire.
0
Axis52401Security AnalystAuthor Commented:
As for the deleted item retention settings I made a screen shot as to what they are set to, it doesn't seem to effect the users deleted items it never has. And yes I have their accounts in AD set in the exchange properties to use Mailbox store defaults
email.docx
0
piattndCommented:
The deleted items is not the same as the items held for recovery during the retention period.

Lets say I'm a user of yours with a 55MB mailbox.  Currently my deleted items has no items in it, so my mailbox size is 55MB and on the database my mailbox is holding 55MB worth of space.

I now deleted 5MB of email and send it to my deleted items.  My mailbox size is still 55MB, though 5MB of it is now held inside of the deleted items folder.  My mailbox size on the database is still 55MB of space.

Now I empty my deleted items.  My mailbox space will now show 50MB of space, but my mailbox space taken in the database is still 55MB.  But why?  Because the items I removed from the deleted items folder are now sitting in a location in the database that's held onto for the 7 days you've set as your deleted item retention.  Assuming I got no new mail for 8 days, my mailbox space requirement would shrink from 55MB to 50MB, because those items I removed would have been purged from the retention hold after the period passed.

Quick fix:

Lower your retention period for deleted items to 3 days.  This should clear some space up, though you really do either need to plan an update or implement lower quotas so the individual mailboxes take less space, allowing for either more mailboxes or a longer retention period on deleted items and mailboxes.
0
Axis52401Security AnalystAuthor Commented:
I set them to 3 days but what do you mean by update? Also we do limit the users mailbox sizes and like I said I went into the system manager and added up the space by all the mail box sizes and its not even close to 75G.
0
piattndCommented:
Right, but in ESM you don't see the space taken by items in the retention window.  What I mean by update is you'll need to upgrade to Exchange Enterprise instead of standard so you can have multiple databases and the databases don't have any limits.
0
Axis52401Security AnalystAuthor Commented:
Yea, I had a friend suggest that and I'd love to but they don't sell it anymore and this client doesn't want to pay to upgrade to Exchange 2010. And I sort of see their point they do actually have enough space if I could figure out how to purge this deleted data from theirs.
0
Simon Butler (Sembee)ConsultantCommented:
Enterprise edition of Exchange 2003 would cost a lot more than Exchange 2010 standard edition.
if the client really refuses to pay for the upgrade, then just setup a scheduled task to restart the information store at 5.10am each morning. The database check is only done once a day - at 5am by default and if you restart the store it will come online again.

How old are those event ID 1221. If you are seeing 30gb of free space then that would probably indicate why the process isn't completing.
Set the overnight task to run for an entire weekend, that may give it enough time to process everything.

Simon.
0
piattndCommented:
When you lower your deleted item retention period, you should see the size of your STM file lower dramatically and your database file should be able to grow to take up that space you freed up.

I think the solution for your situation is pretty simple.  The company doesn't want to upgrade from 2003 standard to enterprise or 2010 to resolve the issue.  The only other way to resolve the issue is to:

-Implement smaller mailbox quotas to force users to archive items off to PSTs and delete unwanted items sooner
-Implement a shorter retention period of deleted items (you've already done this, down to 3 days from 7).
-Implement a shorter retention period for disconnected mailboxes.  This may or may not buy you some valuable space, depending on how frequently you have users leave the company and you end up removing the user and eventually the mailbox.

The first 2 are the primary actions you need to take.  If the company says they don't want smaller mailboxes, then shrink the deleted items retention period to 1 day.  If they want to be able to recover their deleted items further back than 3 days, then lower their mailbox quota so you have enough space to increase the retention period to their desire.

I would never ever resort to rebooting a server daily as a fix.
0
Axis52401Security AnalystAuthor Commented:
I thought of scheduling a task to restart the I store at night but when I manually try and stop it it goes for 30 sec or so then says some error about it can't stop and in the services window shows as stopping. it stays that way till I reboot so I was worried if I just scheduled it it would stop there and be down in the morning before I could get in and reboot it.

I set all the deleted items retention to 3 days last night but so far that stm file size hasn't decreased yet but the server is still running and hasn't done the dismounting problem as of yet today.

I don't see how smaller mailbox quota's will help as most mailboxes are under the current limits and added up together are no where near 75G

I don't see any disconnected mail boxes.
0
Axis52401Security AnalystAuthor Commented:
There seem to be 2 event IDs 1221 every night for about the same size

The database "First Storage Group\Public Folder Store (Server)" has 40 megabytes of free space after online defragmentation has terminated.

some say The database "First Storage Group\Mailbox Store (server)" has 30801 megabytes of free space after online defragmentation has terminated.
0
piattndCommented:
The STM file won't physically shrink until you perform an offline defrag.

http://support.microsoft.com/kb/328804

Try doing an offline defrag now and use the /d /p switches as they explain to create new database files rather than overwriting the old files.  You will need free space on that hard drive approximately 110% of your current database size because the temp file and new database files will be created during the process.  You'll then need to move the new database files back into the original location of your database files and name them to their original name (I'd name the old database files to .old until you confirm the new database files work).
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
piattndCommented:
You will also likely want to include the /s switch to indicate the streaming file (.stm).
0
Axis52401Security AnalystAuthor Commented:
OK, I'm going to have to wait till this weekend for enough off hours time to run these.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Exchange

From novice to tech pro — start learning today.