resourcepc
asked on
Freeing up space on Exchange
I'm running Exchange 2000 on a Windows 2000 Server. I'm in the process of moving some older folders out of our Public folders and into .pst files to burn onto DVD to save space on our Exchange server. However, I've already moved 3GB worth of email from the Public Folders but I'm not seeing the space freeing up on the Exchange Server. Within the Public Folder Store in System Manager, I can verify that the folders I removed are no longer listed there. We're down to 2.3GB on our server (Information Store is backing up right now so we'll free up some space once the Transaction Logs are removed). Any Suggestions?
The Exchange databases will not shrink.
The data you have removed will be marked as white space in the database during the online defrag that takes place overnight. Look for event ID 1221.
What then happens is that that the server will use that white space first, before it increases the physical size of the database.
If you want to physically reduce the size of the database, then you will have to carry out an offline defrag. That will require some space and total downtime of the server. Allowing for a backup of the original file, the file being worked on and the results file, you would need over 3x the physical size of the database - so if the database is 10gb then you will need in excess of 30gb to use.
Simon.
The data you have removed will be marked as white space in the database during the online defrag that takes place overnight. Look for event ID 1221.
What then happens is that that the server will use that white space first, before it increases the physical size of the database.
If you want to physically reduce the size of the database, then you will have to carry out an offline defrag. That will require some space and total downtime of the server. Allowing for a backup of the original file, the file being worked on and the results file, you would need over 3x the physical size of the database - so if the database is 10gb then you will need in excess of 30gb to use.
Simon.
You can run the offline defrag on another drive also using /t switch
http://support.microsoft.com/kb/192185/en-us
Thanks
Kunal
http://support.microsoft.com/kb/192185/en-us
Thanks
Kunal
ASKER
I remember moving folders off the Public Folder last year and putting them into a .pst file and the space increased dramatically on our Exchange server.
The database files don't shrink to prevent physical drive defragmentation. Offline defragmentation is only recommended when a LOT of storage space can be reclaimed.
ESEUTIL /D pub1.edb
will free up the unused database size. The Public Folder store will have to be dismounted (or the Information Service stopped) for this to work.
A temporary, defragemented copy of the database is created, and when successful, the database files are replaced.
See the link from ksharma4 for details on the command line options.
ESEUTIL /D pub1.edb
will free up the unused database size. The Public Folder store will have to be dismounted (or the Information Service stopped) for this to work.
A temporary, defragemented copy of the database is created, and when successful, the database files are replaced.
See the link from ksharma4 for details on the command line options.
ASKER
I know I didn't do anything special last summer when I moved stuff off the Public Folders, and it freed up space.
Trust me, the database engine just doesn't do that by itself. Really.
You don't have to believe me, just the guy with 7-and-a-half-ought million expert points 5 posts ago. ;-)
You don't have to believe me, just the guy with 7-and-a-half-ought million expert points 5 posts ago. ;-)
ASKER
I believe you now, I have found nothing on the net or books that say otherwise.
Another question for you, I have removed about 18GB off our Public Folder store. Yet our Public Folder is continually growing. From my understanding is that Exchange will use the empty white space within the database before it grows. Now I know we haven't filled up 18 GB worth of email in the past week. Any idea as to why it's still growing?
Another question for you, I have removed about 18GB off our Public Folder store. Yet our Public Folder is continually growing. From my understanding is that Exchange will use the empty white space within the database before it grows. Now I know we haven't filled up 18 GB worth of email in the past week. Any idea as to why it's still growing?
What does event ID 1221 overnight say?
You might have a corrupt database - if you aren't getting the white space back, then you may have a problem.
It is a BIG might, I am not saying that is the problem.
Simon.
You might have a corrupt database - if you aren't getting the white space back, then you may have a problem.
It is a BIG might, I am not saying that is the problem.
Simon.
Obviously, deleted item retention can keep the space from being reclaimed. Deleted item retention is disabled by default on Exchange 2000.
Also, public store maintenance needs to run to perform online defragmentation of the store, after which the free space is reclaimed. If the online defragmentation cannot complete a full pass within the scheduled time window, it continues where it left off on the next scheduled interval. Space is not freed until a full pass has completed.
By default, the maintenance interval is from 1:00 to 5:00 AM for both the Mailbox and Public store. On large databases, this concurrency can prevent a full pass from ever completing. Filter the application log for Event 701; this indicates that a full pass has completed. If you don't have any, change the maintenance intervals so maintenance can complete.
Also, public store maintenance needs to run to perform online defragmentation of the store, after which the free space is reclaimed. If the online defragmentation cannot complete a full pass within the scheduled time window, it continues where it left off on the next scheduled interval. Space is not freed until a full pass has completed.
By default, the maintenance interval is from 1:00 to 5:00 AM for both the Mailbox and Public store. On large databases, this concurrency can prevent a full pass from ever completing. Filter the application log for Event 701; this indicates that a full pass has completed. If you don't have any, change the maintenance intervals so maintenance can complete.
ASKER
That makes perfect sense. We do not have any 701 event IDs. We have a bunch of event 702's where online defragmentation resumes, and 704's was online defragmentation was interrupted and terminated. We're only recovering about 40MB a night during the five hours of online defrag. We're also running a brick level and Information Store backup at the same time. I'm thinking I need to set a maintenance schedule to happen over an entire weekend when we're not running any backups so we can recover a lot more free space during an online defrag. Reclaiming 40MB a night just isnt' cutting it. Our deletion retention period is set at 2 days so that isn't an issue.
Thanks! I'll try that this weekend unless you have a better suggestion and let you know how it goes.
Thanks! I'll try that this weekend unless you have a better suggestion and let you know how it goes.
ASKER
This MS article recommends running backups and maintenance at different times, which is probably why we're not reclaiming much space every night, plus the fact that our Public Folder database is 102GB.
This Event 701 is for all mailbox and public folder stores, so the maintenance jobs aren't completed for either public or mailbox store.
Some other ideas:
- extend the maintenance window just until office hours start (1AM-8AM)
- alternate maintenance on the public and private store every other day, don't run them concurrently. One big run is more effective than many small runs.
- use the weekends if you can.
If you still can't get the maintenance jobs to complete, consider not making brick level public store backups (on a daily basis) or at least until your database can be shrunk. Instead, extend deleted item retention to 7 days or whatever the requirement is.
The discussion of brick level backup vs. item retention is another but so far I've done without brick level backups.
Some other ideas:
- extend the maintenance window just until office hours start (1AM-8AM)
- alternate maintenance on the public and private store every other day, don't run them concurrently. One big run is more effective than many small runs.
- use the weekends if you can.
If you still can't get the maintenance jobs to complete, consider not making brick level public store backups (on a daily basis) or at least until your database can be shrunk. Instead, extend deleted item retention to 7 days or whatever the requirement is.
The discussion of brick level backup vs. item retention is another but so far I've done without brick level backups.
ASKER
I think I'll make use of the maintenance window this weekend, because our offices are closed on the weekends, although we have a few that'll work over Citrix. Plus we don't run any backups on the weekends. I'll also try running each store every other day, and maybe increase my window from 8pm - 6am since the majority of our employees are not working during that time.
I don't know if increasing our item retention will work right now, considering the lack of free space we have on our Exchange server. So a brick level backup seems to be our only option right now in case we need to recover a user's mailbox items.
I don't know if increasing our item retention will work right now, considering the lack of free space we have on our Exchange server. So a brick level backup seems to be our only option right now in case we need to recover a user's mailbox items.
ASKER
Is there a reason why the maintenance schedule is set up from 1am to 5am, but I don't see Event 704 until 6am?
Event Type: Information
Event Source: ESE
Event Category: Online Defragmentation
Event ID: 704
Date: 4/19/2006
Time: 6:00:50 AM
User: N/A
Computer: XXXXX
Description:
Information Store (4184) Online defragmentation of database 'D:\Program Files\Exchsrvr\mdbdata\pub
Here is the new schedule I have set up, we've also decided to run brick-level backup only once a week, since we have a 7 day retention period on deleted mailboxes, and a 28 day retention period on deleted items. Let me know if you have any suggestions:
Private Stores from 8pm-5am Mon, Wed, Fri
Public Store from 8pm-5am Tues, Thurs ------ and 6am Saturday until 5am Monday morning.
Event Type: Information
Event Source: ESE
Event Category: Online Defragmentation
Event ID: 704
Date: 4/19/2006
Time: 6:00:50 AM
User: N/A
Computer: XXXXX
Description:
Information Store (4184) Online defragmentation of database 'D:\Program Files\Exchsrvr\mdbdata\pub
Here is the new schedule I have set up, we've also decided to run brick-level backup only once a week, since we have a 7 day retention period on deleted mailboxes, and a 28 day retention period on deleted items. Let me know if you have any suggestions:
Private Stores from 8pm-5am Mon, Wed, Fri
Public Store from 8pm-5am Tues, Thurs ------ and 6am Saturday until 5am Monday morning.
Haven't been able to find a reason why the 704 comes up an hour later than the schedule ends. It doesn't take an hour to stop the defragmentation run so that's not it.
The schedule looks great.
Can you tell us more about the drive configuration, total space per partition, and database sizes? Defragmentation requires a considerable amount of free disk space and you want to have at least 20-25% free space on all database drives to keep up performance.
The schedule looks great.
Can you tell us more about the drive configuration, total space per partition, and database sizes? Defragmentation requires a considerable amount of free disk space and you want to have at least 20-25% free space on all database drives to keep up performance.
ASKER
Sure... We have two partitions (C: and D:).
C: has a total capacity of 34 GB, but only 2.5 GB of free space
D: has a total capacity of 136 GB, but only has 5.5 GB of free space
We have two stores on C:, one is 22 GB, the other is about 1.5 GB.
We have two stores on D:, one is 30 GB, the other (our Public store) is 105 GB.
Should I alternate days with each store? Right now I set it so it does the Public store one night, and all the others on the alternate night.
C: has a total capacity of 34 GB, but only 2.5 GB of free space
D: has a total capacity of 136 GB, but only has 5.5 GB of free space
We have two stores on C:, one is 22 GB, the other is about 1.5 GB.
We have two stores on D:, one is 30 GB, the other (our Public store) is 105 GB.
Should I alternate days with each store? Right now I set it so it does the Public store one night, and all the others on the alternate night.
ASKER
Event Type: Information
Event Source: ESE
Event Category: Logging/Recovery
Event ID: 221
Date: 4/20/2006
Time: 3:17:01 AM
User: N/A
Computer: XXXX
Description:
Information Store (4184) dc83d85b-e5d6-41f2-908e-8e
Event Type: Information
Event Source: ESE
Event Category: Logging/Recovery
Event ID: 221
Date: 4/20/2006
Time: 10:18:15 AM
User: N/A
Computer: XXXX
Description:
Information Store (4184) dc83d85b-e5d6-41f2-908e-8e
Based on the logs shown above, does this mean that it doesn't start backing up the public store until it finished backing up the NPStore at 3:17am? If that's the case, could I run online maintenance on the Public store until 3am and then run maintenance on the NPStore after it's finished backing it up. Or is it just running the backups in general during online maintenance that isn't recommended?
Event Source: ESE
Event Category: Logging/Recovery
Event ID: 221
Date: 4/20/2006
Time: 3:17:01 AM
User: N/A
Computer: XXXX
Description:
Information Store (4184) dc83d85b-e5d6-41f2-908e-8e
Event Type: Information
Event Source: ESE
Event Category: Logging/Recovery
Event ID: 221
Date: 4/20/2006
Time: 10:18:15 AM
User: N/A
Computer: XXXX
Description:
Information Store (4184) dc83d85b-e5d6-41f2-908e-8e
Based on the logs shown above, does this mean that it doesn't start backing up the public store until it finished backing up the NPStore at 3:17am? If that's the case, could I run online maintenance on the Public store until 3am and then run maintenance on the NPStore after it's finished backing it up. Or is it just running the backups in general during online maintenance that isn't recommended?
The smaller stores should normally defragment within an hour or so, but I'm not sure about that when there's so little free space.
Don't take it personal, but my first impression is that you're really tickling the edge of storage space. Not just a little bit, you have no room to maneuvre whatsoever.
Apart from the huge performance penalty on the databases (7% and 4% free), having no disk space really makes any kind of disaster recovery very difficult and very slow. You can't run eseutil on the larger stores, physical defragmentation is impossible, and transaction logs fill all available disk space when you miss one or two online backups.
With these databases on Exchange Enterprise, this data looks important to me. Do you have some sort of service levels that you need to fulfill? Has management ever considered what the consequences are if any of these stores are unavailable for more than 4 hours or even an entire day? I'm asking because I think it's impossible to restore or repair the public store within that time, with this amount of disk space.
I don't know what type of server you have, but if you have 2 or 3 drive bays left to expand storage space, I really recommend you get some nice 300 GB SCSI drives and off-load the public store to that array.
Don't take it personal, but my first impression is that you're really tickling the edge of storage space. Not just a little bit, you have no room to maneuvre whatsoever.
Apart from the huge performance penalty on the databases (7% and 4% free), having no disk space really makes any kind of disaster recovery very difficult and very slow. You can't run eseutil on the larger stores, physical defragmentation is impossible, and transaction logs fill all available disk space when you miss one or two online backups.
With these databases on Exchange Enterprise, this data looks important to me. Do you have some sort of service levels that you need to fulfill? Has management ever considered what the consequences are if any of these stores are unavailable for more than 4 hours or even an entire day? I'm asking because I think it's impossible to restore or repair the public store within that time, with this amount of disk space.
I don't know what type of server you have, but if you have 2 or 3 drive bays left to expand storage space, I really recommend you get some nice 300 GB SCSI drives and off-load the public store to that array.
Running backups during online defrag is not recommended because of performance issues. There is no other technical reason I know of that these shouldn't run concurrently.
ASKER
Do you have an article that you can post where it says I need at least 20-25% of free space for decent performance of online defragmentation? Not that I don't believe you, I would need it to show our IT director, as I'm not finding it on the net.
Unfortunately we only have 1 drive bay available. So we don't have any room for a new array. We are looking at Symantec Enterprise Vault to archive email. I'm hoping we can at least reclaim at least 1GB worth tonight if I start the maintenance at 8pm and the IS backup doesn't start until 1am. I won't be running any maintenance on the other stores tonight so hopefully that goes well. Plus I'm going to run it all weekend.
How long do you think it would take to run an offline defrag over the weekend to a mapped network drive?
Unfortunately we only have 1 drive bay available. So we don't have any room for a new array. We are looking at Symantec Enterprise Vault to archive email. I'm hoping we can at least reclaim at least 1GB worth tonight if I start the maintenance at 8pm and the IS backup doesn't start until 1am. I won't be running any maintenance on the other stores tonight so hopefully that goes well. Plus I'm going to run it all weekend.
How long do you think it would take to run an offline defrag over the weekend to a mapped network drive?
ASKER
Is there anyway for me to tell right now, how much potential free space I could claim with either a successful online defrag, or an offline defrag?
ASKER
Can you even recover the free physcial space from the database during an offline defrag until an online defrag has reclaimed that space first?
ASKER
We have a 300GB drive available. I think I may insert that into our last bay, and run an offline defrag on the public store only. I don't want to have our entire exchange server down over the weekend. So let me get this straight. I can dismount only the Public Folder store and run:
eseutil /d d:\progra~1\exchsrvr\mdbda
Will everything else run on my exchange server just as normal (users can send and receive mail, etc.) but just not have access to the Public folders? And then once this is all completed, I can delete the original pub1.edb database and copy the new defragmented database back to D: How does that sound?
eseutil /d d:\progra~1\exchsrvr\mdbda
Will everything else run on my exchange server just as normal (users can send and receive mail, etc.) but just not have access to the Public folders? And then once this is all completed, I can delete the original pub1.edb database and copy the new defragmented database back to D: How does that sound?
ASKER
Has anyone attempted to run this before... http://support.microsoft.com/kb/163627/
How to Defrag an EDB File on a Non-Exchange Server
How to Defrag an EDB File on a Non-Exchange Server
It is not so much the issue that you need 20% space for optimization of online defrag, because online defrag doesn't cause downtime.
The issue here is that you need the *database size* free to use any kind of database utilities. Offline defrag, physical defrag and database repairs need to make a copy of the database. Also, restoring a full backup of any part of the information requires the replay of transaction logs created since the last backup, so the server must be able to perform a whole day's work as quickly as possible so you can get the stores mounted. File fragmentation is important here: Exchange creates 5 MB log files at once, to prevent log fragmentation. But that doesn't fly when the small amount of space you have is scattered across the drive in 4KB chunks.
<< Is there anyway for me to tell right now, how much potential free space I could claim with either a successful online defrag, or an offline defrag? >>
Not without dismounting the respective store. The command (will take some time!):
eseutil /ms pub1.edb > freespace.txt
will create a text file with very detailed information, including the number of free database pages (multiply by 4096 for bytes, or divide by 256 for MB). See http://support.microsoft.com/kb/q195914/ how to use it.
<< Can you even recover the free physcial space from the database during an offline defrag until an online defrag has reclaimed that space first? >>
Yes, but offline defragmentation requires free space equal to database size.
Inserting a 300 GB drive in the last bay and running defrag on it is a good idea, this is far quicker than copying files over the network. Yes, you can dismount the public store without affecting user mailboxes, and proceed as you mentioned.
- Format the drive completely to make sure it's not defective.
- Make sure you have a working backup
- Perform offline defrag. Have some sleep, this will take more than 8 hours.
- Before copying the 100GB file back, delete the old DB and Windows Defrag the drive to consolidate free space
- Copy the pub1 database files back to D:, this will partially take care of fragmentation.
No, I haven't tried defragging EDB files on a non-exchange server before.
The issue here is that you need the *database size* free to use any kind of database utilities. Offline defrag, physical defrag and database repairs need to make a copy of the database. Also, restoring a full backup of any part of the information requires the replay of transaction logs created since the last backup, so the server must be able to perform a whole day's work as quickly as possible so you can get the stores mounted. File fragmentation is important here: Exchange creates 5 MB log files at once, to prevent log fragmentation. But that doesn't fly when the small amount of space you have is scattered across the drive in 4KB chunks.
<< Is there anyway for me to tell right now, how much potential free space I could claim with either a successful online defrag, or an offline defrag? >>
Not without dismounting the respective store. The command (will take some time!):
eseutil /ms pub1.edb > freespace.txt
will create a text file with very detailed information, including the number of free database pages (multiply by 4096 for bytes, or divide by 256 for MB). See http://support.microsoft.com/kb/q195914/ how to use it.
<< Can you even recover the free physcial space from the database during an offline defrag until an online defrag has reclaimed that space first? >>
Yes, but offline defragmentation requires free space equal to database size.
Inserting a 300 GB drive in the last bay and running defrag on it is a good idea, this is far quicker than copying files over the network. Yes, you can dismount the public store without affecting user mailboxes, and proceed as you mentioned.
- Format the drive completely to make sure it's not defective.
- Make sure you have a working backup
- Perform offline defrag. Have some sleep, this will take more than 8 hours.
- Before copying the 100GB file back, delete the old DB and Windows Defrag the drive to consolidate free space
- Copy the pub1 database files back to D:, this will partially take care of fragmentation.
No, I haven't tried defragging EDB files on a non-exchange server before.
ASKER
Should I use the /t or the /p switch when specifying the temporary location of the new defragmented database? How come you only need to specifiy the .edb file? What happens to the .stm file and the transaction logs, can they be deleted as well after the new database is created? Thanks so much for your help man.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
So when it's all said and done. I'll have a .edb and a .stm file on my F: drive. I'll then delete the two old fragmented files off of D:, and then copy the new database files back to D:, rename them back to pub1.edb and pub1.stm and then mount the store and I should be good to go. Sound right?
What you say is correct.
When you have 30 GB used on D: it can be a good idea to look at a Windows defrag analysis. If the other 30 GB store is scattered in 2500 fragments I'd also run a defrag on the drive to optimize that and consolidate free space. It depends on the amount of time you can spend on that. You can do that when the store is mounted.
When you have 30 GB used on D: it can be a good idea to look at a Windows defrag analysis. If the other 30 GB store is scattered in 2500 fragments I'd also run a defrag on the drive to optimize that and consolidate free space. It depends on the amount of time you can spend on that. You can do that when the store is mounted.
ASKER
I thought I read somewhere not do run a windows defrag on a partition that contains an Exchange database? And did you mean when I have 30 GB free on D:? I can't remember where I read it, but just thought I'd mention it really quick because I have to run somewhere, but I'll check back later this evening...
I'll be going to sleep, different timezone ;-)
http://www.microsoft.com/exchange/techinfo/tips/defragmentation.asp
http://www.microsoft.com/exchange/techinfo/tips/defragmentation.asp
ASKER
Well I dismounted one of our very small private stores (only about 1.4GB) and ran eseutil /MS to see how much space I could free up with an offline defrag. It came out to be about 430MB. Then I ran an offline defrag on the private store and it actually freed up about 700MB. How come the estimate was off when running the /MS switch?
Is that because eseutil /MS can only estimate free space based on what an online defrag has reclaimed already?
I dismounted our Public store and ran eseutil /MS and it only estimated about 3.3GB. Now I know there's more to reclaim from that database because I deleted more than 18GB a couple weeks ago, but I know that an online defrag hasn't even come close to completing. So do you think I can reclaim a lot more space that 3.3GB with an offline defrag?
Is that because eseutil /MS can only estimate free space based on what an online defrag has reclaimed already?
I dismounted our Public store and ran eseutil /MS and it only estimated about 3.3GB. Now I know there's more to reclaim from that database because I deleted more than 18GB a couple weeks ago, but I know that an online defrag hasn't even come close to completing. So do you think I can reclaim a lot more space that 3.3GB with an offline defrag?
Really? There is a section * SLV SPACE DUMP * that gives the statistics of the STM file. Below the graph is a summary that says Free/Reserved/Deleted/Comm
The Free figure from the first table and the Available figure on the bottom added together, divided by 256, should then amount to 700 MB.
The Free figure from the first table and the Available figure on the bottom added together, divided by 256, should then amount to 700 MB.
ASKER
Nope, the total of the two was 110443. Divide that by 256 and it comes to about 431MB. I'm more curious as to why my public store only shows about 3.3GB that can be reclaimed, when I deleted a lot a couple weeks ago.
ASKER
At least I was able to reclaim 1.1GB last night during the online maintenance. I didn't start it until 11pm because I was messing with the Exchange Server and had to reboot it to update our array manager, but it had a good 2 hours to run without any backups going until the IS backup started at 1am. I'm still concerned about only recovering no more than 3.3GB on my public store, it just can't be right....
ASKER
Rant,
You have been extremely helpful. I wish I could award you more than 500 points here! I'm going to start the offline defrag tonight, hopefully we'll reclaim a lot more than 3.3GB.....
You have been extremely helpful. I wish I could award you more than 500 points here! I'm going to start the offline defrag tonight, hopefully we'll reclaim a lot more than 3.3GB.....
You can, I received 2000 because of the A grade ;-)
If the 18 GB you dropped earlier has passed item retention, them I'm sure you will. Glad to be of service!
If the 18 GB you dropped earlier has passed item retention, them I'm sure you will. Glad to be of service!
ASKER
Well, the offline defrag is complete. I had to do it to a mapped network drive when I found out that our PowerEdge 2650 didn't have 6 drive bays like our 2850s do. The defrag only took 5 hours to complete, and freed up 20GB. Everything is fine and I now have about 25GB worth of free space on my D: drive. Now that I'm more familiar with when to run online defrag and backups, hopefully we can keep our Exchange server more under control. Thanks again for your help!
You can run Eseutil /d on the databases.
Check for event ID 1221 , you can check how much space you can get after running eseutil /d ....if you don't find one
Try running eseutil /ms on the Db, it will generate the event id in applog and then you can check the same.
Thanks
Kunal