[2 days left] What’s wrong with your cloud strategy? Learn why multicloud solutions matter with Nimble Storage.Register Now

x
?
Solved

Email system went down (Exchange 2007)

Posted on 2011-03-11
14
Medium Priority
?
632 Views
Last Modified: 2012-05-11
hey guys

just around one hour ago, our entire emails system went down. Nobody could open Outlook (would hang) and the webmail service was the same. Throughout the day, it was constantly slow, very tempermental and evidently it took its toll and everything went down by the end of the day.

We;re running Exchange 2007. It does NOT have SP1. The disk space available for the database is 410GB and the free space on it is at 59GB! Now, around 6months ago we made changes to the registry so that it could accept a database size of 300GB. We're currently using 350GB from what it seems. I've gone into the registry and increased the limit to 400GB.

Nothing in the event viewer indicated anything about database size limit as far as I can see.

However, does this sound like it could have been the database size that caused the issue?

Also, there are so many deleted mailboxes and apparently only a defrag would get rid of them to free up space, but according to some experts this would take around 3 days?!
 
On the disk which holds the databases, we've created them so that it's not just one huge database file but we split into three database files. Is there a way of defraging each one separately so that it doesn't take 3days and going through this procedure without having to take down Exchange entirely?

Any solutions you propose would be much appreciated.
Thanks
Yashy
0
Comment
Question by:Yashy
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 4
  • 4
  • +2
14 Comments
 
LVL 8

Accepted Solution

by:
Acosta Technology Services earned 1000 total points
ID: 35111362
First thing I would do is check the 1211 event log (should be around 7AM) in the application log.  This will tell you how much whitespace is available within the database; that's how much you could free with an offline defrag.

If there is a lot of space available I would recommend an offline defrag for the DB with the most "freeable" space.

Also, how are your logs currently?  Has a full backup finished recently to clear the logs?
0
 
LVL 8

Expert Comment

by:Acosta Technology Services
ID: 35111371
Sorry, that should be event 1221, NOT 1211.
0
 
LVL 10

Expert Comment

by:akhalighi
ID: 35111391
- clear log files , restart exchange services , what do you see in event logs ? any specific errors ?
- how is resource usage on this box , memory , CPU , etc ...
0
Creating Active Directory Users from a Text File

If your organization has a need to mass-create AD user accounts, watch this video to see how its done without the need for scripting or other unnecessary complexities.

 
LVL 44

Expert Comment

by:Amit
ID: 35111621
I would first suggest you to upgrade to lastest Service pack. Which is SP3 for 2007.

For defrag part. Thumb rule is 3-4 GB per hour. I will take 4GB per hour. So 300/4 = 75 and 74/24 = 3.125 days. Which is very long downtime. Best option is to create new DB and move the mailboxes over to them. Try to split them in mulitple DB's rather keeping them in one.

For Server issue. Check the systems and applications logs, what events are generated during that window. Check for Virtual Memory setting also. It has to be Actual + 10MB. Make sure all AV exclusion are in place for Exchange and check for any memory leak issue.

0
 
LVL 8

Expert Comment

by:Acosta Technology Services
ID: 35111694
The timing is based on the logical DB size, not the physical.  If DB1 is 100GB on disk, but has 40GB of whitespace you can calculate the total defrag time based on 60GB.  The total time will depend on the size of each DB in your information store and how much whitespace is available (event 1221).
0
 
LVL 7

Expert Comment

by:OctInv
ID: 35112702
You say the entire system went down - do you mean the databases are dismounted? Can you give us a bit mroe info on your Exchange environment - are all roles on one server, are the log files on a seperate disk/volume (if so how are they doing for space?)....the more info you can give the easier it will be to suggest solutiuons.

Thanks
0
 
LVL 1

Author Comment

by:Yashy
ID: 35114032
i'm looking all of your comments guys and giving feedback in the morning as I restarted the exchange server and things are working, but for how long I don't know.

Also, I have split our databases on our disk drive. So the total is 351gb, however they are split into three databases; around 150gb, 100gb and 151gb. Can they each be defragged separately?
0
 
LVL 1

Author Comment

by:Yashy
ID: 35115578
Guys in the applicatino log I'm getting a lot of warnings like this:

Reset to device, \Device\RaidPort1, was issued.

Also, I have checked the disk array utility and it shows the following, could this be why. Look at the picture I have attached.


disks.JPG
0
 
LVL 8

Expert Comment

by:Acosta Technology Services
ID: 35116106
I would replace the predicted fail drive as soon as possible to avoid any other possible issues.

For Exchange:

-Are you seeing any Exchange related errors in the application event log?
-If you check event's 1221 from this morning how much freespace does it show on each DB?

For your questions:

-Yes, you can defrag each DB indivually
0
 
LVL 44

Expert Comment

by:Amit
ID: 35117438
Open call with Hardware vendor for server hardware issue. Check the system logs...you might see issue related to disk
0
 
LVL 1

Author Comment

by:Yashy
ID: 35118403
Spoke to HP guys and drive will be replaced on Monday. However our database file was 300gb...ouch. So we' re going to change disks and then im going to have to reduce that database size with your helpful comments. Then im going to have to look into doing some sort of disaster recovery so that i can prevent this from happening. Do any of you have a DR for Exchange? Otherwise how is your exchange environment resilient to an event such as one hard disk from failing? Thanks.
0
 
LVL 44

Assisted Solution

by:Amit
Amit earned 1000 total points
ID: 35121630
You can go with Exchange 2007 CCR cluster. This can give you the Server and disk resilency both. If you want to have site resilency also, then you can implement CCR over the WAN.

Storage planning is very important part of any Exchange env. Always use Storage calculator.
http://msexchangeteam.com/archive/2007/01/15/432207.aspx

Secondly, if you have CAS and HUB on different servers. You can use the NLB for load balancing

Finally, If you have any plans to move with Exchange 2010. Then you can use DAG, which is not dependent on Window clustering and easy to implement also.
0
 
LVL 1

Author Comment

by:Yashy
ID: 35122859
Okay, I'll look into this.

By the way, if we have four disks at 146GB each with Raid5 configuration, then how much disk space should be left?

Reason is because i'm looking at the array which holds the four drives and it says:

410gb available, 59gb free to use.

I'm just wondering whether the free space is a correct indication of the current situation?
0
 
LVL 44

Expert Comment

by:Amit
ID: 35123014
In Raid 5 one disk will be used for fault tolerance. You can check with your platform team, what RAID exactly configured currently. As you said it is HP server. Check from HP Array Console.
0

Featured Post

Problems using Powershell and Active Directory?

Managing Active Directory does not always have to be complicated.  If you are spending more time trying instead of doing, then it's time to look at something else. For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The core idea of this article is to make you acquainted with the best way in which you can export Exchange mailbox to PST format.
If something goes wrong with Exchange, your IT resources are in trouble.All Exchange server migration processes are not designed to be identical and though migrating email from on-premises Exchange mailbox to Cloud’s Office 365 is relatively simple…
This tutorial will walk an individual through setting the global and backup job media overwrite and protection periods in Backup Exec 2012. Log onto the Backup Exec Central Administration Server. Examine the services. If all or most of them are stop…
This Experts Exchange video Micro Tutorial shows how to tell Microsoft Office that a word is NOT spelled correctly. Microsoft Office has a built-in, main dictionary that is shared by Office apps, including Excel, Outlook, PowerPoint, and Word. When …
Suggested Courses

656 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question