Exchange store service hangs in SBS2003

For the last few weeks our exchange has been becoming unavailable to clients, outlook just freezes up and cannot retrieve data.

Checking the service list the information store is still running, when I select the service adn either select stop or restart it fails to stop correctly.  At this point if I manually end the store.exe process and then start the service again it comes back up, the pop3 email connector doesn't actually work until it's restarted as well.

I've updated our trend micro anti virus which has been running on the server since day 1 (about 2 years ago) and I've unmounted the stores and run both ESEUTIL and ISUTIL through them with zero errors, they remounted just fine.

I've done everything I can think of, updated all the applications, run all the microsoft updates and service packs, it's still happening every 2-3 days, it doesn't seem to coincide with any particular event like backups or people using OWA.

Any help appreciated
Who is Participating?
Manpreet SIngh KhatraConnect With a Mentor Solutions Architect, Project LeadCommented:
What is the set size of the Database with the Registry "Database size limit in Gb" as the database is about 55Gb. Also the event ID 1005 and 9175 shows that the Stores may be down at this time.

Can you do one thing try stopping the IMAP and also go to regedit
HKLM\System\CCS\Services\MSExchangeIS\VIrusScan (May or may not be)
and then find Enabled and Background scanning and make the value to 0.

Can you also verify that at this time if the stores are mounted or not.
Manpreet SIngh KhatraSolutions Architect, Project LeadCommented:
Can you for a couple of days stop and disable the third party applications?
What is the size of the database (.edb and .stm)?
Manpreet SIngh KhatraSolutions Architect, Project LeadCommented:
Is there any event id for ESE or MSExchangeIS with id as 1000 for hang.
Simplify Active Directory Administration

Administration of Active Directory does not have to be hard.  Too often what should be a simple task is made more difficult than it needs to be.The solution?  Hyena from SystemTools Software.  With ease-of-use as well as powerful importing and bulk updating capabilities.

princess_frostyAuthor Commented:
priv1.stm = 27.7Gb
priv1.edb = 27.1Gb
pub1.stm = 131Mb
pub1.edb = 45Mb

The most recent defrag 1221 event reports "The database "First Storage Group\Mailbox Store (***)" has 3831 megabytes of free space after online defragmentation has terminated."

So it's not a space thing, that I can work out.

I could potentially stop the backup services from running, I'm a bit worried about stopping the Antivirus however, the trend micro mail add on scans incoming mails and strips out any viruses.

We weren't getting a lot of errors surrounding this problem which has made it a lot harder to troubleshoot, I've just checked the event viewer for sunday which is when the store last went down, at about 11am we get errors in the application log with EventID 1023 with a source of IMAP4SVC.

It looks like 2 speciifc users are getting these errors over and over:

Error 0x8004050a occurred while rendering message 0001-000000b4dcf4 for download for user ****@*********

The only thing I can think of that these 2 users have in common that other users do not is that they have their iphones set up to connect via IMAP to the server to retrieve mails (and use SMTP to send), but it's unclear if this is a cause of the problems or seeing these errors is simply a symptom of exchange failing.

Maybe I should stop the IMAP service and see if that helps?  Nothing other than the Iphones use it, and thats only because the exchange setup on the iphone is awkward and fails.
princess_frostyAuthor Commented:
I forgot to mentio, these IMAP4SVC errors continue until about 17:00 at which point one of our admins had logged on and we get a single error below

EventID 1005
Source MSExchangeSA
Unexpected error <<0xc1050000 - Network problems are preventing connection to the Microsoft Exchange Server computer. An unexpected, unknown error has occurred. Microsoft Exchange Server Information Store ID no: 80040115-0514-000006b5>> occurred.

Followed by

EventID 1005
Source MSExchangeSA
Unexpected error <<0xc1050000 - The Microsoft Exchange Server computer is not available. Either there are network problems or the Microsoft Exchange Server computer is down for maintenance. The MAPI provider failed. Microsoft Exchange Server Information Store ID no: 8004011d-0526-00000000>> occurred.

And then multiple instances of:

EventID 9175
Source MSExchangeSA
The MAPI call 'OpenMsgStore' failed with the following error:
The Microsoft Exchange Server computer is not available.  Either there are network problems or the Microsoft Exchange Server computer is down for maintenance.
The MAPI provider failed.
Microsoft Exchange Server Information Store
ID no: 8004011d-0526-00000000

Which I presume is the outlook clients failing to connect...this goes on until 17:16 when the service is manually stopped (which fails) and the process store.exe is killed, then the service is restarted.
Manpreet SIngh KhatraSolutions Architect, Project LeadCommented:
What is the time when the Backup and Antivirus work?
What Antivirus do we have and were any updates done to them?
If its a file scanner do we have exclusions set for the Exchange database and other files to be excuded?
princess_frostyAuthor Commented:
Database Size Limit in GB is 60, this seems consistent as the free space being recoded by online defrag is about 4gb and we're using about 55.

Background scanning was already set to 0, I've set enabled to 0 also now.

Right now the stores are mounted, I unmounted them last night for the integrity checks but they were re-mounted about 2am.

Do i need to restart all the exchange services to pick up this registry change?
Manpreet SIngh KhatraSolutions Architect, Project LeadCommented:
If possible and also as i asked is this a File scanner as well and are exclusions set on them?
Also you can run the msconfig command and hide all microsoft services and verify the other third parties and see if there is anything that might cause the issue
princess_frostyAuthor Commented:
We run Symantec backup service which runs most of time since our backups take all day, and we run trend microscan, the server client is installed and so is the email live scan, but they have been since day1, both are up to date.

I can't find scan settings for the standard antivirus, its a fairly dumb little client agent that just does realtime scanning, I've forced it to scan the exchange dbdata folder with a full scan and it doesn't cause any negative side effects, it's built to work with MS/Exch so I'm not convinced that is the cause.
princess_frostyAuthor Commented:
The only other services running are grouped into

HP services for various features like Proliant and the RAID management
Trend microscan agents for antivirus and proxy service
Backup Exec agents
FTP service
SQL (Sharepoint)

And and few other bits and bobs which shouldn't cause any problems like services to monitor routers etc.
Manpreet SIngh KhatraSolutions Architect, Project LeadCommented:
First of all the Antivirus should never ever scan the Exchange DB, Logs ,Exchsrvr, Inetpub and Inetsrv folders.
Secondly what time is the Backups scheduled to run and what kind of backups are these (Online, Offline, VSS, Brick level or what)?
princess_frostyAuthor Commented:
I can't see any significant control over the scanning options for the security agent, it just realtime scans and thats about it, again scanning the store doesn't result in a problem, I specifically tried this to rule it out.

Backups are scheduled to run at 11pm, tapes are changed at 8-9am in the morning for the 2nd set and 3rd set is usually mid day.  It doesn't run on sunday which is when the store went down last, as I said in my original post it doesn't seem to coincide with backup times.  

Besides these are server based tools designed to work in this environement and have worked fine for 2 years with no change to their configuration.  They're online backups.
Is your backup large or going to slow storage? A backup shouldn't run all day.
Something I see a lot with Backup Exec is where people select the Information Store, Mailboxes and Public Folders, which is just unnecessary. You only need the information store for a valid backup of Exchange, Mailboxes and Public Folders do not need to be selected and is a very slow method of backup. It is also close to useless for most recovery scenarios.

What else is on the machine? I find Exchange is usually very reliable.

Have you run the SBS Best practises tool (free download from MS) against the system?

princess_frostyAuthor Commented:
It's an SBS box it's deisnged to do everything for the business, exchange, domain controller and SQL database are the main functions.

The tape backup is about 40gb per tape and requires 3 tapes to do a full backup, so to get daily backups it runs most of the day.

Exchange HAS been very reliable for basically 2 years, its just the last few weeks this has been happening, I'm going to leave the IMAP4 server stopped and leave the antivirus "enabled" reg key set to 0 for exchange and see if that helps, I will check back in a few days with a yay or nay, thanks.
princess_frostyAuthor Commented:
So far so good, no crashes yet, for some reason the IMAP4 service has started again it maybe because some updates were applied recently (but not restarted yet)

I'll reboot the server tonight and make sure the IMAP4 service is running and see if it remains stable.
princess_frostyAuthor Commented:
Thanks for the help, it looks like since the IMAP4 service started again at some point that it must be the registry change I made...

It doesn't make much sense to me, trend micro is designed for servers and had been running for years without issue, but it seems to be working so at least I have some stability back, thanks!
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.