Solved

Exchange 2010 server 100% CPU - no specific process

Posted on 2015-01-07
7
539 Views
Last Modified: 2015-01-20
We have a Server 2008 R2 Sp1 Hyper-V virtual machine: 4 vCPU's, 16 GB RAM.  Exchange Server 2010 Version: 14.03.0123.003  (SP3).  Single Exchange server running all roles except UM.  It is also running GFI MailEssentials for SPAM control.  The server is pegged at 100% CPU by various processes.  It is not a single process causing the server to become terribly slow.  Mostly it's GFI processes, Store.exe, W3wp.exe,   Activesync users are reporting slowness and it's taking 30+ minutes to receive local or internet emails.

I've beaten my head against this.  We've disabled GFI MailEssentials by pulling the transport rules out of Exchange (they have a utility to do this).  No change.  We've disabled the GFI service and that does provide a small amount of relief, but the server still spikes to 100% enough to cause delays in mail flow.   We loaded Exmon and did find that one user was almost always at the top of the list of Store.exe CPU utilization.  We had that user remove the Activesync account from the phone and close the mailbox from Outlook 2013.  No changes to CPU utilization.  I'd like to add that now EXmon will not run, but crashes.....

Even at this point with GFI services disabled, the server is extremely difficult to work on and takes forever to get anywhere to do anything.  We really need some help in narrowing down where this issue lies.  Also, keeping GFI disabled permanently is really not an option. They receive a significant amount of junk mail and they're practically crippled if that all makes it to the inbox.

One strange item I should mention is that the admin mailbox is receiving notification of messages blocked in GFI that have the SPAM tab appended in the subject line numerous times.  I'm wondering if this is indicating some sort of email loop.  

Help please!
0
Comment
Question by:bkready
  • 3
  • 3
7 Comments
 
LVL 53

Expert Comment

by:Will Szymkowski
ID: 40536508
Does the server act the same way after a reboot? If so, how long does it take before the server is crippled again? Have you checked the logs on the server to see if they are showing any additional info?

Will.
0
 
LVL 41

Expert Comment

by:Amit
ID: 40536515
Do you have hyper threading enabled? If yes, disable it first. Is this a VM or Physical box. Check the CPU guideline
http://technet.microsoft.com/en-in/library/dd346699(v=exchg.141).aspx

What about AV, did you  excluded all Exchange binaries and folder from AV scanning, if not do it.

If it is a VM, make sure all VM patches are up to date.
0
 

Author Comment

by:bkready
ID: 40536542
The server returns to this state very shortly after rebooting it.  

HyperThreading would be set to whatever the default would have been. The Procs are dual Intel® Xeon E5-2440's.

There is currently no AV installed on this box.

It is a VM. We can't really associate any changes to the start of this situation.
0
Do email signature updates give you a headache?

Constantly trying to correctly format email signatures? Spending all of your time at every user’s desk to make updates? Want high-quality HTML signatures on all devices, including on mobiles and Macs? Then, let Exclaimer solve all your email signature problems today!

 
LVL 41

Expert Comment

by:Amit
ID: 40536854
Which process is consuming max CPU? IOS is know to cause such spikes. Make sure your user devices are upto date.
0
 

Accepted Solution

by:
bkready earned 0 total points
ID: 40551588
Sorry for the delays.  A milion other problems since this issue arose....  Amit, it is no specific process, but a mix of Store.exe and various GFI processes.  

However, in the interim, I believe the issue has been resolved.  After disabling GFI MailEssentials for some time, the server CPU usage finally quieted down.  Once able to actually function on the server and access the GFI logs and message tracking in Exchange, we noticed some odd subject lines in emails that were being tagged by GFI.  Some messages had multiple (up to a dozen or so) Spam tags prepended to the subject line.   GFI was causing messages to get stuck in a "loop" and circle back around inside Exchange.  Per GFI support this causes the headers to increase in size to the point where Exchange has difficulty processing these messages. We were/are running MailEssentials 20140627 and were provided a patch by GFI support to resolve this issue.  So far, since the patch has been installed and GFI MailEssentials has been re-enabled, we have not noticed that this issue has reoccurred and CPU usage on the box seems normal.
0
 
LVL 41

Expert Comment

by:Amit
ID: 40551617
Great you found the cause and solution. However, I personally avoid installing anything on Exchange server. For other 3rd party tool, it is good to setup new servers and pass the traffic to Exchange server. This way you avoid creating issues on your Exchange servers and troubleshooting is also easy.
0
 

Author Closing Comment

by:bkready
ID: 40559334
Between our own troubleshooting and contacting vendor support for a product directly, we were able to resolve the issues ourselves.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Following basic email etiquette rules will help you write a professional email and achieve a good, lasting impression with your contacts.
Possible fixes for Windows 7 and Windows Server 2008 updating problem. Solutions mentioned are from Microsoft themselves. I started a case with them from our Microsoft Silver Partner option to open a case and get direct support from Microsoft. If s…
This tutorial will walk an individual through the steps necessary to configure their installation of BackupExec 2012 to use network shared disk space. Verify that the path to the shared storage is valid and that data can be written to that location:…
To add imagery to an HTML email signature, you have two options available to you. You can either add a logo/image by embedding it directly into the signature or hosting it externally and linking to it. The vast majority of email clients display l…

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now