Solved

Exchange 2010 server 100% CPU - no specific process

Posted on 2015-01-07
7
552 Views
Last Modified: 2015-01-20
We have a Server 2008 R2 Sp1 Hyper-V virtual machine: 4 vCPU's, 16 GB RAM.  Exchange Server 2010 Version: 14.03.0123.003  (SP3).  Single Exchange server running all roles except UM.  It is also running GFI MailEssentials for SPAM control.  The server is pegged at 100% CPU by various processes.  It is not a single process causing the server to become terribly slow.  Mostly it's GFI processes, Store.exe, W3wp.exe,   Activesync users are reporting slowness and it's taking 30+ minutes to receive local or internet emails.

I've beaten my head against this.  We've disabled GFI MailEssentials by pulling the transport rules out of Exchange (they have a utility to do this).  No change.  We've disabled the GFI service and that does provide a small amount of relief, but the server still spikes to 100% enough to cause delays in mail flow.   We loaded Exmon and did find that one user was almost always at the top of the list of Store.exe CPU utilization.  We had that user remove the Activesync account from the phone and close the mailbox from Outlook 2013.  No changes to CPU utilization.  I'd like to add that now EXmon will not run, but crashes.....

Even at this point with GFI services disabled, the server is extremely difficult to work on and takes forever to get anywhere to do anything.  We really need some help in narrowing down where this issue lies.  Also, keeping GFI disabled permanently is really not an option. They receive a significant amount of junk mail and they're practically crippled if that all makes it to the inbox.

One strange item I should mention is that the admin mailbox is receiving notification of messages blocked in GFI that have the SPAM tab appended in the subject line numerous times.  I'm wondering if this is indicating some sort of email loop.  

Help please!
0
Comment
Question by:bkready
  • 3
  • 3
7 Comments
 
LVL 53

Expert Comment

by:Will Szymkowski
ID: 40536508
Does the server act the same way after a reboot? If so, how long does it take before the server is crippled again? Have you checked the logs on the server to see if they are showing any additional info?

Will.
0
 
LVL 41

Expert Comment

by:Amit
ID: 40536515
Do you have hyper threading enabled? If yes, disable it first. Is this a VM or Physical box. Check the CPU guideline
http://technet.microsoft.com/en-in/library/dd346699(v=exchg.141).aspx

What about AV, did you  excluded all Exchange binaries and folder from AV scanning, if not do it.

If it is a VM, make sure all VM patches are up to date.
0
 

Author Comment

by:bkready
ID: 40536542
The server returns to this state very shortly after rebooting it.  

HyperThreading would be set to whatever the default would have been. The Procs are dual Intel® Xeon E5-2440's.

There is currently no AV installed on this box.

It is a VM. We can't really associate any changes to the start of this situation.
0
Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

 
LVL 41

Expert Comment

by:Amit
ID: 40536854
Which process is consuming max CPU? IOS is know to cause such spikes. Make sure your user devices are upto date.
0
 

Accepted Solution

by:
bkready earned 0 total points
ID: 40551588
Sorry for the delays.  A milion other problems since this issue arose....  Amit, it is no specific process, but a mix of Store.exe and various GFI processes.  

However, in the interim, I believe the issue has been resolved.  After disabling GFI MailEssentials for some time, the server CPU usage finally quieted down.  Once able to actually function on the server and access the GFI logs and message tracking in Exchange, we noticed some odd subject lines in emails that were being tagged by GFI.  Some messages had multiple (up to a dozen or so) Spam tags prepended to the subject line.   GFI was causing messages to get stuck in a "loop" and circle back around inside Exchange.  Per GFI support this causes the headers to increase in size to the point where Exchange has difficulty processing these messages. We were/are running MailEssentials 20140627 and were provided a patch by GFI support to resolve this issue.  So far, since the patch has been installed and GFI MailEssentials has been re-enabled, we have not noticed that this issue has reoccurred and CPU usage on the box seems normal.
0
 
LVL 41

Expert Comment

by:Amit
ID: 40551617
Great you found the cause and solution. However, I personally avoid installing anything on Exchange server. For other 3rd party tool, it is good to setup new servers and pass the traffic to Exchange server. This way you avoid creating issues on your Exchange servers and troubleshooting is also easy.
0
 

Author Closing Comment

by:bkready
ID: 40559334
Between our own troubleshooting and contacting vendor support for a product directly, we were able to resolve the issues ourselves.
0

Featured Post

Do email signature updates give you a headache?

Do you feel like you are constantly making changes to email signatures? Are the images not formatting how you want them to? Want high-quality HTML signatures on all devices, including on mobiles and Macs? Then, let Exclaimer solve all your email signature problems today.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This process describes the steps required to Import and Export data from and to .pst files using Exchange 2010. We can use these steps to export data from a user to a .pst file, import data back to the same or a different user, or even import data t…
A procedure for exporting installed hotfix details of remote computers using powershell
This tutorial will walk an individual through the steps necessary to configure their installation of BackupExec 2012 to use network shared disk space. Verify that the path to the shared storage is valid and that data can be written to that location:…
This tutorial will walk an individual through the process of transferring the five major, necessary Active Directory Roles, commonly referred to as the FSMO roles to another domain controller. Log onto the new domain controller with a user account t…

914 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now