Solved

Exchange 2010 server 100% CPU - no specific process

Posted on 2015-01-07
7
643 Views
Last Modified: 2015-01-20
We have a Server 2008 R2 Sp1 Hyper-V virtual machine: 4 vCPU's, 16 GB RAM.  Exchange Server 2010 Version: 14.03.0123.003  (SP3).  Single Exchange server running all roles except UM.  It is also running GFI MailEssentials for SPAM control.  The server is pegged at 100% CPU by various processes.  It is not a single process causing the server to become terribly slow.  Mostly it's GFI processes, Store.exe, W3wp.exe,   Activesync users are reporting slowness and it's taking 30+ minutes to receive local or internet emails.

I've beaten my head against this.  We've disabled GFI MailEssentials by pulling the transport rules out of Exchange (they have a utility to do this).  No change.  We've disabled the GFI service and that does provide a small amount of relief, but the server still spikes to 100% enough to cause delays in mail flow.   We loaded Exmon and did find that one user was almost always at the top of the list of Store.exe CPU utilization.  We had that user remove the Activesync account from the phone and close the mailbox from Outlook 2013.  No changes to CPU utilization.  I'd like to add that now EXmon will not run, but crashes.....

Even at this point with GFI services disabled, the server is extremely difficult to work on and takes forever to get anywhere to do anything.  We really need some help in narrowing down where this issue lies.  Also, keeping GFI disabled permanently is really not an option. They receive a significant amount of junk mail and they're practically crippled if that all makes it to the inbox.

One strange item I should mention is that the admin mailbox is receiving notification of messages blocked in GFI that have the SPAM tab appended in the subject line numerous times.  I'm wondering if this is indicating some sort of email loop.  

Help please!
0
Comment
Question by:bkready
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
7 Comments
 
LVL 53

Expert Comment

by:Will Szymkowski
ID: 40536508
Does the server act the same way after a reboot? If so, how long does it take before the server is crippled again? Have you checked the logs on the server to see if they are showing any additional info?

Will.
0
 
LVL 43

Expert Comment

by:Amit
ID: 40536515
Do you have hyper threading enabled? If yes, disable it first. Is this a VM or Physical box. Check the CPU guideline
http://technet.microsoft.com/en-in/library/dd346699(v=exchg.141).aspx

What about AV, did you  excluded all Exchange binaries and folder from AV scanning, if not do it.

If it is a VM, make sure all VM patches are up to date.
0
 

Author Comment

by:bkready
ID: 40536542
The server returns to this state very shortly after rebooting it.  

HyperThreading would be set to whatever the default would have been. The Procs are dual Intel® Xeon E5-2440's.

There is currently no AV installed on this box.

It is a VM. We can't really associate any changes to the start of this situation.
0
Office 365 Training for IT Pros

Learn how to provision tenants, synchronize on-premise Active Directory, implement Single Sign-On, customize Office deployment, and protect your organization with eDiscovery and DLP policies.  Only from Platform Scholar.

 
LVL 43

Expert Comment

by:Amit
ID: 40536854
Which process is consuming max CPU? IOS is know to cause such spikes. Make sure your user devices are upto date.
0
 

Accepted Solution

by:
bkready earned 0 total points
ID: 40551588
Sorry for the delays.  A milion other problems since this issue arose....  Amit, it is no specific process, but a mix of Store.exe and various GFI processes.  

However, in the interim, I believe the issue has been resolved.  After disabling GFI MailEssentials for some time, the server CPU usage finally quieted down.  Once able to actually function on the server and access the GFI logs and message tracking in Exchange, we noticed some odd subject lines in emails that were being tagged by GFI.  Some messages had multiple (up to a dozen or so) Spam tags prepended to the subject line.   GFI was causing messages to get stuck in a "loop" and circle back around inside Exchange.  Per GFI support this causes the headers to increase in size to the point where Exchange has difficulty processing these messages. We were/are running MailEssentials 20140627 and were provided a patch by GFI support to resolve this issue.  So far, since the patch has been installed and GFI MailEssentials has been re-enabled, we have not noticed that this issue has reoccurred and CPU usage on the box seems normal.
0
 
LVL 43

Expert Comment

by:Amit
ID: 40551617
Great you found the cause and solution. However, I personally avoid installing anything on Exchange server. For other 3rd party tool, it is good to setup new servers and pass the traffic to Exchange server. This way you avoid creating issues on your Exchange servers and troubleshooting is also easy.
0
 

Author Closing Comment

by:bkready
ID: 40559334
Between our own troubleshooting and contacting vendor support for a product directly, we were able to resolve the issues ourselves.
0

Featured Post

SharePoint Admin?

Enable Your Employees To Focus On The Core With Intuitive Onscreen Guidance That is With You At The Moment of Need.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Scam emails are a huge burden for many businesses. Spotting one is not always easy. Follow our tips to identify if an email you receive is a scam.
Find out what you should include to make the best professional email signature for your organization.
This tutorial will walk an individual through the steps necessary to install and configure the Windows Server Backup Utility. Directly connect an external storage device such as a USB drive, or CD\DVD burner: If the device is a USB drive, ensure i…
This tutorial will walk an individual through the process of transferring the five major, necessary Active Directory Roles, commonly referred to as the FSMO roles to another domain controller. Log onto the new domain controller with a user account t…

734 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question