Solved

Exchange 2010 server 100% CPU - no specific process

Posted on 2015-01-07
7
669 Views
Last Modified: 2015-01-20
We have a Server 2008 R2 Sp1 Hyper-V virtual machine: 4 vCPU's, 16 GB RAM.  Exchange Server 2010 Version: 14.03.0123.003  (SP3).  Single Exchange server running all roles except UM.  It is also running GFI MailEssentials for SPAM control.  The server is pegged at 100% CPU by various processes.  It is not a single process causing the server to become terribly slow.  Mostly it's GFI processes, Store.exe, W3wp.exe,   Activesync users are reporting slowness and it's taking 30+ minutes to receive local or internet emails.

I've beaten my head against this.  We've disabled GFI MailEssentials by pulling the transport rules out of Exchange (they have a utility to do this).  No change.  We've disabled the GFI service and that does provide a small amount of relief, but the server still spikes to 100% enough to cause delays in mail flow.   We loaded Exmon and did find that one user was almost always at the top of the list of Store.exe CPU utilization.  We had that user remove the Activesync account from the phone and close the mailbox from Outlook 2013.  No changes to CPU utilization.  I'd like to add that now EXmon will not run, but crashes.....

Even at this point with GFI services disabled, the server is extremely difficult to work on and takes forever to get anywhere to do anything.  We really need some help in narrowing down where this issue lies.  Also, keeping GFI disabled permanently is really not an option. They receive a significant amount of junk mail and they're practically crippled if that all makes it to the inbox.

One strange item I should mention is that the admin mailbox is receiving notification of messages blocked in GFI that have the SPAM tab appended in the subject line numerous times.  I'm wondering if this is indicating some sort of email loop.  

Help please!
0
Comment
Question by:bkready
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
7 Comments
 
LVL 53

Expert Comment

by:Will Szymkowski
ID: 40536508
Does the server act the same way after a reboot? If so, how long does it take before the server is crippled again? Have you checked the logs on the server to see if they are showing any additional info?

Will.
0
 
LVL 43

Expert Comment

by:Amit
ID: 40536515
Do you have hyper threading enabled? If yes, disable it first. Is this a VM or Physical box. Check the CPU guideline
http://technet.microsoft.com/en-in/library/dd346699(v=exchg.141).aspx

What about AV, did you  excluded all Exchange binaries and folder from AV scanning, if not do it.

If it is a VM, make sure all VM patches are up to date.
0
 

Author Comment

by:bkready
ID: 40536542
The server returns to this state very shortly after rebooting it.  

HyperThreading would be set to whatever the default would have been. The Procs are dual Intel® Xeon E5-2440's.

There is currently no AV installed on this box.

It is a VM. We can't really associate any changes to the start of this situation.
0
Free NetCrunch network monitor licenses!

Only on Experts-Exchange: Sign-up for a free-trial and we'll send you your permanent license!

Here is what you get: 30 Nodes | Unlimited Sensors | No Time Restrictions | Absolutely FREE!

Act now. This offer ends July 14, 2017.

 
LVL 43

Expert Comment

by:Amit
ID: 40536854
Which process is consuming max CPU? IOS is know to cause such spikes. Make sure your user devices are upto date.
0
 

Accepted Solution

by:
bkready earned 0 total points
ID: 40551588
Sorry for the delays.  A milion other problems since this issue arose....  Amit, it is no specific process, but a mix of Store.exe and various GFI processes.  

However, in the interim, I believe the issue has been resolved.  After disabling GFI MailEssentials for some time, the server CPU usage finally quieted down.  Once able to actually function on the server and access the GFI logs and message tracking in Exchange, we noticed some odd subject lines in emails that were being tagged by GFI.  Some messages had multiple (up to a dozen or so) Spam tags prepended to the subject line.   GFI was causing messages to get stuck in a "loop" and circle back around inside Exchange.  Per GFI support this causes the headers to increase in size to the point where Exchange has difficulty processing these messages. We were/are running MailEssentials 20140627 and were provided a patch by GFI support to resolve this issue.  So far, since the patch has been installed and GFI MailEssentials has been re-enabled, we have not noticed that this issue has reoccurred and CPU usage on the box seems normal.
0
 
LVL 43

Expert Comment

by:Amit
ID: 40551617
Great you found the cause and solution. However, I personally avoid installing anything on Exchange server. For other 3rd party tool, it is good to setup new servers and pass the traffic to Exchange server. This way you avoid creating issues on your Exchange servers and troubleshooting is also easy.
0
 

Author Closing Comment

by:bkready
ID: 40559334
Between our own troubleshooting and contacting vendor support for a product directly, we were able to resolve the issues ourselves.
0

Featured Post

Free NetCrunch network monitor licenses!

Only on Experts-Exchange: Sign-up for a free-trial and we'll send you your permanent license!

Here is what you get: 30 Nodes | Unlimited Sensors | No Time Restrictions | Absolutely FREE!

Act now. This offer ends July 14, 2017.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article will help to fix the below errors for MS Exchange Server 2013 I. Certificate error "name on the security certificate is invalid or does not match the name of the site" II. Out of Office not working III. Make Internal URLs and Externa…
Check out this step-by-step guide for using the newly updated Experts Exchange mobile app—released on May 30.
In this Micro Video tutorial you will learn the basics about Database Availability Groups and How to configure one using a live Exchange Server Environment. The video tutorial explains the basics of the Exchange server Database Availability grou…
This video shows how to quickly and easily add an email signature for all users on Exchange 2016. The resulting signature is applied on a server level by Exchange Online. The email signature template has been downloaded from: www.mail-signatures…

717 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question