Solved

2 x ESXi hosts on IBM System X3550 Hardware - CPU Spike on the hour every hour

Posted on 2015-02-17
4
351 Views
Last Modified: 2015-03-03
we have 2 ESXi hosts running 4 VMs each

the hardware in both hosts consists of:

45GB RAM
6 x 2.26 GHZ Processors

The storage is:

IBMv3700 connected by ISCSI

Every hour on the hour the 2 hosts experience a spike in CPU which causes all servers to also experience high CPU. This is, unsurprisingly, most noticeable on the RDS server.

I've checked for scheduled tasks and the like on the Windows VMs ( it doesn't affect the vCenter VA ) and cannot see a regular pattern. There is no backup, AV, tasks, etc running to this schedule.

The fact that it happens on both hosts is quite bizarre.

There is a 50/50 split in VMs across the hosts. Configured as follows:


HOST 1

Exchange
4 vCPU
12GB RAM

Print
1 vCPU
1GB RAM

WebServer
1 vCPU
2GB RAM

CRM
4 vCPU
7GB RAM


HOST 2

RDS
6 vCPU
18GB RAM

VC
2 vCPU
8GB RAM

SQL
2 vCPU
12GB RAM

DC
1 vCPU
4GB RAM

When viewing the performance tab within vSphere, we see an identical pattern on VMs as the hosts.

Any help would be greatly appreciated.
0
Comment
Question by:utilize
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
4 Comments
 
LVL 28

Accepted Solution

by:
asavener earned 500 total points
ID: 40615098
It sounds to me as if you have the causality wrong.  High CPU on the guests would cause high CPU usage on the hosts.

High CPU usage on the hosts would cause a spike in CPU_Ready on the guests.  (CPU_Ready measures when the guest is demanding CPU and the host cannot provide it.)

If all of your windows machines are spiking at the same time, then I'd look at what processes are common to all of them (such as antivirus).

If it's this regular, you should be able to just run performance monitor, sort by CPU, and wait for the spike to happen.  That should identify the process causing the CPU spike.
0
 

Author Comment

by:utilize
ID: 40616005
I've identified that the issue seems to be caused by TrustedInstaller.exe on the hour within each VM.

I've also identified an issue with Group Policies and PolicyDefinitions which I believe is related. I'll come back shortly
0
 

Assisted Solution

by:utilize
utilize earned 0 total points
ID: 40620970
Following on from this.

It did indeed end up being a third party monitoring tool (N-Able) that was checking for updates every hour. Checking for Windows Updates, that is.

Disabling these services, after checking the requirements, sorted out the issue.

Accepting your solution as an answer, you alluded to third parties.
0
 

Author Closing Comment

by:utilize
ID: 40641324
Self resolved.
0

Featured Post

When ransomware hits your clients, what do you do?

MSPs: Endpoint security isn’t enough to prevent ransomware.
As the impact and severity of crypto ransomware attacks has grown, Webroot fought back, not just by building a next-gen endpoint solution capable of preventing ransomware attacks but also by being a thought leader.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Giving access to ESXi shell console is always an issue for IT departments to other Teams, or Projects. We need to find a way so that teams can use ESXTOP for their POCs, or tests without giving them the access to ESXi host shell console with a root …
For anyone that has accidentally used newSID with Server 2008 R2 (like I did) and hasn't been able to get the server running again because you were unlucky (as I was) and had no backups - I was able to get things working by doing a Registry Hive rec…
This tutorial will give a short introduction and overview of Backup Exec 2012 and how to navigate and perform basic functions. Click on the Backup Exec button in the upper left corner. From here, are global settings for the application such as conne…
There are cases when e.g. an IT administrator wants to have full access and view into selected mailboxes on Exchange server, directly from his own email account in Outlook or Outlook Web Access. This proves useful when for example administrator want…

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question