We help IT Professionals succeed at work.
Get Started

High CPU Load

672 Views
Last Modified: 2015-10-25
HI

I am running Xen-PV with SolusVM on Centos 6 and hosting about 11 Virtual machines. 5. Since yesterday the cpu load on my host machine goes very high and reaches between 28-35. Connecting with SSH becomes very hard. I run top command but did not find any processes taking very high CPU.
I am puzzled. I also ran xm top command (screenshot attached).
Also /var/log/messages show below message:

Oct 19 16:47:01 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320
Oct 19 16:47:01 slave-wahad kernel: audit: audit_lost=150 audit_rate_limit=0 audit_backlog_limit=320
Oct 19 16:47:01 slave-wahad kernel: audit: backlog limit exceeded
Oct 19 16:47:01 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320
Oct 19 16:47:01 slave-wahad kernel: audit: audit_lost=151 audit_rate_limit=0 audit_backlog_limit=320
Oct 19 16:47:01 slave-wahad kernel: audit: backlog limit exceeded
Oct 19 16:47:01 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320
Oct 19 16:47:01 slave-wahad kernel: audit: audit_lost=152 audit_rate_limit=0 audit_backlog_limit=320
Oct 19 16:47:01 slave-wahad kernel: audit: backlog limit exceeded
Oct 19 16:47:01 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320
Oct 19 16:47:20 slave-wahad kernel: audit_log_start: 2 callbacks suppressed
Oct 19 16:47:20 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320
Oct 19 16:47:20 slave-wahad kernel: audit: audit_lost=154 audit_rate_limit=0 audit_backlog_limit=320
Oct 19 16:47:20 slave-wahad kernel: audit: backlog limit exceeded
Oct 19 16:47:20 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320
Oct 19 16:47:20 slave-wahad kernel: audit: audit_lost=155 audit_rate_limit=0 audit_backlog_limit=320
Oct 19 16:47:20 slave-wahad kernel: audit: backlog limit exceeded
Oct 19 16:47:20 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320
Oct 19 16:47:20 slave-wahad kernel: audit: audit_lost=156 audit_rate_limit=0 audit_backlog_limit=320
Oct 19 16:47:20 slave-wahad kernel: audit: backlog limit exceeded
Oct 19 16:47:20 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320

Open in new window


and

# aureport --start today --event --summary -i

Event Summary Report
======================
total  type
======================
6751  NETFILTER_CFG
1470  USER_START
1462  LOGIN
1462  USER_ACCT
1462  CRED_ACQ
1446  USER_END
1445  CRED_DISP
1415  CRYPTO_KEY_USER
748  USER_AUTH
584  CRYPTO_SESSION
538  USER_LOGIN
265  USER_ERR
11  ANOM_PROMISCUOUS
8  CRED_REFR
2  USER_LOGOUT
1  CONFIG_CHANGE

Open in new window


I would be really thankful if someone can please help me identify what is causing this high CPU load.
Screen-Shot-2015-10-19-at-8.42.17-PM.png
Comment
Watch Question
Software Engineer
CERTIFIED EXPERT
Distinguished Expert 2019
Commented:
This problem has been solved!
Unlock 2 Answers and 3 Comments.
See Answers
Why Experts Exchange?

Experts Exchange always has the answer, or at the least points me in the correct direction! It is like having another employee that is extremely experienced.

Jim Murphy
Programmer at Smart IT Solutions

When asked, what has been your best career decision?

Deciding to stick with EE.

Mohamed Asif
Technical Department Head

Being involved with EE helped me to grow personally and professionally.

Carl Webster
CTP, Sr Infrastructure Consultant
Ask ANY Question

Connect with Certified Experts to gain insight and support on specific technology challenges including:

  • Troubleshooting
  • Research
  • Professional Opinions
Did You Know?

We've partnered with two important charities to provide clean water and computer science education to those who need it most. READ MORE