HI
I am running Xen-PV with SolusVM on Centos 6 and hosting about 11 Virtual machines. 5. Since yesterday the cpu load on my host machine goes very high and reaches between 28-35. Connecting with SSH becomes very hard. I run top command but did not find any processes taking very high CPU.
I am puzzled. I also ran xm top command (screenshot attached).
Also /var/log/messages show below message:
Oct 19 16:47:01 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320
Oct 19 16:47:01 slave-wahad kernel: audit: audit_lost=150 audit_rate_limit=0 audit_backlog_limit=320
Oct 19 16:47:01 slave-wahad kernel: audit: backlog limit exceeded
Oct 19 16:47:01 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320
Oct 19 16:47:01 slave-wahad kernel: audit: audit_lost=151 audit_rate_limit=0 audit_backlog_limit=320
Oct 19 16:47:01 slave-wahad kernel: audit: backlog limit exceeded
Oct 19 16:47:01 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320
Oct 19 16:47:01 slave-wahad kernel: audit: audit_lost=152 audit_rate_limit=0 audit_backlog_limit=320
Oct 19 16:47:01 slave-wahad kernel: audit: backlog limit exceeded
Oct 19 16:47:01 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320
Oct 19 16:47:20 slave-wahad kernel: audit_log_start: 2 callbacks suppressed
Oct 19 16:47:20 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320
Oct 19 16:47:20 slave-wahad kernel: audit: audit_lost=154 audit_rate_limit=0 audit_backlog_limit=320
Oct 19 16:47:20 slave-wahad kernel: audit: backlog limit exceeded
Oct 19 16:47:20 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320
Oct 19 16:47:20 slave-wahad kernel: audit: audit_lost=155 audit_rate_limit=0 audit_backlog_limit=320
Oct 19 16:47:20 slave-wahad kernel: audit: backlog limit exceeded
Oct 19 16:47:20 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320
Oct 19 16:47:20 slave-wahad kernel: audit: audit_lost=156 audit_rate_limit=0 audit_backlog_limit=320
Oct 19 16:47:20 slave-wahad kernel: audit: backlog limit exceeded
Oct 19 16:47:20 slave-wahad kernel: audit: audit_backlog=321 > audit_backlog_limit=320
Select all Open in new window
and
# aureport --start today --event --summary -i
Event Summary Report
======================
total type
======================
6751 NETFILTER_CFG
1470 USER_START
1462 LOGIN
1462 USER_ACCT
1462 CRED_ACQ
1446 USER_END
1445 CRED_DISP
1415 CRYPTO_KEY_USER
748 USER_AUTH
584 CRYPTO_SESSION
538 USER_LOGIN
265 USER_ERR
11 ANOM_PROMISCUOUS
8 CRED_REFR
2 USER_LOGOUT
1 CONFIG_CHANGE
Select all Open in new window
I would be really thankful if someone can please help me identify what is causing this high CPU load.
Screen-Shot-2015-10-19-at-8.42.17-PM.png
> The problem causing processes might also cause a lot of logging, that may help. Look for processes with a lot of fast changing pid's
How can I see those processes when top doesn't show that? Sorry if the the question looks noob.