Solved

xenserver 6.2 - centos 6.5 xentools and cpu load balancing

Posted on 2015-02-19
6
103 Views
Last Modified: 2016-04-06
Hi experts!

I have a mysql-based monitoring system that runs on CentOS 6.5 which is on a vm hosted by xenserver 6.2. It monitors a large number of interfaces and requires a lot of cpu power. I have 10 cores provisioned for it. Currently 24GB of RAM, which is way more than it needs. RAID 10 disk array. It seems that all cores are being used to a degree when I use the 'top' tool in centos to monitor them, but the 1st cpu is being hammered much more than any other. I have contacted the monitoring system vendor about it, and they do not have much support for xenserver installations, but they said the lack of CPU load balancing is not typical for physical and vmware installations. I don't think xencenter 6.2 yet supports CentOS6.5 as a guest OS.  XenCenter always shows less than 25% CPU usage, and averages around 10% or less. I can tell the the web interface is less responsive when that 1st CPU is being hammered. (see attached screenshot which shows typical scenario.)

I do not have the XenTools installed because I couldn't get it to install. There was an error. I tried for a couple hours to install it from the CLI , and I thought I had it, but it turns out I was not successful.

I used the generic Linux template to install it because using the CentOS template caused the GUI OS interface not to load properly--possibly the reason why I could not install XenTools is because I installed it this way.

Also, note that not all the physical CPU cores are currently allocated to guest VMs. There are 40 total, and 18 are free.


My first question is where do I start looking for the cause of the load balancing problem? mysql? centos? xentools? the xenserver host?

Second question: has anyone else had these issues with a CentOS 6.5 on xenserver 6.2?

Any help is appreciated. Please let me know if more info is needed.


Centos-top-021915
0
Comment
Question by:jdfuller
  • 4
  • 2
6 Comments
 
LVL 23

Accepted Solution

by:
Dirk Kotte earned 500 total points
ID: 40620584
wa times means waiting for IO completion.
http://www.computerhope.com/unix/top.htm
possible the missing paravirtualisation-drivers (xentool) are the problem.
without these tools standard-hardware is emulated.

take a look to the xencenters performance view and add the "latency" counter for the disk.
0
 

Author Comment

by:jdfuller
ID: 40622180
Thanks!  I'll look into it.
JDF
0
 

Author Comment

by:jdfuller
ID: 40628813
Would the wa mean the same thing in a virtual environment as in a physical environment?

I mean, If CentOS sees physical CPUs, it is really seeing vCPUs. If it sees a physical disk, it really means it sees a virtual disk, correct? So if it sees physical I/O, it is really looking at virtual I/O, right? So I need to understand, if virtual I/O is the problem, what does this translate to, physically?
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 
LVL 23

Expert Comment

by:Dirk Kotte
ID: 40631571
i can comment this only from windows perspective.
using windows (like XP) with standard drivers is slower in disk IO than the same windows with specialized vendor drivers installed.
The XenServer guest without paravirtualisation-drivers uses the emulated interface for disk- / hardware- access. This is slower than the other option.
0
 

Author Comment

by:jdfuller
ID: 40631631
Thank you.  XenTools are not installed in the CentOS instance.  We have had a lot of difficulty in getting these loaded.  Perhaps with those installed we will see an improvement - if we can get them installed at all.  We have kind of hit a wall with that.  Going to Xen support for help.
0
 

Author Closing Comment

by:jdfuller
ID: 41538268
We  concluded that it is indeed an I/O performance related issue. So, I marked the first I/O comment as the best solution. Thanks everyone, and sorry for the late close.
0

Featured Post

Netscaler Common Configuration How To guides

If you use NetScaler you will want to see these guides. The NetScaler How To Guides show administrators how to get NetScaler up and configured by providing instructions for common scenarios and some not so common ones.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Citrix XenDesktop 7.6 Citrix Policies Disable Peripherals
Fine Tune your automatic Updates for Ubuntu / Debian
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to moveā€¦
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

840 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question