Solved

xenserver 6.2 - centos 6.5 xentools and cpu load balancing

Posted on 2015-02-19
6
99 Views
Last Modified: 2016-04-06
Hi experts!

I have a mysql-based monitoring system that runs on CentOS 6.5 which is on a vm hosted by xenserver 6.2. It monitors a large number of interfaces and requires a lot of cpu power. I have 10 cores provisioned for it. Currently 24GB of RAM, which is way more than it needs. RAID 10 disk array. It seems that all cores are being used to a degree when I use the 'top' tool in centos to monitor them, but the 1st cpu is being hammered much more than any other. I have contacted the monitoring system vendor about it, and they do not have much support for xenserver installations, but they said the lack of CPU load balancing is not typical for physical and vmware installations. I don't think xencenter 6.2 yet supports CentOS6.5 as a guest OS.  XenCenter always shows less than 25% CPU usage, and averages around 10% or less. I can tell the the web interface is less responsive when that 1st CPU is being hammered. (see attached screenshot which shows typical scenario.)

I do not have the XenTools installed because I couldn't get it to install. There was an error. I tried for a couple hours to install it from the CLI , and I thought I had it, but it turns out I was not successful.

I used the generic Linux template to install it because using the CentOS template caused the GUI OS interface not to load properly--possibly the reason why I could not install XenTools is because I installed it this way.

Also, note that not all the physical CPU cores are currently allocated to guest VMs. There are 40 total, and 18 are free.


My first question is where do I start looking for the cause of the load balancing problem? mysql? centos? xentools? the xenserver host?

Second question: has anyone else had these issues with a CentOS 6.5 on xenserver 6.2?

Any help is appreciated. Please let me know if more info is needed.


Centos-top-021915
0
Comment
Question by:jdfuller
  • 4
  • 2
6 Comments
 
LVL 23

Accepted Solution

by:
Dirk Kotte earned 500 total points
ID: 40620584
wa times means waiting for IO completion.
http://www.computerhope.com/unix/top.htm
possible the missing paravirtualisation-drivers (xentool) are the problem.
without these tools standard-hardware is emulated.

take a look to the xencenters performance view and add the "latency" counter for the disk.
0
 

Author Comment

by:jdfuller
ID: 40622180
Thanks!  I'll look into it.
JDF
0
 

Author Comment

by:jdfuller
ID: 40628813
Would the wa mean the same thing in a virtual environment as in a physical environment?

I mean, If CentOS sees physical CPUs, it is really seeing vCPUs. If it sees a physical disk, it really means it sees a virtual disk, correct? So if it sees physical I/O, it is really looking at virtual I/O, right? So I need to understand, if virtual I/O is the problem, what does this translate to, physically?
0
Backup Your Microsoft Windows Server®

Backup all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

 
LVL 23

Expert Comment

by:Dirk Kotte
ID: 40631571
i can comment this only from windows perspective.
using windows (like XP) with standard drivers is slower in disk IO than the same windows with specialized vendor drivers installed.
The XenServer guest without paravirtualisation-drivers uses the emulated interface for disk- / hardware- access. This is slower than the other option.
0
 

Author Comment

by:jdfuller
ID: 40631631
Thank you.  XenTools are not installed in the CentOS instance.  We have had a lot of difficulty in getting these loaded.  Perhaps with those installed we will see an improvement - if we can get them installed at all.  We have kind of hit a wall with that.  Going to Xen support for help.
0
 

Author Closing Comment

by:jdfuller
ID: 41538268
We  concluded that it is indeed an I/O performance related issue. So, I marked the first I/O comment as the best solution. Thanks everyone, and sorry for the late close.
0

Featured Post

Netscaler Common Configuration How To guides

If you use NetScaler you will want to see these guides. The NetScaler How To Guides show administrators how to get NetScaler up and configured by providing instructions for common scenarios and some not so common ones.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Citrix XenDesktop 7.6 Citrix Policies Graphics
ADCs have gained traction within the last decade, largely due to increased demand for legacy load balancing appliances to handle more advanced application delivery requirements and improve application performance.
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

785 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question