Solved

xenserver 6.2 - centos 6.5 xentools and cpu load balancing

Posted on 2015-02-19
6
91 Views
Last Modified: 2016-04-06
Hi experts!

I have a mysql-based monitoring system that runs on CentOS 6.5 which is on a vm hosted by xenserver 6.2. It monitors a large number of interfaces and requires a lot of cpu power. I have 10 cores provisioned for it. Currently 24GB of RAM, which is way more than it needs. RAID 10 disk array. It seems that all cores are being used to a degree when I use the 'top' tool in centos to monitor them, but the 1st cpu is being hammered much more than any other. I have contacted the monitoring system vendor about it, and they do not have much support for xenserver installations, but they said the lack of CPU load balancing is not typical for physical and vmware installations. I don't think xencenter 6.2 yet supports CentOS6.5 as a guest OS.  XenCenter always shows less than 25% CPU usage, and averages around 10% or less. I can tell the the web interface is less responsive when that 1st CPU is being hammered. (see attached screenshot which shows typical scenario.)

I do not have the XenTools installed because I couldn't get it to install. There was an error. I tried for a couple hours to install it from the CLI , and I thought I had it, but it turns out I was not successful.

I used the generic Linux template to install it because using the CentOS template caused the GUI OS interface not to load properly--possibly the reason why I could not install XenTools is because I installed it this way.

Also, note that not all the physical CPU cores are currently allocated to guest VMs. There are 40 total, and 18 are free.


My first question is where do I start looking for the cause of the load balancing problem? mysql? centos? xentools? the xenserver host?

Second question: has anyone else had these issues with a CentOS 6.5 on xenserver 6.2?

Any help is appreciated. Please let me know if more info is needed.


Centos-top-021915
0
Comment
Question by:jdfuller
  • 4
  • 2
6 Comments
 
LVL 23

Accepted Solution

by:
Dirk Kotte earned 500 total points
Comment Utility
wa times means waiting for IO completion.
http://www.computerhope.com/unix/top.htm
possible the missing paravirtualisation-drivers (xentool) are the problem.
without these tools standard-hardware is emulated.

take a look to the xencenters performance view and add the "latency" counter for the disk.
0
 

Author Comment

by:jdfuller
Comment Utility
Thanks!  I'll look into it.
JDF
0
 

Author Comment

by:jdfuller
Comment Utility
Would the wa mean the same thing in a virtual environment as in a physical environment?

I mean, If CentOS sees physical CPUs, it is really seeing vCPUs. If it sees a physical disk, it really means it sees a virtual disk, correct? So if it sees physical I/O, it is really looking at virtual I/O, right? So I need to understand, if virtual I/O is the problem, what does this translate to, physically?
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 
LVL 23

Expert Comment

by:Dirk Kotte
Comment Utility
i can comment this only from windows perspective.
using windows (like XP) with standard drivers is slower in disk IO than the same windows with specialized vendor drivers installed.
The XenServer guest without paravirtualisation-drivers uses the emulated interface for disk- / hardware- access. This is slower than the other option.
0
 

Author Comment

by:jdfuller
Comment Utility
Thank you.  XenTools are not installed in the CentOS instance.  We have had a lot of difficulty in getting these loaded.  Perhaps with those installed we will see an improvement - if we can get them installed at all.  We have kind of hit a wall with that.  Going to Xen support for help.
0
 

Author Closing Comment

by:jdfuller
Comment Utility
We  concluded that it is indeed an I/O performance related issue. So, I marked the first I/O comment as the best solution. Thanks everyone, and sorry for the late close.
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

Citrix XenDesktop, gold image, VMware, vSphere.
#Citrix #Citrix Policies #XenDesktop #VDI #POC #Citrix Univeral Printer Driver #Citrix UPD
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

8 Experts available now in Live!

Get 1:1 Help Now