Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

That NUMA and vNUMA question

Posted on 2016-11-10
4
Medium Priority
?
163 Views
Last Modified: 2016-11-15
This question is based on settings to for NUMA and vNUMA and what is the best way to set Cores Per Sockets and Virtual Sockets in the VM settings.

So, when enabled vNUMA exposes the VM OS to the physical NUMA.
But is it automatically set?

By default, if I recall, VMware recommends setting Core Per socket to 1 and changing Virtual Sockets thus vNUMA will be enabled to select the best NUMA configuration to be used?

So, with that said, we have a requirement to test customer software on systems with multiple CPUs and various amounts or RAM.

Our ESXi host has 2 cpu sockets, 8 cores, and 32 Logical processors with 512 GB of RAM.
I logged onto the ESXi host and see that there are 2 NUMA Nodes available.

VM requirements :

Win7 Vm with 2 vCPUs
Win7 Vm with 4 vCPUs
Win7 Vm with 8 vCPUs
Centos Server with 4 vCPUs
Centos Server with 8 vCPUs
Centos Server with 16 vCPUs

One test VM that is Centos Server, we initially have set as 4 Core Per Sockets and 2 Virtual Sockets giving 8 vCPUS.. If I understood the above info I stated, this would not enable vNUMA? Is this still a valid set up for vCPUs?

So, should we set the VMs vCPUS as:
  1 Core Per Socket and 8 Virtual Cores,
  1 Core Per Socket and 16 Virtual Cores,
or
  Stay with something like

  4 Core Per Sockets and 2 Virtual Cores, etc?
0
Comment
Question by:sjniii
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
4 Comments
 
LVL 123

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 2000 total points
ID: 41882729
The best method is to let VMware vSphere decide and accept the defaults.

e.g. sockets.

So this is 2 sockets x 1 core = 2 vCPU (2 CPUs in the VM)

always use sockets, unless you have a specific reason to use cores.

Recommended Practices

#1 When creating a virtual machine, by default, vSphere will create as many virtual sockets as you’ve requested vCPUs and the cores per socket is equal to one. I think of this configuration as “wide” and “flat.” This will enable vNUMA to select and present the best virtual NUMA topology to the guest operating system, which will be optimal on the underlying physical topology.

#2 When you must change the cores per socket though, commonly due to licensing constraints, ensure you mirror physical server’s NUMA topology. This is because when a virtual machine is no longer configured by default as “wide” and “flat,” vNUMA will not automatically pick the best NUMA configuration based on the physical server, but will instead honor your configuration – right or wrong – potentially leading to a topology mismatch that does affect performance.

Source

Does corespersocket Affect Performance?

vSMP (virtual SMP) can affect virtual machine performance, when adding too many vCPUs to virtual machines that cannot use the vCPUs effectly, e.g. Servers than can use vSMP correctly :- SQL Server, Exchange Server.

This is true, many VMware Administrators, think adding lots of processors, will increase performance - wrong! (and because they can, they just go silly!). Sometimes there is confusion between cores and processors. But what we are adding is additional processors in the virtual machine.

So 4 vCPU, to the VM is a 4 Way SMP (Quad Processor Server), if you have Enterprise Plus license you can add 8, (and only if you have the correct OS License will the OS recognise them all).

If applications, can take advantage e.g. Exchange, SQL, adding additional processors, can/may increase performance.

So usual rule of thumb is try 1 vCPU, then try 2 vCPU, knock back to 1 vCPU if performance is affected. and only use vSMP if the VM can take advantage.

Example, VM with 4 vCPUs allocated!

My simple laymans explaination of the "scheduler!"

As you have assigned 4 vCPUs, to this VM, the VMware scheulder, has to wait until 4 cores are free and available, to do this, it has to pause the first cores, until the 4th is available, during this timeframe, the paused cores are not available for processes, this is my simplistic view, but bottom line is adding more vCPUs to a VM, may not give you the performance benefits you think, unless the VM, it's applications are optimised for additional vCPUs.

See here
http://www.vmware.com/resources/techresources/10131

see here
http://www.gabesvirtualworld.com/how-too-many-vcpus-can-negatively-affect-your-performance/

http://www.zdnet.com/virtual-cpus-the-overprovisioning-penalty-of-vcpu-to-pcpu-ratios-4010025185/

also there is a document here about the CPU scheduler

www.vmware.com/files/pdf/perf-vsphere-cpu_scheduler.pdf
1
 
LVL 1

Assisted Solution

by:sjniii
sjniii earned 0 total points
ID: 41886755
We have a vSphere Enterprise Plus license.

Per the customer request, we have to run their code on VMs with different CPU settings.

in the example, VM with 4 vCPUS allocated, it does not show what settings were used to get the 4 vCPUS (Number of virtual sockets 4 + Number of cores per socket 1, Number of virtual sockets 2 + Number of cores per socket 2, or Number of virtual sockets 1 + Number of cores per socket 4,?)

We are focusing on the Centos 64-bit VMs as this is where the code is running and need to test 4, 8, 16 CPUs.

It sounds as though we should set the  3 VMs and let vSphere decide
as in
Number of Virtual Sockets number of virtual sockets 4, 8, 16 and leave  Number of cores per socket set t 1 = vNUMA wide and flat?
0
 
LVL 123
ID: 41886830
That's correct.

Or, you will have to do the configuration, and work it out yourself, if you manually allocate VMs per core, which defeats the money you've paid for the hypervisor.

Or build a cluster of Centos Machines using a HPC, using bare metal and no hypervisors.
0
 
LVL 1

Author Closing Comment

by:sjniii
ID: 41887827
Thanks for the help.
0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article outlines why you need to choose a backup solution that protects your entire environment – including your VMware ESXi and Microsoft Hyper-V virtualization hosts – not just your virtual machines.
Giving access to ESXi shell console is always an issue for IT departments to other Teams, or Projects. We need to find a way so that teams can use ESXTOP for their POCs, or tests without giving them the access to ESXi host shell console with a root …
Teach the user how to configure vSphere Replication and how to protect and recover VMs Open vSphere Web Client: Verify vsphere Replication is enabled: Enable vSphere Replication for a virtual machine: Verify replicated VM is created: Recover replica…
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial

715 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question