That NUMA and vNUMA question

Posted on 2016-11-10
Last Modified: 2016-11-15
This question is based on settings to for NUMA and vNUMA and what is the best way to set Cores Per Sockets and Virtual Sockets in the VM settings.

So, when enabled vNUMA exposes the VM OS to the physical NUMA.
But is it automatically set?

By default, if I recall, VMware recommends setting Core Per socket to 1 and changing Virtual Sockets thus vNUMA will be enabled to select the best NUMA configuration to be used?

So, with that said, we have a requirement to test customer software on systems with multiple CPUs and various amounts or RAM.

Our ESXi host has 2 cpu sockets, 8 cores, and 32 Logical processors with 512 GB of RAM.
I logged onto the ESXi host and see that there are 2 NUMA Nodes available.

VM requirements :

Win7 Vm with 2 vCPUs
Win7 Vm with 4 vCPUs
Win7 Vm with 8 vCPUs
Centos Server with 4 vCPUs
Centos Server with 8 vCPUs
Centos Server with 16 vCPUs

One test VM that is Centos Server, we initially have set as 4 Core Per Sockets and 2 Virtual Sockets giving 8 vCPUS.. If I understood the above info I stated, this would not enable vNUMA? Is this still a valid set up for vCPUs?

So, should we set the VMs vCPUS as:
  1 Core Per Socket and 8 Virtual Cores,
  1 Core Per Socket and 16 Virtual Cores,
  Stay with something like

  4 Core Per Sockets and 2 Virtual Cores, etc?
Question by:sjniii
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
LVL 121

Accepted Solution

Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 41882729
The best method is to let VMware vSphere decide and accept the defaults.

e.g. sockets.

So this is 2 sockets x 1 core = 2 vCPU (2 CPUs in the VM)

always use sockets, unless you have a specific reason to use cores.

Recommended Practices

#1 When creating a virtual machine, by default, vSphere will create as many virtual sockets as you’ve requested vCPUs and the cores per socket is equal to one. I think of this configuration as “wide” and “flat.” This will enable vNUMA to select and present the best virtual NUMA topology to the guest operating system, which will be optimal on the underlying physical topology.

#2 When you must change the cores per socket though, commonly due to licensing constraints, ensure you mirror physical server’s NUMA topology. This is because when a virtual machine is no longer configured by default as “wide” and “flat,” vNUMA will not automatically pick the best NUMA configuration based on the physical server, but will instead honor your configuration – right or wrong – potentially leading to a topology mismatch that does affect performance.


Does corespersocket Affect Performance?

vSMP (virtual SMP) can affect virtual machine performance, when adding too many vCPUs to virtual machines that cannot use the vCPUs effectly, e.g. Servers than can use vSMP correctly :- SQL Server, Exchange Server.

This is true, many VMware Administrators, think adding lots of processors, will increase performance - wrong! (and because they can, they just go silly!). Sometimes there is confusion between cores and processors. But what we are adding is additional processors in the virtual machine.

So 4 vCPU, to the VM is a 4 Way SMP (Quad Processor Server), if you have Enterprise Plus license you can add 8, (and only if you have the correct OS License will the OS recognise them all).

If applications, can take advantage e.g. Exchange, SQL, adding additional processors, can/may increase performance.

So usual rule of thumb is try 1 vCPU, then try 2 vCPU, knock back to 1 vCPU if performance is affected. and only use vSMP if the VM can take advantage.

Example, VM with 4 vCPUs allocated!

My simple laymans explaination of the "scheduler!"

As you have assigned 4 vCPUs, to this VM, the VMware scheulder, has to wait until 4 cores are free and available, to do this, it has to pause the first cores, until the 4th is available, during this timeframe, the paused cores are not available for processes, this is my simplistic view, but bottom line is adding more vCPUs to a VM, may not give you the performance benefits you think, unless the VM, it's applications are optimised for additional vCPUs.

See here

see here

also there is a document here about the CPU scheduler

Assisted Solution

sjniii earned 0 total points
ID: 41886755
We have a vSphere Enterprise Plus license.

Per the customer request, we have to run their code on VMs with different CPU settings.

in the example, VM with 4 vCPUS allocated, it does not show what settings were used to get the 4 vCPUS (Number of virtual sockets 4 + Number of cores per socket 1, Number of virtual sockets 2 + Number of cores per socket 2, or Number of virtual sockets 1 + Number of cores per socket 4,?)

We are focusing on the Centos 64-bit VMs as this is where the code is running and need to test 4, 8, 16 CPUs.

It sounds as though we should set the  3 VMs and let vSphere decide
as in
Number of Virtual Sockets number of virtual sockets 4, 8, 16 and leave  Number of cores per socket set t 1 = vNUMA wide and flat?
LVL 121
ID: 41886830
That's correct.

Or, you will have to do the configuration, and work it out yourself, if you manually allocate VMs per core, which defeats the money you've paid for the hypervisor.

Or build a cluster of Centos Machines using a HPC, using bare metal and no hypervisors.

Author Closing Comment

ID: 41887827
Thanks for the help.

Featured Post

[Live Webinar] The Cloud Skills Gap

As Cloud technologies come of age, business leaders grapple with the impact it has on their team's skills and the gap associated with the use of a cloud platform.

Join experts from 451 Research and Concerto Cloud Services on July 27th where we will examine fact and fiction.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this step by step tutorial with screenshots, we will show you HOW TO: Enable SSH Remote Access on a VMware vSphere Hypervisor 6.5 (ESXi 6.5). This is important if you need to enable SSH remote access for additional troubleshooting of the ESXi hos…
Ransomware is a malware that is again in the list of security  concerns. Not only for companies, but also for Government security and  even at personal use. IT departments should be aware and have the right  knowledge to how to fight it.
Teach the user how to use create log bundles for vCenter Server or ESXi hosts Open vSphere Web Client: Generate vCenter Server and ESXi host log bundle:  Open vCenter Server Appliance Web Management interface and generate log bundle: Open vCenter Se…
This video shows you how easy it is to boot from ISO images for virtual machines with the ISO images stored on a local datastore on the ESXi host.

626 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question