That NUMA and vNUMA question

Posted on 2016-11-10
Last Modified: 2016-11-15
This question is based on settings to for NUMA and vNUMA and what is the best way to set Cores Per Sockets and Virtual Sockets in the VM settings.

So, when enabled vNUMA exposes the VM OS to the physical NUMA.
But is it automatically set?

By default, if I recall, VMware recommends setting Core Per socket to 1 and changing Virtual Sockets thus vNUMA will be enabled to select the best NUMA configuration to be used?

So, with that said, we have a requirement to test customer software on systems with multiple CPUs and various amounts or RAM.

Our ESXi host has 2 cpu sockets, 8 cores, and 32 Logical processors with 512 GB of RAM.
I logged onto the ESXi host and see that there are 2 NUMA Nodes available.

VM requirements :

Win7 Vm with 2 vCPUs
Win7 Vm with 4 vCPUs
Win7 Vm with 8 vCPUs
Centos Server with 4 vCPUs
Centos Server with 8 vCPUs
Centos Server with 16 vCPUs

One test VM that is Centos Server, we initially have set as 4 Core Per Sockets and 2 Virtual Sockets giving 8 vCPUS.. If I understood the above info I stated, this would not enable vNUMA? Is this still a valid set up for vCPUs?

So, should we set the VMs vCPUS as:
  1 Core Per Socket and 8 Virtual Cores,
  1 Core Per Socket and 16 Virtual Cores,
  Stay with something like

  4 Core Per Sockets and 2 Virtual Cores, etc?
Question by:sjniii
  • 2
  • 2
LVL 118

Accepted Solution

Andrew Hancock (VMware vExpert / EE MVE) earned 500 total points
ID: 41882729
The best method is to let VMware vSphere decide and accept the defaults.

e.g. sockets.

So this is 2 sockets x 1 core = 2 vCPU (2 CPUs in the VM)

always use sockets, unless you have a specific reason to use cores.

Recommended Practices

#1 When creating a virtual machine, by default, vSphere will create as many virtual sockets as you’ve requested vCPUs and the cores per socket is equal to one. I think of this configuration as “wide” and “flat.” This will enable vNUMA to select and present the best virtual NUMA topology to the guest operating system, which will be optimal on the underlying physical topology.

#2 When you must change the cores per socket though, commonly due to licensing constraints, ensure you mirror physical server’s NUMA topology. This is because when a virtual machine is no longer configured by default as “wide” and “flat,” vNUMA will not automatically pick the best NUMA configuration based on the physical server, but will instead honor your configuration – right or wrong – potentially leading to a topology mismatch that does affect performance.


Does corespersocket Affect Performance?

vSMP (virtual SMP) can affect virtual machine performance, when adding too many vCPUs to virtual machines that cannot use the vCPUs effectly, e.g. Servers than can use vSMP correctly :- SQL Server, Exchange Server.

This is true, many VMware Administrators, think adding lots of processors, will increase performance - wrong! (and because they can, they just go silly!). Sometimes there is confusion between cores and processors. But what we are adding is additional processors in the virtual machine.

So 4 vCPU, to the VM is a 4 Way SMP (Quad Processor Server), if you have Enterprise Plus license you can add 8, (and only if you have the correct OS License will the OS recognise them all).

If applications, can take advantage e.g. Exchange, SQL, adding additional processors, can/may increase performance.

So usual rule of thumb is try 1 vCPU, then try 2 vCPU, knock back to 1 vCPU if performance is affected. and only use vSMP if the VM can take advantage.

Example, VM with 4 vCPUs allocated!

My simple laymans explaination of the "scheduler!"

As you have assigned 4 vCPUs, to this VM, the VMware scheulder, has to wait until 4 cores are free and available, to do this, it has to pause the first cores, until the 4th is available, during this timeframe, the paused cores are not available for processes, this is my simplistic view, but bottom line is adding more vCPUs to a VM, may not give you the performance benefits you think, unless the VM, it's applications are optimised for additional vCPUs.

See here

see here

also there is a document here about the CPU scheduler

Assisted Solution

sjniii earned 0 total points
ID: 41886755
We have a vSphere Enterprise Plus license.

Per the customer request, we have to run their code on VMs with different CPU settings.

in the example, VM with 4 vCPUS allocated, it does not show what settings were used to get the 4 vCPUS (Number of virtual sockets 4 + Number of cores per socket 1, Number of virtual sockets 2 + Number of cores per socket 2, or Number of virtual sockets 1 + Number of cores per socket 4,?)

We are focusing on the Centos 64-bit VMs as this is where the code is running and need to test 4, 8, 16 CPUs.

It sounds as though we should set the  3 VMs and let vSphere decide
as in
Number of Virtual Sockets number of virtual sockets 4, 8, 16 and leave  Number of cores per socket set t 1 = vNUMA wide and flat?
LVL 118
ID: 41886830
That's correct.

Or, you will have to do the configuration, and work it out yourself, if you manually allocate VMs per core, which defeats the money you've paid for the hypervisor.

Or build a cluster of Centos Machines using a HPC, using bare metal and no hypervisors.

Author Closing Comment

ID: 41887827
Thanks for the help.

Featured Post

Comprehensive Backup Solutions for Microsoft

Acronis protects the complete Microsoft technology stack: Windows Server, Windows PC, laptop and Surface data; Microsoft business applications; Microsoft Hyper-V; Azure VMs; Microsoft Windows Server 2016; Microsoft Exchange 2016 and SQL Server 2016.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
vmware 4 64
Changing ESXi Path Selection Policy implications and caveats ? 4 68
Understanding VMware NSX 7 30
VMware 6.0 3 26
This is an issue that we can get adding / removing permissions in the vCSA 6.0. We can also have issues searching for users / groups in the AD (using your identify sources). This is how one of the ways to handle this issues and fix it.
In this article, I will show you HOW TO: Suppress Configuration Issues and Warnings Alert displayed in Summary status for ESXi 6.5 after enabling SSH or ESXi Shell.
Teach the user how to convert virtaul disk file formats and how to rename virtual machine files on datastores. Open vSphere Web Client: Review VM disk settings: Migrate VM to new datastore with a thick provisioned (lazy zeroed) disk format: Rename a…
Teach the user how to use configure the vCenter Server storage filters Open vSphere Web Client:  Navigate to vCenter Server Advanced Settings: Add the four vCenter Server storage filters: Review the advanced settings: Modify the values of the four v…

896 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now