Solved

That NUMA and vNUMA question

Posted on 2016-11-10
4
43 Views
Last Modified: 2016-11-15
This question is based on settings to for NUMA and vNUMA and what is the best way to set Cores Per Sockets and Virtual Sockets in the VM settings.

So, when enabled vNUMA exposes the VM OS to the physical NUMA.
But is it automatically set?

By default, if I recall, VMware recommends setting Core Per socket to 1 and changing Virtual Sockets thus vNUMA will be enabled to select the best NUMA configuration to be used?

So, with that said, we have a requirement to test customer software on systems with multiple CPUs and various amounts or RAM.

Our ESXi host has 2 cpu sockets, 8 cores, and 32 Logical processors with 512 GB of RAM.
I logged onto the ESXi host and see that there are 2 NUMA Nodes available.

VM requirements :

Win7 Vm with 2 vCPUs
Win7 Vm with 4 vCPUs
Win7 Vm with 8 vCPUs
Centos Server with 4 vCPUs
Centos Server with 8 vCPUs
Centos Server with 16 vCPUs

One test VM that is Centos Server, we initially have set as 4 Core Per Sockets and 2 Virtual Sockets giving 8 vCPUS.. If I understood the above info I stated, this would not enable vNUMA? Is this still a valid set up for vCPUs?

So, should we set the VMs vCPUS as:
  1 Core Per Socket and 8 Virtual Cores,
  1 Core Per Socket and 16 Virtual Cores,
or
  Stay with something like

  4 Core Per Sockets and 2 Virtual Cores, etc?
0
Comment
Question by:sjniii
  • 2
  • 2
4 Comments
 
LVL 117

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE) earned 500 total points
ID: 41882729
The best method is to let VMware vSphere decide and accept the defaults.

e.g. sockets.

So this is 2 sockets x 1 core = 2 vCPU (2 CPUs in the VM)

always use sockets, unless you have a specific reason to use cores.

Recommended Practices

#1 When creating a virtual machine, by default, vSphere will create as many virtual sockets as you’ve requested vCPUs and the cores per socket is equal to one. I think of this configuration as “wide” and “flat.” This will enable vNUMA to select and present the best virtual NUMA topology to the guest operating system, which will be optimal on the underlying physical topology.

#2 When you must change the cores per socket though, commonly due to licensing constraints, ensure you mirror physical server’s NUMA topology. This is because when a virtual machine is no longer configured by default as “wide” and “flat,” vNUMA will not automatically pick the best NUMA configuration based on the physical server, but will instead honor your configuration – right or wrong – potentially leading to a topology mismatch that does affect performance.

Source

Does corespersocket Affect Performance?

vSMP (virtual SMP) can affect virtual machine performance, when adding too many vCPUs to virtual machines that cannot use the vCPUs effectly, e.g. Servers than can use vSMP correctly :- SQL Server, Exchange Server.

This is true, many VMware Administrators, think adding lots of processors, will increase performance - wrong! (and because they can, they just go silly!). Sometimes there is confusion between cores and processors. But what we are adding is additional processors in the virtual machine.

So 4 vCPU, to the VM is a 4 Way SMP (Quad Processor Server), if you have Enterprise Plus license you can add 8, (and only if you have the correct OS License will the OS recognise them all).

If applications, can take advantage e.g. Exchange, SQL, adding additional processors, can/may increase performance.

So usual rule of thumb is try 1 vCPU, then try 2 vCPU, knock back to 1 vCPU if performance is affected. and only use vSMP if the VM can take advantage.

Example, VM with 4 vCPUs allocated!

My simple laymans explaination of the "scheduler!"

As you have assigned 4 vCPUs, to this VM, the VMware scheulder, has to wait until 4 cores are free and available, to do this, it has to pause the first cores, until the 4th is available, during this timeframe, the paused cores are not available for processes, this is my simplistic view, but bottom line is adding more vCPUs to a VM, may not give you the performance benefits you think, unless the VM, it's applications are optimised for additional vCPUs.

See here
http://www.vmware.com/resources/techresources/10131

see here
http://www.gabesvirtualworld.com/how-too-many-vcpus-can-negatively-affect-your-performance/

http://www.zdnet.com/virtual-cpus-the-overprovisioning-penalty-of-vcpu-to-pcpu-ratios-4010025185/

also there is a document here about the CPU scheduler

www.vmware.com/files/pdf/perf-vsphere-cpu_scheduler.pdf
1
 

Assisted Solution

by:sjniii
sjniii earned 0 total points
ID: 41886755
We have a vSphere Enterprise Plus license.

Per the customer request, we have to run their code on VMs with different CPU settings.

in the example, VM with 4 vCPUS allocated, it does not show what settings were used to get the 4 vCPUS (Number of virtual sockets 4 + Number of cores per socket 1, Number of virtual sockets 2 + Number of cores per socket 2, or Number of virtual sockets 1 + Number of cores per socket 4,?)

We are focusing on the Centos 64-bit VMs as this is where the code is running and need to test 4, 8, 16 CPUs.

It sounds as though we should set the  3 VMs and let vSphere decide
as in
Number of Virtual Sockets number of virtual sockets 4, 8, 16 and leave  Number of cores per socket set t 1 = vNUMA wide and flat?
0
 
LVL 117
ID: 41886830
That's correct.

Or, you will have to do the configuration, and work it out yourself, if you manually allocate VMs per core, which defeats the money you've paid for the hypervisor.

Or build a cluster of Centos Machines using a HPC, using bare metal and no hypervisors.
0
 

Author Closing Comment

by:sjniii
ID: 41887827
Thanks for the help.
0

Featured Post

Complete VMware vSphere® ESX(i) & Hyper-V Backup

Capture your entire system, including the host, with patented disk imaging integrated with VMware VADP / Microsoft VSS and RCT. RTOs is as low as 15 seconds with Acronis Active Restore™. You can enjoy unlimited P2V/V2V migrations from any source (even from a different hypervisor)

Join & Write a Comment

Last article we focus in how to VMware: How to create and use VMs TAGs – Part 1 so before follow this article and perform the next tasks, you should read the first article how to create the TAG before using them in Veeam Backup Jobs.
HOW TO: Install and Configure VMware vSphere Hypervisor 6.5 (ESXi 6.5), Step by Step Tutorial with screenshots. From Download, Checking Media, to Completed Installation.
Teach the user how to rename, unmount, delete and upgrade VMFS datastores. Open vSphere Web Client: Rename VMFS and NFS datastores: Upgrade VMFS-3 volume to VMFS-5: Unmount VMFS datastore: Delete a VMFS datastore:
Teach the user how to install and configure the vCenter Orchestrator virtual appliance Open vSphere Web Client: Deploy vCenter Orchestrator virtual appliance OVA file: Verify vCenter Orchestrator virtual appliance boots successfully: Connect to the …

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

27 Experts available now in Live!

Get 1:1 Help Now