SHIFT

--- Sjoerd Hooft's InFormation Technology ---

User Tools

Site Tools


Sidebar

Recently Changed Pages:

View All Pages


View All Tags


LinkedIn




WIKI Disclaimer: As with most other things on the Internet, the content on this wiki is not supported. It was contributed by me and is published “as is”. It has worked for me, and might work for you.
Also note that any view or statement expressed anywhere on this site are strictly mine and not the opinions or views of my employer.


Pages with comments

View All Comments

vmwarecpumgmt

Vmware CPU Management

On an ESX server run multiple virtual machines which are all competing for the same resources. The first resources that are obvious to the end user if underperforming are CPU and memory. In two articles I will explain a little bit about what lies beneath, and then how to find out what is causing the decreased performance. Of course, you should always keep in mind that performance issues are subjective to the end users view. Some users will complain about their systems being slow just on a feeling that “it was faster yesterday”, and they have no benchmark to prove it. Sometimes how ever they can show you that the number of processed transactions in their database is down, or throughput on their file or web server is down.

CPU Scheduling

A VM can be configured with up to 8 vCPUs. The VMkernel includes a VPU scheduler that dynamically schedules vCPUs on the physical processor of the host system. These physical processors can be defined as a logical processor when multiple cores are involved, each logical processor represents a core. Then, when a vCPU or a single vCPU or multiple vCPU VM must be scheduled, the VMkernel maps the vCPU to an available logical processor. Note however, when working with VMs with multiple vCPUs (which is calles a SMP/Symmetric MultiProcessing VM ), if you do not have enough logical processors available this will cause congestion of CPU resources. This is because a guest OS with multiple vCPUs assumes these CPUs to be physical and evenly fast. That means that if it uses one, it assumes it can use both, which is called co-processing. Without coscheduling, the vCPUs associated with a SMP VM would be scheduled independently, breaking the guest's assumptions regarding uniform progress. So consider this example, an ESX Server with two physical cores running one dual-vCPU VM and one single-vCPU VM. When the single-vCPU VM is running, the scheduler cannot use the remaining physical core to run just one of the SMP VM's two vCPUs. As you can imagine, this will cause congestion and high CPU ready times on the dual-vCPU VM. For more information about this see the resources section at the end of this article.

CPU Graphs for a VM

Perform the following steps to create a CPU graph that shows data for a virtual machine from the last hour:

  1. Inside vCenter select a virtual machine
  2. Select the Performance tab and change the view to “Advanced”
  3. Click the “Chart Options” link.
  4. Select CPU → Custom interval
  5. Below the Chart Options you can set the interval to “Last 1 Hours”
  6. Set the “Chart Type” to “Line graph”
  7. Select the virtual machine in the Objects Section
  8. From the list of counters, select “CPU Usage In MHz (Average)” and “CPU Ready”. See CPU Counters for what this means
  9. Click OK to see the chart.

CPU Graph Examples

Quiet VM

This is an example of a CPU Graph for an virtual machine that is quiet:
vmwarecpumgmt-quietvm.jpg

Busy VM

This is an example of a CPU Graph for an virtual machine that is busy and has a (really) long CPU Ready time:
vmwarecpumgmt-busyvm.jpg

CPU Graphs for an ESX Host

Perform the following steps to create a real-time graph for a host's CPU usage:

  1. Inside vCenter select a host
  2. Select the Performance tab and change the view to “Advanced”
  3. Click the “Chart Options” link.
  4. Select CPU → Real-time
  5. Set the “Chart Type” to “Stacked Graph (Per VM)”
  6. Select all objects. There should be a separate object for each virtual machine on the ESX host.
  7. Select the “CPU Usage (Average)” counter.
  8. Click OK to see the chart.

This chart shows the usage of all the virtual machines on the selected ESX host in a stacked fashion. It allows for a clear view to see whether one VM uses an abnormal percentage of resources.

CPU Graph Examples

Quiet ESX Host

This is an example of a CPU Graph for an ESX host is quiet:
vmwarecpumgmt-quiethost.jpg

Busy ESX Host - Peaks

This is an example of a CPU Graph for an ESX host that is busy and has trouble handling peaks:
vmwarecpumgmt-busyhost01.jpg

Busy ESX Host - Busy VM

This is an example of a CPU Graph for an ESX host that is busy because of one virtual machine that demands too much CPU. Note that this is an percentage of the CPU resources assigned to the virtual machine:
vmwarecpumgmt-busyhost02.jpg

Busy ESX Host - Busy VM - NOT

This is an example of a CPU Graph for the same ESX host as before with the same virtual machine highlighted, but now with the counter “CPU Usage in Mhz”. As you can see, this machine is not as demanding as it seems:
vmwarecpumgmt-busyhost03.jpg

As you can see from the previous example you can't jump to conclusions too fast. The virtual machine in question is assigned one virtual cpu which it claims almost, but not completely. You should check with end-users how the virtual performs, and you could check whether the virtual machine performs better with for example 2 virtual CPUs.

CPU Counters

CPU Ready

Percentage of time that the virtual machine was ready, but could not get scheduled to run on the physical CPU
If this value is high as in more than thousand milliseconds it might indicate that the ESX host is overloaded or it's resource pool or the virtual machine itself does not have enough shares. Some CPU Ready time is not a problem though, on physical machines there is also a wait time for CPU resources.

CPU Usage in MHz

CPU usage, as measured in megahertz, during the interval.
This one speaks for itself.

Resources

CPU Scheduler (pdf)
Co-scheduling SMP VMs in ESX

This story continues with VMWare Memory Management which handles memory performance issues.

You could leave a comment if you were logged in.
vmwarecpumgmt.txt · Last modified: 2021/09/24 00:25 (external edit)