A little Friday esxtop trivia for the performance massive: did you ever notice your Hyper-Threaded systems have three rows showing CPU utilization in the CPU panel header? They are labeled “PCPU USED(%)”, “PCPU UTIL(%)”, and “CORE UTIL(%)”. Here is a screen shot to jog your memory:
esxtop shows three processor utilization rows. What do they mean?
This capture shows utilization of each physical and logical processor core in the system. The first row, PCPU USED, provides the percent of each physical core used by the logical core, multiplied by turbo mode, a processor feature that temporarily increases the core’s internal clock frequency. This means two threads running at full, turbo speed might produce a number like 55% for each entry. It also means that one thread can drive its logical CPU to 100% only if the logical core’s sibling is unused. The second row is a straightforward calculation of the utilization of each logical core and the third row similarly shows utilization of physical cores.
Where things get really confusing is when these results are combined into three system-wide, aggregate utilization numbers, as seen by esxtop’s batch printout. The three utilization types above generated different utilization numbers. Unfortunately esxtop’s batch mode labels these counters slightly differently. But this table includes both names:
| esxtop Interactive | esxtop Batch Output (and esxplot) | Description | Single Core Example |
|---|---|---|---|
| PCPU USED(%) | % Processor Time | The average of each hardware thread’s use of the physical core multiplied by turbo mode. | One thread running fully: 108%, two threads running fully: 50% |
| PCPU UTIL(%) | % Util Time | Percent utilization of logical cores. | One thread running fully: 50%, two threads running fully: 100% |
| CORE UTIL(%) | % Core Util Time | Utilization of the physical core. | One thread running fully: 100%, two threads running fully: 100% |
The “Single Core Example” column provides an example calculation based on threads running as fast as possible on a single Hyper-Threaded physical core. There are some interesting observations on these calculations:
- Even a great number of threads running full bore on an HT system will not produce a PCPU USED(%) number much over 50%.
- Two running threads will produce a lower PCPU USED(%) than one running thread. This is because each thread’s utilization is calculated against the physical core. With two running, each is averaging 50% of the core. But a single thread that is not sharing the core can drive this to 100%. In both cases the actual number could be a little higher if turbo mode is on.
- You must have at least two threads–one on each logical core–to drive PCPU UTIL(%) to 100%.
- CORE UTIL(%) can be driven to 100% with only one thread per physical core.
Look for this content to be rolled into our esxtop documentation soon.
Excellent post Scott!! I have recently been using resxtop to closely monitor a recent terminal server migration I performed to a Nehalem host.
Also, any comments on the statements made here by AMD referencing their opinions on HT and Nehalem/Westmere on vSphere vs True Cores?
http://blogs.amd.com/work/2010/03/30/intel-hyper-threading-vs-amd-true-core-scalability/
Jonathan,
If my summary of that article is correct, AMD is saying nothing more than “physical cores are better than logical cores”. I can find no reason to disagree with that sentiment. Seeing as Magny Cours has the highest core density per package, it is logical that AMD wants to tout their advantage with blogs like this.
Scott
[...] Drummonds – Processor Utilization CalculationsWhere things get really confusing is when these results are combined into three system-wide, [...]
[...] vPivot [...]