Scott Drummonds on Virtualization

Hyper-Threading on vSphere


I continue to receive many questions from our customers on the expected performance gains of the new version of Hyper-Threading in Intel’s Core i7 processors. The answer requires a little bit of discussion on Hyper-Threading, a little bit on ESX, and comes with some performance data. If you are still interested, read on.

On VI3, many of VMware’s customers disabled Hyper-Threading on their older, Netburst architecture Intel processors. Intel has vaguely described the new Hyper-Threading as more efficient than the previous generation and I believe this to be due to a shorter pipeline and an improved ability to context switch pipeline stage data. Long pipelines–such as the Netburst era Xeons of model numbers x1xx and x2xx–are more likely to suffer bubbles during context switches and are therefore penalized versus shorter pipeline products, such as the Core i7. Furthermore, by pushing and restoring pipeline stage data during a hardware context switch, the new HT can reduce pipeline bubbles.

But the gains vSphere users experience as a result of the new Hyper-Threading also comes from changes in ESX. ESX’s scheduler must make decisions as to when to co-locate two worlds on a physical core to take advantage of Hyper-Threading. In some conditions the scheduler will perform this co-location and in others it will allow a world to run on the core by itself. The decision to execute worlds concurrently instead of serially on a physical core can be informally called the scheduler’s trust of Hyper-Threading. The vSphere scheduler trusts Hyper-Threading more than the VI3 scheduler did. This amplifies the effect of HT.

I am now going to bore you with a disclaimer before I give you any data showing the effect of Hyper-Threading. The value of HT will vary from workload to workload and the ultimate authority of HT’s value is the end-user. The following numbers are the result of informal analysis and VMware that should only be used as a guide in your own analysis. Please do not make purchasing decisions on this information, which is devoid of the detail we would normally commit to a white paper.

Workload Observed Throughput Gain Due to HT
VMmark 24%
SPECjbb 10%
Consolidated SQL 19%

In addition to the gains we informally cite here, I can say that we have not yet seen a workload where the new Hyper-Threading slows down consolidated performance. As far as we can tell, the new Hyper-Threading should be left enabled in 100% of virtualized environments.

11 Responses

On the topic of HT:
I guess you are aware of it, but in the recently published report of Project VRC, comparing ESX, HyperV and XenServer for TS workloads, ESX4 leads by about 5% without HT. However, with HT enabled it’s about 15% slower then the other 2, which puzzled me quite bit.

It would be nice to get a comment on the whole thing from you or other VMware folks.

    • orz,

      Yeah, I am aware of that work. We researched the reason for the disappointing results and discovered something interesting about our scheduler. I wrote up a summary that was distributed internally. I think that I will share those comments on this blog.

      The interesting thing about those VRC results is that ESX did not benefit from HT the way it should. HT did not slow things down but nor did it provide value (on vSphere).

      More to come…


    • Thanks for the follow up, Scott. Great to hear you guys are already on it, wouldn’t have expected any less.

      Now I just hope your findings on this and more general performance insights on here are soon to be publicized, in the awesome fashion as usual.

  • […] Unlike the old E5430 hosts, hyper threading is possible on the x5550 hosts, and according to VMWare’s documentation, is recommended.  Whether it actually improves performance is subject to some debate, as found here. […]

  • […] intel, scheduler, terminal services, vsphere — Scott @ 3:23 pm I recently wrote a blog article detailing Hyper-Threading (HT) and its effect on vSphere.  An astute reader pointed out, a recent update to Project VRC’s terminal services analysis […]

  • Why should it recommended that HT be enabled? Should you do any FT then HT should be switched off in any case. Although it is always nice to see plenty of CPU’s (with HT), it might just cause obstacles in future

  • […] of my favorite bloggers and performance guru Scott Drummonds has posted some info on Hyper-Threading and […]