vPivot

Scott Drummonds on Virtualization

Performance Survey from VMworld (US)

1 Comment »

The VMworld events in both San Francisco and Copenhagen included a new discussion forum where subject matter experts spent time interacting with a small group of similarly-minded enthusiasts.  VMware was kind enough to invite me to host a performance discussion and the interactions I had with each group were fantastic.  At the organizers’ request, I seeded the discussions with an interactive survey that touched off deep conversation and engaging debate.  The results were collected by local staff and just recently I received those from the San Francisco VMworld’s group discussion on performance.

I asked about nine questions of the forum’s attendees and four of them generated rich discussion.  There were about 25 people in attendance, which were self-selected to have both opinions and total comfort sharing them.  Here are the questions, the results, and my commentary.

Question: What do you think is the state of performance of VMware vSphere 4.1?

Answer options:

  1. I am not concerned about performance in any of my virtualized applications
  2. I am concerned about performance in a very small number (<2%) of my applications
  3. I do not think that vSphere can perform well enough for my big databases
  4. I virtualize IT applications, but fear the virtual performance of my lines of business applications
  5. I fear performance in general, and only virtualize after rigorous testing

The results:

My perspective on VMware performance is that it is sufficient to virtualize all but about 0.1% of application instances in today’s enterprise since vSphere 4, nevertheless vSphere 4.1.  This means a theoretical penetration of 99.9% for VMware’s hypervisor.  It is clear from these results that VMware’s customers do not yet believe VMware is so close to ubiquitous virtualization.  But often when I dig into performance objections the root of concern is dated results on older hardware and experiences on previous versions of VMware’s products.  My mantra to those of you that remain skeptical of what VMware can do with your big applications is “give it another try”.

Question: How do you feel VMware vSphere 4.1 performance compares against the competition?

Answer options:

  1. vSphere 4.1 is generally much better than competing hypervisors
  2. vSphere 4.1 is sometimes better and other times similarly performing to competing hypervisors
  3. vSphere 4.1 performance is equivalent to other hypervisors, sometimes winning and sometimes losing
  4. vSphere 4.1 is consistently slower than at least one other hypervisor
  5. Who cares?  I am not making a purchasing decision based on performance anyway

The results:

A few times in my years at VMware I directly engaged in competitive debate on the relative merits of VMware’s products.  But my position on the need of direct competitive comparison has mellowed recently.  I much prefer to put my company’s products in their best light and let our customers decide if its merits outweigh those of our competitors.  But I am always fascinated by the public perception of our work, even when evaluated at a conference largely composed of enthusiasts.

The large portion of respondents that believe vSphere in a dominant position with respect to performance will extol the incredible pace of innovation coming from Palo Alto.  And the sizable portion of attendees that consider most hypervisors at performance parity with each other will surely site all products’ heavy reliance on hardware assist.  I believe that both of these perspectives are true.  When ESX is “dumbed down” to be feature equivalent to another product (by doing something like removing the balloon driver or turning off memory compression) it is possible for a less mature product to deliver comparable performance.  But it is because of the barrage of features included in each launch that VMware can generate more work per server than its competition.

Question: Which of the following represents the most important performance accomplishment in the past three years?

Answer options:

  1. The ESX scheduler’s continual improvement
  2. Improved hardware assist in the CPU
  3. A better vSphere IO stack, including the paravirtualized SCSI device
  4. Paravirtualization of the operating system (including Windows enlightenments)
  5. Other

The results:

Intel and AMD’s dedication to improving the efficiency of virtual computing environments cannot be overstated.  But it is often lost in that discussion that VMware put tremendous engineering effort to making good use of these features, including rebuilding a scheduler to leverage Hyper-threading more aggressively than ever before.

From my perspective the most important improvement in VMware’s history was the storage stack optimizations that came with vSphere 4.  It took those improvements before one could seriously consider running intense, business-critical, IO-bound applications in a virtual environment.  But now you can.  Storage performance is not longer a problem.  Case closed on this, believe me.

Question: What is the coolest performance feature added to vSphere 4.1?

Answer options:

  1. Memory compression
  2. Storage IO control (SIOC)
  3. Network IO control (NetIOC)
  4. Faster vMotion, with more concurrent vMotions
  5. Other

The results:

I was glad to see how popular the new vMotion is in vSphere 4.1.  I was surprised that many people took note of this improvement.  But very glad.

I am not surprised that Storage IO Control (SIOC) figures prominently with customers of new features.  Given that storage causes so many performance problems, VMware clearly has a lot of customers that want help correcting them.

My erstwhile colleague and VMworld co-presenter, Kaushik “K-ban” Banerjee, votes strongly for Network IO Control (NetIOC) as the more important addition to the vSphere 4.1 arsenal.  Kaushik argues that 10 Gb networking is exploding in IT and that application performance management on it is very difficult without bandwidth prioritization.  One attendee of the Copenhagen group discussion agreed with K-ban in principle but claimed that 10 Gb networks were not yet prevalent enough to demand the presence of NetIOC.  If this is so, then we can thank VMware for having the foresight to correct a problem before it is widespread.

I hope to soon receive the results from the VMworld in Copenhagen.  When I do I will certainly post a follow-up blog.  The sample size is relative small as a portion of VMware’s US and EMEA business but it could be fun to draw some generalizations from each region’s responses.

One Response

Scott,

I am interested in taking this survey. Performance is our primary concern at my organization. Perhaps you could add it to your blog similar to the “Chargeback” poll you posted a while back.

Cheers,
Chris.