vPivot

Scott Drummonds on Virtualization

Server Flash + Array Flash

3 Comments »

With the recent general availability of VFCache, EMC has buzzed with ideas about what to do with server-based solid state storage.  Server-based solid state has been around for years.  I remember when Fusion-io visited us at VMware in 2009.  I spent a lot of time thinking about use cases, value, costs, and features. Now at EMC I am asking myself even bigger questions: how far can we go with this technology?  How much can we federate it, migrate data among nodes and within shared storage, protect it, and replicate it?  There are a lot of smart people in EMC that are way ahead of me on this.

But for the time being, the world is using server cache to speed up applications while living with mobility limitations.  Because of my performance background I am still a speed junky.  My long time in that field makes me a bit of a cynic, too.  When I saw a version of the following chart used in an internal EMC presentation I was skeptical.  Take at look at this and ask yourself if you believe it.

The benchmark is an Oracle OLTP workload.  VFCache is EMC’s PCI card that provides solid state cache at the server.  FAST Cache is EMC’s array-based solid state read/write cache.  At the request of EMC engineering I am only showing relative versions of IOPS measured from the application. This graph says that VFCache and FAST Cache provide multiplicative benefits to performance.  By multiplicative benefit I mean that when VFCache accelerates baseline performance by 30% it can also improve FAST Cache performance again by 30%.

How could this be?  I originally reasoned that the value of VFCache should be additive, not multiplicative.  I thought it would provide a total speedup to some fixed portion of the data.  A large array cache would speed up the rest of the data. In other words, VFCache speeds up 10% of the data a lot and FAST Cache speeds up 90% of the data to a lesser degree. The aggregate gain is the summary of the two, right?

Wrong.

VFCache can provide a multiplicative affect on performance. This means VFCache improves FAST Cache utilization and vice versa.  Here is why: by speeding up the rest of the data you are decreasing the application wait time.  When wait times go down, efficiency goes up.  When efficiency goes up, the utilization to each of these capabilities increases. We saw this efficiency gain when we saw VFCache throughput increase once FAST Cache at the array was enabled.

Because of the incredible performance benefits server cache provides, many customers are willing to live with the mobility limitations it imposes. Because of those mobility limitations we can really only think of today’s server cache as a precursor to the revolutionary technology we are waiting for. Eventually we will be able to federate the data on server cache, protect it, and promote and demote blocks between server and array. Eventually flash will be thought of as a true catalyst for change. But even today in its current form you can enjoy some screaming fast performance when you combine server cache and array cache.

Update: A Happy Addendum

After seeing my post one of the talented folks involved in this analysis sent me an email. He gently pointed out that the numbers I quote above are IOPS, as I wrote. These are indirect numbers for performance. The real application performance numbers, transactions per second, showed an even more dramatic benefit of the FAST Cache and VFCache configuration. The total application performance improvement for for these two technologies was 10x for this particular workload. So, take everything I said above, and multiply it again for this configuration.

3 Responses

[…] combine EMC’s array-based cache (FAST Cache) with EMC’s server-based cache (VFCache)? Read to find out, and be sure to note the addendum, which makes the story even […]

  • Have you seen Nutanix? It’s got the replicated and protected bit nailed. But I hear EMC has something to compete with it as well, which is good to hear.

    • From my perspective, I would not consider VFCache something to compete with Nutanix. EMC still believes that a centralized, shared storage device has value. That’s the opposite of Nutanix’s position. If customers want shared storage, EMC will help them federate flash strewn throughout their datacenter. If customers do not want shared storage, they will be talking to vendors like Nutanix. Whether or not flash is present is, I think, a small part of the equation.

      Or do you think I have read this wrong?