Scott Drummonds on Virtualization

Simpler, Better Chargeback


In a recent discussion with VKernel’s Bryan Semple I argued for a better chargeback tool.  I think that VKernel, VMware, and anyone in the virtualization business can improve the chargeback process through expanded automation and inclusion of capacity analysis metrics.  This would simply chargeback, increase its accuracy, and reduce its maintenance. Chargeback would become so simple and valuable that customers might actually start doing it.  Doubt me?  Read on.

Bryan sees among VKernel’s customers the same lethargic, half support of chargeback that I see across the industry.  But we both agree that chargeback–or at least showback–is critical to datacenter efficiency.  By layering cost on top of capacity, chargeback is the ultimate capacity optimization that creates lean datacenters.  But very few companies implement chargeback in their virtual environments.  I want to show you why customers hate chargeback and how enterprise software companies can easily fix the problem.

First, let’s take a look at the process today’s customers use to produce invoices and cost sheets for their lines of business (LOB).

Limited Automation of Current Chargeback

In the figure above I have documented with my world famous PowerPoint skills today’s partially automated chargeback process. Many customers lack the experience to generate per-VM costs by translating IT costs into RAM, CPU, and GB charges.  Once they go through that tedious exercise, changes to capacity or IT costs (both capital and operational) require an update of the VM price sheet.  This again requires difficult calculations and estimations that unsettle IT and their internal customers.

I argue that this process pain is why you, VMware’s customers, do not want to deal with chargeback or showback.  Now can I tell you how easy this process could be for you?  You need not do most of these steps. The only thing you should have to do is collect your IT costs.  A good chargeback tool, with the benefit of capacity information, can automate everything else.

Complete chargeback automation requires the customer only collect IT costs

This new flow automates the process of creating a per-VM cost sheet. This means customers do not need to manually determine the costs of memory, CPU, storage, or network connectivity. This is done by using IT costs and properly mapping them to current or expected capacity utilization. The only unavoidable manual step is collecting IT costs.

Producing a per-VM cost sheet from these costs is well understood by experts so any software vendor could implement it.  The software would use a wizard to interrogate you and collect vital cost numbers and physical connectivity characteristics. The calculation produces a cost for an atom of infrastructure sizing called a slice.  Using capacity information, the tool then maps slices to virtual machines to produce a VM price sheet.  VMware, VKernel, and others already have capacity information in their products.

But benefits of this improved automation improve with time.  Good capacity tools will predict trends to help you schedule future price sheet changes.  Those predictive capabilities can alert to future loss or profit.  This trend analysis will notify you in advance that pricing changes will soon be needed.

I do not think my ideas here are novel or represent intellectual property that only one vendor could implement.  I think that all admins of virtualized datacenters will benefit if any ISV increases its chargeback automation.  I bet the first vendor to implement this better chargeback will have a little bit easier time selling it.

4 Responses

Scott- thinking more about this, I am actually not sure if using “real costs” is the right approach. I reserve the right to change my mind over the coming weeks, but here goes some of my thinking.

Clearly, the chargeback amount needs to be related to the actual costs incurred – agreed. But, the danger here is that an organization creates a cost model that doesn’t reflect the entire value of the service they provide. A private cloud has some value to an organization that an Amazon won’t have in terms of security, compliance etc. So if the private cloud infrastructure costs $10/VM and Amazon is $5, this would lead end users to conclude that IT is ripping them off.

In addition, you raised this point during our chat — the pricing will vary based on the vacancy rate of the infrastructure. The lower the VM vacancy rate, the more you can spread the costs across more operating VMs. The higher the vacancy rate, the more expensive the VMs. Can an private cloud really operate with low vacancy rates?

Finally, I will call this last point, the greater good theory. Since private clouds just transfer costs internal to a company, the real goal here needs to be to lower overall IT costs for the company. If I request overallocated VMs, and I “pay for them”, I may not care if IT calls me up and says I could save $300/month, for example, by reducing my memory allocation. In the big scheme of my marketing departments budget, that is noise.

But from an IT perspective, if every department is oversubscribed, that is a lot of money for the company. The greater good of the company would be served if IT just reallocated my memory and told me to deal with it vs. trying to negotiate. Once you “pay” for something, you feel you have a right to it.

More on this —– I am on a panel in July and am trying to collect more thoughts on how people are doing this. Also just published a revised private cloud paper that contains many of these thoughts:


    • Great stuff as always, Bryan. I agree with your first comment about the intangible benefits of private clouds. But I’m not sure if that discussion belongs in a Chargeback tool or process. It seems to me that somewhere way up the people management stack, someone says, “I am prepared to pay 10% or 20% more to have my applications in-house for my own piece of mind.” Chargeback works the same regardless of add-on.

      I love your “greater good” idea. This is an even stronger reason to tie capacity measurements to chargeback. VKernel could, for instance, calculate an “efficiency” number that could be reported in each monthly show/bill cycle. That metric would identify that although business unit #1 paid $100K, it only consumed 50% of what it paid for. That could be a powerful force for greater business efficiency.

      Good luck on the panel!

  • Chargeback is basically the breakdown of Capex into Opex – and utility companies have been doing this with their billing software for years.

    Mainframes have had chargeback for years.

    Chargeback and Capacity are the two disciplines that have to work together and if a product can tie in these two disciplines – they will lead the market.

    Before you can do Chargeback – you have to have good resource accounting – and tha’ts lacking – even with all these system management tools, its hard to agree on a resource consumption model.

    Take memory metrics – if you charge $10 per month per Gigabyte – how do you do it? Memory Allocated? Memory Consumed? Memory Active?

    If you do it based on allocation – you can charge out at the beginning of each month – simple accounting. Someone wants a VM – its 4 GB – its $40 per month.

    If you do it based on usage – then you need hourly or daily reporting, etc. So it could range $1-$40 per hour? per day? and what happens with Java or Linux VMs where extra memory is used for cache or heap.

    I think the majority of the public cloud vendors run with an allocation model – you ask for 8 GB and you use 1 GB – you pay for 8 GB, if you want to pay less – you need to rightsize your VM.

    So its a mix of “Resource Accounting” then “Chargeback” then “Rightsizing” and Capacity (Replentishment) (when do I add physical hosts to my cloud, when do I increase RAM or CPU Cores per host.

    BMS Computer, which was founded in 1974 and which created resource accounting and chargeback software for MVS and VSE operating systems on mainframes. In the 1980s, the company added chargeback functions for DB2 and CICS and, as the decade ended, expanded into Unix systems. In the early 1990s, the company announced a desktop resource accounting and chargeback program called CIMS Desktop, and in 1996, Platinum Technology (which was later eaten by Computer Associates) acquired CIMS Lab, but divested itself of the company in 1999. The dot-com crash and slowdown in the IT business actually helped make CIMS Lab’s products more relevant, as has the ideas of grid and utility computing. In 2002, the company rolled out a Web-based version of its tools called CIMS Server and delivered support for Microsoft’s Windows platform; in 2004 the company delivered an extended product line that modularized the offering into agents and various modules for measuring, analyzing, reporting, and billing for IT usage.

    Tivoli / IBM owns them today – and all the big vendors (CA, BMC (recently acquired Neptuny)) have chargeback solutions – but they lack resource account models, the hypervisors lack rightsizing features and other than VCE’s vBlock which will SKU up the Compute/Network/Storage as one SKU for replentishment – adding a host to a Cloud is still the same traditional crapshoot that its been.

    Until we fix that – its nothing new.

    • Hi, Rob, thanks for the comments and background. I think you have touched on something in your mention of allocation/metered chargeback methods. I believe that most customers have little idea where to start. This is where I look to VMware and its partners such as all the ones you and I mentioned to take control of the discussion.

      I think we are providing too much flexibility in chargeback tools and asking VMware admins to do too much modeling. My idea of taking infrastructure costs as an input suggest that the only thing the customer decide is the kind of model. The the tool spits out the costs.

      Today, I find that 99% of customers roll their eyes when you talk about chargeback. Everyone wants to do it, but no one wants to assign costs to RAM or CPU for the reasons you have mentioned. Companies with experience doing this (VMware’s partner community) have an opportunity to say, “we understand this, trust us”. I think they should.