vPivot

Scott Drummonds on Virtualization

esxtop Analysis With esxplot

16 Comments »

esxtop remains a popular performance troubleshooting tool because of its fine granularity, expansive counter list, and support for interactive and off-line analysis. The biggest problem with esxtop is the huge CSV files generated in batch mode.  The output is so large that Excel is unable to open the file and Perfmon can take hours to do so. But now we have a better way to manage esxtop batch files.

A colleague of mine at VMware, Geoff White, wrote a Python-based tool for esxtop batch file analysis. His tool, dubbed esxplot, provides a graphical interface to the data generated by esxtop’s batch mode. esxplot provides search capabilities to find relevant counters, can plot different counters on the same graph, and does not suffer from the size limitations of Excel and Perfmon.

Geoff has graciously provided unsupported access to this tool’s beta version (version removed — see update below) for my readers. Please test esxplot much as possible and comment here with your observations. We would like to know of any bugs you discover as well as ideas for improvement.

The esxplot package comes with Windows and Ubuntu binaries as well as the Python source. The binaries will run without need of additional libraries or tools.  But if you decide to use the Python script, as I have done on MacOS, you will need to verify the following:

  1. Ensure that you have Python 2.6 or greater, but not Python 3.x.
  2. esxplot uses wxPython, which you can download and install from here.
  3. esxplot also requires NumPy, available here.

A manual is included in the package but I think you should find the interface simple and intuitive.  Here are a few tips to get you going:

  • Load an esxtop batch file by specifying it on the command line or through File->Import->Dataset.
  • Using the counter navigator in the lower left corner, navigate to the counter of interest.  For instance, to view percent ready you would first expand the host  (use the triangle to the left of the host name), then expand “Group Cpu (GID:VM)”, then select “% Ready”.  The graph will be displayed on the right.
  • You can quickly find counters by entering a query search in the middle of the left panel.  This creates a new entry in the counter navigator that contains only counters whose name matches the expression you entered.  For instance, entering “ready” will create a counter query that contains all ready times in the batch file.

Please let us know what you think of this wonderful tool.

Update: 1/13/09

esxplot version 1.0 was released on 1/13/09. See the updated blog article for information on that release and continued discussion on this tool.

16 Responses

this looks very promising. esxtop logs can be a pain.

one thing i’d really like to see (feature request) is the ability to combine several logs in one view. every server logs it’s own counters but several of them write to the same LUNs. if we could combine the counters from several hosts for one LUN it would truly be a killer app!

    • This is a feature planed for the next release after 1.0. The current app does everything by “brute force”, i.e. it loads all of the data into memory, which is not a problem for the manipulation of the data but is a problem for the GUI widgets, which start to slow down considerably under MS windows, when there are 150K elements in the tree control widget, some refactoring is required to be more “lazy” laying out the GUI. Stay tuned.

  • hi,
    im using a win 64 to load the tool..after loading the dataset, what query do i need to put??

  • Itzik,

    I have updated the blog entry with some instructions that address your question. In short, you do not need to ever enter a query.

    However, should you get tired of searching through the thousands of counters that esxtop can provide, you can limit the counter list in the query box by specifying a counter of interest.

    Scott

  • Awesome tool! Looking forward to version 1.0 with ability to load multiple CSV files. Let me know if you need help testing or applying rusty PHP/Perl/Ruby skills to help out. 🙂

    Thanks!

    Sean Clark – vSeanClark

    • Sean,
      Thanks. 1.0 should be out around Thanksgiving. It has a few enhancements, you can plot up to 16 plots on a graph, you can close a dataset and open another without exiting the tool, metric categories that have all zero data are greyed out at the top level so you don’t have to drill down only to discover that all of the metrics are zero.

      Being able to open multiple data sets at one time will require some refactoring and I’m not going to hold up 1.0 for that, it is planned as the first major feature for 1.1, due out around early 2010.

  • Any news on the 1.0 version ?

  • Look forward to the update – the tool is very helpful and gets round the problems with Excel and Perfmon.

    Thank you for all your work.

  • Great tool! Look forward to version 1.0
    Dave

  • […] to announce that Geoff White has completed the first official release of esxplot, version 1.0.  In an earlier blog article we discussed a beta version of this product and the response from VMware’s customers has been […]

  • Version 1.0 of esxplot has been released. Please see the updated blog article for information and to comment on this tool:

    http://vpivot.com/2010/01/13/esxplot-1-0-released/

  • […] Scott Drummonds of vPivot.com posted on ESXPlot, a great tool that is far more efficient and less aggravating than Perfmon for working with […]

  • […] esxtop Batch Output (and esxplot) […]