Scott Drummonds on Virtualization

Custom Alarms for VMware SIOC

Comments Off on Custom Alarms for VMware SIOC

Over a year and a half ago I previewed VMware’s unreleased feature, Storage IO Control (SIOC).  SIOC creates new intelligent latency metrics to evaluate the health of VMFS volumes.  The same latency measurements are used in storage DRS, which VMware released in vSphere 5.  While automated performance correction is great, vCenter should warn VMware admins when latency crosses defined thresholds.  Custom vCenter alarms can do this.

With hardest work of making vSphere 5 generally available behind him, one of VMware’s engineers, Balaji Parimi, recently sent me scripts he wrote to create SIOC alarms.  These alarms can be used to tell administrators that SIOC is throttling some virtual machines to save high priority applications from ailing datastores.

Balaji’s script creates vCenter alarms that trigger when:

  1. SIOC configuration is changed (enable / disable, congestionThreshold) on any datastore. The user does not need to do anything if a new datastore is added. The current vCenter alarm infrastructure does not allow to create alarms based on specific properties (e.g. iorm configuration enabled / disabled or congestion threshold etc.). So, I created one alarm that gets triggered when DatastoreIORMReconfiguredEvent is generated. This event gets generated whenever the StorageIORMConfiguration on any datastore in the vCenter inventory is changed.
  2. SIOC Normalized Datastore Latency exceeds the congestionThreshold for a given datastore on a host. This alarm is a bit tricky as it depends on the performance metric. The performance metric based alarms are per instance only. So, this creates one alarm object per datastore in the vCenter. The script allows you to create for all datastores in the vCenter inventory in one shot. If more datastores are added later, the user needs to create an alarm for each of those datastores. If the user changes the congestionThreshold for a datastore, the user needs to delete the old alarm and re-create the alarm on that datastore for the new congestionThreshold value to be recognized.

The second alarm will tell you that SIOC is taking action, slowing some virtual machines to improve the performance of others.  This alarm only works on the datastores in existence at the time of its creation.  Thus the first alarm, which will tell the administrator that SIOC or the vSphere storage configuration has changed.  When the first alarm triggers it is time to re-register the second one.

Balaji included in his email to me the following instructions on setting the alarms:

Here are the two commands:

./siocalarms.sh Administrator password all

This covers all the Datastores in the vCenter. This creates the SIOC configuration changed Alarm as well.

I added a new Datastore with the name FCLun0. And ran this command to cover the new Datastore.

./siocalarms.sh Administrator password FCLun0

You can use the GUI to delete any of these Alarms. You can use the GUI to set up a specific action (like sending an email) for any of these Alarms.

I am attaching a zip file containing a README, shell and a batch script. All you need is JRE 1.6 or higher to run this.

Please give it a try and let me know your comments.

You may notice that my corporate Exchange server’s malware detection software removed the batch file Balaji references above.  But the batch file only contained a single line:

java -jar SIOCAlarms.jar <VCServer> <Username> <Password> <datastoreName>

As the instructions above state, the datastoreName “all” will add the alarms for all datastores.

Comments are closed.