Templating Monitoring of Windows Services

About a year ago, I was inspired by Raymond Kuiper’s Zen and the Art of Zabbix Template Design, and decided to start applying its practices to my own environment. If you’re not familiar with it, I would strongly recommend looking at the linked page, as well his video presentation on the subject. The idea is to create small “Task” templates for very specialized purposes, and then group those into more generalized “Duty” templates, which are grouped into even more generalized “Role” templates that provide a scope for the function of the generalized server, and that finally nests inside of a “Profile” template which gets applied to the monitored server. The process creates an extremely modular system that is very easy to manage. One of the key concepts is to rely on discovery as much as possible to maintain this modularity.

Among the first places I put this into practice was to monitor Windows services. I had envisioned several task templates for monitoring Windows services – one for DFS (distributed file services), one for SQL services, one for Skype For Business services, etc. The only functional differences between them would be that the filters used in each rule would pick just the services for that particular task.

In practice, just doing this created a problem, because while Zabbix does have a very nice Windows service.discovery item, it can only exist once per server. It was easy to create the Tplt::Task::Win::Services::DFS and Tplt::Task::Win::Services::SQL “Task” templates, however trying to link these into the same parent “Duty” template Tplt::Duty::MSSQLServer gave an error like this:

- Discovery rule "service.discovery" already exists on ...

So – how to address this?

Aliasing to the rescue!

The Zabbix agent has a parameter named “Alias” that can be used to, “substitute long and complex item key with a smaller and simpler one.” The documentation also notes that “Different Alias keys may reference the same item key.” The documentation for alias even goes so far as to recommend it’s use for, “Running multiple low-level discovery rules processing the same discovery items.” This is exactly what was needed!

The biggest downside here is that the parameter needs to be added in the Zabbix agent configuration, so it requires updating the configuration file(s) on each monitored host. The use of a configuration manager (such as Puppet, Ansible, Salt, or even GPO – this is Windows after all…), makes this easier. Simply add a line like this to each agent’s config:

Alias=my.service.discovery[*]:service.discovery

Then use service discovery rules like one of these:

my.service.discovery[SQL_Services]
my.service.discovery[DFS_Services]
my.service.discovery[SfB_Services]

It really doesn’t matter what the parameter name is, because it’s dropped by the agent. What does matter is simply that no two discovery rules have the same parameter name, which should be very easy to accomplish.

Duplicate Item Prototypes

After setting the Alias, and the service discovery rule in two different “Task” templates, I still had problems adding these templates to a parent “Duty” template. One of the reasons I want to monitor Windows services is to see if the service is running. The Zabbix service.info[] item key is designed specifically for this purpose, and I had an item with key service.info[{#SERVICE.NAME}] in each of the duty templates. This caused the following error when trying to link the templates:

-Item prototype "service.info[{#SERVICE.NAME}]" already exists on ...

My first thought was to use another alias in the config file and do something like this:

Alias=my.service.info[*]:service.info[$1]

And then use item keys like service.info[{#SERVICE.NAME},sql] in one of the Task templates, and service.info[{#SERVICE.NAME},dfs] in another Task template, however it looks like the Alias parameter can either pass all parameters or zero paramters. It does not provide the ability to access individual parameters, however UserParameter does allow this access, so it solved the problem. It’s somewhat ugly looking, as it involves the Zabbix agent making a call to zabbix_get, but it does work. The following line was also added to the agent config:

UserParameter=my.service.info[*],"%PROGRAMFILES%\Zabbix Agent\zabbix_get" -s 127.0.0.1 -k service.info["$1"]

Note that your path to the location of zabbix_get may be different than what is shown here.

Now each template can have a discovery rule that creates independent service checks. The item prototype that creates checks for the status of the services desired in the Tplt::Task::Win::Services::SQL template will be:

service.info[{#SERVICE.NAME},sql]

And the item prototype that creates checks for the status of the services desired in the Tplt::Task::Win::Services::DFS template will be:

service.info[{#SERVICE.NAME},dfs]

And the item prototype that creates checks for the status of the services desired in the Tplt::Task::Win::Services::SfB template will be:

service.info[{#SERVICE.NAME},sfb]

…and so on.

In all of the items created by these prototypes, the second parameter is discarded by the user parameter, so it can be anything. The only limitation is that the literal key value of the item prototype in any given discovery should be unique within the Zabbix installation. Now it’s just a case of setting the filters in each discovery to find only the desired services.

Leave a Reply