Zabbix stopped sending out alerts, HALP!

Zabbix was this reliable friend, always sending you an email, SMS or both when something went down. It sometimes sent you a lot of emails, but you never got angry at Zabbix about that – it was just eager to help you, make sure you did not miss the weekly disaster. But then… last week… Zabbix did not send you an SMS. It did not send you an email. It did not telepathically inform you. But things were DOWN. Server was not RESPONDING.

A fat cat at a table with paws up

Zabbix knew about this. As you review the data, sitting in a dark room, the graphs clearly show the downtime. But there was-no-alert. How is that possible? Wait, what, this is impossible. You can see on the glowing screen that the main action, a crucial piece in getting those alerts, is disabled. That just cannot be, as nobody, NOBODY would ever disable that. How, oh how. Why, oh why.

Zabbix frontend row, showing a disabled action

Continue reading Zabbix stopped sending out alerts, HALP!

Visiting the Open Source Monitoring Conference 2016, Part 3

Monitoring most often deals with IT infrastructure. Sometimes it diverges a bit and starts caring about temperature and humidity, but in most cases that’s still limited to datacentre monitoring. In a talk at the Open Source Monitoring Conference 2016, Antony Stone covered some real world monitoring that goes a bit further than temperature monitoring. On a more classic-IT note, Shlomi Zadok covered system management with Foreman and security/compliance reporting by integrating with OpenSCAP. Let’s see what these fine gentlemen talked about.

Two cows

Continue reading Visiting the Open Source Monitoring Conference 2016, Part 3

Zabbix API example: creating a graph with LLD-generated items

We talked about the second most-voted feature request for Zabbix, an ability to create a graph containing an item for all LLD-created entities like network interfaces or filesystems. There was a teaser on using the Zabbix API to implement something like that. Now is the time to check out a simple example that shows such a functionality using a Perl module for the Zabbix API, Zabbix::Tiny.

Don’t despair if you have never used Zabbix::Tiny or even Perl before – everybody starts that way, and questions are always welcome. You might also want to review the posts that introduced Zabbix::Tiny:

Our script will do these things:

  • connect to the Zabbix API and obtain the host ID
  • query items by the specified key pattern that belong to the specified host (using the host ID obtained earlier)
  • create a custom graph

Our goal is to automatically create graphs with LLD-generated items like these:
Zabbix graph configuration, showing 4 free diskspace items
Continue reading Zabbix API example: creating a graph with LLD-generated items

Zabbix feature request TOP20, January 2017

We recently covered the most desirable feature request in Zabbix – ability to tell an item to collect a value right now, instead of waiting for the next polling interval. But there are many more feature requests – actually, there are almost 2000 open feature requests. Let’s look at TOP20 out of those, and let’s figure out what’s the second most desirable thing to be implemented for Zabbix.

Zabbix TOP3 feature requests
There are 17 more of these below!

Continue reading Zabbix feature request TOP20, January 2017

Visiting the Open Source Monitoring Conference 2016, Part 2

WeMos D1 Mini boardContinuing the visit of Open Source Monitoring Conference 2016, it was time for Jan-Piet Mens to talk about using small things for monitoring. These specific small things communicate using MQTT, a messaging protocol that has been around since 1999. After that, Remo Rickli introduced a tool called NeDi – short for Network Discovery. Being around since 2003, it has received more development effort recently.

Continue reading Visiting the Open Source Monitoring Conference 2016, Part 2

Visiting the Open Source Monitoring Conference 2016, Part 1

A witch on a broom over a crescent moonOpen Source Monitoring Conference (OSMC) is an event in Nuremberg, Germany. It started back in 2006 as a Nagios Conference, and got renamed to OSMC in 2009. As the name implies, it started out very focused on Nagios, then slowly became more generic with various other monitoring-related topics being included. I had the pleasure of attending the conference this year and here’s a small summary of a few of the very interesting talks at OSMC 2016.

Continue reading Visiting the Open Source Monitoring Conference 2016, Part 1

Monitoring items for uneven values, how odd is that?

As someone working in IT infrastructure, every now and then you are confronted with a problem that you are not certain how to solve. Often times I have found myself overthinking things and ending up with a complex solution that isn’t very elegant but get’s the job done.

One such occasion was my solution to monitor Link Aggregation Group (or LAG) interfaces on switches.

dice
A game of Chō-han, anyone?

Continue reading Monitoring items for uneven values, how odd is that?