Visiting the Open Source Monitoring Conference 2016, Part 2

WeMos D1 Mini boardContinuing the visit of Open Source Monitoring Conference 2016, it was time for Jan-Piet Mens to talk about using small things for monitoring. These specific small things communicate using MQTT, a messaging protocol that has been around since 1999. After that, Remo Rickli introduced a tool called NeDi – short for Network Discovery. Being around since 2003, it has received more development effort recently.

Small Things for Monitoring

Jan-Piet started with a quick overview of MQTT. Users may choose one of the QoS levels for the client behaviour depending on their needs:

  • at most once just sends the messages
  • assured delivery sends the message and waits for an acknowledgement
  • once only sends the message, waits for an acknowledgement. Then it sends info that acknowledgement has been received and waits for this info to be acknowledged

The more precision is needed, the more verbose the communication becomes. MQTT communication participants consist of loosely-coupled publishers/subscribers – that is, they can appear/disappear at any time. Publishers/subscribers connect to the MQTT server – broker. A fairly common analogy seems to be one of a large cauldron. Publishers throw topic-tagged messages in the cauldron (at the broker side). Any subscribers to that topic get these messages. If they aren’t connected to the broker at that time, they won’t get the messages – unless he messages are marked as “retained”. In this case, any subscribers that connect later will get the last known value for that topic. Specifically, the Mosquitto broker (written in C) was covered.

Usage is very simple, especially with the mosquitto_pub and mosquitto_sub tools.

There’s also a rudimentary monitoring/alerting built-in where publishers can instruct that some operation should be carried out if a topic stops receiving values. Brokers can also share the data in a “bridging” mode – they will mirror messages from one to another. The brokers can mirror all messages or filter on a smaller subset only.

And then we got the subject of the talk – small devices that communicate using MQTT. The main hero of the talk was the ESP8266 family – a small module than can connect over WiFi. The appeal here is the possibility for these small, inexpensive devices to monitor anything, at a distance. Inexpensive meant starting at 1.5 EUR per piece and up to ~5 EUR for more advanced options. Specifically, Wemos produced modules were passed around. Micro-USB powered, with an external antenna connector – that makes them easily deployable also at a longer range.

WeMos D1 Mini board

But all these small devices, they must send the data in somehow. As they are WiFi capable, they could just connect to some central WiFi network – but that is often unavailable in the datacenter, or requires heavy authentication. The solution – having a separate WiFi network that we could even call a management plane. Having an expensive WiFi router on each location would soon destroy our grand plan of cheap, ubiquitous monitoring. Here a GL-Technologies device GL-AR150 was suggested as a cheap-enough solution. Obtainable for ~23 EUR this tiny router sports a couple of ethernet ports for WAN/LAN, a USB port and a micro-USB port for power – as well as an external antenna port.

There seems to be a healthy competition in the tiny-router market – you can also get a Mikrotik device RB941-2nD-TC for a few euros less, but it lacks the USB port, and comes with half the RAM – 32MB instead of 64MB the AR-150 has.

Such WiFi connected modules could report temperature, humidity or anything else they have the sensors for. During the talk, a few practical examples were demonstrated that used large LED panels for a visual effect. One of them was a WiFi-enabled button that was passed around and every attendee was supposed to press it exactly once. Pressing it more than once meant that the offender had to buy a drink for the speaker. Well, there were rumours that this aspect was slightly rigged, though… That might have been a lesson in not trusting systems one cannot inspect before using them 🙂

This talk was inspiring in the “could tinker with that…” way. The low pricing of the components seems to allow easier entry to the “Internet of Things” hype for smaller players, too.

NeDi update and more

OSMC usually has talks about specific monitoring tools as well, and this year was no exception. A talk by Remo Rickli introduced a tool called NeDi – short for Network Discovery.

The development on NeDi started back in 2001 with the first public release happening in 2003. While it has been around for a long time, the development was a bit stagnant previously, picking up speed in the last few years. For example, problem acknowledgements got implemented not too long ago.

NeDi, as the name implies, seems to be more oriented towards network device monitoring, and discovering such devices. The focus on network shows with some of the built-in features, like the ability to alert when an unknown MAC address appears on the network, or even cutting PoE off for unknown MAC addresses.

User interface wise, almost all tools have some feature that other tools either miss or do not implement as well. In case of NeDi, the one that caught my eye was something seemingly simple – using simple icons as buttons to perform manual actions on monitored devices. Zabbix does have a feature that allows executing manual actions – it is called global scripts.

These scripts are available in the user interface as a text-based menu, which looks something like this:

Zabbix global script menu

 

The ability to have these as buttons in NeDi inspired this Zabbix feature request 🙂

NeDi also has automatic maps, showing the monitored devices. This functionality seemed simplistic, but still pretty nice. What got me more enthusiastic was heatmaps, based on the Leaflet library. The problems are aggregated based on the geographical location, and the impact is communicated with coloured areas.

Leaflet map, showing problem clustering

Depending on the interaction between individual problems either a heatmap or plain clustering might be more appropriate, but in any case it is great to see open and OpenStreetMap based technologies being integrated in more and more products.

Overall NeDi seemed like a project to follow, except maybe the licensing approach. According to the Wikipedia page, NeDi is both delayed opensource and open core: “Access to the latest version can be bought with support, on a yearly subscription basis. The last year’s version is released to the public”. That’s the delayed opensource aspect. It then continues with “Some premium modules are not included in this free “community edition”, which is the “open core” approach. For many companies and individuals these two business models might be a barrier to investigating NeDi more.

3 thoughts on “Visiting the Open Source Monitoring Conference 2016, Part 2”

  1. I’ve been playing a lot with ESP8266 and MQTT, since Jan-Piet Mens inspired me to do so with a talk at OSDC2015. The one thing that’s missing is a convenient bridge from MQTT to Zabbix. At the moment, I’m using JP’s mqttwarn, but I would really appreciate something more flexible. Is there any feasible way to send data from Node Red to Zabbix?

Leave a Reply