Skip to content

fix(librenms-health): report numeric sensors that breach their limits #1329

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
NavidSassan wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix(librenms-health): report numeric sensors that breach their limits #1329

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ Monitoring Plugins:

* disk-usage: performance data now carries the effective warning and critical thresholds again, so graphing tools such as Icinga Graphite can draw the warn/crit lines ([#1310](https://github.com/Linuxfabrik/monitoring-plugins/issues/1310))
* journald-query: a relative `--since` value such as `-8h` configured in the Icinga Director now works instead of making the check fail ([#1264](https://github.com/Linuxfabrik/monitoring-plugins/issues/1264))
* librenms-health: numeric sensors (temperature, humidity, voltage, power, etc.) that breach their configured LibreNMS limits are now reported, matching the severity LibreNMS shows on its device health pages (WARNING at the warning limit, CRITICAL at the critical limit); previously only discrete state sensors could raise an alert, while an over- or under-limit numeric reading was silently reported as OK
* lynis: audits now produce a report on Debian, Ubuntu and other distributions that keep the lynis plugins outside `/usr/share` ([#1262](https://github.com/Linuxfabrik/monitoring-plugins/issues/1262))
* lynis: when an audit produces no report, the plugin now shows the underlying lynis error instead of a generic message
* redfish-\*: the Redfish API URL is now a mandatory `--url` parameter; the previous placeholder default silently pointed unconfigured checks at `localhost` instead of the target BMC, producing misleading connection errors ([#1306](https://github.com/Linuxfabrik/monitoring-plugins/issues/1306))
Expand Down
52 changes: 35 additions & 17 deletions check-plugins/librenms-health/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Retrieves hardware sensor information (temperature, humidity, voltage, power, st
* Queries the LibreNMS MySQL/MariaDB database directly (the API is too resource-intensive for large-scale environments)
* Joins `devices`, `sensors`, `sensors_to_state_indexes`, `state_translations`, `device_groups`, and `locations` tables
* For state-class sensors, displays the state description instead of the raw numeric value
* For numeric sensors with defined limits, displays the value together with its low/high range
* For numeric sensors, compares the value against its four configured limits the same way LibreNMS does on its device health pages (warning and critical, low and high) and displays the value together with its low/high range
* Supports filtering by device group (`--device-group`, with SQL wildcards), device hostname (`--device-hostname`, repeatable), and device type (`--device-type`, repeatable)
* In default (compact) mode, only sensors with alerts are shown; use `--lengthy` to display all sensors with extended details (type, location, sensor class, last update time)

Expand Down Expand Up @@ -79,38 +79,56 @@ options:
## Usage Examples

```bash
./librenms-health --defaults-file=/var/spool/icinga2/.my.cnf --device-group="%storage%"
./librenms-health --defaults-file=/var/spool/icinga2/.my.cnf
```

Output:

```text
There are 4 alerts. Checked 113 sensors.

Hostname ! SysName ! Sensor ! Val (Range) ! State
-------------+----------+--------------------------+--------------------+-----------
192.0.2.11 ! synoRZ02 ! Disk 17 MZILT1T9HAJQ/007 ! NotInitialized ! [WARNING]
192.0.2.11 ! synoRZ02 ! Disk 18 MZILT1T9HAJQ/007 ! NotInitialized ! [WARNING]
storinator02 ! synoRZ04 ! Upgrade Availability ! Available ! [WARNING]
storinator02 ! synoRZ04 ! Disk 5 WD4000FYYZ-01UL1B3 ! NotInitialized ! [WARNING]
There are 2 alerts. Checked 4 sensors.

Hostname ! SysName ! Sensor ! Val (warn/crit Range) ! State
-------------+----------+--------------------------+------------------------------+-----------
192.0.2.11 ! pdu-a1 ! lgpEnvSupplyAirHumidity ! 71.7 (35.0..65.0/30.0..70.0) ! [CRITICAL]
192.0.2.11 ! pdu-a1 ! lgpEnvReturnAirHumidity ! 82.0 (-/20.0..70.0) ! [CRITICAL]
storinator02 ! synoRZ04 ! Disk 17 MZILT1T9HAJQ/007 ! NotInitialized ! [WARNING]
```

With `--lengthy`:

```text
There are 4 alerts. Checked 113 sensors.
There are 2 alerts. Checked 4 sensors.

Hostname ! SysName ! Type ! Location ! Sensor ! Class ! Changed ! Val (warn/crit Range) ! State
-------------+----------------+---------+-------------+--------------------------+----------+---------+------------------------------+-----------
192.0.2.11 ! pdu-a1 ! power ! DC1 Rack A1 ! lgpEnvSupplyAirHumidity ! humidity ! 3m 22s ! 71.7 (35.0..65.0/30.0..70.0) ! [CRITICAL]
192.0.2.11 ! pdu-a1 ! power ! DC1 Rack A1 ! lgpEnvReturnAirHumidity ! humidity ! 23m 27s ! 82.0 (-/20.0..70.0) ! [CRITICAL]
storinator02 ! synoRZ04 ! storage ! DC1 Rack B3 ! Disk 17 MZILT1T9HAJQ/007 ! state ! 2h 15m ! NotInitialized ! [WARNING]
sw01 ! core-switch-01 ! network ! DC1 Rack A2 ! PSU 1 Voltage ! voltage ! 5m 1s ! 12.0 (11.8..12.6/11.4..13.0) ! [OK]
```

The value column shows each sensor's current reading followed by its two limit ranges, the warning range first and the critical range second:

Hostname ! SysName ! Type ! Location ! Sensor ! Class ! Changed ! Val (Range) ! State
-------------+----------+---------+----------+--------------------------+-------------+---------+--------------------+-----------
192.0.2.11 ! synoRZ02 ! storage ! DC1 ! Disk 17 MZILT1T9HAJQ/007 ! state ! 2h 15m ! NotInitialized ! [WARNING]
...
```text
71.7 (35.0..65.0/30.0..70.0)
`----------'`----------'
warn range crit range
low..high low..high
```

* A range that has a limit on only one side is left open on the other, for example `31.0 (..30.0/..35.0)` for a sensor with high limits but no low limits.
* A range that is not configured at all is shown as `-`, for example `82.0 (-/20.0..70.0)` for a sensor that has critical limits but no warning limits. This is common: LibreNMS auto-discovery sets only the critical limits for most sensors and leaves the warning limits empty, so a lot of sensors report only a critical range and can therefore only ever reach CRITICAL, never WARNING.
* Discrete state sensors have no numeric limits and show only their state description.

Warning (and critical) limits can be set per sensor in LibreNMS: open the device, go to its settings (the gear icon), and edit the values on the **Health** tab. Once a warning limit is filled in, that sensor starts reporting WARNING before it reaches its critical limit, and its warning range appears here in place of the `-`.


## States

* OK if all sensor values are within their LibreNMS-configured thresholds.
* WARN, CRIT, or UNKNOWN based on the sensor's `state_generic_value` in LibreNMS (mirrors the alert state from LibreNMS).
* OK if all sensors are within their LibreNMS-configured limits.
* Numeric sensors (temperature, humidity, voltage, power, etc.) report CRIT when the value is at or beyond its critical low/high limit, and WARN when it is at or beyond its warning low/high limit.
* Discrete state sensors (fan, power supply, disk state, etc.) report the severity LibreNMS assigned to the current state: WARN, CRIT or UNKNOWN.
* A sensor without a current reading is UNKNOWN.
* Sensors with alerting disabled in LibreNMS are excluded and do not affect the check state.
* `--always-ok` suppresses all alerts and always returns OK.

Expand Down
Loading