Note, this is a first run. What I want to do is reduce as many files required as possible for the running, but I have not done so (e.g. mapping direct files, instead of entire directories). But, the first run at this is functions.
Note, the jasonrivers/nagios image runs a web service of its own that requires cgi to be in the root. This means that you won't be able to utilize proxy pass in Apache HTTPd unless you are mapping root for root (e.g. "/"), and not adding a "/nagios" onto the URL.
Okay, down to the nitty gritty. Here's how we'll go about this.
Here we go.
Set up NRPE (Nagios Remote Plugin Executor)
First, install nrpe (this is so that the nagios instance can monitor our docker server that this is running on) on your docker host. This is specifically because you probably want to monitor it, and host-to-host monitoring is easiest through nrpe.
sudo yum install nrpe
yum install nagios-plugins-swap.x86_64 nagios-plugins-uptime.x86_64 nagios-plugins-load.x86_64 nagios-plugins-disk.x86_64 nagios-plugins-ping.x86_64 nagios-plugins-check-updates.x86_64 nagios-plugins-users.x86_64 nagios-plugins-procs.x86_64
Edit your NRPE configuration (/etc/nagios/nrpe.cfg) and configure the commands you want. Additionally, I changed my allowed hosts to match the docker network. If you are absolutely crazy, comment out allowed_hosts out, which will allow ANYTHING to run checks against the service. My commands are :
command[check_users]=/usr/lib64/nagios/plugins/check_users -w 5 -c 10
command[check_load]=/usr/lib64/nagios/plugins/check_load -r -w .15,.10,.05 -c .30,.25,.20
command[check_root]=/usr/lib64/nagios/plugins/check_disk -w 20% -c 10% -p /dev/mapper/rhel-root
command[check_home]=/usr/lib64/nagios/plugins/check_disk -w 20% -c 10% -p /dev/mapper/rhel-home
command[check_swap]=/usr/lib64/nagios/plugins/check_swap
command[check_zombie_procs]=/usr/lib64/nagios/plugins/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/lib64/nagios/plugins/check_procs -w 275 -c 300
command[check_uptime]=/usr/lib64/nagios/plugins/check_uptime
command[check_gateway]=/usr/lib64/nagios/plugins/check_ping -H 10.0.0.1 -w 5,1% -c 10,5%
command[check_internet]=/usr/lib64/nagios/plugins/check_ping -H 8.8.8.8 -w 20,1% -c 30,5%then enable and start NRPE :
systemctl enable nrpe
systemctl start nrpeAnd the service should be listening on port 5666
Nagios Docker
Next, create the docker container directory structure :
cd /docker/images/path
mkdir nagios
cd nagios
mkdir {etc,var,plugins,graphvar,graphetc}launch the basic, no frills docker image first :
docker run --name nagios4 jasonrivers/nagiosThen, hit ctrl-c to break out of it. This installs the image with the name nagios4 that we can reference and copy out the configs we'll need. Now you can copy the files out that you need :
docker cp nagios4:opt/nagios/etc etc/
docker cp nagios4:opt/nagios/var var/
chown polkitd var
chown polkitd graphvar
Clean up that image now :
docker container rm nagios4Now, you probably want to add the NRPE checks for the docker host at this point. Edit the etc/objects/commands.cfg file, and add the following :
define command {
command_name check_nrpe
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
}Then, edit the localhost.cfg (or create a new cfg file and ensure the nagios.cfg includes it) and add the host for your docker and any checks you want to run that you created in your nrpe configuration, e.g. :
define service{
use local-service,graphed-service ; Name of service template to use
host_name docker
service_description Docker Host Internet
check_command check_nrpe!check_internet
}Next, you can modify those configurations, and launch your custom image using :
docker run --name nagios -d --restart unless-stopped -e NAGIOS_FQDN=sharktooth.net -e NAGIOS_TIMEZONE=America/Boise -v /opt/docker/nagios/etc:/opt/nagios/etc -v /opt/docker/nagios/var:/opt/nagios/var -v /opt/docker/nagios/plugins:/opt/Custom-Nagios-Plugins -v /opt/docker/nagios/graphetc:/opt/nagiousgraph/etc -v /opt/docker/nagios/graphvar:/opt/nagiosgraph/var -p 0.0.0.0:6080:80 jasonrivers/nagios:latestThat should give it a quick start and have you online. You can now customize those configurations and restart the container as needed.
What is installed
Now, I wanted some customizations, such as monitoring mqtt, but I kinda need to know what is included. Sure, we have a custom plugin directory we can stuff things into, but if it's already there, then why should I also create a plugin?
While a simple directory listing would suffice, I just used the typical docker cp command, and copied out the plugin directory to see what had been installed.
[username@docker-host ~]$ docker cp nagios:/opt/nagios/libexec tmp/
Successfully copied 8.42MB to /home/username/tmp/
[username@docker-host ~]$ ls -l tmp/libexec/
total 8300
-rwxr-xr-x. 1 username 1026 194024 Nov 6 2024 check_apt
-rwxr-xr-x. 1 username 1026 2342 Nov 6 2024 check_breeze
-rwxr-xr-x. 1 username 1026 205824 Nov 6 2024 check_by_ssh
lrwxrwxrwx. 1 username 1026 9 Nov 6 2024 check_clamd -> check_tcp
-rwxr-xr-x. 1 username 1026 139256 Nov 6 2024 check_cluster
-rwxr-xr-x. 1 username 1026 204664 Nov 6 2024 check_dbi
-r-sr-xr-x. 1 username 1026 210952 Nov 6 2024 check_dhcp
-rwxr-xr-x. 1 username 1026 193304 Nov 6 2024 check_dig
-rwxr-xr-x. 1 username 1026 336488 Nov 6 2024 check_disk
-rwxr-xr-x. 1 username 1026 10130 Nov 6 2024 check_disk_smb
-rwxr-xr-x. 1 username 1026 220352 Nov 6 2024 check_dns
-rwxr-xr-x. 1 username 1026 97440 Nov 6 2024 check_dummy
-rwxr-xr-x. 1 username 1026 5062 Nov 6 2024 check_file_age
-rwxr-xr-x. 1 username 1026 6500 Nov 6 2024 check_flexlm
-rwxr-xr-x. 1 username 1026 203824 Nov 6 2024 check_fping
lrwxrwxrwx. 1 username 1026 9 Nov 6 2024 check_ftp -> check_tcp
-rwxr-xr-x. 1 username 1026 156760 Nov 6 2024 check_game
-rwxr-xr-x. 1 username 1026 191392 Nov 6 2024 check_hpjd
-rwxr-xr-x. 1 username 1026 342280 Nov 6 2024 check_http
-r-sr-xr-x. 1 username 1026 252320 Nov 6 2024 check_icmp
-rwxr-xr-x. 1 username 1026 146576 Nov 6 2024 check_ide_smart
-rwxr-xr-x. 1 username 1026 15271 Nov 6 2024 check_ifoperstatus
-rwxr-xr-x. 1 username 1026 14477 Nov 6 2024 check_ifstatus
lrwxrwxrwx. 1 username 1026 9 Nov 6 2024 check_imap -> check_tcp
-rwxr-xr-x. 1 username 1026 6981 Nov 6 2024 check_ircd
lrwxrwxrwx. 1 username 1026 9 Nov 6 2024 check_jabber -> check_tcp
-rwxr-xr-x. 1 username 1026 2363 Nov 6 2024 check_jenkins
-rwxr-xr-x. 1 username 1026 204960 Nov 6 2024 check_ldap
lrwxrwxrwx. 1 username 1026 10 Nov 6 2024 check_ldaps -> check_ldap
-rwxr-xr-x. 1 username 1026 181336 Nov 6 2024 check_load
-rwxr-xr-x. 1 username 1026 8182 Nov 6 2024 check_log
-rwxr-xr-x. 1 username 1026 25719 Nov 6 2024 check_mailq
-rwxr-xr-x. 1 username 1026 19217 Nov 6 2024 check_mem.pl
-rwxr-xr-x. 1 username 1026 14163 Nov 6 2024 check-mqtt.py
-rwxr-xr-x. 1 username 1026 146112 Nov 6 2024 check_mrtg
-rwxr-xr-x. 1 username 1026 148776 Nov 6 2024 check_mrtgtraf
-rwxr-xr-x. 1 username 1026 14953 Nov 6 2024 check_mssql_database.py
-rwxr-xr-x. 1 username 1026 21856 Nov 6 2024 check_mssql_server.py
-rwxr-xr-x. 1 username 1026 158376 Nov 6 2024 check_nagios
-rwxr-xr-x. 1 username 1026 12200 Nov 6 2024 check_ncpa.py
lrwxrwxrwx. 1 username 1026 9 Nov 6 2024 check_nntp -> check_tcp
lrwxrwxrwx. 1 username 1026 9 Nov 6 2024 check_nntps -> check_tcp
-rwxr-xr-x. 1 username 1026 156328 Nov 6 2024 check_nrpe
-rwxr-xr-x. 1 username 1026 206624 Nov 6 2024 check_nt
-rwxr-xr-x. 1 username 1026 225112 Nov 6 2024 check_ntp
-rwxr-xr-x. 1 username 1026 203008 Nov 6 2024 check_ntp_peer
-rwxr-xr-x. 1 username 1026 207672 Nov 6 2024 check_ntp_time
-rwxr-xr-x. 1 username 1026 219552 Nov 6 2024 check_nwstat
-rwxr-xr-x. 1 username 1026 9495 Nov 6 2024 check_oracle
-rwxr-xr-x. 1 username 1026 172160 Nov 6 2024 check_overcr
-rwxr-xr-x. 1 username 1026 195272 Nov 6 2024 check_pgsql
-rwxr-xr-x. 1 username 1026 207440 Nov 6 2024 check_ping
lrwxrwxrwx. 1 username 1026 9 Nov 6 2024 check_pop -> check_tcp
-rwxr-xr-x. 1 username 1026 209304 Nov 6 2024 check_procs
-rwxr-xr-x. 1 username 1026 170464 Nov 6 2024 check_real
-rwxr-xr-x. 1 username 1026 9675 Nov 6 2024 check_rpc
-rwxr-xr-x. 1 username 1026 1629 Nov 6 2024 check_sensors
lrwxrwxrwx. 1 username 1026 9 Nov 6 2024 check_simap -> check_tcp
-rwxr-xr-x. 1 username 1026 237160 Nov 6 2024 check_smtp
-rwxr-xr-x. 1 username 1026 250280 Nov 6 2024 check_snmp
lrwxrwxrwx. 1 username 1026 9 Nov 6 2024 check_spop -> check_tcp
-rwxr-xr-x. 1 username 1026 8481 Nov 6 2024 check_sql
-rwxr-xr-x. 1 username 1026 166736 Nov 6 2024 check_ssh
-rwxr-xr-x. 1 username 1026 12544 Nov 6 2024 check_ssl_validity
lrwxrwxrwx. 1 username 1026 9 Nov 6 2024 check_ssmtp -> check_tcp
-rwxr-xr-x. 1 username 1026 143696 Nov 6 2024 check_swap
-rwxr-xr-x. 1 username 1026 218216 Nov 6 2024 check_tcp
-rwxr-xr-x. 1 username 1026 167368 Nov 6 2024 check_time
lrwxrwxrwx. 1 username 1026 9 Nov 6 2024 check_udp -> check_tcp
-rwxr-xr-x. 1 username 1026 186936 Nov 6 2024 check_ups
-rwxr-xr-x. 1 username 1026 133536 Nov 6 2024 check_uptime
-rwxr-xr-x. 1 username 1026 131696 Nov 6 2024 check_users
-rwxr-xr-x. 1 username 1026 8193 Nov 6 2024 check_vpn
-rwxr-xr-x. 1 username 1026 3266 Nov 6 2024 check_wave
lrwxrwxrwx. 1 username 1026 20 Nov 6 2024 mibs -> /usr/share/snmp/mibs
-rwxr-xr-x. 1 username 1026 131912 Nov 6 2024 negate
-rwxr-xr-x. 1 username 1026 124112 Nov 6 2024 remove_perfdata
-rwxr-xr-x. 1 username 1026 123440 Nov 6 2024 urlize
-rwxr-xr-x. 1 username 1026 1920 Nov 6 2024 utils.pm
-rwxr-xr-x. 1 username 1026 2792 Nov 6 2024 utils.sh
[username@docker-host ~]$
This tells me exactly what we have available to us straight out of the box, and lists things like check_jabber, check-mqtt.py, and check_snmp (so, in theory, I COULD monitor my NAS without customization - that's cool).
Monitoring MQTT and SNMP
Let's get those set up. Now, in the commands.cfg file, there is already a check_snmp command configured. That means we don't have to define a command, but we just point to our NAS for HDD monitoring. We have three, just because I can :
# NAS stuff
# the overall health
define service{
use local-service,graphed-service
host_name nas
service_description NAS Overall Health
check_command check_snmp!-C community -o .1.3.6.1.4.1.6574.1.1 -r 1
}
# the NAS filesystems
define service{
use local-service,graphed-service
host_name nas
service_description NAS Volume 1 Health
check_command check_snmp!-C NASHealthCheck -o .1.3.6.1.4.1.6574.3.1.1.3.0 -r 1
}
# NAS temperature
define service{
use local-service,graphed-service
host_name nas
service_description NAS Temperature
check_command check_snmp!-C NASHealthCheck -o .1.3.6.1.4.1.6574.1.2 -r 3
} I loaded it, and all the snmp just failed. Bah! Wait a minute.... didn't I have to open UDP ports for the weatherflow crap?
Let's add it to the firewall. I threw the entire pot of spaghetti at the wall hoping something stuck :
sudo firewall-cmd --zone public --permanent --add-service=snmp
sudo firewall-cmd --permanent --add-port=161/udp
sudo firewall-cmd --reloadNagios eventually loaded those and they started checking OK.
On to MQTT. This one doesn't have a command defined, so we'll have to do that first, and we can (in theory) run the check-mqtt.py file to get output.
Note, to be able to run the check-mqtt.py from a server that didn't have this properly installed to (e.g. not in the container), I had to install the python3-paho-mqtt.noarch package. Once done, I could run the "./check-mqtt.py" to see output.
[username@docker-host nagios]$ /home/username/tmp/libexec/check-mqtt.py -?
usage: check-mqtt.py [-h] [-d] [-H ] [-P ] [-u ] [-p ] [-m ] [-e ] [--sleep ] [-a ] [-C ] [-k ] [-n] [-t ]
[-s ] [-r] [-l ] [-v ] [-o ] [-w ] [-c ] [-S] [-V]
check-mqtt.py: error: unrecognized arguments: -?
[username@docker-host nagios]$ This gives us the basics we need to build a command.
define command {
command_name check_mqtt
command_line $USER1$/check-mqtt.py -H $HOSTNAME$ $ARG1$
}Slap together a quick command like (I put the options from the output into there, but they are also listed at https://github.com/check-plugins/check_mqtt ) :
# check MQTT queues define service{ use local-service,graphed-service host_name docker service_description MQTT check_command check_mqtt!-t topic # usage: check-mqtt.py [-h] [-d] [-H ] [-P ] [-u ] [-p ] [-m ] [-e ] [--sleep ] [-a ] [-C ] [-k ] [-n] [-t ] # [-s ] [-r] [-l ] [-v ] [-o ] [-w ] [-c ] [-S] [-V] }
and restart Nagios. I do have some tweaking to get it to work right, since it went into a yellow/WARNING state with some mumbo jumbo about "No output on stdout". So, let's fix that hurdle next.
[username@docker-host nagios]$ docker container exec nagios /opt/nagios/libexec/check-mqtt.py -H docker -t topic
Traceback (most recent call last):
File "/opt/nagios/libexec/check-mqtt.py", line 257, in
mqttc = paho.Client('nagios-%d' % (os.getpid()), clean_session=True, userdata=userdata, protocol=4)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/paho/mqtt/client.py", line 772, in __init__
raise ValueError(
ValueError: Unsupported callback API version: version 2.0 added a callback_api_version, see docs/migrations.rst for details
[username@docker-host nagios]$
Quick research, tells me the Client declaration needs modification after an update to a version, so I copied the check-mqtt.py into the custom plugins folder for the docker instance with the following patch applied to it :
-- /home/username/tmp/libexec/check-mqtt.py 2024-11-06 02:35:46.000000000 -0700
+++ plugins/check-mqtt.py 2026-08-13 08:22:59.243414998 -0600
@@ -254,7 +254,7 @@ userdata = {
'have_response' : False,
'start_time' : time.time(),
}
-mqttc = paho.Client('nagios-%d' % (os.getpid()), clean_session=True, userdata=userdata, protocol=4)
+mqttc = paho.Client(paho.CallbackAPIVersion.VERSION1,'nagios-%d' % (os.getpid()), clean_session=True, userdata=userdata, protocol=4)
mqttc.on_message = on_message
mqttc.on_connect = on_connect
mqttc.on_disconnect = on_disconnect
this alloed to run
[username@docker-host nagios]$ docker container exec nagios /opt/Custom-Nagios-Plugins/check-mqtt.py -H docker -t topic
/opt/Custom-Nagios-Plugins/check-mqtt.py:257: DeprecationWarning: Callback API version 1 is deprecated, update to latest version
mqttc = paho.Client(paho.CallbackAPIVersion.VERSION1,'nagios-%d' % (os.getpid()), clean_session=True, userdata=userdata, protocol=4)
OK - message from topic at docker in 0.31s | response_time=0.31 value=PiNG
[username@docker-host nagios]$And then I updated my command to point to the new module :
define command {
command_name check_mqtt
command_line /opt/Custom-Nagios-Plugins/check-mqtt.py -H $HOSTNAME$ $ARG1$
#command_line $USER1$/check-mqtt.py -H $HOSTNAME$ $ARG1$
}Success! I'm working, even with the deprecation warning. I did create an issue in the jasonrivers/nagios github for it.
