Showing posts with label cloudflare. Show all posts
Showing posts with label cloudflare. Show all posts

Thursday, October 15, 2026

Removing Basic Auth from Nagios Behind Proxy

I recently installed checkmk to compare checkmk with Nagios (both under Docker and behind a Cloudflare proxy), and found Nagios runs a lot cooler on the CPU side.  So, considering keeping Nagios, I decided I needed to tackle one problem that has been bugging me.

Every time I connect to Nagios through the service, I also get prompted for a username and password via basic authentication.

If I keep Nagios, the above issue is unacceptable.  I need to resolve it.  So, since it is already behind Cloudflare, I know the header is getting there.  That means I simply need to load the rewrite module and ensure the header gets copied into the REMOTE_USER (just in case - may not be needed), and then set the "default_user_name" in cgi.cfg.  So, I grabbed a few files :

    docker cp nagios:/opt/nagios/etc/cgi.cfg cgi.cfg
    docker exec -it nagios cat /etc/apache2/sites-available/nagios.conf

I then modified the nagios.conf file in the following way :

    --- apache-nagios.conf.orig	2026-10-15 16:22:42.366466648 -0600
    +++ apache-nagios.conf	2026-10-15 16:47:12.198454588 -0600
    @@ -7,6 +7,12 @@
     
     ScriptAlias /nagios/cgi-bin "/opt/nagios/sbin"
     
    +# the following two lines are to get the authenticated user into
    +# the REMOTE USER
    +LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so
    +RewriteRule .* - [E=REMOTE_USER:%{HTTP:Cf-Access-Authenticated-User-Email}]
    +
    +
     <Directory "/opt/nagios/sbin">
     #  SSLRequireSSL
        Options ExecCGI
    @@ -16,10 +22,10 @@ ScriptAlias /nagios/cgi-bin "/opt/nagios
              Require all granted
     #        Require host 127.0.0.1
     
    -         AuthName "Nagios Access"
    -         AuthType Basic
    -         AuthUserFile /opt/nagios/etc/htpasswd.users
    -         Require valid-user
    +#         AuthName "Nagios Access"
    +#         AuthType Basic
    +#         AuthUserFile /opt/nagios/etc/htpasswd.users
    +#         Require valid-user
           </RequireAll>
        </IfVersion>
        <IfVersion < 2.3>
    @@ -29,10 +35,10 @@ ScriptAlias /nagios/cgi-bin "/opt/nagios
     #     Deny from all
     #     Allow from 127.0.0.1
     
    -      AuthName "Nagios Access"
    -      AuthType Basic
    -      AuthUserFile /opt/nagios/etc/htpasswd.users
    -      Require valid-user
    +#      AuthName "Nagios Access"
    +#      AuthType Basic
    +#      AuthUserFile /opt/nagios/etc/htpasswd.users
    +#      Require valid-user
        </IfVersion>
     </Directory>
     
    @@ -47,10 +53,10 @@ Alias /nagios "/opt/nagios/share"
              Require all granted
     #        Require host 127.0.0.1
     
    -         AuthName "Nagios Access"
    -         AuthType Basic
    -         AuthUserFile /opt/nagios/etc/htpasswd.users
    -         Require valid-user
    +#         AuthName "Nagios Access"
    +#         AuthType Basic
    +#         AuthUserFile /opt/nagios/etc/htpasswd.users
    +#         Require valid-user
           </RequireAll>
        </IfVersion>
        <IfVersion < 2.3>
    @@ -60,9 +66,9 @@ Alias /nagios "/opt/nagios/share"
     #     Deny from all
     #     Allow from 127.0.0.1
     
    -      AuthName "Nagios Access"
    -      AuthType Basic
    -      AuthUserFile /opt/nagios/etc/htpasswd.users
    -      Require valid-user
    +#      AuthName "Nagios Access"
    +#      AuthType Basic
    +#      AuthUserFile /opt/nagios/etc/htpasswd.users
    +#      Require valid-user
        </IfVersion>
    </Directory>

And, I modified the cgi.conf file in the following way :

    --- cgi.cfg.orig	2026-10-15 16:54:45.686779272 -0600
    +++ cgi.cfg	2026-10-15 16:52:28.798266410 -0600
    @@ -105,6 +105,7 @@ use_ssl_authentication=0
     # server will inherit all rights you assign to this user!
      
     #default_user_name=guest
    +default_user_name=nagiosadmin
     
     
    

Now, stop your container, then delete it, and re-run adding two new parameters to map those files into place.

And, you should no longer prompt for authentication after already authenticating. 

Trying Checkmk under Docker

Please note this up front - when I installed checkmk under Docker, I did a comparison on CPU utilization between checkmk and Nagios, and the comparison (while not apples-to-apples, simply because Nagios checks on a per-minute basis, while checkmk looks like it is grabbing a lot more with regularity) is not close.  Both containers run with 4 "CPU's", so that is a direct comparison.


It appears that the checkmk runs a lot hotter than Nagios itself.  Sure, it is a much nicer interface, and it also potentially exposes things through a livestatus API in ways that might be better (I haven't played with it), but it is still a much busier application.  Proceed with that being a known point.

NetworkChuck rolled a video about a "new" monitoring package called "Checkmk".  It was actually fairly intriguing, as it uses a web-based configuration front-end (don't need to change monitoring using the command-line with a restart).  The video : 

https://www.youtube.com/watch?v=Bjifz55TTPI

Now, some of the added stuff is that it looks like a much nicer integration with metrics, not just the CRITICAL/WARNING/OK/UNKNOWN status.  I have started using docker a LOT more lately, just because it might allow things to be portable if I have to move to new hardware, or if I want to update.  So, we'll give this one a shot.  Let's see if we can do this.

It took me a few commands to find out how the image starts and what ports to forward, because nothing about it was intuitive.  So, after starting the image without it, I found a nice little command that dumps the TCP ports of the container itself.  We can use that to figure out the port to forward using trial and error, frankly.

    [username@dockerhost ~]$ docker container inspect checkmk --format '{{json .Config.ExposedPorts}}'
    {"5000/tcp":{},"6557/tcp":{},"80/tcp":{}}
    [username@dockerhost ~]$

This told me it listens on ports 5000, 6557, and 80 (Apache's HTTPd, if I've read the documentation properly).  Unfortunately, port 80 didn't actually work like I anticipated when I forwarded that.  So, I had to delete the container, and then re-create it.  The  command to start it for me :

    [username@dockerhost ~]$ docker run --name checkmk -d -p 0.0.0.0:5000:5000 checkmk/check-mk-community
    1b54e26451cc125f2a26b1298a9cfdf9a3b3843ecd3f142dc968c17ffd54acb9
    [username@dockerhost ~]$
    

It dumps the container ID, and the -d tells it to run it in the background. I recommend you then grab the logs, because the first run dumps some information we're going to need such as the default username and password (this password is NOT consistent between images - you really need to obtain that).

    [username@dockerhost ~]$ docker logs checkmk -f
    ### CREATING SITE 'cmk'
    Adding /opt/omd/sites/cmk/tmp to /etc/fstab.
    Going to set TMPFS to off.
    Preparing tmp directory /omd/sites/cmk/tmp...Generating configuration for core (type nagios)...
    Precompiling host checks...OK
    Executing post-create script "01_create-sample-config.py"...OK
    Executing post-create script "02_message-broker-certs"...OK
    Updating core configuration...
    Adding /opt/omd/sites/cmk/tmp to /etc/fstab.
    Going to set TMPFS to off.
    Skipping Apache restart.
    Created new site cmk with version 2.5.0p15.community.
    
      The site can be started with omd start cmk.
      The default web UI is available at http://1b54e26451cc/cmk/
    
      The admin user for the web applications is cmkadmin with password: Bo06iHX7qugF
      For command line administration of the site, log in with 'omd su cmk'.
      After logging in, you can change the password for cmkadmin with 'cmk-passwd cmkadmin'.
    
    ### STARTING XINETD
     * Starting internet superserver xinetd
       ...done.
    ### STARTING SITE
    Preparing tmp directory /omd/sites/cmk/tmp...Starting agent-receiver...OK
    Starting mkeventd...OK
    Starting rrdcached...OK
    Starting redis...OK
    Starting npcd...OK
    Starting automation-helper...OK
    Starting ui-job-scheduler...OK
    Starting nagios...OK
    Starting apache...OK
    Starting crontab...OK
    ### STARTING CRON
    ### CONTAINER STARTED
    ^C[username@dockerhost ~]$

What you want is the line that starts with "The admin user for the web applications" (I highlighted it in the example I'm working through).  It tells you the username, and the password you are going to need. Get this logged into your password manager.

Now, since I added the port forwarding of 5000 to 5000, I point to my docker host on port 5000, and was then presented with the login.  Entering the credentials from above gives me the wizard to create my first host (I could create a folder - I skipped that).

Monitoring our first host 

The first host I created was my NAS, because I do want to add docker stuff, and I want to port my Nagios into this thing as well. This should let me see how the metrics differ between the two.  I saw the "Problems with 1 data source detected", and immediately panicked.

Breathe, ya big nerd. It means that the agent connection failed, and I had ensured that was NOT connected (it wasn't checked on the form).  No worries.

Well, you can go back and check the box for that checkmk agent thing, and then select the "No API integrations, no checkmk agent" option, save, and re-scan.

I scanned down the list below that big scary error message, and found that it looks like it did pick up everything via SNMP, except for the temperature.  I could add the "undecided" in that list by clicking the "Monitor undecided services".  But, I opted to keep it fairly clean, and only added CPU, fans, memory, RAID status, and the volume that runs on the RAID.  Just click the green "plus" right next to  the undecided monitor you want.

Once you have them selected, you should see a "Changes" on the right side with a yellow number.  It simply means you have to click in there when you are done and activate the pending changes.

We are now monitoring our first "host".

Getting everything into checkmk

Now, as often as I searched the Googles for "migrating from Nagios to Checkmk", they all simply talked about manually putting in the hosts - e.g., there are no real tools built for migration to make it one fell swoop.  So, since the Docker instance of Nagios has all hosts in one single file in my configuration, I found that I could simply grab the unique hostnames from custom.cfg .

    grep host_name /opt/docker/nagios/etc/objects/custom.cfg |awk '{print $2}'|sort|uniq

This gave me a quick list of hostnames to add. What makes this easy is that you can just add the agent, then point to the new host, and then have it look.  We'll do that with docker, just for the sanity.  On the left, click "Setup", followed by "Linux" under "Agents".  Then, right click on the RPM (I'm doing this with RHEL), and copy the link.

Then, on your docker host, run "wget " and paste the URL, e.g. :

    [username@dockerhost nagios]$ wget http://dockerhost:5000/cmk/check_mk/agents/check-mk-agent-2.5.0p15-1.noarch.rpm
    --2026-10-14 17:23:53--  http://docker:5000/cmk/check_mk/agents/check-mk-agent-2.5.0p15-1.noarch.rpm
    Resolving docker (docker)... 192.168.1.219
    Connecting to docker (docker)|192.168.1.219|:5000... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 5877930 (5.6M) [application/x-redhat-package-manager]
    Saving to: ‘check-mk-agent-2.5.0p15-1.noarch.rpm’
    
    check-mk-agent-2.5.0p15-1.noarch.rpm 100%[======================================================================>]   5.61M  --.-KB/s    in 0.02s   
    
    2026-10-14 17:23:53 (348 MB/s) - ‘check-mk-agent-2.5.0p15-1.noarch.rpm’ saved [5877930/5877930]
    
    [username@dockerhost nagios]$

Now install it using "sudo rpm -ivh check-mk-agent-2.5.0p15-1.noarch.rpm".  Mine gave me a bright-red error about "Failed to reload xinetd.service: Unit xinetd.service not found.", but it had no impact (utilizes systemd now, and RHEL9 has NOTHING to support xinetd, anyway).

On that setup->agents side bar again, select "Other operating systems", and find the mk_docker.py under "Plug-ins".  RIght click on it, and copy the link.  Then, wget that again on your docker host to copy the plug-in down.  Install it :

    sudo install -m 0755 mk_docker.py /usr/lib/check_mk_agent/plugins/

Also make sure the Python module is installed :

    sudo yum install python3-docker.noarch

Download the example configuration file (the same way we downloaded the agent - from the browser, get to "Setup" -> "Agents" -> "Linux", and search for "docker.cfg".  Right click on that, copy the link, then in a shell on the docker host, run "wget URL" (paste the URL in place of the URL, right?).  Copy this into the /etc/check_mk directory on your docker host.

Enter the host in the hostname field, select the "Checkmx agent" under "Monitoring agents", and then click the test-connection to the right.  It should say it was successful.  If so, click the "Save & run service discovery".

Now, re-scan or perform a service discovery. 

In Checkmk, create a new folder for your docker containers ("Setup"->"Hosts"->"Add folder").  I named my folder "Docker Containers", because I have a simple mind.  If you don't have the "IP address family" checkbox showing, expand the "more settings" on the "Network address", and check the box for "IP address family", and then set the dropdown to "No IP".  Under "Monitoring agents", check the override for the "Check mkagent / API integrations", and then select "no API integrations, no checkmk agent".  If you don't see the "piggy back" option, hit the more on the right again.  Under piggy back, select it, then choose the "Always use and expect piggy back data", and hit save.

Let's get a list of container names now.  In a shell on the docker host, run a command to show them all.

    [username@dockerhost ~]$ docker ps --format '{{.Names}}'
    checkmk
    httpd
    nagios
    grafana
    cloudflare
    mqtt
    mysql-container
    [username@dockerhost ~]$

Copy the names of your hosts.

The next step is to click back into "Setup"->"Hosts", and then into the "Docker Containers" folder.  From there, click the "Hosts" at the top, followed by the "Import hosts via CSV file".

Paste the list of hosts into your contents box, and then throw a "hostname" at the top of the list (it needs a header row), click the "Perform automatic service discovery" box, then click "Upload".

Next, make sure the CSV has recognized the "host name" column, then click on the "Import" button that shows up.  Note, this screen shot has an ugly error message because I missed the hostname assignment field due to me not knowing that it needed a header row.  I only noticed when I realized I was missing a container name.

Click the "Start" button, then wait for it to finish, and then do the usual "Changes"->"Activate pending changes" step.

Give it time, and it should load all the information on your docker containers.

The Lazy Way - Scanning The Network

By this point, my second-grader attention span has waned, and I'm just "done".  So, I went into the "hosts" from under "setup", and added a folder, and enabled a network scan for my non-Internet-routeable setup.  I did specify the IP range for that network, and then I went all "Frozen" on it and "just let it go".  Eventually pulled everything up.

Setting This Up To Use A Proxied Authentication

Now for my big frustration with Nagios in docker - I would have had to really hack to get REMOTE_USER in as the user you are in for it.  It continually asked for authentication via Basic-Auth, even after performing the Cloudflare authentication.  I REALLY wanted to keep it tied to who you actually authenticated with (auditing).  Here's how.

Log in to your Checkmk instance.  Go into the Users, and add a user in the format your proxy provides (e.g. e-mail address).

Then, under "Setup", under "General", should be a "global settings". Expand "User interface", and locate the "Authenticate users by incoming HTTP requests".

Next, check the box for "Activate HTTP header authentication", and in the "HTTP request header variable" field that pops up, enter the header name used by your access manager/proxy.

If you have Cloudflare in front of it, you would use "Cf-Access-Authenticated-User-Email" for the header.  Save that.

Now you can do into the Cloudflare portal and create a new hostname/mapping, and you have access through your portal.

Friday, July 24, 2026

Cloudflare (Personal) for Secure Tunnel + Specific Google Account Access

I'm working on getting rid of my virtual server at a service provider (cheaper, right), and I don't want to shift to a static IP or an open firewall.  NetworkChuck posted a video about setting up a "free" account with Cloudflare for Zero Trust, and that resulted in this direction.  So, here we go.

I signed up at Cloudflare following NetworkChuck's video (there were some differences - they've changed their website since he posted the video).  Then, once I had the DNS transferred over and detected, it was time to start setting it up.

Note, you still have to provide a credit card (or other method for payment such as PayPal or a bank) and "sign" the form indicating that you will pay for services that exceed the "free" version limits.  That means you could stop now if you don't want to proceed.

Okay.  Once you have your site added and the DNS transferred, on the left side menu (toward the bottom) is a "Zero Trust" link.  Click that.  You will work through their wizard for activating Zero Trust (including the above bank/card information).

Once complete, click on "Networks" on the side to expand it, and you'll see "Tunnels and Mesh".  Click on that.

Then, click "Add a Tunnel".  I selected the Cloudflare tunnel rather than a mesh tunnel.  Provide a name, then save the tunnel.

Now, if you aren't using Docker, you can select the OS type (e.g. RedHat, Debian, Windows), all of which have the instructions on installation.  Even though I am working on RedHat, I went with the Docker OS type. This will present you with the commands to run.  In order to make the docker image background, after the FIRST run in there (because the Docker image has another command for "run"), add a -d (for detach).

    docker run -d cloudflare/cloudflared:latest tunnel --no-autoupdate run --token [TOKEN]

I'd recommend storing your token somewhere safe.  Really.  Safe.

The command will start a docker image.  Now, just to be sure that this image starts after a power outage, I ran an update to the policy.  Get the policy name from docker first, then run the update :

    docker ps
    docker update --restart unless-stopped container_name

This should make it permanent.  I did rename my "docker container rename CONTAINER NEW_CONTAINER_NAME".

 Now, I followed NetChuck's instructions on the last step of setting up the tunnel a little too exact, and had some problems (bad gateway error).

So, getting the logs revealed that :

    docker container logs cloudflare -f

The above command dumped the logs and kept watching for more log updates (like tail -f), so I knew what the problem was.

    2026-07-23T15:39:03Z ERR  error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: dial tcp 10.0.17.45:8006: connect: connection refused" connIndex=2 event=1 ingressRule=0 originService=https://10.0.17.45:8006
    2026-07-23T15:39:03Z ERR Request failed error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: dial tcp 10.0.17.45:8006: connect: connection refused" connIndex=2 dest=https://hostname.domain/favicon.ico event=0 ip=131.67.22.37 type=http

It turns out the URL wasn't for the Cloudflare service, but actually for the web service I needed to connect to.  Duh.

Okay, under "Tunnels & Mesh", click on "Published application routes" at the top, then the three dots to the right of the connection we just set up, and then "Edit". 

Once fixed, save that thing, and reload it.

Now, since we don't have authentication, we'd better get OpenIDC authentication going.  I fought long and hard trying to get the Apache HTTPd docker image to load the mod_auth_openidc module so I could configure it that way, but I stopped short of rebuilding an entire httpd specifically for that.  I was getting errors loading the module if I manually mapped it from an existing ubuntu image, but that gave me errors about dynamic .so file load failures for libcrypt as well.

I rapidly gave up and in the Cloudflare dashboard (https://dash.cloudflare.com/), I went under Zero-Trust, and then hit "integrations".

In there, click on "Identity Providers" followed by "Add an Identity Provider", and then fill out the information you get from your Google (or whatever service provider you are using).  Hit the "test" button.

If it gives you an error when testing, such as :

If it gives you an error, get more details (Google likes to add a link about if you are the application developer, and that will provide a redirect URL - copy that).  Then, go into your application provider where you set up your OAuth2.0 key, and add it as a redirection URL.  If it complains about a space in it and refuses to let you save it, you've likely copied too much - just whack everything after the "callback" where the space starts, and then save it.

Next, go under "Access Controls", and "Applications".  Follow the wizard to create a new "application" - it can be a sub URI, or you can lock down the whole site (which is what I did).

Here, you will also create a policy (I did a policy that only allows specific Google e-mail addresses, both those that are in a google domain as well as one or two that are outside of it).

Once that is done, before you finish the application, if you are only authenticating against one source, I'd recommend selecting that source and not showing the prompt for multiple sources.  However, if you are going to accept users from other locations (such as Entra, etc), leave those as-is. 

Suddenly, you should have immediate access if you've logged in to Google.

The next thing I had to figure out was how to get Grafana to recognize the user.  And that meant going into the Grafana docker and changing the GF_AUTH_PROXY_HEADER_NAME value :

     - GF_AUTH_PROXY_HEADER_NAME=Cf-Access-Authenticated-User-Email

After restarting the docker image, this let me use the username in the dashboards for Grafana, and view my data.  Sweet!