Docker, Apache HTTPd, and Perl with DBI

Okay.  I swore about crap all the day long, trying to get CGI's to write to a database using the default httpd:2.4 docker image.

NOTHING worked.

Until I found some obscure post referencing extending an image, I was at a loss.  Then, with the reference to "extending an image", I knew what to search for, and finally got it.  Here's what to do.  You can "create" a new image using a "build", e.g. create a Dockerfile containing :

# Sample Dockerfile for extending a container
FROM httpd:latest
RUN apt-get update && apt-get install -y libdbd-mysql-perl
CMD ["httpd-foreground"]

The file says to take the httpd:latest, run a command on it, then set the command to run (could already be there, I don't know - it just worked and make me very happy).

Then, run :

docker build -t httpd:mysql .

After that, change your docker-compose.yml to reference your new image, and then you should be able to make it function.  Granted, CGI's need the REMOTE_USER if you are doing anything specific with that, so add the config where you need it to pull the authenticated user from the Cloudflare header containing the username and stuff it.

# set the REMOTE_USER variable
RewriteCond %{HTTP:Cf-Access-Authenticated-User-Email} ^(.*)$
RewriteRule .* - [E=REMOTE_USER:%1]

Then stop the container and start it back up, and that part should be functional.  The bonus is you can extend all you want. 

Cloudflare (Personal) for Secure Tunnel + Specific Google Account Access

I'm working on getting rid of my virtual server at a service provider (cheaper, right), and I don't want to shift to a static IP or an open firewall.  NetworkChuck posted a video about setting up a "free" account with Cloudflare for Zero Trust, and that resulted in this direction.  So, here we go.

I signed up at Cloudflare following NetworkChuck's video (there were some differences - they've changed their website since he posted the video).  Then, once I had the DNS transferred over and detected, it was time to start setting it up.

Note, you still have to provide a credit card (or other method for payment such as PayPal or a bank) and "sign" the form indicating that you will pay for services that exceed the "free" version limits.  That means you could stop now if you don't want to proceed.

Okay.  Once you have your site added and the DNS transferred, on the left side menu (toward the bottom) is a "Zero Trust" link.  Click that.  You will work through their wizard for activating Zero Trust (including the above bank/card information).

Once complete, click on "Networks" on the side to expand it, and you'll see "Tunnels and Mesh".  Click on that.

Then, click "Add a Tunnel".  I selected the Cloudflare tunnel rather than a mesh tunnel.  Provide a name, then save the tunnel.

Now, if you aren't using Docker, you can select the OS type (e.g. RedHat, Debian, Windows), all of which have the instructions on installation.  Even though I am working on RedHat, I went with the Docker OS type. This will present you with the commands to run.  In order to make the docker image background, after the FIRST run in there (because the Docker image has another command for "run"), add a -d (for detach).

docker run -d cloudflare/cloudflared:latest tunnel --no-autoupdate run --token [TOKEN]

I'd recommend storing your token somewhere safe.  Really.  Safe.

The command will start a docker image.  Now, just to be sure that this image starts after a power outage, I ran an update to the policy.  Get the policy name from docker first, then run the update :

docker ps
docker update --restart unless-stopped container_name

This should make it permanent.  I did rename my "docker container rename CONTAINER NEW_CONTAINER_NAME".

 Now, I followed NetChuck's instructions on the last step of setting up the tunnel a little too exact, and had some problems (bad gateway error).

So, getting the logs revealed that :

docker container logs cloudflare -f

The above command dumped the logs and kept watching for more log updates (like tail -f), so I knew what the problem was.

2026-07-23T15:39:03Z ERR  error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: dial tcp 10.0.17.45:8006: connect: connection refused" connIndex=2 event=1 ingressRule=0 originService=https://10.0.17.45:8006
2026-07-23T15:39:03Z ERR Request failed error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: dial tcp 10.0.17.45:8006: connect: connection refused" connIndex=2 dest=https://hostname.domain/favicon.ico event=0 ip=131.67.22.37 type=http

It turns out the URL wasn't for the Cloudflare service, but actually for the web service I needed to connect to.  Duh.

Okay, under "Tunnels & Mesh", click on "Published application routes" at the top, then the three dots to the right of the connection we just set up, and then "Edit". 

Once fixed, save that thing, and reload it.

Now, since we don't have authentication, we'd better get OpenIDC authentication going.  I fought long and hard trying to get the Apache HTTPd docker image to load the mod_auth_openidc module so I could configure it that way, but I stopped short of rebuilding an entire httpd specifically for that.  I was getting errors loading the module if I manually mapped it from an existing ubuntu image, but that gave me errors about dynamic .so file load failures for libcrypt as well.

I rapidly gave up and in the Cloudflare dashboard (https://dash.cloudflare.com/), I went under Zero-Trust, and then hit "integrations".

In there, click on "Identity Providers" followed by "Add an Identity Provider", and then fill out the information you get from your Google (or whatever service provider you are using).  Hit the "test" button.

If it gives you an error when testing, such as :

If it gives you an error, get more details (Google likes to add a link about if you are the application developer, and that will provide a redirect URL - copy that).  Then, go into your application provider where you set up your OAuth2.0 key, and add it as a redirection URL.  If it complains about a space in it and refuses to let you save it, you've likely copied too much - just whack everything after the "callback" where the space starts, and then save it.

Next, go under "Access Controls", and "Applications".  Follow the wizard to create a new "application" - it can be a sub URI, or you can lock down the whole site (which is what I did).

Here, you will also create a policy (I did a policy that only allows specific Google e-mail addresses, both those that are in a google domain as well as one or two that are outside of it).

Once that is done, before you finish the application, if you are only authenticating against one source, I'd recommend selecting that source and not showing the prompt for multiple sources.  However, if you are going to accept users from other locations (such as Entra, etc), leave those as-is. 

Suddenly, you should have immediate access if you've logged in to Google.

The next thing I had to figure out was how to get Grafana to recognize the user.  And that meant going into the Grafana docker and changing the GF_AUTH_PROXY_HEADER_NAME value :

 - GF_AUTH_PROXY_HEADER_NAME=Cf-Access-Authenticated-User-Email

After restarting the docker image, this let me use the username in the dashboards for Grafana, and view my data.  Sweet!

Docker and MQTT (Mosquito)

My Mosquito implementation is not heavily modified.  In fact, it's so simple, I could do a :

docker run -d ---name mosquitto -p 1883:1883 eclipse-mosquitto:2

This would be just fine.  However, I want my docker image to restart after reboots, and I'm too lazy to go about it any other way than a docker-compose.yml :

# docker-compose.yml - Mosquitto MQTT broker
services:
  mosquitto:
    image: eclipse-mosquitto:2
    container_name: mqtt
    ports:
      # Standard MQTT protocol
      - "1883:1883"
      # MQTT over WebSocket
      - "9001:9001"
    #volumes:
      # Mount configuration directory
      #- ./config:/mosquitto/config
      # Persist message data
      #- ./data:/mosquitto/data
      # Persist log files
      #- ./log:/mosquitto/log
    restart: unless-stopped

Note the entire "volumes" section has been commented out.  I started to add it, and realized I didn't care about persisting message or log data, nor was I running a custom configuration.  So, I left it out.

Then, start it up in the usual fashion with a :

docker compose up -d

And off you go! 

Tempest Weatherflow - Data Mapping into MySQL

I'm slowly migrating off of a virtual server at a service provider.  Most things have been moving to docker.  In this case, it's going onto the docker server, but not going to run in docker itself.

It has to be on the same network as the Weatherflow device to be able to pick up data, and me keeping it on the unrouteable network meant this docker platform needed to be on two separate networks.  So, I fired off the command to create a new connection on the same interface.

nmcli connection add con-name unrouteable ifname enp1s0 type ethernet ipv4.method manual ipv4.addresses 192.168.2.10/24 ipv4.never-default yes

Unfortunately, this resulted in the system not being able to boot onto the network.  So, it was backed out, and I moved the weather hub off of the unrouteable to the regular network.  The code below, I simply could NOT get to arrive in any docker image, or running directly on the docker server (outside of a container).  However, after a lot of troubleshooting to verify that the broadcast packets were getting there using :

sudo tcpdump -i enp1s0 udp port 50222 -Xvvv

I finally realized that RedHat had firewall-cmd running by default.  Now, I'm trying to make all this work with defaults, which means that instead of disabling it, I'm going to open the port :

firewall-cmd --permanent --add-port=50222/udp
firewall-cmd --reload
firewall-cmd --list-all

You should see the ports: line now containing "50222/udp".  And, if you run the server, it should show it's receiving packets.  Okay, back to building this "listener".  I went down this whole path of making it run an exec() to curl, which made it really easy to send the received JSON to a perl script and customize whatever you needed from there.  Yes, at the very bottom of the code I have commented out a "sendJSON" function, and instead I'm using a handleJSON function that will do the parsing, so if you need to flip that switch because you are using a non-docker version of HTTPd and Perl, you can install the package and make it happen.

Since the Perl instances in the Apache/HTTPd 2.4 instances ("httpd 2.4") do NOT include either the Perl JSON module, or the DBD::MySQL module, it ultimately means you can't send this to Apache and deal with it nicely there if you are also using native docker.  That leaves one choice - write a UDP "listener" (duh, UDP packages don't "listen" on UDP ports, because there is no connection/handshake).

So, I wrote the dumb thing in perl. I'd already had the UDP side, but since it might as well do the local database insertion, whelp, that's what I did.

Fancy thing is that I can specify which datagram types to accept from the Tempest system.  I really only care about lightning strikes and obs_st (standard metrics).  But, I slapped SQL for all, and did a fancy ".disabled" extension to hide the ones I wanted.  If I don't have a disabled, when it sees a packet, it will dump the hash_ref so that I can easily write one if I need it.  I just wrote what I saw, then disabled most of them.

Here's the code for the listener :

#!/usr/bin/perl -w

# written by Joe Lewis <joe@joe-lewis.com> on November 29, 2022
# A UDP listener for the tempest weather system. The API for weatherflow (maker
# of the tempest system) can be found at :
#     https://weatherflow.github.io/Tempest/api/udp.html
# if on RHEL, install :
#   perl-libwww-perl
#   perl-Sys-Syslog.x86_64
# for RHEL, this uses the template tool kit (available in the EPEL repository)
#   sudo subscription-manager repos --enable codeready-builder-for-rhel-9-x86_64-rpms
#   sudo dnf install https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm
#   sudo yum install perl-Template-Toolkit.x86_64


use strict;
use warnings;
use Data::Dumper;
use DBI;
use Getopt::Long;
use HTTP::Request;
use HTTP::Response;
use IO::Socket;
use JSON;
use LWP::UserAgent;
use POSIX;
use Sys::Syslog qw(:DEFAULT :standard :macros);
use Template;


my $template_directory = 'sql';
my $mysql_hostname = '127.0.0.1'; # if you use "localhost", it will try the unix socket
my $mysql_username = 'user_only_having_insert_or_update';
my $mysql_password = 'users_password';
my $mysql_database = 'weather';

my $verbose = undef;
my $daemonize = undef;

GetOptions(
  'sql-directory=s' => \$template_directory,
  'daemonize' => \$daemonize,
  'verbose' => \$verbose,
);


my($sock, $newmsg, $MAXLEN, $PORTNO);
my $json = JSON->new->allow_nonref;
my $userAgent = LWP::UserAgent->new(timeout => 10);
my $postURL = 'https://protected.silverhawk.net/cgi-bin/weatherflow.pl';
my $postContentType = 'application/json';
my $name = 'WeatherFlow-UDP-conv';
my $nagios_command_file = '/var/lib/nagios3/rw/nagios.cmd';
my $error_log_qfn = '/var/log/'.$name.'.log';
my $pid_file_qfn = '/var/run/'.$name.'.pid';

$MAXLEN = 1024;
$PORTNO  = 50222;

sub sighup_handler {
	exit;
};

sub sigusr1_handler {
	exit;
};

sub logMessage {
	my $msg = shift;
	$msg .= "\n" unless $msg =~ /\n/;
	syslog('info',$msg);
};

sub handleUDPPacket {
	my $weatherstationname = shift;
	my $weatherstationclient = shift;
	my $data = shift;
	my $href = $json->decode($data);
	my $weatherstationip = inet_ntoa($weatherstationclient);
	# just for my sanity, insert the hostname
	$href->{'client_hostname'} = $weatherstationname;
	$href->{'client_ip'} = $weatherstationip;

	my $sqlTemplate = $template_directory.'/'.$href->{type}.'.sql';
	if (-f $sqlTemplate) {
		my $tt = Template->new({INCLUDE_PATH=>$template_directory,INTERPOLATE => 1}) or die "Error launching the template toolkit : $Template::ERROR\n";
		# establish a MySQL connection
		my $dbh = DBI->connect('dbi:mysql:host='.$mysql_hostname.':database='.$mysql_database.':mysql_ssl=1',$mysql_username,$mysql_password);
		$href->{dbh} = $dbh;
		my $SQL;
		$tt->process($href->{type}.'.sql',$href,\$SQL) or warn 'Template Processing Failure: '.$tt->error()."\n";
		if ($SQL) {
			$dbh->do($SQL) or warn "Error running SQL on type $href->{type}: $DBI::errstr, SQL :\n\t$SQL\n";
		}
		$dbh->disconnect();
	} else {
		# if we have a disabled SQL file, do nothing because we're intentionally swallowing it.
		if (!-f $sqlTemplate.'.disabled') {
			# no template defined, warn it
			warn "non-existant SQL template $sqlTemplate containing:\n".Dumper($href);
		}
	};
};

openlog("",'ndelay','daemon');
if (defined($daemonize)) {
	my $pid = fork();
	exit if ($pid > 0);
	setsid();
	$pid = fork();
	exit if ($pid > 0);
};
$SIG{HUP} = \&sighup_handler;
$SIG{USR1} = \&sigusr1_handler;

$sock = IO::Socket::INET->new(
           LocalPort => $PORTNO, 
       Proto => 'udp') or die "socket: $@";
       
logMessage("Awaiting UDP messages on port $PORTNO");

while ($sock->recv($newmsg, $MAXLEN)) {
	my($clientport, $clientaddr) = sockaddr_in($sock->peername);
	my $clienthost = gethostbyaddr($clientaddr, AF_INET);
	handleUDPPacket($clienthost,$clientaddr,$newmsg);
}
my $status = $!;
logMessage("Loop completed, exiting with $status");
die "recv: $status";

Now, that's relatively simple.  I created the separate directory for the SQL's specific to the Tempest functionality, and then piled a bunch of files in there :

*** sql/device_status.sql.disabled
INSERT INTO device_status (timestamp,serial_number,client_ip,client_hostname,hub_sn,voltage,rssi,hub_rssi,sensor_status,firmware_revision) VALUES (from_unixtime([% timestamp %]),[% dbh.quote(serial_number) %],[% dbh.quote(client_ip) %],[% dbh.quote(client_hostname) %],[% dbh.quote(hub_sn) %],[% voltage %],[% rssi %],[% hub_rssi %],[% sensor_status %],[% firmware_revision %])

*** sql/evt_precip.sql.disabled
INSERT INTO evt_precip (timestamp,station_id,hub_sn) VALUES (from_unixtime([% evt.0 %]),[% dbh.quote(serial_number) %],[% dbh.quote(hub_sn) %)

*** sql/evt_strike.sql
INSERT INTO lightning_events (epoch,station_id,energy,distance_km,distance_miles) VALUES (from_unixtime([% evt.0 %]),[% dbh.quote(serial_number) %],[% evt.2 %],[% evt.1 %],[% evt.1/1.6093445 %])

*** sql/hub_status.sql.disabled
INSERT INTO hub_status (timestamp,serial_number,frequency,rssi,client_ip,uptime,firmware_revision,hw_version,client_hostname,radio_version,reboot_count,i2c_bus_error_count,radio_status,radio_network_id) VALUES (from_unixtime([% timestamp %]),[% dbh.quote(serial_number) %],[% freq %],[% rssi %],[% dbh.quote(client_ip) %],[% uptime %],[% dbh.quote(firmare_revision) %],[% hw_version %],[% dbh.quote(client_hostname) %],[% radio_stats.0 %],[% radio_stats.1 %],[% radio_stats.2 %],[% radio_stats.3 %],[% radio_stats.4 %])

*** sql/obs_air.sql.disabled
INSERT INTO obs_air (timestamp,serial_number,firmware_revision,hub_sn,station_pressure_mb,station_pressure_inhg,temperature_celcius,temperature_farenheit,rel_humidity,lightning_strike_count,lightning_strike_avg_distance_km,lightning_strike_avg_distance_miles,battery_voltage,report_interval) VALUES (from_unixtime([% obs.0.0 %]),[% dbh.quote(serial_number) %],[% firmware_revision %],[% dbh.quote(hub_sn) %],[% obs.0.1 %],([% obs.0.1 %]*0.029530)+4.7634625,[% obs.0.2 %],([% obs.0.2 %]*1.8)+32,[% obs.0.3 %],[% obs.0.4 %],[% obs.0.5 %],[% obs.0.5 %]/0.621371,[% obs.0.6],[% obs.0.7 %])

*** sql/obs_sky.sql.disabled
INSERT INTO obs_sky (timestamp,serial_number,firmware_revision,hub_sn,illuminance_lux,uv_index,rain_previous_minute_mm,rain_previous_minute_in,wind_lull_mps,wind_lull_mph,wind_avg_mps,wind_avg_mph,wind_gust_mps,wind_gust_mph,wind_direction,battery_voltage,report_interval,solar_radiation,day_rain_accumulation_mm,precipitation_type,wind_sample_interval) VALUES (from_unixtime([% obs.0.0 %]),[% dbh.quote(serial_number) %],[% firmware_revision %],[% dbh.quote(hub_sn) %],[% obs.0.1 %],[% obs.0.2 %],[% obs.0.3 %],[% obs.0.3 %]/25.4,[% obs.0.4 %],[% obs.0.4 %]*2.23694,[% obs.0.5 %],[% obs.0.5 %]*2.23694,[% obs.0.6 %],[% obs.0.6 %]*2.23694,[% obs.0.7 %],[% obs.0.8 %],[% obs.0.9 %],[% obs.0.10 %],[% obs.0.11 %],[% obs.0.12 %],[% obs.0.13 %])

*** sql/obs_st.sql
INSERT INTO twenty_four_hour (epoch,station_id,wind_lull_metric,wind_lull_standard,wind_avg_metric,wind_avg_standard,wind_gust_metric,wind_gust_standard,wind_direction,wind_sample_interval,barometric_pressure_milibars,barometric_pressure_inHg,temperature_celcius,temperature_farenheit,relative_humidity,illuminance,ultraviolet_index,solar_radiation,rain_amount_metric,rain_amount_standard,precipitation_type,lightning_avg_distance_metric,lightning_avg_distance_standard,lightning_strike_count,battery_voltage,report_interval) VALUES (from_unixtime([% obs.0.0 %]),[% dbh.quote(serial_number) %],[% obs.0.1 %],[% obs.0.1 %]*2.23694,[% obs.0.2 %],[% obs.0.2 %]*2.23694,[% obs.0.3 %],[% obs.0.3 %]*2.23694,[% obs.0.4 %],[% obs.0.5 %],[% obs.0.6 %],([% obs.0.6 %]*0.029530)+4.7634625,[% obs.0.7 %],([% obs.0.7 %]*1.8)+32,[% obs.0.8 %],[% obs.0.9 %],[% obs.0.10 %],[% obs.0.11 %],[% obs.0.12 %],[% obs.0.12 %]/25.4,[% obs.0.13 %],[% obs.0.14 %],[% obs.0.14 %]/0.621371,[% obs.0.15 %],[% obs.0.16 %],[% obs.0.17 %])

*** sql/obs_st.sql.original
INSERT INTO obs_st (timestamp,serial_number,firmware_revision,hub_sn,wind_lull_mps,wind_lull_mph,wind_avg_mps,wind_avg_mph,wind_gust_mps,wind_gust_mph,wind_direction,wind_sample_interval,station_pressure_mb,station_pressure_inhg,temperature_c,temperature_f,relative_humidity,illuminance_lux,uv,solar_radiation,rain_previous_minute_mm,rain_previous_minute_in,precip_type,lightning_strike_avg_distance_km,lightning_avg_distance_miles,lightning_strike_count,battery,report_interval) VALUES (from_unixtime([% obs.0.0 %]),[% dbh.quote(serial_number) %],[% firmware_revision %],[% dbh.quote(hub_sn) %],[% obs.0.1 %],[% obs.0.1 %]*2.23694,[% obs.0.2 %],[% obs.0.2 %]*2.23694,[% obs.0.3 %],[% obs.0.3 %]*2.23694,[% obs.0.4 %],[% obs.0.5 %],[% obs.0.6 %],([% obs.0.6 %]*0.029530)+4.7634625,[% obs.0.7 %],([% obs.0.7 %]*1.8)+32,[% obs.0.8 %],[% obs.0.9 %],[% obs.0.10 %],[% obs.0.11 %],[% obs.0.12 %],[% obs.0.12 %]/25.4,[% obs.0.13 %],[% obs.0.14 %],[% obs.0.14 %]/0.621371,[% obs.0.15 %],[% obs.0.16 %],[% obs.0.17 %])

*** sql/rapid_wind.sql.disabled
INSERT INTO rapid_wind (epoch,station_id,speed_mps,speed_mph,direction) VALUES (from_unixtime([% ob.0 %]),[% dbh.quote(serial_number) %],[% ob.1 %],[% ob.1 %]*2.236936,[% ob.2 %])

And to ensure it starts on boot (this is outside of docker - meh, I was so frustrated that I'm fine with it), I created an /etc/systemd/system/weatherflow_udp.service file containing :

[Unit]
Description=UDP Listener for Tempest Weatherflow
After=default.target

[Service]
Type=oneshot
ExecStart=/where/ever/you/stored/it/tempest/perl/udp_listener.pl --daemonize --sql-directory /where/ever/you/stored/it/tempest/perl/sql/

[Install]
WantedBy=default.target

And then enabled the service :

systemctl daemon-reload
systemctl enable weatherflow_udp

Viola (or Cello, I don't care), I have data!

One step closer!

Docker and WeeWx

WeeWX - Weather Station Data Gathering

Note, I eventually gave up on installing the weatherflow-udp extention under docker, and just wrote a UDP-to-CGI bridge tool, documented later.  However, I opted to post this anyway because of what I ran into and it could be useful for someone out there (I got fairly close, actually).

This is an interesting tool to accept data for the weather system. I'm using a weatherflow tempest, which means I have a UDP port I have to translate.  Plus, it is a little more complicated than the others we've dealt with.  First, let's create a pre-run form of docker-connect.yaml :

services:
  weewx:
    image: felddy/weewx:latest
    container_name: tempest
    restart: unless-stopped
    ports:
      - "50222:50222/udp" # Map WeatherFlow UDP port
    environment:
      - TZ=America/Boise
    volumes:
      - type: bind
        source: ./weewx-data
        target: /data 

Then, let's create our data directory to grab configuarion files from :

mkdir weewx-data/
chown 1000:1000 weewx-data

Now we can run a "fake" start of the docker image to get the configurations we need :

docker compose run --rm weewx

At this point, the weewx-data/weewx.conf file is ready for manipulation.  Copy the weewx.conf into the docker image directory, then modify it by changing the station name, the latitude and longitude, etc.  Once ready, change your docker-compose.yml file by adding the bind volume for the weewx.conf file, e.g. :

    volumes:
      - type: bind
        source: ./weewx-data
        target: /data

becomes :

    volumes:
      - type: bind
        source: ./weewx-data
        target: /data

Then, start the docker (usual "docker compose up -d").

It is now possible to run and install extensions.  First, just to make sure it's showing, run :

docker exec -it tempest weectl extension list

Then install with : 

docker compose run --rm weewx extension install --yes https://github.com/captain-coredump/weatherflow-udp/archive/refs/heads/master.zip 

Now, it was right about here that I gave up on getting the extension installed.  I'm leaving my notes, just in case.  But really, all I need is a UDP translator, anyway.  I'm not trying to do anything fancy like serve weather data (that will all happen through Grafana, so, yeah).\

On giving up, I just wrote a UDP listener.

Docker and Grafana

Managers love pretty pictures, right?  I know, this is a personal intranet, but hey, why not?

Grafana was a bit more of a challenge to run in docker.  I have issues with it being a default non-SSL port, but we can potentially fix that by throwing Apache in front of it down the road.  But, it's the same process as before, but with a major caveat.

We're not doing much special here, but I do need to make a quick statement.  My Grafana was an older 9.something version.  If you run it in Docker, and top are not starting from scratch (e.g. you are going to copy from a backup), it will fail.  In this case, you MUST download a 12.3 package, and do a debian upgrade on that package, and stop/restart so that it can migrate.  THEN you can move to the Docker 12.4 image base.

Now, back to the regularly scheduled programming.

Simply copy over your /var/lib/grafana/grafana.db SQLite file.  If you have customizations in your grafana.ini file, you can make those changes to a grafana.ini file and map it just like we did with Apache HTTPd above.  Note that it will be better to do that old "cat" command we mentioned in Apache to get a copy and then modify that one.

My docker-compose.yml looks like :

 services:
  web:
    image: grafana/grafana
    container_name: grafana
    restart: unless-stopped
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_USER: admin
      - GF_SECURITY_ADMIN_PASSWORD: admin
      - GF_SERVER_ROOT_URL=%(protocol)s://%(domain)s:%(http_port)s/grafana/
      - GF_SERVER_ENFORCE_DOMAIN=false
      - GF_AUTH_PROXY_ENABLED=true
      - GF_AUTH_PROXY_HEADER_NAME=X-WEBAUTH-USER
      - GF_AUTH_PROXY_HEADER_PROPERTY=username
      - GF_AUTH_PROXY_AUTO_SIGN_UP=true
    volumes:
      - /opt/docker/storage/grafana:/var/lib/grafana:rw

Note the environment variables.

• The SECURITY_ADMIN stuff sets  your default username/password.  If you are not migrating, and are instead creating a brand-new instance, these will set the default username/password.
• The GF_SERVER_ROOT_URL is used to change the default URL (since Grafana is so smart it is going to redirect you to the Grafana port number, and this specifies whoat to hit with).
• The GF_SERVER_ENFORCE_DOMAIN - this is where the AI engines kept biting me.  They had NO CLUE about this.  Grafana LOVES to redirect you, and if the enforce_domain in the grafana.ini uses the default or is set to true, it's doing to redirect you if you didn't hit it with the domain (could be hostname, and in docker, this defaults to "localhost").  So, if you are using docker, and if you are also reverse proxy'ing Grafana, which, in a reverse proxy environment, is not good).  It kept redirecting me to http://localhost:3000/ .  Setting this value via GF_SERVER_ENFORCE_DOMAIN allowed me to override the enforce_domain and shut that stupid thing off.
• The GF_AUTH_PROXY_* environmment variables are to disable forcing a login.  If you are reverse proxy'ing Grafana, and the parent process is handling the authentication, this is likely what you will need to do in combination with the ReverseProxy configs I showed earlier that use the rewrite_module under Apache HTTPd.

Then it's a matter of starting it up again (the "docker compose up -d" command).

Docker and MySQL

In my pursuit of an easier, personal "intranet", my next task was to get MySQL running.  It's critical for the Grafana and Nagios instances that run my environmental monitoring.  There is a lot of historical (hysterical?) data involved, so we'll make it easier and just migrate it over, right?

First, create your docker directory for MySQL :

mkdir -p /opt/docker/mysql

Then, create only one file in there, the infamous docker-compose.yml :

services:
  mysql:
    image: mysql:latest
    container_name: mysql-container
    environment:
      MYSQL_ROOT_PASSWORD: super-secret-pwd
      # uncomment below if you want to automatically create and select a database
      # MYSQL_DATABASE: database_name
    ports:
      - "3306:3306"
    volumes:
      - /opt/docker/storage/mysql:/var/lib/mysql

Change the MYSQL_ROOT_PASSWORD row in the docker-compose.yml, and you're done.

Now, we are ready to spin that sucker up already - it really is that simple.

docker compose up -d

Give it a minute to spin it up.  Then, test the connection (no MySQL Unix-socket will be available, so your MySQL command has to use the "--protocol TCP" option :

[jlewis@docker mysql]$ mysql -u root -p -h localhost --protocol TCP
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 9
Server version: 9.7.1 MySQL Community Server - GPL

Copyright (c) 2000, 2026, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
4 rows in set (0.01 sec)

mysql> exit 

Create whatever you need in your database, and you have a modular database that can be quickly moved over to another instance of docker if CPU gets too high.  In my case, I'm migrating an existing MySQL database, so I'll run : 

mysqldump -u root -p databasename > mysqldump-databasename.sql

Copy that to the location that can write this into the database, create the database ("CREATE DATABASE databasename" in the new MySQL server first) and import it :

mysql -u root -p --protocol TCP databasename > mysqldump-databasename.sql

Doing pretty good so far. 

Docker - Apache HTTPd for my tiny Intranet

I'm rebuilding my network stack for my little intranet, and the first thing out of the gate is the HTTPd web service.  So, let's get started.

I'm not using NginX because there will be a lot more CGI, and also because I want to implement my template mechanism, a custom module for the Apache HTTPd serer.  Converting from an existing HTTPd configuration is a bit more difficult when it comes to migrating it in to Docker.

Now, RedHat and other Linux vendors/distributors love to part the httpd.conf file out into a whole lot of directories.

Docker doesn't do that.  However, we have a little secret.  So, let's create our docker image first.  Create your docker directory and then the container for this specific image.

mkdir -p /opt/docker/httpd

Because the docker image is actually based on a Debian version, and because it includes an httpd.conf file already (by default), we only need three files in this directory.

For the httpd.conf, I found it easier to grab the httpd.conf file out of the docker image first, then add my custom module.  And, since I wanted the image to serve content from the NAS, I needed to get the web content from that mounted up.  Let's grab a copy of the docker container httpd.conf file.

docker run --rm httpd:2.4 cat /usr/local/apache2/conf/httpd.conf > httpd.conf

Now, with that, I can add my own module using the LoadModule with any custom directives.  Then, we simply add the NFS mounts for the mount the NAS up via NFS, and then use volumes, too.  Enter the docker compose. 

services:
  web:
    image: httpd:2.4
    container_name: httpd
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /opt/docker/storage/http/html:/usr/local/apache2/htdocs/:ro
      - /opt/docker/storage/http/cgi-bin:/usr/local/apache2/cgi-bin/:ro
      - /opt/docker/storage/http/logs:/usr/local/apache2/logs/:rw
      - ./certs:/etc/apache2/ssl
      - ./httpd.conf:/usr/local/apache2/conf/httpd.conf:ro
      - ./mod_template.so:/usr/local/apache2/modules/mod_template.so:ro

The container name/image, are fairly inconsequential - you can name them anything you want.  The default is "apache_service" for this from the Docker image, but I like to get very specific because Apache likes to do a lot of services.

The "restart" line is there in case the EliteDesk hardware loses power and reboots - I want this container to start back up immediately.

The ports are very easy to understand if you are used to TCP.  It is formatted as "HOST PORT:CONTAINER PORT".

Also, we need to set up SSL (note we have port 443 in there).  Create a certificate :

mkdir certs
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout certs/httpd.key -out certs/httpd.crt -subj "/C=US/ST=Idaho/L=Boise/O=SilverHawk/CN=silverhawk.net"

You will need to set up the standard VirtualHost for it (make sure you uncomment the LoadModule line for the ssl extension while in there) :

<VirtualHost *:443>
        DocumentRoot "/usr/local/apache2/htdocs"
        #ServerName www.example.com
        SSLEngine On
        SSLCertificateFile /etc/apache2/ssl/httpd.crt
        SSLCertificateKeyFile /etc/apache2/ssl/httpd.key
</VirtualHost>

While I was there, I also added proxy and proxypass information for Grafana, (that's coming - note you need to enable xml2enc_module, proxy_http_module, and proxy_module, and make sure you disable proxy for everything except your own proxt so that someone doesn't suddenly use your site as a personal proxy to mask what they are doing on the Internet).

A grafana note :

I really strugged with proxy pass in this situation.  all of the AI engines kept telling me things that were not accurate.  So, here's what I have.  In my httpd.conf for docker, I have the following excerpt :

ProxyRequests Off
<VirtualHost *:443>
        DocumentRoot "/usr/local/apache2/htdocs"
        #ServerName www.example.com
        SSLEngine On
        SSLCertificateFile /etc/apache2/ssl/httpd.crt
        SSLCertificateKeyFile /etc/apache2/ssl/httpd.key

        RewriteEngine On
        RewriteRule .* - [E=PROXY_USER:%{LA-U:REMOTE_USER},NS]
        RequestHeader set X-WEBAUTH-USER "%{REMOTE_USER}e"
        ProxyPreserveHost On
        ProxyPass /grafana/ http://172.17.0.1:3000/
        ProxyPassReverse /grafana/ http://172.17.0.1:3000/
        RequestHeader unset Accept-Encoding
        <Location /charts>
                TemplateEnabled off
        </Location>
</VirtualHost>

Then, in my Grafana docker-compose.yml file, I had to add the GF_SERVER_ROOT_URL and GF_SERVER_ENFORCE_DOMAIN environment variables (there are some others, to set up anonymous browsing since this will be front-ended by an authentication later) :

    environment:
      - GF_SERVER_ROOT_URL=%(protocol)s://%(domain)s:%(http_port)s/grafana/
      - GF_SERVER_ENFORCE_DOMAIN=false
      - GF_AUTH_PROXY_ENABLED=true
      - GF_AUTH_PROXY_HEADER_NAME=X-WEBAUTH-USER
      - GF_AUTH_PROXY_HEADER_PROPERTY=username
      - GF_AUTH_PROXY_AUTO_SIGN_UP=true

Volumes are where the magic happens for the custom module and configuration as well as the web root.  Each row represents a specific file/directory that is mapped from the docker image.  For example, I have my mod_template.so in there so it gets mapped into the container, plus I am mapping an httpd.conf file, as well as two NFS directories so I don't have to restart the docker every time I change the website.

Then, it's simply a matter of launching it up.

sudo docker compose up -d  

Note, if you run into errors, you can use (and there is a -f option to tail -f them) :

docker compose logs

This will dump the containers log files (I suppose you COULD re-map log files if you want those pulled out of the image naturally, but this was easier).