Showing posts with label grafana. Show all posts
Showing posts with label grafana. Show all posts

Friday, July 24, 2026

Cloudflare (Personal) for Secure Tunnel + Specific Google Account Access

I'm working on getting rid of my virtual server at a service provider (cheaper, right), and I don't want to shift to a static IP or an open firewall.  NetworkChuck posted a video about setting up a "free" account with Cloudflare for Zero Trust, and that resulted in this direction.  So, here we go.

I signed up at Cloudflare following NetworkChuck's video (there were some differences - they've changed their website since he posted the video).  Then, once I had the DNS transferred over and detected, it was time to start setting it up.

Note, you still have to provide a credit card (or other method for payment such as PayPal or a bank) and "sign" the form indicating that you will pay for services that exceed the "free" version limits.  That means you could stop now if you don't want to proceed.

Okay.  Once you have your site added and the DNS transferred, on the left side menu (toward the bottom) is a "Zero Trust" link.  Click that.  You will work through their wizard for activating Zero Trust (including the above bank/card information).

Once complete, click on "Networks" on the side to expand it, and you'll see "Tunnels and Mesh".  Click on that.

Then, click "Add a Tunnel".  I selected the Cloudflare tunnel rather than a mesh tunnel.  Provide a name, then save the tunnel.

Now, if you aren't using Docker, you can select the OS type (e.g. RedHat, Debian, Windows), all of which have the instructions on installation.  Even though I am working on RedHat, I went with the Docker OS type. This will present you with the commands to run.  In order to make the docker image background, after the FIRST run in there (because the Docker image has another command for "run"), add a -d (for detach).

    docker run -d cloudflare/cloudflared:latest tunnel --no-autoupdate run --token [TOKEN]

I'd recommend storing your token somewhere safe.  Really.  Safe.

The command will start a docker image.  Now, just to be sure that this image starts after a power outage, I ran an update to the policy.  Get the policy name from docker first, then run the update :

    docker ps
    docker update --restart unless-stopped container_name

This should make it permanent.  I did rename my "docker container rename CONTAINER NEW_CONTAINER_NAME".

 Now, I followed NetChuck's instructions on the last step of setting up the tunnel a little too exact, and had some problems (bad gateway error).

So, getting the logs revealed that :

    docker container logs cloudflare -f

The above command dumped the logs and kept watching for more log updates (like tail -f), so I knew what the problem was.

    2026-07-23T15:39:03Z ERR  error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: dial tcp 10.0.17.45:8006: connect: connection refused" connIndex=2 event=1 ingressRule=0 originService=https://10.0.17.45:8006
    2026-07-23T15:39:03Z ERR Request failed error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: dial tcp 10.0.17.45:8006: connect: connection refused" connIndex=2 dest=https://hostname.domain/favicon.ico event=0 ip=131.67.22.37 type=http

It turns out the URL wasn't for the Cloudflare service, but actually for the web service I needed to connect to.  Duh.

Okay, under "Tunnels & Mesh", click on "Published application routes" at the top, then the three dots to the right of the connection we just set up, and then "Edit". 

Once fixed, save that thing, and reload it.

Now, since we don't have authentication, we'd better get OpenIDC authentication going.  I fought long and hard trying to get the Apache HTTPd docker image to load the mod_auth_openidc module so I could configure it that way, but I stopped short of rebuilding an entire httpd specifically for that.  I was getting errors loading the module if I manually mapped it from an existing ubuntu image, but that gave me errors about dynamic .so file load failures for libcrypt as well.

I rapidly gave up and in the Cloudflare dashboard (https://dash.cloudflare.com/), I went under Zero-Trust, and then hit "integrations".

In there, click on "Identity Providers" followed by "Add an Identity Provider", and then fill out the information you get from your Google (or whatever service provider you are using).  Hit the "test" button.

If it gives you an error when testing, such as :

If it gives you an error, get more details (Google likes to add a link about if you are the application developer, and that will provide a redirect URL - copy that).  Then, go into your application provider where you set up your OAuth2.0 key, and add it as a redirection URL.  If it complains about a space in it and refuses to let you save it, you've likely copied too much - just whack everything after the "callback" where the space starts, and then save it.

Next, go under "Access Controls", and "Applications".  Follow the wizard to create a new "application" - it can be a sub URI, or you can lock down the whole site (which is what I did).

Here, you will also create a policy (I did a policy that only allows specific Google e-mail addresses, both those that are in a google domain as well as one or two that are outside of it).

Once that is done, before you finish the application, if you are only authenticating against one source, I'd recommend selecting that source and not showing the prompt for multiple sources.  However, if you are going to accept users from other locations (such as Entra, etc), leave those as-is. 

Suddenly, you should have immediate access if you've logged in to Google.

The next thing I had to figure out was how to get Grafana to recognize the user.  And that meant going into the Grafana docker and changing the GF_AUTH_PROXY_HEADER_NAME value :

     - GF_AUTH_PROXY_HEADER_NAME=Cf-Access-Authenticated-User-Email

After restarting the docker image, this let me use the username in the dashboards for Grafana, and view my data.  Sweet!

Wednesday, July 22, 2026

Docker and Grafana

Managers love pretty pictures, right?  I know, this is a personal intranet, but hey, why not?

Grafana was a bit more of a challenge to run in docker.  I have issues with it being a default non-SSL port, but we can potentially fix that by throwing Apache in front of it down the road.  But, it's the same process as before, but with a major caveat.

We're not doing much special here, but I do need to make a quick statement.  My Grafana was an older 9.something version.  If you run it in Docker, and top are not starting from scratch (e.g. you are going to copy from a backup), it will fail.  In this case, you MUST download a 12.3 package, and do a debian upgrade on that package, and stop/restart so that it can migrate.  THEN you can move to the Docker 12.4 image base.

Now, back to the regularly scheduled programming.

Simply copy over your /var/lib/grafana/grafana.db SQLite file.  If you have customizations in your grafana.ini file, you can make those changes to a grafana.ini file and map it just like we did with Apache HTTPd above.  Note that it will be better to do that old "cat" command we mentioned in Apache to get a copy and then modify that one.

My docker-compose.yml looks like :

     services:
      web:
        image: grafana/grafana
        container_name: grafana
        restart: unless-stopped
        ports:
          - "3000:3000"
    environment:
    - GF_SECURITY_ADMIN_USER: admin
    - GF_SECURITY_ADMIN_PASSWORD: admin
    - GF_SERVER_ROOT_URL=%(protocol)s://%(domain)s:%(http_port)s/grafana/
    - GF_SERVER_ENFORCE_DOMAIN=false
    - GF_AUTH_PROXY_ENABLED=true
    - GF_AUTH_PROXY_HEADER_NAME=X-WEBAUTH-USER
    - GF_AUTH_PROXY_HEADER_PROPERTY=username
    - GF_AUTH_PROXY_AUTO_SIGN_UP=true
        volumes:
          - /opt/docker/storage/grafana:/var/lib/grafana:rw

Note the environment variables.

  • The SECURITY_ADMIN stuff sets  your default username/password.  If you are not migrating, and are instead creating a brand-new instance, these will set the default username/password.
  • The GF_SERVER_ROOT_URL is used to change the default URL (since Grafana is so smart it is going to redirect you to the Grafana port number, and this specifies whoat to hit with).
  • The GF_SERVER_ENFORCE_DOMAIN - this is where the AI engines kept biting me.  They had NO CLUE about this.  Grafana LOVES to redirect you, and if the enforce_domain in the grafana.ini uses the default or is set to true, it's doing to redirect you if you didn't hit it with the domain (could be hostname, and in docker, this defaults to "localhost").  So, if you are using docker, and if you are also reverse proxy'ing Grafana, which, in a reverse proxy environment, is not good).  It kept redirecting me to http://localhost:3000/ .  Setting this value via GF_SERVER_ENFORCE_DOMAIN allowed me to override the enforce_domain and shut that stupid thing off.
  • The GF_AUTH_PROXY_* environmment variables are to disable forcing a login.  If you are reverse proxy'ing Grafana, and the parent process is handling the authentication, this is likely what you will need to do in combination with the ReverseProxy configs I showed earlier that use the rewrite_module under Apache HTTPd.

Then it's a matter of starting it up again (the "docker compose up -d" command).

Docker - Apache HTTPd for my tiny Intranet

I'm rebuilding my network stack for my little intranet, and the first thing out of the gate is the HTTPd web service.  So, let's get started.

I'm not using NginX because there will be a lot more CGI, and also because I want to implement my template mechanism, a custom module for the Apache HTTPd serer.  Converting from an existing HTTPd configuration is a bit more difficult when it comes to migrating it in to Docker.

Now, RedHat and other Linux vendors/distributors love to part the httpd.conf file out into a whole lot of directories.

Docker doesn't do that.  However, we have a little secret.  So, let's create our docker image first.  Create your docker directory and then the container for this specific image.

    mkdir -p /opt/docker/httpd

Because the docker image is actually based on a Debian version, and because it includes an httpd.conf file already (by default), we only need three files in this directory.

For the httpd.conf, I found it easier to grab the httpd.conf file out of the docker image first, then add my custom module.  And, since I wanted the image to serve content from the NAS, I needed to get the web content from that mounted up.  Let's grab a copy of the docker container httpd.conf file.

    docker run --rm httpd:2.4 cat /usr/local/apache2/conf/httpd.conf > httpd.conf

Now, with that, I can add my own module using the LoadModule with any custom directives.  Then, we simply add the NFS mounts for the mount the NAS up via NFS, and then use volumes, too.  Enter the docker compose. 

    services:
      web:
        image: httpd:2.4
        container_name: httpd
        restart: unless-stopped
        ports:
          - "80:80"
          - "443:443"
        volumes:
          - /opt/docker/storage/http/html:/usr/local/apache2/htdocs/:ro
          - /opt/docker/storage/http/cgi-bin:/usr/local/apache2/cgi-bin/:ro
          - /opt/docker/storage/http/logs:/usr/local/apache2/logs/:rw
          - ./certs:/etc/apache2/ssl
          - ./httpd.conf:/usr/local/apache2/conf/httpd.conf:ro
          - ./mod_template.so:/usr/local/apache2/modules/mod_template.so:ro

The container name/image, are fairly inconsequential - you can name them anything you want.  The default is "apache_service" for this from the Docker image, but I like to get very specific because Apache likes to do a lot of services.

The "restart" line is there in case the EliteDesk hardware loses power and reboots - I want this container to start back up immediately.

The ports are very easy to understand if you are used to TCP.  It is formatted as "HOST PORT:CONTAINER PORT".

Also, we need to set up SSL (note we have port 443 in there).  Create a certificate :

    mkdir certs
    openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout certs/httpd.key -out certs/httpd.crt -subj "/C=US/ST=Idaho/L=Boise/O=SilverHawk/CN=silverhawk.net"

You will need to set up the standard VirtualHost for it (make sure you uncomment the LoadModule line for the ssl extension while in there) :

    <VirtualHost *:443>
    DocumentRoot "/usr/local/apache2/htdocs"
    #ServerName www.example.com
    SSLEngine On
    SSLCertificateFile /etc/apache2/ssl/httpd.crt
    SSLCertificateKeyFile /etc/apache2/ssl/httpd.key
    </VirtualHost>

While I was there, I also added proxy and proxypass information for Grafana, (that's coming - note you need to enable xml2enc_module, proxy_http_module, and proxy_module, and make sure you disable proxy for everything except your own proxt so that someone doesn't suddenly use your site as a personal proxy to mask what they are doing on the Internet).

A grafana note :

I really strugged with proxy pass in this situation.  all of the AI engines kept telling me things that were not accurate.  So, here's what I have.  In my httpd.conf for docker, I have the following excerpt :

    ProxyRequests Off
    <VirtualHost *:443>
    DocumentRoot "/usr/local/apache2/htdocs"
    #ServerName www.example.com
    SSLEngine On
    SSLCertificateFile /etc/apache2/ssl/httpd.crt
    SSLCertificateKeyFile /etc/apache2/ssl/httpd.key

    RewriteEngine On
    RewriteRule .* - [E=PROXY_USER:%{LA-U:REMOTE_USER},NS]
    RequestHeader set X-WEBAUTH-USER "%{REMOTE_USER}e"
    ProxyPreserveHost On
    ProxyPass /grafana/ http://172.17.0.1:3000/
    ProxyPassReverse /grafana/ http://172.17.0.1:3000/
    RequestHeader unset Accept-Encoding
    <Location /charts>
    TemplateEnabled off
    </Location>
    </VirtualHost>

Then, in my Grafana docker-compose.yml file, I had to add the GF_SERVER_ROOT_URL and GF_SERVER_ENFORCE_DOMAIN environment variables (there are some others, to set up anonymous browsing since this will be front-ended by an authentication later) :

        environment:
    - GF_SERVER_ROOT_URL=%(protocol)s://%(domain)s:%(http_port)s/grafana/
    - GF_SERVER_ENFORCE_DOMAIN=false
    - GF_AUTH_PROXY_ENABLED=true
    - GF_AUTH_PROXY_HEADER_NAME=X-WEBAUTH-USER
    - GF_AUTH_PROXY_HEADER_PROPERTY=username
    - GF_AUTH_PROXY_AUTO_SIGN_UP=true

Volumes are where the magic happens for the custom module and configuration as well as the web root.  Each row represents a specific file/directory that is mapped from the docker image.  For example, I have my mod_template.so in there so it gets mapped into the container, plus I am mapping an httpd.conf file, as well as two NFS directories so I don't have to restart the docker every time I change the website.

Then, it's simply a matter of launching it up.

    sudo docker compose up -d  

Note, if you run into errors, you can use (and there is a -f option to tail -f them) :

    docker compose logs

This will dump the containers log files (I suppose you COULD re-map log files if you want those pulled out of the image naturally, but this was easier). 

Saturday, September 25, 2021

Setting Up Apache for Federated Identity

I was loading my environmental data up on a web server through Grafana.  I'd been running LDAP-based authentication for years, and wanted to be able to shut that service down because I was growing weary of updating SSL certificates.  I had Grafana running on it's own TCP port inside, but I desired to expose it (while protecting the data).  I finally broke down and did it.

I started out with two "tutorials" :

Both of these were lacking in information.  For example, there are no instructions on setting up the Google API, and there was a configuration option of "OIDCCryptoPassphrase" that was a variable and no one explained what it needed to be set to.  But, I wanted to get it done.

First, I went through Google.  I'd not set up google cloud for my domain before, so this was new.  First, log in to the https://console.cloud.google.com/apis/dashboard?pli=1 (it's the cloud platform).  Once in there, if you don't have a project already, create one.  This is done using the drop down at the top :

Click on "New Project" in the upper right hand corner :

Now, you can create a credential.  Click on "Credentials" on the left, and then "Create Credential" at the top :

 

Follow the set up guide.  The type will be what you need, in my case, I was doing Apache's HTTPd server, so I went with "Web Application".  The redirect URI setting must match what you use for your OAuth configuration (in the configuration file, actually).  Make sure you have your domains listed, etc.


At this point, copy the ID as well as the client secret.  These need to go into your configuration file for Apache's HTTPd.

You might need to create an "OAuth Consent Screen", too.  Those three configurations in Google are all you need.

Load up your editor you use to change the HTTPd configuration.  The basic lines you are going to need are :

    OIDCProviderMetadataURL https://accounts.google.com/.well-known/openid-configuration
    OIDCClientID CLIENT_ID_FROM_CONFIGURATION
    OIDCClientSecret CLIENT_SECRET_FROM_CONFIGURATION

    # OIDCRedirectURI is a vanity URL, and should not point to any actual content
    OIDCRedirectURI http://hostname.example.com/grafana/redirect_uri
    OIDCCryptoPassphrase PERSONALLY_CHOSEN_PASSPHRASE
    OIDCScope "openid email profile"
    OIDCRemoteUserClaim email

    <Location /grafana/>
        TemplateEnabled off
        AuthType openid-connect
        <RequireAny>
            # not just anyone signed in from google
            # Require valid-user

            # network
            Require ip 10.0.0.

            # signed in with domain
            Require claim hd:silverhawk.net

            # or, signed in with domain (e-mail fall through)
            Require claim "email~^(.*)@silverhawk.net$"

            # or Someone External
            Require claim "email~^username@gmail.com$"
        </RequireAny>

        ProxyPass http://localhost:3000/
        ProxyPassReverse http://localhost:3000/
        Order allow,deny
        Allow from all

        # grafana requires the username to be in a header
        RewriteEngine On
        RewriteRule .* - [E=PROXY_USER:%{LA-U:REMOTE_USER},NS]
        RequestHeader set X-WEBAUTH-USER "%{PROXY_USER}e"
    </Location>

The OIDCClientID and the OIDCClientSecret configuration items are where you stuff the respective items from your OAuth configuration we copied above.  The OIDCCryptoPassphrase is where I was getting lost - this is going to be something you choose, and is specific to the cluster (e.g. so that the cluster can keep state if you hit other servers).

I used a RequireAny to set up multiple options - so, if you are on the local network and sign in to google, you'll get in.  If your primary google address is a silverhawk.net domain, you can get in.  If your e-mail address is username@gmail.com, you can get in.

The rewrite directives are there specifically for Grafana so that Grafana can see the remote_user as the e-mail of the individual who just authenticated.  In your grafana.ini, locate your root_url, and make sure we've added the URI piece we are proxying, e.g. :

    root_url = %(protocol)s://%(domain)s:%(http_port)s/grafana/

Next, locate your "[users]" section, and set the following :

    allow_sign_up = false
    auto_assign_org = true

Next, locate the auth.proxy in your Grafana configuration.  Since we are rolling through HTTPd and it will be doing the authentication, we can let Grafana accept whatever HTTPd feeds to us.

    [auth.proxy]
    enabled = true
    header_name = X-WEBAUTH-USER
    header_property = username
    auto_sign_up = true

The header name should match what was in our rewrite rule, and the header property is the username that is going to get set up.  auto_sign_up needs to be set to true so that we can create accounts on the fly.

Now, restart any processes and give it a test!