Dock(er) Wok

Docker is a simple way to create virtualization without having full systems running.  I took a moment to figure out how to run docker in relation to Postgres.

Note you need at least three servers for this for it to have any sort of redundancy. One is the "master".  You can probably get away with two (run the manager on one of the nodes), but I do not recommend this.  Really, you could get away with one server that runs the manager and two instances (I did), but know that if you don't run the docker instances on separate hosts, you've now lost all high availability, because if that server fails, the whole stack will cease to exist.

First, install Docker :

sudo apt-get install docker.io
sudo systemctl enable docker
sudo systemctl start docker

Then, create your swarm (anything recent should have swarm built in, you just need to set it up).

sudo docker swarm init --advertise-addr 192.168.x.x

Next, join a docker worker to the swarm

username@server1:~$ sudo docker swarm init --advertise-addr 192.168.0.3
Swarm initialized: current node (NODE IDENTIFIER) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join --token TOKEN_STRING 192.168.0.3:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
username@server1:~$

Then, on each worker node that isn't the manager, run the command you were provided (use sudo) :

sudo docker swarm join --token TOKEN_STRING 192.168.1.237:2377

Next is your basic configuration.  Each host is relatively identical, with just a few modifications.  Each host has 3 different configuration files, pg_hba.conf, postgresql.conf, and pg_ident.conf.  Let's start by creating our directory structure.

I created a directory to house all of this so I can clear it out quickly after my learning curve.

mkdir cluster-postgres
cd cluster-postgres
mkdir -p {master,slave-1,slave-2}/config

Next, create our three files for each host :

touch {master,slave-1,slave-2}/config/{pg_hba.conf,postgresql.conf,pg_ident.conf}

With the files created, let's populate them.  Here's the gist for each host, but at the end, I'll identify differences as needed.

postgresql.conf

This file contains :

# -----------------------------
# PostgreSQL configuration file
# -----------------------------
#

data_directory = '/data'
hba_file = '/config/pg_hba.conf'
ident_file = '/config/pg_ident.conf'

port = 5432
listen_addresses = '*'
max_connections = 100
shared_buffers = 128MB
dynamic_shared_memory_type = posix
max_wal_size = 1GB
min_wal_size = 80MB
log_timezone = 'Etc/UTC'
datestyle = 'iso, mdy'
timezone = 'Etc/UTC'

#locale settings
lc_messages = 'en_US.utf8'   # locale for system error message
lc_monetary = 'en_US.utf8'   # locale for monetary formatting
lc_numeric = 'en_US.utf8'    # locale for number formatting
lc_time = 'en_US.utf8'       # locale for time formatting

default_text_search_config = 'pg_catalog.english'

#replication
wal_level = replica
wal_keep_size = 512MB  # Adjust this value as needed
archive_mode = on
archive_command = 'test ! -f /mnt/server/archive/%f && cp %p /mnt/server/archive/%f'
max_wal_senders = 3

However, on the slave nodes, lines 28-34, which contain :

#replication
wal_level = replica
wal_keep_size = 512MB  # Adjust this value as needed
archive_mode = on
archive_command = 'test ! -f /mnt/server/archive/%f && cp %p /mnt/server/archive/%f'

are removed.

pg_hba.conf

This file contains :

# TYPE  DATABASE        USER            ADDRESS                 METHOD

host     replication     replicationUser         0.0.0.0/0        md5

# "local" is for Unix domain socket connections only
local   all             all                                     trust
# IPv4 local connections:
host    all             all             127.0.0.1/32            trust
# IPv6 local connections:
host    all             all             ::1/128                 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
local   replication     all                                     trust
host    replication     all             127.0.0.1/32            trust
host    replication     all             ::1/128                 trust

host all all all scram-sha-256

However, on the slave nodes, line 3 :

host     replication     replicationUser         0.0.0.0/0        md5

is removed.

pg_ident.conf

This file contains :

# PostgreSQL User Name Maps
# =========================
#
# Refer to the PostgreSQL documentation, chapter "Client
# Authentication" for a complete description.  A short synopsis
# follows.
#
# This file controls PostgreSQL user name mapping.  It maps external
# user names to their corresponding PostgreSQL user names.  Records
# are of the form:
#
# MAPNAME  SYSTEM-USERNAME  PG-USERNAME
#
# (The uppercase quantities must be replaced by actual values.)
#
# MAPNAME is the (otherwise freely chosen) map name that was used in
# pg_hba.conf.  SYSTEM-USERNAME is the detected user name of the
# client.  PG-USERNAME is the requested PostgreSQL user name.  The
# existence of a record specifies that SYSTEM-USERNAME may connect as
# PG-USERNAME.
#
# If SYSTEM-USERNAME starts with a slash (/), it will be treated as a
# regular expression.  Optionally this can contain a capture (a
# parenthesized subexpression).  The substring matching the capture
# will be substituted for \1 (backslash-one) if present in
# PG-USERNAME.
#
# Multiple maps may be specified in this file and used by pg_hba.conf.
#
# No map names are defined in the default configuration.  If all
# system user names and PostgreSQL user names are the same, you don't
# need anything in this file.
#
# This file is read on server startup and when the postmaster receives
# a SIGHUP signal.  If you edit the file on a running system, you have
# to SIGHUP the postmaster for the changes to take effect.  You can
# use "pg_ctl reload" to do that.

# Put your actual configuration here
# ----------------------------------

# MAPNAME       SYSTEM-USERNAME         PG-USERNAME

There are no differences here for each node.

Last Configurations

Create the docker "network" :

sudo docker network create postgres-cluster-network

This will print a fairly large alphanumeric ID. 

Starting the "master"

At this point in time, start the master node using the following command :

sudo docker run -d --name postgres-master  --net postgres-cluster-network \
-e POSTGRES_USER=postgresadmin -e POSTGRES_PASSWORD=admin123 \
-e POSTGRES_DB=postgresdb -e PGDATA="/data" -v ${PWD}/master/pgdata:/data \
-v ${PWD}/master/config:/config -v ${PWD}/master/archive:/mnt/server/archive \
-p 5000:5432 postgres:latest -c 'config_file=/config/postgresql.conf'

If you haven't downloaded the postgres docker image yet, this will actually cause it to try and install.  If you get an error about it already existing because you tried to start it one and it gave you an error :

docker: Error response from daemon: Conflict. The container name "/postgres-master" is already in use by container "6362ca473d3f29b7fe1bf02558f5a871fb6cf6827eff23fe01714f34cb386951". You have to remove (or rename) that container to be able to reuse that name.

Then list the dockers, and delete it :

username@server1:~/postgres-cluster$ sudo docker ps -a
CONTAINER ID   IMAGE             COMMAND                  CREATED          STATUS    PORTS     NAMES
6362ca473d3f   postgres:latest   "docker-entrypoint.s…"   10 minutes ago   Created             postgres-master
username@server1:~/postgres-cluster$ sudo docker rm 6362ca473d3f
6362ca473d3f
username@server1:~/postgres-cluster$ sudo docker ps -a
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
username@server1:~/postgres-cluster$

Now try to start it if it had previous failed again, and it should simply give you a large alphanumeric ID again.

Next, create a replication user :

username@server1:~$ sudo docker exec -it postgres-master bash
[sudo] password for username:          
root@63919c16e356:/# createuser -U postgresadmin -P -c 5 --replication replicationUser
Enter password for new role: 
Enter it again: 
root@63919c16e356:/# exit
exit
username@server1:~$

Just type exit and get back to the prompt.

Now, we can move on to starting the slaves. 

Starting the slaves

For each node, run :

username@server1:~$ sudo docker run -it --name postgres-slave1 --rm \
> --net postgres-cluster-network \
> -v ${PWD}/slave-1/pgdata:/data \
> --entrypoint /bin/bash postgres:latest
root@447571e3e11e:/#

This will put you in a bash prompt as "interactive", where you then run :

pg_basebackup -h postgres-master -p 5432 -U replicationUser -D /data/ -Fp -Xs -R

The command initiates replication, and will use the password you specified on the master node when you started that up with the "createuser" command.

Repeat for slave-2 (of course, replacing postgres-slave1 with postgres-slave2, and the slave-1 folder names with slave-2).

Create standby instances 

Run the following :

sudo docker run -d --name postgres-slave1 --net postgres-cluster-network \
-e POSTGRES_USER=postgresadmin -e POSTGRES_PASSWORD=admin123 \
-e POSTGRES_DB=postgresdb -e PGDATA="/data"  \
-v ${PWD}/slave-2/pgdata:/data -v ${PWD}/slave-2/config:/config \
-v ${PWD}/slave-2/archive:/mnt/server/archive -p 5002:5432 \
postgres:latest -c 'config_file=/config/postgresql.conf'

Repeat for slave-2 (of course, replacing postgres-slave1 with postgres-slave2, and the slave-1 folder names with slave-2).

Test It

Connect to the master node using :

sudo docker exec -it postgres-master bash

From in here, you can run your psql commands to create databases and manipulate whatever you need.

psql --username=postgresadmin postgresdb

Exit, and then check the other slave nodes by connecting to them (sudo docker exec) and running psql to query any tables you've created and populated with data.

Just Pulley-ing My Leg

 A year or so back, I had heavily modified am early 1940's tiny lathe with the intent of using it to turn pens.  There are a number of small lathes that are likely cheaper than the cost put into that one, but for some odd reason, I love the old iron, and did it anyway.

The results were, less than stellar.  Indeed, it did work, but I had to take my time because the stepper motor did not have enough torque.  And that is with a small pen.

So, enter the next chapter.  Someone over on the hobby machinist forum sent an old sewing machine motor to me (think Pfaff 130, and you have the footprint of it), just for the cost of shipping.  With the pulley that was on it, I found myself in a bit of a quandary - I needed more modifications to see if this would work.  So, I set about creating a new belt tensioning system.  I patterned this variation off of a cars alternator - where it is installed and then tightened over an arc-bracket.

I made the bar using my drill press, because, I didn't have a rotary table large enough for the arc (mine is 4", the arc is 8" - but that is due to only having a scrap bar with an 8" arc that was an off cut for the 127-tooth compound gear), and also because I was too lazy to drag out the milling machine onto a cold back porch and cut it.  Really, it was because I was way too lazy.

But, it was close enough for what I needed.

So, I drilled a series of holes, then used an endill in the drill press to smooth out the arcs.  I drilled a few extra holes in some bar stock, and used a 1/4"-20 bolt to put it all together.

Next, I needed a pulley.  So I grabbed another off-cut/drop (2" diameter aluminum), chucked it up, and made a 2L pulley that was as large as this lathe could accept, but small enough to fit (1.950" with the belt).  This was faced, drilled, then reamed.  It was then turned on the boss, then the flat part of the pulley, then the angles.

The result is a usable 2L pulley to match the one on the sewing machine motor.

I wired the motor to test it, and once that checked out okay, it was time to put it all together.

This immediately feels like more torque than the stepper.  But, the proof is always in the pudding.  What is the end result?  Let's give it a try. I ran through the normal steps preparing a pen blank (this time, I chose a  gun-metal "mini" bolt action pen kit from Penn State Industries - I have a lot of family that love the regular ones, so I thought I'd give this more "unique" variant a whack.)

1. Usually, mark the blank lengthwise to keep grain lined up.  In this case, it's small, and I was fixing another pen while I was at it, so this step didn't matter.
2. Cut off two chunks to fit your pen tubes, about 1/4" longer than the tubes themselves.  Again, this was small enough, and I was using a scrap blank I had from another project.
3. Drill the centering hole for the blank(s).
4. Glue in the tubes.  Many people just use CA glue.  I have always loved 5-minute epoxy for this.
5. Barrel-trim the ends to get it flush with the brass tubes (don't take off brass or it might not fit).
6. Put the tubes on the appropriate bushings, then on the mandrel, and turn.
7. Once to slightly larger than size, use the lathe to sand the blanks.  Get as high a grit as possible for the best finishes.
8. Remove from the lathe and assemble according to the instructions. 

So, once I got to the lathe part, I grabbed a photo.

They do look fantastic (I had two on there - I was repairing another pen while I was at it).  Then, assembly prep :

Then, the final assembly.

While the motor was slightly "under-powered" (It IS a 1/5 HP sewing machine motor), it still did the trick.  I do have to take things a little slower with this, but it does mean I don't have to drag out the larger twin for this little thing.  I can simply pick it up, clamp it to a work bench, plug it in, and turn a pen. 

Finding God in Failure

Many religions believe we are being punished for Adam's transgression.  We do not.

"And now, behold, if Adam had not transgressed he would not have fallen, but he would have remained in the garden of Eden. And all things which were created must have remained in the same state in which they were after they were created; and they must have remained forever, and had no end.

"And they would have had no children; wherefore they would have remained in a state of innocence, having no joy, for they knew no misery; doing no good, for they knew no sin.

"But behold, all things have been done in the wisdom of him who knoweth all things.

"Adam fell that men might be; and men are, that they might have joy.

"And the Messiah cometh in the fulness of time, that he may redeem the children of men from the fall. And because that they are redeemed from the fall they have become free forever, knowing good from evil; to act for themselves and not to be acted upon, save it be by the punishment of the law at the great and last day, according to the commandments which God hath given." - 2 Nephi 2:22-26

Man is fallen.  The earth is Telestial - imperfect.

It is this imperfect world that allows us to be imperfect. 

Were it not for the imperfection of the world in which we live, we could not know how to hope for a better one.  Two verses before the famous "faith" scripture in Ether, we have this little gem.

"Wherefore, whoso believeth in God might with surety hope for a better world, yea, even a place at the right hand of God, which hope cometh of faith, maketh an anchor to the souls of men, which would make them sure and steadfast, always abounding in good works, being led to glorify God." - Ether 12:4 

All of us find themselves in times of "hoping for a better world".

Sermon on the mount - after stating that we should pray for those who curse us, states :

"for he maketh his sun to rise on the evil and on the good, and sendeth rain on the just and on the unjust." - Matthew 5:45

Do I mention cancer?  Heartbreak?  Heart Failure?

By Elder Gong tells the following story. 

A Chinese story begins as a man’s son finds a beautiful horse.

“How fortunate,” the neighbors say.

“We’ll see,” says the man.

Then the son falls off the horse and is permanently injured.

“How unfortunate,” the neighbors say.

“We’ll see,” says the man.

A conscripting army comes but doesn’t take the injured son.

“How fortunate,” the neighbors say.

“We’ll see,” says the man.

This fickle world often feels tempest tossed, uncertain, sometimes fortunate, and—too often—unfortunate. Yet, in this world of tribulation, “we know that all things work together for good to them that love God.” Indeed, as we walk uprightly and remember our covenants, “all things shall work together for your good.” - Elder Gerrit W. Gong, Apr 2024, "All Things for Our Good"

Do we want to be "tempest tossed", or to wonder "why me?"  How do we avoid this?

"We become our truest, best selves only as we put off the natural man or woman and become a child before God." -  Elder Gerrit W. Gong, Apr 2024, "All Things for Our Good"

How do we put off the natural man?

For the natural man is an enemy to God, and has been from the fall of Adam, and will be, forever and ever, unless he yields to the enticings of the Holy Spirit, and putteth off the natural man and becometh a saint through the atonement of Christ the Lord, and becometh as a child, submissive, meek, humble, patient, full of love, willing to submit to all things which the Lord seeth fit to inflict upon him, even as a child doth submit to his father. - Mosiah 3:19

Yield to the enticings of the Holy Spirit.

1. Create an environment where the world can be quiet.  Temple, at home in your personal room, or sitting in your car while waiting.
2. Commit to follow the direction you are about to receive. 
3. Pray for the enticing of the Spirit of God to show how you can serve someone. 
4. Remain in the quiet environment until you feel that you have something to do.
5. Obey immediately.
6. Repeat.

As you put this effort in to interact with the Spirit of God and to follow through, you are becoming a saint through the atonement of Christ the Lord.  You become meek, humble, patient, full of love, and willing to submit to all things the Lord seeth fit to inflict.

You are building a trusting relationship with Christ. 

"When we trust God and His love for us, even our greatest heartbreaks can, in the end, work together for our good." -  Elder Gerrit W. Gong, Apr 2024, "All Things for Our Good"

This trusting relationship does not obliterate our enemies, or push obstacles out of our way.

Shadrach, Meshach, and Abed-nego, would not cede their covenants to a King, and were to be thrown into a furnace of fire.  When the King questioned the power of their God, their response was :

"If it be so, our God whom we serve is able to deliver us from the burning fiery furnace, and he will deliver us out of thine hand, O king.

"But if not, be it known unto thee, O king, that we will not serve thy gods, nor worship the golden image which thou hast set up." - Daniel 3:17-18 

The second half of that, "But if not", is the crucial component.  Whether something happens or not does not cause a break in what our commitment should be.

Remember, we are mortal, living in a mortal world.  But when we get lost in that "forest for the trees", pause, to also remember this.

"Even if you cannot always see that silver lining on your clouds, God can, for He is the very source of the light you seek. He does love you, and He knows your fears. He hears your prayers. He is your Heavenly Father, and surely He matches with His own the tears His children shed." - Jeff Holland, Oct 1999, "An High Priest of Good Things to Come"

God knows you, deeply and fully.  If he is "endless", pause and realize that his love for you is also endless.

"He is the light and the life of the world; yea, a light that is endless, that can never be darkened; yea, and also a life which is endless, that there can be no more death." - Mosiah 16:9

We must recognize that our timing is never God's timing.

"Some misunderstand the promises of God to mean that obedience to Him yields specific outcomes on a fixed schedule. They might think, “If I diligently serve a full-time mission, God will bless me with a happy marriage and children” or “If I refrain from doing schoolwork on the Sabbath, God will bless me with good grades” or “If I pay tithing, God will bless me with that job I’ve been wanting.” If life doesn’t fall out precisely this way or according to an expected timetable, they may feel betrayed by God. But things are not so mechanical in the divine economy. We ought not to think of God’s plan as a cosmic vending machine where we (1) select a desired blessing, (2) insert the required sum of good works, and (3) the order is promptly delivered.

"God will indeed honor His covenants and promises to each of us. We need not worry about that. The atoning power of Jesus Christ—who descended below all things and then ascended on high and who possesses all power in heaven and in earth—ensures that God can and will fulfill His promises. It is essential that we honor and obey His laws, but not every blessing predicated on obedience to law is shaped, designed, and timed according to our expectations. We do our best but must leave to Him the management of blessings, both temporal and spiritual." - Elder D. Todd Christofferson, April 2022, "Our Relationship with God"

And once you recognize this, you will also recognize that, while we are imperfect and mortal, there is a full regalia of blessings waiting.

"Some blessings come soon, some come late, and some don’t come until heaven; but for those who embrace the gospel of Jesus Christ, they come." - Jeffrey Holland - Oct 1999, "An High Priest of Good Things to Come"

Recommend Jan 2009, Thomas Monson's speech to graduating students called "Great Expectations".

Rudyard Kipling, "If" :

If you can keep your head when all about you   
    Are losing theirs and blaming it on you,   
If you can trust yourself when all men doubt you,
    But make allowance for their doubting too;   
If you can wait and not be tired by waiting,
    Or being lied about, don’t deal in lies,
Or being hated, don’t give way to hating,
    And yet don’t look too good, nor talk too wise:

If you can dream—and not make dreams your master;   
    If you can think—and not make thoughts your aim;   
If you can meet with Triumph and Disaster
    And treat those two impostors just the same;   
If you can bear to hear the truth you’ve spoken
    Twisted by knaves to make a trap for fools,
Or watch the things you gave your life to, broken,
    And stoop and build ’em up with worn-out tools:

If you can make one heap of all your winnings
    And risk it on one turn of pitch-and-toss,
And lose, and start again at your beginnings
    And never breathe a word about your loss;
If you can force your heart and nerve and sinew
    To serve your turn long after they are gone,   
And so hold on when there is nothing in you
    Except the Will which says to them: ‘Hold on!’

If you can talk with crowds and keep your virtue,   
    Or walk with Kings—nor lose the common touch,
If neither foes nor loving friends can hurt you,
    If all men count with you, but none too much;
If you can fill the unforgiving minute
    With sixty seconds’ worth of distance run,   
Yours is the Earth and everything that’s in it,   
    And—which is more—you’ll be a Man, my son! 

Elder Holland spoke about one of his own "travails" - he and his young family had car troubles.  When he drove past that spot later in life, how he wished he could tell himself :

"Don't you quit. You keep walking, you keep trying, there is help and happiness ahead. Some blessings come soon. Some come late. Some don't come until heaven. But for those who embrace the gospel of Jesus Christ, they come. It will be alright in the end. Trust God and believe in Good Things to Come." - Jeff Holland, Oct 1999, "An High Priest of Good Things to Come"

And in the same talk, he continued :

To any who may be struggling to see that light and find that hope, I say: Hold on. Keep trying. God loves you. Things will improve. Christ comes to you in His “more excellent ministry” with a future of “better promises.” He is your “high priest of good things to come.” - Jeffrey Holland - Oct 1999, "An High Priest of Good Things to Come"