Databases controller

A walkthrough of a Kubernetes controller on humus that provisions a database and the role that owns it in PostgreSQL for each Database resource, writes the role’s credentials to a Secret, and drops both on delete as the Database’s deletionPolicy says. These pages build it step by step.

The databases controller reconciles Database resources, with kube. For each one, it:

ensures a role, and a database the role owns, in a PostgreSQL server;
applies a Secret, which the Database controls, holding the role’s credentials;
sets the Database’s Ready condition, saying which step failed if one did.

Deleting a Database drops its database and role, or keeps them, as its deletionPolicy says. The controller runs two replicas, and the one that holds the Lease reconciles.

The `Database` resource

A Database, as the kind overlay applies it:

apiVersion: postgres.example.com/v1alpha1
kind: Database
metadata:
  name: orders
spec:
  name: orders
  deletionPolicy: Delete

Once it’s provisioned, its status.secretName is orders-postgres, and its Ready condition is True, with the reason Provisioned.

Its names

Name	What it is
`postgres.example.com`, `v1alpha1`, `Database`, `databases`	the resource’s group, version, kind and plural
`spec.name`	the database’s name in the server, and its role’s
`<metadata.name>-postgres`	the Secret holding the role’s credentials, in the `Database`’s namespace
`postgres.example.com/finalizer`	the finalizer that keeps a deleted `Database` until it’s cleaned up
`postgres.example.com/failure-scenario`	the annotation that carries a failure-injection scenario
`databases-controller`	the controller’s name, its Lease’s, and the field manager of its applies

The files it ends with

Its code is a module of its own, at example/kube/databases:

example/kube/databases/
├── main.go             humus.Main(app.Init, ...)
├── api/v1alpha1/       the Database resource and its rules
├── postgres/           the admin interface, and the Server that implements
│                       it on PostgreSQL
├── controller/         the Reconciler and its finalizer
├── app/                Init, the settings, BuildTelemetry and Faults
├── internal/certfile/  the probes' certificate, read again when it's renewed
├── cases/              the test cases the in-process tests and kind share
├── functional/         the cases against kind, and the Grafana check
├── deploy/             the golden CRD, the kustomization, the alerts and
│                       dashboard, and the kind overlay
└── Dockerfile          the databases image

These pages build api/v1alpha1, postgres and controller, then app and main, then test it, deploy it and observe it, and end with the production-ready checklist.

The Database resource

The databases controller’s custom resource: the Database’s spec and status as Go types, its rules, including a CEL rule that makes the name immutable, and the golden CRD reflected from them.

Talking to PostgreSQL

The databases controller’s admin interface to PostgreSQL: Provision and Drop, the Server that implements them on a pgx pool, their injection points, postgresql:provision and postgresql:drop, and how no credential reaches the SQL text, a span or an error.

Reconciling

The databases controller’s Reconciler: the password from the Secret, the role and database provisioned, the Secret applied with server-side apply, the Ready condition and its reasons, and rotation by deleting the Secret.

Cleaning up

How the databases controller cleans up a deleted Database: a finalizer keeps it until the role and the database are dropped, or kept, as its deletionPolicy says, and a failed drop keeps it until a retry succeeds.

Init

The databases controller’s Init: its settings, sourced and checked; its secrets, read from files; the Connection to the API server; the Settings and the Lease; TrustScenario; the Controller; the probes; BuildTelemetry and Faults; and main.

Testing

How the databases controller is tested: its Reconciler as a unit, on a fake clientset and a fake Admin; the golden CRD; the shared cases and their Target, run in process against k3s and PostgreSQL in containers and on kind; and the failure injection cases.

Deploying

How the databases controller is deployed: its image, the kustomization, RBAC, the Deployment and its disruption budget, the grace period, and the kind overlay. Then running it on kind with up.sh, and applying a Database to it.

Observing

What the running databases controller tells you in Grafana on kind: a reconcile’s trace in Tempo, its log records in Loki, humus’s controller instruments and the containers’ restarts in Mimir, the alert rules in deploy/alerts.yaml, the dashboard in deploy/dashboard.json, and the check that Grafana loaded them.

Production-ready

The checklist the databases controller meets, and how it meets each item, with the page that builds it. Check your own Kubernetes controller against it.