From ecf63a62a2713831e861d1de2d7fd7e3c5f56ef9 Mon Sep 17 00:00:00 2001 From: Lucas Winther Date: Sat, 3 Oct 2026 17:59:47 +0200 Subject: [PATCH] Write a README into /srv/stacks, and an adoption checklist The README says what the directory is, links repo_url as the source of truth, and covers deploying, stackctl and OpenBao. stackctl.yml refuses to run while repo_url is a placeholder; AGENTS.md gains "Adopting this repository". Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 57 ++++++++- README.md | 12 +- ansible/inventory/group_vars/all.yml | 6 + ansible/playbooks/site.yml | 2 +- ansible/playbooks/stackctl.yml | 30 ++++- .../playbooks/templates/stacks-README.md.j2 | 112 ++++++++++++++++++ 6 files changed, 211 insertions(+), 8 deletions(-) create mode 100644 ansible/playbooks/templates/stacks-README.md.j2 diff --git a/AGENTS.md b/AGENTS.md index 057bf79..e0a87a0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -8,6 +8,12 @@ and `.claude/skills/` points at `skills/`. Put content here or in `skills/`, never in a pointer — the whole arrangement exists so that every agent, and every human, reads the same text. +> **Setting this up for your own hosts?** If you are an agent that has just +> copied, forked or initialised this repository for someone, do +> [Adopting this repository](#adopting-this-repository) first, before any other +> change. Every host, address, hostname and URL in it is a placeholder, and a +> setup that still contains them is not finished. + ## What this repository is A pattern for deploying Docker Compose stacks to a small number of Linux hosts @@ -26,6 +32,53 @@ Everything under `server/` and every host name is an **example**. `example.com` is reserved by RFC 2606 and can never resolve to a real service. Replace them with yours; the framework around them is the part meant to be reused. +## Adopting this repository + +When this repository is copied for real hosts, the examples have to become that +setup's own. Leaving one behind is not harmless: a CI button for a stack that +does not exist, a connectivity check against `edge.example.com`, or a README on +the host naming `git.example.com` as the source of truth all look like they +work and mislead whoever comes next. + +**Ask the person for these. Do not invent them**, and do not leave an +`example.com` value "to fill in later": + +- each host's inventory name, address, and SSH user and port if not `deploy`/22; +- where their copy of this repository lives (`repo_url`); +- which of the example stacks, if any, they actually want, and their hostnames. + +Then: + +1. **Hosts.** `ansible/inventory/hosts.yml`, and rename the directories under + `server/` to match — the role resolves `server//`. Update + the `hosts:` of every playbook you keep, and the host table at the top of + this file. +2. **Repository URL.** `repo_url` in `ansible/inventory/group_vars/all.yml`. + `stackctl.yml` refuses to run until it is no longer a placeholder. +3. **Stacks.** For each example stack not wanted, remove all of: its playbook, + its payload under `server/`, its `site.yml` import, its CI buttons (three + places — step 4 of the `adding-a-stack` skill), its `stack-groups` entries, + and its rows in the tables in `README.md`, `server/README.md`, the + worked-examples paragraph below, and the skill. Then add the person's real + stacks with the `adding-a-stack` skill. +4. **Hostnames and addresses.** Every `VIRTUAL_HOST`, `LETSENCRYPT_HOST`, vhost + file under `reverse-proxy/proxy-data/vhost/`, and the default `hosts` input + of both `connectivity.yml` workflows. +5. **CI.** Delete the platforms not in use (`.github/`, `.gitea/`, + `.gitlab-ci.yml`, and the matching `docs/ci-*.md`), then set the secrets in + `docs/secrets.md` on the one that is. +6. **Groups.** `server//stack-groups` for each host. + +Done means both of these hold: + +```bash +make check # passes +grep -rn 'example\.com' --exclude-dir=.git . # only prose explaining the convention +``` + +Then report what was changed, and what still needs a person: the deploy key, +`bootstrap.yml` on each host, the vault, and the CI secrets. + ## Vocabulary - **host** — one machine (`edge`, `app`). The top-level directories under @@ -110,8 +163,8 @@ Worked examples for each: `static-site.yml` (the minimum), `reverse-proxy.yml` `openbao.yml` (owner-only escalation; a notice rather than a restart on `compose_stack_synced`), `webapp-staging.yml` (`stack_src` + `stack_dest` + `stack_env`), `banner.yml` (not a stack at all), -`stackctl.yml` (not a stack either: one shared file and one per-host file into -`stack_root`), `bootstrap.yml` (preparing a host; see `docs/bootstrap.md`). +`stackctl.yml` (not a stack either: one shared file, one per-host file and a +templated README into `stack_root`), `bootstrap.yml` (preparing a host; see `docs/bootstrap.md`). #### `stack_dirs` diff --git a/README.md b/README.md index 46e7e2c..17ff337 100644 --- a/README.md +++ b/README.md @@ -65,7 +65,8 @@ ansible/ └── playbooks/ ├── site.yml # every stack, in dependency order ├── bootstrap.yml # prepares a host; run once, by an admin - ├── stackctl.yml # installs stackctl and each host's stack-groups + ├── stackctl.yml # installs stackctl, each host's stack-groups, and a README + ├── templates/ # that README └── .yml # one per stack, runnable on its own server/// # the payload: compose files and config, copied verbatim server/shared/ # payloads used by more than one host @@ -92,6 +93,10 @@ TODO.md # known defects and deferred decisions > lives in a world-writable directory, and a runner's build directory is exactly > that. +**Using this for your own hosts?** Every host, hostname and URL here is a +placeholder. Start with *Adopting this repository* in [AGENTS.md](AGENTS.md) — a +checklist of everything to replace, written so an agent can follow it too. + **A host that has never been deployed to needs preparing first** — a `deploy` user, its key, the `docker` group, rsync, and a writable `/srv/stacks`. That is `scripts/new-deploy-key.sh` and `playbooks/bootstrap.yml`, once per host; see @@ -247,7 +252,10 @@ Groups are per host, in `server//stack-groups`, along with the start order (`down` reverses it) and the stacks that need confirming before `down` or `restart`. The script is one file for every host, in `server/shared/components/stackctl`. `playbooks/stackctl.yml`, which `site.yml` -runs, installs both; edit them here, not on the host. +runs, installs both; edit them here, not on the host. The same playbook writes +`/srv/stacks/README.md`, which tells anyone who logs in to a host what the +directory is, how it is deployed, and that your repository — `repo_url` in +`group_vars/all.yml` — is the source of truth. ## Secrets diff --git a/ansible/inventory/group_vars/all.yml b/ansible/inventory/group_vars/all.yml index 7ae3387..f3779df 100644 --- a/ansible/inventory/group_vars/all.yml +++ b/ansible/inventory/group_vars/all.yml @@ -19,6 +19,12 @@ stack_root: /srv/stacks # See "Deploy versus update" in the README. pull: policy +# Where your copy of this repository lives, for people who find its files on a +# host: the README that stackctl.yml writes into stack_root links here as the +# single source of truth. A placeholder until you change it, and stackctl.yml +# refuses to run while it is one — see "Adopting this repository" in AGENTS.md. +repo_url: https://git.example.com/you/config + # Repository root, so playbooks and the role can find payloads under server/ # regardless of the directory ansible-playbook was invoked from. repo_root: "{{ (inventory_dir ~ '/../..') | realpath }}" diff --git a/ansible/playbooks/site.yml b/ansible/playbooks/site.yml index 83b8ae4..9c5374d 100644 --- a/ansible/playbooks/site.yml +++ b/ansible/playbooks/site.yml @@ -15,7 +15,7 @@ ansible.builtin.import_playbook: reverse-proxy.yml - name: Default vhost banner ansible.builtin.import_playbook: banner.yml -- name: Stackctl +- name: Stackctl and the stack root README ansible.builtin.import_playbook: stackctl.yml - name: Metrics ansible.builtin.import_playbook: metrics.yml diff --git a/ansible/playbooks/stackctl.yml b/ansible/playbooks/stackctl.yml index 5ba54e0..ea73137 100644 --- a/ansible/playbooks/stackctl.yml +++ b/ansible/playbooks/stackctl.yml @@ -1,13 +1,19 @@ --- -# stackctl, and each host's groups for it, into the stack root. Not a Compose -# stack: two files and no `docker compose up`, so plain tasks rather than the -# role, as with banner.yml. +# stackctl, each host's groups for it, and a README, into the stack root. Not a +# Compose stack: three files and no `docker compose up`, so plain tasks rather +# than the role, as with banner.yml. # # The script is one file shared by every host, under server/shared/components/. # The groups are per host, at server//stack-groups, because each host runs # different stacks; a host without one still gets the script, and "all" and # stack names work there without any groups. See "On the host: stackctl" in the # README. +# +# The README is for whoever logs in to the host: what the directory is, that the +# repository at repo_url is the source of truth, how to deploy, and stackctl. +# It is a template, which payloads never are, because it is not a payload: it +# names the host and the repository. Nothing in it changes between runs, so a +# second run still reports no changes. - name: Install stackctl hosts: - edge @@ -20,6 +26,18 @@ stackctl_groups_src: "{{ repo_root }}/server/{{ inventory_hostname }}/stack-groups" tasks: + # A README sending people to git.example.com is worse than none: it names a + # source of truth that does not exist. Fail until repo_url is real. + - name: Verify repo_url points at your repository + ansible.builtin.assert: + that: + - "'example.com' not in repo_url" + fail_msg: >- + repo_url in inventory/group_vars/all.yml is still the placeholder + {{ repo_url }}. Set it to where your copy of this repository lives — + see "Adopting this repository" in AGENTS.md. + quiet: true + - name: Install the stackctl script ansible.builtin.copy: src: "{{ stackctl_src }}" @@ -33,3 +51,9 @@ dest: "{{ stack_root }}/stack-groups" mode: "0644" when: stackctl_groups_src is file + + - name: Install the README for the stack root + ansible.builtin.template: + src: stacks-README.md.j2 + dest: "{{ stack_root }}/README.md" + mode: "0644" diff --git a/ansible/playbooks/templates/stacks-README.md.j2 b/ansible/playbooks/templates/stacks-README.md.j2 new file mode 100644 index 0000000..eceeead --- /dev/null +++ b/ansible/playbooks/templates/stacks-README.md.j2 @@ -0,0 +1,112 @@ +# /srv/stacks on {{ inventory_hostname }} + +**Everything in this directory is managed by Ansible. Do not edit it here.** + +The single source of truth is the repository: + + {{ repo_url }} + +This file is written by `ansible/playbooks/stackctl.yml` in that repository. +Change it there. + +## What is here + +Every subdirectory with a compose file is one Docker Compose stack. Each is +copied from the repository — `server/{{ inventory_hostname }}//`, or +`server/shared//` for one that runs on several hosts — and brought up by +`ansible/playbooks/.yml`. + +| Path | What it is | +| --- | --- | +| `/docker-compose.yml`, config | Copied from the repository on every deploy. A local edit is overwritten by the next one | +| `/.env` | Never in the repository. For a stack whose playbook sets `stack_secrets`, rendered from the vault on every deploy: change the value there and redeploy, never this file. Otherwise placed here by hand | +| `/` | The stack's own state: databases, uploads, config the app writes. Not in the repository | +| `stackctl`, `stack-groups` | The by-hand runner below, and this host's groups | +| `README.md` | This file | + +## Changing something + +1. Change it in the repository and push. +2. Deploy it from CI: run the **deploy** workflow (or the stack's `check:` and + `deploy:` jobs on GitLab), in check mode first, and read what it would change. + Pull `always` fetches newer images; a normal deploy keeps the ones already + here. +3. Or, from a checkout, in its `ansible/` directory: + + ansible-playbook playbooks/.yml --check --diff + ansible-playbook playbooks/.yml + ansible-playbook playbooks/.yml -e pull=always # update images + +A stack that exists here but not in the repository was deployed by hand, and +nothing will keep it up to date. Bring it under the repository; the +`adding-a-stack` skill there walks through it. + +## Starting and stopping by hand: stackctl + +For maintenance on this host. It runs plain `docker compose` in each stack's +directory, with the same `.env` and project name a deploy uses. It does not +deploy: it never copies a file. + + ./stackctl list # the groups, and every stack found here + ./stackctl ps # running/total containers per stack + ./stackctl up # a group, a stack, or "all" + ./stackctl down # groups and stacks mix + ./stackctl pull && ./stackctl up + ./stackctl logs # follows one stack + ./stackctl -y restart all # -y skips the confirmation below + +`up` starts the stacks under `order:` in `stack-groups` first; `down` runs in +reverse. `down` and `restart` ask before touching a stack under `confirm:` — +read that line before restarting anything. A vault restarts sealed, and every +deploy that reads a secret fails until it is unsealed. Groups are edited in +`server/{{ inventory_hostname }}/stack-groups` in the repository, not here. + +## OpenBao: unseal, log in, add a secret + +Only on a host that runs the `openbao` stack; elsewhere, skip this section. +The vault runs in the `openbao` container. Every command below goes through +`docker exec`, which inherits the container's `BAO_ADDR`, so none of them needs +an address. Keep the five unseal keys and the root token in a password manager, +never on this host. + +### Unseal, after any restart or reboot + +OpenBao starts sealed. Until it is unsealed, every deploy that reads a secret +fails. Three of the five unseal keys, each a different one: + + docker exec -it openbao bao operator unseal # paste key 1 + docker exec -it openbao bao operator unseal # paste key 2 + docker exec -it openbao bao operator unseal # paste key 3 + docker exec openbao bao status # expect: Sealed false + +`-it` makes it prompt, so a key never lands in the shell history or the process +table. Never pass one as an argument. + +### Log in, to add or change a secret + +Writing needs the root token. The deploy token can only read, so with it every +write fails with `403 permission denied`. + + docker exec -it openbao bao login # paste the root token + + # a generated value; tr strips the newline, which the deploy refuses + openssl rand -hex 32 | tr -d '\n' \ + | docker exec -i openbao bao kv put kv/stacks/ =- + + # a value you have, typed without echo and kept out of the history + read -rs SECRET + printf '%s' "$SECRET" | docker exec -i openbao bao kv patch kv/stacks/ =- + unset SECRET + + docker exec openbao sh -c 'rm -f ~/.bao-token' # log out: drop the saved root token + +`kv put` replaces every field at that path; `kv patch` adds or changes one and +keeps the rest. Then redeploy the stack, which renders the new value into its +`.env`. + +The root token from `bao operator init` does not expire; it is the one to log +in with, every time. Only if it is lost or revoked does a new one need making, +from three unseal keys with `bao operator generate-root`. + +The full documentation, including secrets and preparing a new host, is in +`README.md` and `docs/` in the repository.