Write a README into /srv/stacks, and an adoption checklist
The README says what the directory is, links repo_url as the source of truth, and covers deploying, stackctl and OpenBao. stackctl.yml refuses to run while repo_url is a placeholder; AGENTS.md gains "Adopting this repository". Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
35b1378080
commit
ecf63a62a2
@@ -8,6 +8,12 @@ and `.claude/skills/` points at `skills/`. Put content here or in `skills/`,
|
||||
never in a pointer — the whole arrangement exists so that every agent, and every
|
||||
human, reads the same text.
|
||||
|
||||
> **Setting this up for your own hosts?** If you are an agent that has just
|
||||
> copied, forked or initialised this repository for someone, do
|
||||
> [Adopting this repository](#adopting-this-repository) first, before any other
|
||||
> change. Every host, address, hostname and URL in it is a placeholder, and a
|
||||
> setup that still contains them is not finished.
|
||||
|
||||
## What this repository is
|
||||
|
||||
A pattern for deploying Docker Compose stacks to a small number of Linux hosts
|
||||
@@ -26,6 +32,53 @@ Everything under `server/` and every host name is an **example**. `example.com`
|
||||
is reserved by RFC 2606 and can never resolve to a real service. Replace them
|
||||
with yours; the framework around them is the part meant to be reused.
|
||||
|
||||
## Adopting this repository
|
||||
|
||||
When this repository is copied for real hosts, the examples have to become that
|
||||
setup's own. Leaving one behind is not harmless: a CI button for a stack that
|
||||
does not exist, a connectivity check against `edge.example.com`, or a README on
|
||||
the host naming `git.example.com` as the source of truth all look like they
|
||||
work and mislead whoever comes next.
|
||||
|
||||
**Ask the person for these. Do not invent them**, and do not leave an
|
||||
`example.com` value "to fill in later":
|
||||
|
||||
- each host's inventory name, address, and SSH user and port if not `deploy`/22;
|
||||
- where their copy of this repository lives (`repo_url`);
|
||||
- which of the example stacks, if any, they actually want, and their hostnames.
|
||||
|
||||
Then:
|
||||
|
||||
1. **Hosts.** `ansible/inventory/hosts.yml`, and rename the directories under
|
||||
`server/` to match — the role resolves `server/<inventory_hostname>/`. Update
|
||||
the `hosts:` of every playbook you keep, and the host table at the top of
|
||||
this file.
|
||||
2. **Repository URL.** `repo_url` in `ansible/inventory/group_vars/all.yml`.
|
||||
`stackctl.yml` refuses to run until it is no longer a placeholder.
|
||||
3. **Stacks.** For each example stack not wanted, remove all of: its playbook,
|
||||
its payload under `server/`, its `site.yml` import, its CI buttons (three
|
||||
places — step 4 of the `adding-a-stack` skill), its `stack-groups` entries,
|
||||
and its rows in the tables in `README.md`, `server/README.md`, the
|
||||
worked-examples paragraph below, and the skill. Then add the person's real
|
||||
stacks with the `adding-a-stack` skill.
|
||||
4. **Hostnames and addresses.** Every `VIRTUAL_HOST`, `LETSENCRYPT_HOST`, vhost
|
||||
file under `reverse-proxy/proxy-data/vhost/`, and the default `hosts` input
|
||||
of both `connectivity.yml` workflows.
|
||||
5. **CI.** Delete the platforms not in use (`.github/`, `.gitea/`,
|
||||
`.gitlab-ci.yml`, and the matching `docs/ci-*.md`), then set the secrets in
|
||||
`docs/secrets.md` on the one that is.
|
||||
6. **Groups.** `server/<host>/stack-groups` for each host.
|
||||
|
||||
Done means both of these hold:
|
||||
|
||||
```bash
|
||||
make check # passes
|
||||
grep -rn 'example\.com' --exclude-dir=.git . # only prose explaining the convention
|
||||
```
|
||||
|
||||
Then report what was changed, and what still needs a person: the deploy key,
|
||||
`bootstrap.yml` on each host, the vault, and the CI secrets.
|
||||
|
||||
## Vocabulary
|
||||
|
||||
- **host** — one machine (`edge`, `app`). The top-level directories under
|
||||
@@ -110,8 +163,8 @@ Worked examples for each: `static-site.yml` (the minimum), `reverse-proxy.yml`
|
||||
`openbao.yml` (owner-only escalation; a notice rather than a restart on
|
||||
`compose_stack_synced`), `webapp-staging.yml`
|
||||
(`stack_src` + `stack_dest` + `stack_env`), `banner.yml` (not a stack at all),
|
||||
`stackctl.yml` (not a stack either: one shared file and one per-host file into
|
||||
`stack_root`), `bootstrap.yml` (preparing a host; see `docs/bootstrap.md`).
|
||||
`stackctl.yml` (not a stack either: one shared file, one per-host file and a
|
||||
templated README into `stack_root`), `bootstrap.yml` (preparing a host; see `docs/bootstrap.md`).
|
||||
|
||||
#### `stack_dirs`
|
||||
|
||||
|
||||
@@ -65,7 +65,8 @@ ansible/
|
||||
└── playbooks/
|
||||
├── site.yml # every stack, in dependency order
|
||||
├── bootstrap.yml # prepares a host; run once, by an admin
|
||||
├── stackctl.yml # installs stackctl and each host's stack-groups
|
||||
├── stackctl.yml # installs stackctl, each host's stack-groups, and a README
|
||||
├── templates/ # that README
|
||||
└── <stack>.yml # one per stack, runnable on its own
|
||||
server/<host>/<stack>/ # the payload: compose files and config, copied verbatim
|
||||
server/shared/ # payloads used by more than one host
|
||||
@@ -92,6 +93,10 @@ TODO.md # known defects and deferred decisions
|
||||
> lives in a world-writable directory, and a runner's build directory is exactly
|
||||
> that.
|
||||
|
||||
**Using this for your own hosts?** Every host, hostname and URL here is a
|
||||
placeholder. Start with *Adopting this repository* in [AGENTS.md](AGENTS.md) — a
|
||||
checklist of everything to replace, written so an agent can follow it too.
|
||||
|
||||
**A host that has never been deployed to needs preparing first** — a `deploy`
|
||||
user, its key, the `docker` group, rsync, and a writable `/srv/stacks`. That is
|
||||
`scripts/new-deploy-key.sh` and `playbooks/bootstrap.yml`, once per host; see
|
||||
@@ -247,7 +252,10 @@ Groups are per host, in `server/<host>/stack-groups`, along with the start order
|
||||
(`down` reverses it) and the stacks that need confirming before `down` or
|
||||
`restart`. The script is one file for every host, in
|
||||
`server/shared/components/stackctl`. `playbooks/stackctl.yml`, which `site.yml`
|
||||
runs, installs both; edit them here, not on the host.
|
||||
runs, installs both; edit them here, not on the host. The same playbook writes
|
||||
`/srv/stacks/README.md`, which tells anyone who logs in to a host what the
|
||||
directory is, how it is deployed, and that your repository — `repo_url` in
|
||||
`group_vars/all.yml` — is the source of truth.
|
||||
|
||||
## Secrets
|
||||
|
||||
|
||||
@@ -19,6 +19,12 @@ stack_root: /srv/stacks
|
||||
# See "Deploy versus update" in the README.
|
||||
pull: policy
|
||||
|
||||
# Where your copy of this repository lives, for people who find its files on a
|
||||
# host: the README that stackctl.yml writes into stack_root links here as the
|
||||
# single source of truth. A placeholder until you change it, and stackctl.yml
|
||||
# refuses to run while it is one — see "Adopting this repository" in AGENTS.md.
|
||||
repo_url: https://git.example.com/you/config
|
||||
|
||||
# Repository root, so playbooks and the role can find payloads under server/
|
||||
# regardless of the directory ansible-playbook was invoked from.
|
||||
repo_root: "{{ (inventory_dir ~ '/../..') | realpath }}"
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
ansible.builtin.import_playbook: reverse-proxy.yml
|
||||
- name: Default vhost banner
|
||||
ansible.builtin.import_playbook: banner.yml
|
||||
- name: Stackctl
|
||||
- name: Stackctl and the stack root README
|
||||
ansible.builtin.import_playbook: stackctl.yml
|
||||
- name: Metrics
|
||||
ansible.builtin.import_playbook: metrics.yml
|
||||
|
||||
@@ -1,13 +1,19 @@
|
||||
---
|
||||
# stackctl, and each host's groups for it, into the stack root. Not a Compose
|
||||
# stack: two files and no `docker compose up`, so plain tasks rather than the
|
||||
# role, as with banner.yml.
|
||||
# stackctl, each host's groups for it, and a README, into the stack root. Not a
|
||||
# Compose stack: three files and no `docker compose up`, so plain tasks rather
|
||||
# than the role, as with banner.yml.
|
||||
#
|
||||
# The script is one file shared by every host, under server/shared/components/.
|
||||
# The groups are per host, at server/<host>/stack-groups, because each host runs
|
||||
# different stacks; a host without one still gets the script, and "all" and
|
||||
# stack names work there without any groups. See "On the host: stackctl" in the
|
||||
# README.
|
||||
#
|
||||
# The README is for whoever logs in to the host: what the directory is, that the
|
||||
# repository at repo_url is the source of truth, how to deploy, and stackctl.
|
||||
# It is a template, which payloads never are, because it is not a payload: it
|
||||
# names the host and the repository. Nothing in it changes between runs, so a
|
||||
# second run still reports no changes.
|
||||
- name: Install stackctl
|
||||
hosts:
|
||||
- edge
|
||||
@@ -20,6 +26,18 @@
|
||||
stackctl_groups_src: "{{ repo_root }}/server/{{ inventory_hostname }}/stack-groups"
|
||||
|
||||
tasks:
|
||||
# A README sending people to git.example.com is worse than none: it names a
|
||||
# source of truth that does not exist. Fail until repo_url is real.
|
||||
- name: Verify repo_url points at your repository
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- "'example.com' not in repo_url"
|
||||
fail_msg: >-
|
||||
repo_url in inventory/group_vars/all.yml is still the placeholder
|
||||
{{ repo_url }}. Set it to where your copy of this repository lives —
|
||||
see "Adopting this repository" in AGENTS.md.
|
||||
quiet: true
|
||||
|
||||
- name: Install the stackctl script
|
||||
ansible.builtin.copy:
|
||||
src: "{{ stackctl_src }}"
|
||||
@@ -33,3 +51,9 @@
|
||||
dest: "{{ stack_root }}/stack-groups"
|
||||
mode: "0644"
|
||||
when: stackctl_groups_src is file
|
||||
|
||||
- name: Install the README for the stack root
|
||||
ansible.builtin.template:
|
||||
src: stacks-README.md.j2
|
||||
dest: "{{ stack_root }}/README.md"
|
||||
mode: "0644"
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
# /srv/stacks on {{ inventory_hostname }}
|
||||
|
||||
**Everything in this directory is managed by Ansible. Do not edit it here.**
|
||||
|
||||
The single source of truth is the repository:
|
||||
|
||||
{{ repo_url }}
|
||||
|
||||
This file is written by `ansible/playbooks/stackctl.yml` in that repository.
|
||||
Change it there.
|
||||
|
||||
## What is here
|
||||
|
||||
Every subdirectory with a compose file is one Docker Compose stack. Each is
|
||||
copied from the repository — `server/{{ inventory_hostname }}/<stack>/`, or
|
||||
`server/shared/<stack>/` for one that runs on several hosts — and brought up by
|
||||
`ansible/playbooks/<stack>.yml`.
|
||||
|
||||
| Path | What it is |
|
||||
| --- | --- |
|
||||
| `<stack>/docker-compose.yml`, config | Copied from the repository on every deploy. A local edit is overwritten by the next one |
|
||||
| `<stack>/.env` | Never in the repository. For a stack whose playbook sets `stack_secrets`, rendered from the vault on every deploy: change the value there and redeploy, never this file. Otherwise placed here by hand |
|
||||
| `<stack>/<data dirs>` | The stack's own state: databases, uploads, config the app writes. Not in the repository |
|
||||
| `stackctl`, `stack-groups` | The by-hand runner below, and this host's groups |
|
||||
| `README.md` | This file |
|
||||
|
||||
## Changing something
|
||||
|
||||
1. Change it in the repository and push.
|
||||
2. Deploy it from CI: run the **deploy** workflow (or the stack's `check:` and
|
||||
`deploy:` jobs on GitLab), in check mode first, and read what it would change.
|
||||
Pull `always` fetches newer images; a normal deploy keeps the ones already
|
||||
here.
|
||||
3. Or, from a checkout, in its `ansible/` directory:
|
||||
|
||||
ansible-playbook playbooks/<stack>.yml --check --diff
|
||||
ansible-playbook playbooks/<stack>.yml
|
||||
ansible-playbook playbooks/<stack>.yml -e pull=always # update images
|
||||
|
||||
A stack that exists here but not in the repository was deployed by hand, and
|
||||
nothing will keep it up to date. Bring it under the repository; the
|
||||
`adding-a-stack` skill there walks through it.
|
||||
|
||||
## Starting and stopping by hand: stackctl
|
||||
|
||||
For maintenance on this host. It runs plain `docker compose` in each stack's
|
||||
directory, with the same `.env` and project name a deploy uses. It does not
|
||||
deploy: it never copies a file.
|
||||
|
||||
./stackctl list # the groups, and every stack found here
|
||||
./stackctl ps # running/total containers per stack
|
||||
./stackctl up <group> # a group, a stack, or "all"
|
||||
./stackctl down <group> <stack> # groups and stacks mix
|
||||
./stackctl pull <stack> && ./stackctl up <stack>
|
||||
./stackctl logs <stack> # follows one stack
|
||||
./stackctl -y restart all # -y skips the confirmation below
|
||||
|
||||
`up` starts the stacks under `order:` in `stack-groups` first; `down` runs in
|
||||
reverse. `down` and `restart` ask before touching a stack under `confirm:` —
|
||||
read that line before restarting anything. A vault restarts sealed, and every
|
||||
deploy that reads a secret fails until it is unsealed. Groups are edited in
|
||||
`server/{{ inventory_hostname }}/stack-groups` in the repository, not here.
|
||||
|
||||
## OpenBao: unseal, log in, add a secret
|
||||
|
||||
Only on a host that runs the `openbao` stack; elsewhere, skip this section.
|
||||
The vault runs in the `openbao` container. Every command below goes through
|
||||
`docker exec`, which inherits the container's `BAO_ADDR`, so none of them needs
|
||||
an address. Keep the five unseal keys and the root token in a password manager,
|
||||
never on this host.
|
||||
|
||||
### Unseal, after any restart or reboot
|
||||
|
||||
OpenBao starts sealed. Until it is unsealed, every deploy that reads a secret
|
||||
fails. Three of the five unseal keys, each a different one:
|
||||
|
||||
docker exec -it openbao bao operator unseal # paste key 1
|
||||
docker exec -it openbao bao operator unseal # paste key 2
|
||||
docker exec -it openbao bao operator unseal # paste key 3
|
||||
docker exec openbao bao status # expect: Sealed false
|
||||
|
||||
`-it` makes it prompt, so a key never lands in the shell history or the process
|
||||
table. Never pass one as an argument.
|
||||
|
||||
### Log in, to add or change a secret
|
||||
|
||||
Writing needs the root token. The deploy token can only read, so with it every
|
||||
write fails with `403 permission denied`.
|
||||
|
||||
docker exec -it openbao bao login # paste the root token
|
||||
|
||||
# a generated value; tr strips the newline, which the deploy refuses
|
||||
openssl rand -hex 32 | tr -d '\n' \
|
||||
| docker exec -i openbao bao kv put kv/stacks/<stack> <field>=-
|
||||
|
||||
# a value you have, typed without echo and kept out of the history
|
||||
read -rs SECRET
|
||||
printf '%s' "$SECRET" | docker exec -i openbao bao kv patch kv/stacks/<stack> <field>=-
|
||||
unset SECRET
|
||||
|
||||
docker exec openbao sh -c 'rm -f ~/.bao-token' # log out: drop the saved root token
|
||||
|
||||
`kv put` replaces every field at that path; `kv patch` adds or changes one and
|
||||
keeps the rest. Then redeploy the stack, which renders the new value into its
|
||||
`.env`.
|
||||
|
||||
The root token from `bao operator init` does not expire; it is the one to log
|
||||
in with, every time. Only if it is lost or revoked does a new one need making,
|
||||
from three unseal keys with `bao operator generate-root`.
|
||||
|
||||
The full documentation, including secrets and preparing a new host, is in
|
||||
`README.md` and `docs/` in the repository.
|
||||
Reference in New Issue
Block a user