Add the host bootstrap: deploy user, key, docker access, stack root

bootstrap.yml prepares a host as an admin via -u; new-deploy-key.sh makes the
key pair and refuses to write inside the repo. CI deploys refuse the bootstrap
playbook, since the dispatch API ignores the dropdown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/bootstrap.md

@@ -0,0 +1,155 @@
+# Preparing a host
+
+Everything else in this repository assumes a host that already has a `deploy`
+user, a key it trusts, access to Docker, and a writable stack root. This is how
+that gets made. It runs once per host.
+
+There is a chicken-and-egg problem in the middle of it: every other playbook
+connects **as** `deploy`, so the playbook that creates `deploy` cannot. It
+connects as an administrator you already have access with, and it is the only
+thing in this repository that does.
+
+## 1. Generate the deploy key
+
+```bash
+scripts/new-deploy-key.sh                 # ~/.ssh/deploy_key
+scripts/new-deploy-key.sh ~/.ssh/other    # somewhere else
+```
+
+`ed25519`, and no passphrase — nothing in CI or in a playbook run can type one,
+and an encrypted key fails much later with an authentication error that says
+nothing about a passphrase.
+
+The script prints the paths and the public key. It does not print the private
+key unless you pass `--print-secret`, for the same reason
+[`load-ssh-key.sh`](../scripts/load-ssh-key.sh) does not: key material should
+stay out of terminals, logs and the process list.
+
+It refuses two things. It will not overwrite an existing key without `--force`,
+because replacing a live deploy key locks you out of every host that still
+trusts the old one. And it will not write anywhere inside this repository:
+gitleaks scans the working tree **and** the full history, so a key committed
+once stays leaked after you delete the file.
+
+One key serves every host and every stack. Rotating it is therefore an
+all-hosts operation — replace the line in each `authorized_keys`, then update
+the one CI secret.
+
+## 2. Prepare the host
+
+```bash
+cd ansible
+ansible-playbook playbooks/bootstrap.yml -u <admin-user> -k -K -e deploy_sudo=true
+```
+
+`deploy_sudo=true` is what grants the deploy user passwordless sudo, and it is
+off by default. Leave it off only on a host where no stack sets `owner` or
+`group` in `stack_dirs` — those entries are the only thing the role escalates
+for, and `metrics.yml` and `openbao.yml` both have them. Omit it and the run
+still ends green, with `skipped=1` in the recap as the only sign; the
+consequence turns up much later, in a different playbook, as
+`Missing sudo password`.
+
+`-u` is honoured because the deploy account is `remote_user` in `ansible.cfg`
+and not `ansible_user` in the inventory. That distinction exists for this
+playbook: an inventory variable outranks a command-line `-u`, so naming the
+account there would make the flag a silent no-op and send the run in as the
+user that does not exist yet — or, on a host already bootstrapped, as the one
+that cannot escalate.
+
+Running it on the host itself avoids SSH altogether:
+
+```bash
+cd ansible
+ansible-playbook playbooks/bootstrap.yml -c local -K -e deploy_sudo=true
+```
+
+`-c local` executes as whoever invoked it and ignores `ansible_user` altogether.
+
+`-k` asks for that account's SSH password, `-K` for its sudo password; drop
+either if it already uses a key or has passwordless sudo. `SSH_KEY_PATH` is not
+needed here. The public key is read from whichever machine runs Ansible, so an
+on-host run needs `deploy_pubkey_file` to point at a copy that is there.
+
+| Variable | Default | |
+| --- | --- | --- |
+| `deploy_user` | `deploy` | Must match `remote_user` in `ansible/ansible.cfg` |
+| `deploy_pubkey_file` | `~/.ssh/deploy_key.pub` | Read from the control node |
+| `deploy_sudo` | `false` | Passwordless sudo for the deploy user |
+
+What it does, and why each part matters:
+
+- **Creates `deploy` in the `docker` group.** The `compose_stack` role runs
+  Compose unescalated, which is what stops bind mounts being created
+  root-owned. Without group membership every Docker call fails on the socket.
+- **Installs rsync.** The payload sync is `ansible.posix.synchronize`, which
+  shells out to rsync **at both ends**. Missing it on the target fails the first
+  deploy; missing it on the machine running Ansible fails the same task with the
+  same message, so read the PATH in the error to tell the two apart.
+- **Installs the public key**, never exclusively — rewriting the whole
+  `authorized_keys` would drop a key that is currently somebody's only way in.
+- **Creates `{{ stack_root }}` owned by `deploy`.** This is the one that bites
+  silently: the role creates `/srv/stacks/<stack>` unescalated and `/srv` is
+  root-owned, so skipping it produces a permission error on first deploy that
+  looks nothing like its cause.
+- **Grants passwordless sudo only when you ask.** The role escalates when a
+  `stack_dirs` entry sets `owner` or `group`, which is how a bind mount gets the
+  uid its container runs as. `metrics.yml` and `openbao.yml` both need it, so
+  any host running either wants `-e deploy_sudo=true`. The sudoers file is
+  written through `visudo -cf`, because a malformed one can lock everyone out of
+  a host permanently. `NOPASSWD:ALL` is broad — narrowing it to the commands the
+  `file` module actually invokes is a reasonable hardening once a host settles.
+
+It is not imported by `site.yml` and not offered in the CI deploy dropdown.
+Both authenticate as `deploy` — which on a fresh host does not exist yet, and on
+a bootstrapped one is the unprivileged account this playbook exists to grant
+privilege to. Either way it cannot bootstrap itself. Making CI able to would
+mean storing an administrator's credential where the runner can reach it, a
+larger and permanent grant in place of one interactive command.
+
+**`--check` does not work on a host that has never been bootstrapped.** It stops
+at *Authorise the deploy key* with `Either user must exist or you must provide
+full path to key file in check mode`: check mode did not really create the user,
+so the module has no home directory to inspect. That is Ansible's limitation,
+not a fault in the run. On an already-bootstrapped host check mode is clean.
+
+### rsync on the machine running Ansible
+
+The half this playbook cannot do for you. `synchronize` runs rsync locally and
+pushes, so whatever drives the deploy needs the binary as well:
+
+- **A CI runner.** GitHub's hosted runners ship it. The GitLab jobs install it
+  in the `.ansible` `before_script`. The Gitea deploy installs it through the
+  toolchain action, because the act_runner images do not carry it. A runner
+  without it fails with `Failed to find required executable "rsync"` and a PATH
+  full of `/opt/hostedtoolcache/...`, which is the giveaway that the message is
+  about the runner and not about your host.
+- **Your workstation**, if you deploy from it: `apt install rsync`, or the
+  equivalent.
+
+### The first deploy of a stack cannot be a check run
+
+`--check` creates nothing, so on a host that has never run a given stack there
+is no project directory for Compose to inspect. The role notices and skips the
+compose step with a note rather than failing, and the payload diff above it is
+still reported — but the stack itself is only really created by a real deploy.
+
+## 3. Hand the key to CI
+
+```bash
+base64 -w0 < ~/.ssh/deploy_key
+```
+
+Store it as `SSH_PRIVATE_KEY_B64`. Where that lives differs per platform —
+[secrets.md](secrets.md) has the table, the other backends, and what not to use
+on Windows.
+
+## 4. Confirm it before deploying anything
+
+Run the **connectivity** workflow first (`ansible:ping` on GitLab).
+`SUCCESS => {"ping": "pong"}` proves
+the runner decoded the key, reached port 22, authenticated, and ran Python on
+the far side — everything a deploy needs except the playbook.
+
+If it fails, the message tells you which of three different problems you have;
+[connectivity.md](connectivity.md) has the table.