config-public/docs/bootstrap.md

Preparing a host

Everything else in this repository assumes a host that already has a deploy user, a key it trusts, access to Docker, and a writable stack root. This is how that gets made. It runs once per host.

There is a chicken-and-egg problem in the middle of it: every other playbook connects as deploy, so the playbook that creates deploy cannot. It connects as an administrator you already have access with, and it is the only thing in this repository that does.

1. Generate the deploy key

scripts/new-deploy-key.sh                 # ~/.ssh/deploy_key
scripts/new-deploy-key.sh ~/.ssh/other    # somewhere else

ed25519, and no passphrase — nothing in CI or in a playbook run can type one, and an encrypted key fails much later with an authentication error that says nothing about a passphrase.

The script prints the paths and the public key. It does not print the private key unless you pass --print-secret, for the same reason load-ssh-key.sh does not: key material should stay out of terminals, logs and the process list.

It refuses two things. It will not overwrite an existing key without --force, because replacing a live deploy key locks you out of every host that still trusts the old one. And it will not write anywhere inside this repository: gitleaks scans the working tree and the full history, so a key committed once stays leaked after you delete the file.

One key serves every host and every stack. Rotating it is therefore an all-hosts operation — replace the line in each authorized_keys, then update the one CI secret.

2. Prepare the host

cd ansible
ansible-playbook playbooks/bootstrap.yml -u <admin-user> -k -K -e deploy_sudo=true

deploy_sudo=true is what grants the deploy user passwordless sudo, and it is off by default. Leave it off only on a host where no stack sets owner or group in stack_dirs — those entries are the only thing the role escalates for, and metrics.yml and openbao.yml both have them. Omit it and the run still ends green, with skipped=1 in the recap as the only sign; the consequence turns up much later, in a different playbook, as Missing sudo password.

-u is honoured because the deploy account is remote_user in ansible.cfg and not ansible_user in the inventory. That distinction exists for this playbook: an inventory variable outranks a command-line -u, so naming the account there would make the flag a silent no-op and send the run in as the user that does not exist yet — or, on a host already bootstrapped, as the one that cannot escalate.

Running it on the host itself avoids SSH altogether:

cd ansible
ansible-playbook playbooks/bootstrap.yml -c local -K -e deploy_sudo=true

-c local executes as whoever invoked it and ignores ansible_user altogether.

-k asks for that account's SSH password, -K for its sudo password; drop either if it already uses a key or has passwordless sudo. SSH_KEY_PATH is not needed here. The public key is read from whichever machine runs Ansible, so an on-host run needs deploy_pubkey_file to point at a copy that is there.

Variable	Default
deploy_user	deploy	Must match remote_user in ansible/ansible.cfg
deploy_pubkey_file	~/.ssh/deploy_key.pub	Read from the control node
deploy_sudo	false	Passwordless sudo for the deploy user

What it does, and why each part matters:

• Creates deploy in the docker group. The compose_stack role runs Compose unescalated, which is what stops bind mounts being created root-owned. Without group membership every Docker call fails on the socket.
• Installs rsync. The payload sync is ansible.posix.synchronize, which shells out to rsync at both ends. Missing it on the target fails the first deploy; missing it on the machine running Ansible fails the same task with the same message, so read the PATH in the error to tell the two apart.
• Installs the public key, never exclusively — rewriting the whole authorized_keys would drop a key that is currently somebody's only way in.
• Creates {{ stack_root }} owned by deploy. This is the one that bites silently: the role creates /srv/stacks/<stack> unescalated and /srv is root-owned, so skipping it produces a permission error on first deploy that looks nothing like its cause.
• Grants passwordless sudo only when you ask. The role escalates when a stack_dirs entry sets owner or group, which is how a bind mount gets the uid its container runs as. metrics.yml and openbao.yml both need it, so any host running either wants -e deploy_sudo=true. The sudoers file is written through visudo -cf, because a malformed one can lock everyone out of a host permanently. NOPASSWD:ALL is broad — narrowing it to the commands the file module actually invokes is a reasonable hardening once a host settles.

It is not imported by site.yml and not offered in the CI deploy dropdown. Both authenticate as deploy — which on a fresh host does not exist yet, and on a bootstrapped one is the unprivileged account this playbook exists to grant privilege to. Either way it cannot bootstrap itself. Making CI able to would mean storing an administrator's credential where the runner can reach it, a larger and permanent grant in place of one interactive command.

--check does not work on a host that has never been bootstrapped. It stops at Authorise the deploy key with Either user must exist or you must provide full path to key file in check mode: check mode did not really create the user, so the module has no home directory to inspect. That is Ansible's limitation, not a fault in the run. On an already-bootstrapped host check mode is clean.

rsync on the machine running Ansible

The half this playbook cannot do for you. synchronize runs rsync locally and pushes, so whatever drives the deploy needs the binary as well:

• A CI runner. GitHub's hosted runners ship it. The GitLab jobs install it in the .ansible before_script. The Gitea deploy installs it through the toolchain action, because the act_runner images do not carry it. A runner without it fails with Failed to find required executable "rsync" and a PATH full of /opt/hostedtoolcache/..., which is the giveaway that the message is about the runner and not about your host.
• Your workstation, if you deploy from it: apt install rsync, or the equivalent.

The first deploy of a stack cannot be a check run

--check creates nothing, so on a host that has never run a given stack there is no project directory for Compose to inspect. The role notices and skips the compose step with a note rather than failing, and the payload diff above it is still reported — but the stack itself is only really created by a real deploy.

3. Hand the key to CI

base64 -w0 < ~/.ssh/deploy_key

Store it as SSH_PRIVATE_KEY_B64. Where that lives differs per platform — secrets.md has the table, the other backends, and what not to use on Windows.

4. Confirm it before deploying anything

Run the connectivity workflow first (ansible:ping on GitLab). SUCCESS => {"ping": "pong"} proves the runner decoded the key, reached port 22, authenticated, and ran Python on the far side — everything a deploy needs except the playbook.

If it fails, the message tells you which of three different problems you have; connectivity.md has the table.