config-public/docs/secrets.md

# Secrets

Three separate things, handled three different ways. The split matters: the first
is needed to *reach* a host, the second is needed by the software *on* it, and
the third is a mistake.

## 1. The deploy SSH key

Ansible knows exactly one thing about credentials: **`SSH_KEY_PATH`**, a path to
a `0600` private key. `inventory/group_vars/all.yml` feeds it to
`ansible_ssh_private_key_file` and nothing else in the repository touches key
material.

Everything about *where the key comes from* lives in `scripts/load-ssh-key.sh`,
which prints a path and never prints the key:

```bash
export SSH_KEY_PATH="$(scripts/load-ssh-key.sh)"         # env:   already on disk
export SSH_KEY_PATH="$(scripts/load-ssh-key.sh ci)"      # ci:    from a CI secret
export SSH_KEY_PATH="$(scripts/load-ssh-key.sh vault)"   # vault: from Vault / OpenBao
```

Add a backend there rather than teaching a playbook about secrets. That seam is
the reason the same playbooks run unchanged from a laptop, from GitHub Actions,
from GitLab CI and from Gitea Actions.

Whichever backend you use, the matching **public** key must be in the deploy
user's `~/.ssh/authorized_keys` on every host, or the key is useless.

### Storing it in CI

Store the key **base64-encoded**, as a single line:

```bash
base64 -w0 < ~/.ssh/deploy_key            # Linux
base64 < ~/.ssh/deploy_key | tr -d '\n'   # macOS
```

```powershell
# Windows PowerShell
[Convert]::ToBase64String([IO.File]::ReadAllBytes("$HOME\.ssh\deploy_key"))
```

On Windows use exactly that. `certutil -encode` wraps its output in a header and
line breaks, and `Get-Content` re-encodes the bytes — both produce something that
looks like base64 and decodes to an unusable key. Do not re-wrap the output
either; it is one long line by design.

Then set it as `SSH_PRIVATE_KEY_B64`:

| Platform | Where | Notes |
| --- | --- | --- |
| GitHub | Settings → Environments → `production` → Secrets | Scoped to the environment, so only jobs naming it can read it |
| GitLab | Settings → CI/CD → Variables | Mark **Masked** and **Protected** |
| Gitea | Settings → Actions → Secrets | No environments and no masking controls: any run in the repository can read it. See [ci-gitea.md](ci-gitea.md) |

Why base64 rather than the raw key? GitLab can only *mask* a value that is a
single line with no whitespace, and an OpenSSH private key is neither. GitHub
redacts secrets from logs regardless, but a single-line value survives being
passed between steps without surprises. One encoding satisfies both.

A raw `SSH_PRIVATE_KEY` still works — as a GitLab File-type variable, as a
GitHub multi-line secret, or as a path on disk. The loader accepts all three. It
just cannot be masked on GitLab, so the key can appear in a job log if something
echoes it.

### Storing it in a vault

`vault` reads a HashiCorp Vault or OpenBao KV v2 secret. The expected layout,
overridable with `VAULT_KV_MOUNT` / `VAULT_SECRET_PATH` / `VAULT_SECRET_FIELD`
(or the `BAO_*` spellings):

```
kv/infra/ansible/deploy-key      field: private_key
```

A read-only policy is enough:

```hcl
path "kv/data/infra/ansible/deploy-key" {
  capabilities = ["read"]
}
```

**Keep the vault off the deploy path.** If the vault runs on a host this
repository deploys to, and it seals on reboot, then a deploy that fetches its key
from the vault cannot run — including the deploy that would fix that host. Host
it elsewhere, or auto-unseal it. A CI secret is the dependable route.

## 2. Per-stack `.env` files

Stack secrets live on the hosts, not here.

The role excludes `.env` from every sync (`stack_exclude`), so editing one in
this repository has no effect, and a host's `.env` is never overwritten by a
deploy. To change one, edit it on the host and redeploy.

Each example stack that needs credentials ships a `.env.example` listing the
variable names with no values, and reads them in its compose file with Compose's
required-variable syntax:

```yaml
    environment:
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set it in .env on the host}
```

That is worth copying. Without the `:?`, a missing variable expands to the empty
string and Postgres starts with **no password at all** — a failure that looks
like success until someone finds it. With it, Compose refuses to start and names
the variable.

`webapp.yml` goes one step further and asserts the file exists on the host before
doing anything, so the error arrives before the payload is synced rather than
halfway through starting the stack:

```yaml
  pre_tasks:
    - name: Look for the webapp .env on the host
      ansible.builtin.stat:
        path: "{{ webapp_dir }}/.env"
      register: webapp_env
      changed_when: false

    - name: Verify the webapp .env is present on the host
      ansible.builtin.assert:
        that:
          - webapp_env.stat.exists
        fail_msg: >-
          {{ webapp_dir }}/.env is missing on {{ inventory_hostname }}.
        quiet: true
```

Do this for any stack whose compose file has no safe defaults.

## 3. Secrets committed to the repository

There is no third category. This section exists because it is the failure mode
this layout is designed to prevent, and it is worth being blunt about why.

A credential written directly into a compose file:

- **cannot be excluded from the sync.** `stack_exclude` keeps `.env` off the
  wire; it can do nothing about a value inside `docker-compose.yml`.
- **is in every clone, forever.** Git history is the point of git. Deleting the
  file in a later commit changes nothing about the credential.
- **cannot be rotated without a deploy**, so it usually is not rotated at all.
- **escapes its blast radius.** A personal access token carries its creator's
  access to everything they can reach, not just the project it was added to. A
  cloud application secret outlives the VM it was used on.

If you find one, the fix is to **rotate the credential**, not to delete the file.
Removing it stops new clones carrying it and changes nothing else. Then move it
to a `.env` on the host and read it with `${VAR:?}`.

### The guard

The `lint` workflow — [`.github/workflows/lint.yml`](../.github/workflows/lint.yml),
[`.gitea/workflows/lint.yml`](../.gitea/workflows/lint.yml) and the `lint` job in
`.gitlab-ci.yml` — runs [gitleaks](https://github.com/gitleaks/gitleaks) on every
push, over the working tree **and** the full history: a secret that was committed
and then removed is still leaked, and the scan should say so. `.gitleaks.toml`
allowlists only `*.env.example` files and this document.

On GitHub it runs the upstream Docker image rather than the marketplace action on
purpose: the action requires a licence key for organisation-owned repositories
and silently does nothing without one, which is the worst possible behaviour for
a secret scanner. The Gitea port runs the pinned release binary instead, because
a job there is itself a container with no Docker socket.

Do not add an allowlist entry to make a finding go away. Rotate the credential.