# compose-stack-deploy

A small, complete pattern for deploying Docker Compose stacks to a handful of
Linux hosts with Ansible — and for letting CI do it safely, one stack at a time,
on purpose rather than on every push.

It is deliberately not a framework. There is **one** Ansible role, about 90 lines
of tasks, and a convention about where files live. Everything else in this
repository is documentation and six worked examples.

```
server/app/webapp/          →  rsync  →  /srv/stacks/webapp/  on app
ansible/playbooks/webapp.yml            docker compose up -d
```

## Is this for you?

It fits if you have somewhere between two and twenty services on a few hosts,
each one a `docker-compose.yml`, and you are currently deploying them by hand or
with a shell script you no longer trust.

It is the wrong tool if you want orchestration, scheduling, or rolling updates.
That is Kubernetes or Nomad, and you should use one of those.

## The idea, in three parts

**Payloads are data.** A stack's compose file and its configuration live under
`server/<host>/<stack>/` and are copied to the host *verbatim*. Nothing is
templated. You can read the file in the repo and know exactly what is on the
host, and you can `docker compose up` it by hand in an emergency.

**One role, one contract.** Everything a stack can need — networks to create,
directories to pre-create, which compose files, what to exclude from the sync,
whether to pull — is a variable on a single role. A new service is usually four
lines:

```yaml
- name: Deploy the static site
  hosts: app
  gather_facts: false
  tags: [static-site]

  roles:
    - role: compose_stack
      vars:
        stack_name: static-site
        stack_networks: [proxy-net]
```

**One key, one seam.** Ansible knows exactly one thing about credentials:
`SSH_KEY_PATH` points at a `0600` private key. A single script decides where that
key comes from — a file, a CI secret, a vault — so adding a backend never touches
a playbook.

## Layout

```
ansible/
├── ansible.cfg              # read only from this directory — see below
├── requirements.yml         # Galaxy collections
├── inventory/
│   ├── hosts.yml
│   └── group_vars/all.yml   # connection settings, stack_root, the pull default
├── roles/compose_stack/     # the one shared "sync payload, bring stack up" role
└── playbooks/
    ├── site.yml             # every stack, in dependency order
    └── <stack>.yml          # one per stack, runnable on its own
server/<host>/<stack>/       # the payload: compose files and config, copied verbatim
server/shared/               # payloads used by more than one host
scripts/load-ssh-key.sh      # produces the deploy key path (file / CI / vault)
scripts/check.sh             # the full static gate — what CI runs
skills/adding-a-stack/       # step-by-step guide to adding a service
docs/                        # secrets, CI setup, connectivity, triggering deploys
AGENTS.md                    # the conventions and the role contract, in full
```

## Getting started

> **Run every ansible command from the `ansible/` directory.** `ansible.cfg` is
> only discovered in the current working directory, and it is what points at the
> inventory and puts `roles/` on the roles path. From the repository root you
> will get `The role 'compose_stack' was not found`.
>
> Alternatively export `ANSIBLE_CONFIG=/path/to/repo/ansible/ansible.cfg`, which
> works from anywhere. CI has to do this: Ansible ignores an `ansible.cfg` that
> lives in a world-writable directory, and a runner's build directory is exactly
> that.

```bash
cd ansible
ansible-galaxy install -r requirements.yml
export SSH_KEY_PATH=~/.ssh/your_deploy_key     # see Secrets, below
```

Then check what a deploy *would* do before doing it:

```bash
ansible-playbook --check --diff playbooks/metrics.yml
```

### On Windows

Ansible has no native Windows control node — `ansible-core` needs a POSIX system,
and the scripts here are shell scripts. Use **WSL2** and run everything inside
it; the commands above then work unchanged.

Two things that will cost you an afternoon if you skip them:

- **Clone into the WSL filesystem** (`~/config`), not `/mnt/c/...`. On the
  Windows drive `chmod 600` does not stick, and OpenSSH refuses a private key
  whose permissions are too open. It is also markedly slower.
- **Never open a private key or a shell script in Notepad.** It rewrites the file
  with CRLF line endings, which makes SSH reject the key and makes deployed
  scripts fail on the host with `bad interpreter: /bin/bash^M`. The
  `.gitattributes` here keeps LF endings on checkout, but it cannot protect a
  file you edit outside git.

## Deploying

```bash
# One stack
ansible-playbook playbooks/metrics.yml

# Everything, in dependency order
ansible-playbook playbooks/site.yml

# One stack out of site.yml
ansible-playbook playbooks/site.yml --tags metrics
```

### Deploy versus update

This distinction is the one piece of the design worth internalising.

A plain run brings the stack up with the images already on the host. It never
pulls, migrates, backs up, or destroys anything. Run it as often as you like.

```bash
ansible-playbook playbooks/webapp.yml -e pull=always
```

`pull=always` fetches newer images and recreates changed containers. It is *also*
the switch for the heavier per-stack work: `webapp` only runs its backup and
database migration on an update run.

If your images are tagged `latest`, a plain deploy will **not** pick up a new
build — Compose is content with the image it already has. Use `pull=always` when
you mean to update.

Verify with `--check --diff` first. These are production servers.

## The examples

Six stacks, each chosen to demonstrate one thing. Read them in this order.

| Stack | Hosts | What it shows |
| --- | --- | --- |
| [`static-site`](ansible/playbooks/static-site.yml) | app | The minimum: a name and a network. **Copy this one.** |
| [`reverse-proxy`](ansible/playbooks/reverse-proxy.yml) | edge | Pre-creating bind mounts; reloading after a config-only change |
| [`metrics`](ansible/playbooks/metrics.yml) | edge, app | One payload shared by two hosts; per-host variable maps; directories owned by a container's uid |
| [`webapp`](ansible/playbooks/webapp.yml) | app | Backup and migrate *between* sync and up, gated on an update run; a second compose file; building from a Dockerfile; requiring a host-side `.env` |
| [`webapp-staging`](ansible/playbooks/webapp-staging.yml) | app | A second instance of the same payload, elsewhere, with different settings |
| [`banner`](ansible/playbooks/banner.yml) | edge, app | The escape hatch: something that is not a Compose stack at all |

`server/` is data — see [`server/README.md`](server/README.md).

## Secrets

Three separate things, handled three different ways. Full detail in
[`docs/secrets.md`](docs/secrets.md).

**1. The deploy SSH key.** Ansible only ever reads `SSH_KEY_PATH`.
`scripts/load-ssh-key.sh` is the only thing that knows where the key comes from;
it prints a path and never prints key material.

```bash
export SSH_KEY_PATH="$(scripts/load-ssh-key.sh)"         # key already on disk
export SSH_KEY_PATH="$(scripts/load-ssh-key.sh ci)"      # from a CI secret
export SSH_KEY_PATH="$(scripts/load-ssh-key.sh vault)"   # from Vault / OpenBao
```

**2. Per-stack `.env` files.** Stack secrets live on the hosts, not here. The
role excludes `.env` from every sync, so editing one in the repo has no effect
and a host's `.env` is never overwritten. The example stacks use Compose's
`${VAR:?message}` syntax, so a stack refuses to start with a missing credential
rather than booting with an empty password.

**3. Nothing else.** There is no third category, and that is the point. A secret
written directly into a compose file cannot be excluded from the sync, cannot be
rotated without a commit, and is in every clone of the repository forever. CI
runs [gitleaks](https://github.com/gitleaks/gitleaks) over the working tree *and*
the history on every push to keep it that way.

## CI/CD

All three platforms are maintained, and all three do the same two things.

|  | GitHub Actions | GitLab CI | Gitea Actions |
| --- | --- | --- | --- |
| Static gate | [`lint.yml`](.github/workflows/lint.yml) | `lint` job | [`lint.yml`](.gitea/workflows/lint.yml) |
| Deploy | [`deploy.yml`](.github/workflows/deploy.yml) | per-stack buttons | [`deploy.yml`](.gitea/workflows/deploy.yml) |
| Diagnostics | [`connectivity.yml`](.github/workflows/connectivity.yml) | `ansible:ping` | [`connectivity.yml`](.gitea/workflows/connectivity.yml) |
| Setup guide | [`ci-github-actions.md`](docs/ci-github-actions.md) | [`ci-gitlab.md`](docs/ci-gitlab.md) | [`ci-gitea.md`](docs/ci-gitea.md) |

Gitea reads `.gitea/workflows/` *instead of* `.github/workflows/` — it stops at
the first workflow directory that exists — so the two never both run. Delete the
platforms you are not using rather than letting them rot.

The rules that matter, on all three:

- **The lint gate runs on every push** and must pass. It needs no access to any
  host. It is `scripts/check.sh` — the same command you run locally — so it
  cannot drift from what you tested.
- **Deploys are never automatic.** Merging changes what *would* be deployed; a
  person still decides when. Every deploy path is manual or externally triggered.
- **Check before deploy.** All three expose a check mode that runs
  `--check --diff` and touches nothing.
- **To update rather than deploy**, set the pull policy to `always`.
- **Untrusted input is validated.** The stack name and pull policy reach a shell
  command and, for externally triggered deploys, come from another repository.
  Both are matched against a fixed pattern before anything runs.

Another repository can ask this one to redeploy a stack after building a new
image — see [`docs/triggering-deploys.md`](docs/triggering-deploys.md).

## Adding a stack

1. Put the compose file and its configuration in `server/<host>/<stack>/`.
2. Add `ansible/playbooks/<stack>.yml`. Copy `static-site.yml`.
3. Add it to `site.yml`, in dependency order.
4. Add it to CI: the `stack` dropdown in `.github/workflows/deploy.yml` and in
   `.gitea/workflows/deploy.yml`, and a `check:`/`deploy:` pair in
   `.gitlab-ci.yml`. Skip this and the stack works from a workstation but is
   invisible in CI, which is how services end up hand-deployed.
5. `make check`, then `--check --diff` against the host.

The long version, with a table of which contract variable solves which problem,
is in [`skills/adding-a-stack/SKILL.md`](skills/adding-a-stack/SKILL.md).
[`AGENTS.md`](AGENTS.md) has the conventions and the full role contract.

## Troubleshooting

| Symptom | Cause |
| --- | --- |
| `The role 'compose_stack' was not found` | Not running from the `ansible/` directory |
| `SSH_KEY_PATH does not point at a readable file` | Key not loaded, or a CI job that cannot see the secret |
| A deploy succeeds but the service is unchanged | Image tag is `latest` and you did not pass `-e pull=always` |
| Config file copied but nothing happens | Bind-mounted config; the stack needs a reload, which the playbook does only when the sync changed something |
| `no payload directory at ...` | `stack_src` does not match where the files actually are |
| A stack refuses to start, naming a variable | Its `.env` is missing on the host — that is the design working |
| `Permission denied (publickey)` | The matching public key is not in the deploy user's `authorized_keys` |
| `Connection timed out` | A network path problem, not a key problem — see [`docs/connectivity.md`](docs/connectivity.md) |

## License

MIT. See [LICENSE](LICENSE).