compose-stack-deploy

A small, complete pattern for deploying Docker Compose stacks to a handful of Linux hosts with Ansible — and for letting CI do it safely, one stack at a time, on purpose rather than on every push.

It is deliberately not a framework. There is one Ansible role, a couple of hundred lines of tasks, and a convention about where files live. Everything else in this repository is documentation and seven worked examples.

server/app/webapp/          →  rsync  →  /srv/stacks/webapp/  on app
ansible/playbooks/webapp.yml            docker compose up -d

Is this for you?

It fits if you have somewhere between two and twenty services on a few hosts, each one a docker-compose.yml, and you are currently deploying them by hand or with a shell script you no longer trust.

It is the wrong tool if you want orchestration, scheduling, or rolling updates. That is Kubernetes or Nomad, and you should use one of those.

The idea, in three parts

Payloads are data. A stack's compose file and its configuration live under server/<host>/<stack>/ and are copied to the host verbatim. Nothing is templated. You can read the file in the repo and know exactly what is on the host, and you can docker compose up it by hand in an emergency.

One role, one contract. Everything a stack can need — networks to create, directories to pre-create, which compose files, what to exclude from the sync, whether to pull — is a variable on a single role. A new service is usually four lines:

- name: Deploy the static site
  hosts: app
  gather_facts: false
  tags: [static-site]

  roles:
    - role: compose_stack
      vars:
        stack_name: static-site
        stack_networks: [proxy-net]

One key, one seam. Ansible knows exactly one thing about credentials: SSH_KEY_PATH points at a 0600 private key. A single script decides where that key comes from — a file, a CI secret, a vault — so adding a backend never touches a playbook.

Layout

ansible/
├── ansible.cfg              # read only from this directory — see below
├── requirements.yml         # Galaxy collections
├── inventory/
│   ├── hosts.yml
│   └── group_vars/all.yml   # connection settings, stack_root, pull/start defaults
├── roles/compose_stack/     # the one shared role: sync, render secrets, bring up
└── playbooks/
    ├── site.yml             # every stack, in dependency order
    ├── bootstrap.yml        # prepares a host; run once, by an admin
    └── <stack>.yml          # one per stack, runnable on its own
server/<host>/<stack>/       # the payload: compose files and config, copied verbatim
server/shared/               # payloads used by more than one host
scripts/load-ssh-key.sh      # produces the deploy key path (file / CI / vault)
scripts/new-deploy-key.sh    # generates the deploy key pair
scripts/check.sh             # the full static gate — what CI runs
skills/adding-a-stack/       # step-by-step guide to adding a service
docs/                        # bootstrap, secrets, CI setup, connectivity, deploys
AGENTS.md                    # the conventions and the role contract, in full
TODO.md                      # known defects and deferred decisions

Getting started

Run every ansible command from the ansible/ directory. ansible.cfg is only discovered in the current working directory, and it is what points at the inventory and puts roles/ on the roles path. From the repository root you will get The role 'compose_stack' was not found.

Alternatively export ANSIBLE_CONFIG=/path/to/repo/ansible/ansible.cfg, which works from anywhere. CI has to do this: Ansible ignores an ansible.cfg that lives in a world-writable directory, and a runner's build directory is exactly that.

A host that has never been deployed to needs preparing first — a deploy user, its key, the docker group, rsync, and a writable /srv/stacks. That is scripts/new-deploy-key.sh and playbooks/bootstrap.yml, once per host; see docs/bootstrap.md.

cd ansible
ansible-galaxy install -r requirements.yml
export SSH_KEY_PATH=~/.ssh/deploy_key          # see Secrets, below
export VAULT_TOKEN=...                         # for stacks with stack_secrets

Then check what a deploy would do before doing it:

ansible-playbook --check --diff playbooks/metrics.yml

Passwordless sudo

Almost everything a deploy does runs as deploy, unescalated, and that is deliberate: a bind mount created by root is one the container cannot write. The exception is a stack_dirs entry that sets owner or group. Handing a directory to another uid needs root, so the role escalates for exactly those entries and nothing else. metrics.yml does it for Grafana (uid 472), because an image that does not run as root cannot repair its own data directory from the inside.

Without it, those deploys stop here:

TASK [compose_stack : Create the extra directories the stack needs]
[ERROR]: Task failed: Missing sudo password

There is no password to supply — deploy has none. Grant NOPASSWD once, as an administrator, either over SSH or on the host itself:

cd ansible
ansible-playbook playbooks/bootstrap.yml -u <admin-user> -k -K -e deploy_sudo=true
ansible-playbook playbooks/bootstrap.yml -c local -K -e deploy_sudo=true

Leaving -e deploy_sudo=true off is the easy mistake: the run still ends green and the only sign is skipped=1 in the recap. Confirm it took with ssh deploy@<host> sudo -n true && echo ok.

This cannot come from CI, and that is the point: the runner holds only the unprivileged deploy key. Why -u works, what NOPASSWD:ALL costs and how to narrow it are in docs/bootstrap.md.

On Windows

Ansible has no native Windows control node — ansible-core needs a POSIX system, and the scripts here are shell scripts. Use WSL2 and run everything inside it; the commands above then work unchanged.

Two things that will cost you an afternoon if you skip them:

• Clone into the WSL filesystem (~/config), not /mnt/c/.... On the Windows drive chmod 600 does not stick, and OpenSSH refuses a private key whose permissions are too open. It is also markedly slower.
• Never open a private key or a shell script in Notepad. It rewrites the file with CRLF line endings, which makes SSH reject the key and makes deployed scripts fail on the host with bad interpreter: /bin/bash^M. The .gitattributes here keeps LF endings on checkout, but it cannot protect a file you edit outside git.

Deploying

# One stack
ansible-playbook playbooks/metrics.yml

# Everything, in dependency order
ansible-playbook playbooks/site.yml

# One stack out of site.yml
ansible-playbook playbooks/site.yml --tags metrics

Deploy versus update

This distinction is the one piece of the design worth internalising.

A plain run brings the stack up with the images already on the host. It never pulls, migrates, backs up, or destroys anything. Run it as often as you like.

ansible-playbook playbooks/webapp.yml -e pull=always

pull=always fetches newer images and recreates changed containers. It is also the switch for the heavier per-stack work: webapp only runs its backup and database migration on an update run.

If your images are tagged latest, a plain deploy will not pick up a new build — Compose is content with the image it already has. Use pull=always when you mean to update.

Verify with --check --diff first. These are production servers.

Deploy without starting

ansible-playbook playbooks/webapp.yml -e start=false

Syncs the payload and creates the directories and networks, then stops before Compose. It is for the gap between the files landing and the stack opening its bind mounts — copying data across from a hand-deployed instance, or restoring a database dump, before anything starts on top of it. Run again without the flag to start the stack.

The examples

Seven stacks, each chosen to demonstrate one thing. Read them in this order.

Stack	Hosts	What it shows
static-site	app	The minimum: a name and a network. Copy this one.
reverse-proxy	edge	Pre-creating bind mounts; reloading after a config-only change
metrics	edge, app	One payload shared by two hosts; per-host variable maps; directories owned by a container's uid; requiring a hand-placed .env
openbao	app	The secret store, on 127.0.0.1:8200 only. Reports a changed config instead of restarting, because a restart seals it
webapp	app	Backup and migrate between sync and up, gated on an update run; a second compose file; building from a Dockerfile; credentials from the vault with stack_secrets
webapp-staging	app	A second instance of the same payload, elsewhere, with different settings
banner	edge, app	The escape hatch: something that is not a Compose stack at all

server/ is data — see server/README.md.

Secrets

Three separate things, handled three different ways. Full detail in docs/secrets.md.

1. The deploy SSH key. Ansible only ever reads SSH_KEY_PATH. scripts/load-ssh-key.sh is the only thing that knows where the key comes from; it prints a path and never prints key material.

export SSH_KEY_PATH="$(scripts/load-ssh-key.sh)"         # key already on disk
export SSH_KEY_PATH="$(scripts/load-ssh-key.sh ci)"      # from a CI secret
export SSH_KEY_PATH="$(scripts/load-ssh-key.sh vault)"   # from Vault / OpenBao

2. Per-stack secrets. They live in OpenBao on the host, under kv/stacks/. A playbook names what it needs with stack_secrets, and the role reads it on the host during the sync and writes /srv/stacks/<stack>/.env at 0600. The vault is published to 127.0.0.1 only, and VAULT_TOKEN — a read-only token — comes from the environment of whoever runs the deploy. webapp works this way. A stack without stack_secrets reads a .env placed on the host by hand instead, as metrics does; the sync never touches it either way. Compose files read ${VAR:?message}, so a stack refuses to start with a missing credential rather than booting with an empty password.

Adding or rotating one is a kv put and a redeploy. Keep the value off the command line, and strip the trailing newline — <field>=- stores stdin byte for byte:

docker exec -it openbao bao login
openssl rand -hex 32 | tr -d '\n' \
  | docker exec -i openbao bao kv put kv/stacks/<stack> <field>=-

3. Nothing else. There is no third category, and that is the point. A secret written directly into a compose file cannot be excluded from the sync, cannot be rotated without a commit, and is in every clone of the repository forever. CI runs gitleaks over the working tree and the history on every push to keep it that way.

CI/CD

All three platforms are maintained, and all three do the same two things.

	GitHub Actions	GitLab CI	Gitea Actions
Static gate	lint.yml	lint job	lint.yml
Deploy	deploy.yml	per-stack buttons	deploy.yml
Diagnostics	connectivity.yml	ansible:ping	connectivity.yml
Setup guide	ci-github-actions.md	ci-gitlab.md	ci-gitea.md

Gitea reads .gitea/workflows/ instead of .github/workflows/ — it stops at the first workflow directory that exists — so the two never both run. Delete the platforms you are not using rather than letting them rot.

The rules that matter, on all three:

• The lint gate runs on every push and must pass. It needs no access to any host. It is scripts/check.sh — the same command you run locally — so it cannot drift from what you tested.
• Deploys are never automatic. Merging changes what would be deployed; a person still decides when. Every deploy path is manual or externally triggered.
• Check before deploy. All three expose a check mode that runs --check --diff and touches nothing.
• To update rather than deploy, set the pull policy to always.
• Untrusted input is validated. The stack name and pull policy reach a shell command and, for externally triggered deploys, come from another repository. Both are matched against a fixed pattern before anything runs.

Another repository can ask this one to redeploy a stack after building a new image — see docs/triggering-deploys.md.

Adding a stack

1. Put the compose file and its configuration in server/<host>/<stack>/.
2. Add ansible/playbooks/<stack>.yml. Copy static-site.yml.
3. Add it to site.yml, in dependency order.
4. Add it to CI: the stack dropdown in .github/workflows/deploy.yml and in .gitea/workflows/deploy.yml, and a check:/deploy: pair in .gitlab-ci.yml. Skip this and the stack works from a workstation but is invisible in CI, which is how services end up hand-deployed.
5. make check, then --check --diff against the host.

The long version, with a table of which contract variable solves which problem, is in skills/adding-a-stack/SKILL.md. AGENTS.md has the conventions and the full role contract.

Troubleshooting

Symptom	Cause
The role 'compose_stack' was not found	Not running from the ansible/ directory
SSH_KEY_PATH does not point at a readable file	Key not loaded, or a CI job that cannot see the secret
A deploy succeeds but the service is unchanged	Image tag is latest and you did not pass -e pull=always
Config file copied but nothing happens	Bind-mounted config; the stack needs a reload, which the playbook does only when the sync changed something
no payload directory at ...	stack_src does not match where the files actually are
A secrets task fails naming the vault	OpenBao is sealed (e.g. after a reboot), or VAULT_TOKEN is unset or expired — see docs/secrets.md
A stack refuses to start, naming a variable	Its .env is missing or incomplete on the host — that is the design working
Permission denied (publickey)	The matching public key is not in the deploy user's authorized_keys
Connection timed out	A network path problem, not a key problem — see docs/connectivity.md
Failed to find required executable "rsync"	synchronize needs rsync at both ends; read the PATH in the error to tell the runner from the host
Missing sudo password on Create the extra directories	The host was bootstrapped without -e deploy_sudo=true — see Passwordless sudo
"/srv/stacks/<stack>" is not a directory	A file is in the way; the role says which. A first --check skips Compose instead — see docs/bootstrap.md

License

MIT. See LICENSE.