Archived
fix: isolate registry publishing and pin image revisions
This commit is contained in:
@@ -32,31 +32,8 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Resolve publish tags
|
||||
id: tags
|
||||
run: |
|
||||
python3 scripts/image_tags.py "$GITHUB_REF" > image-tags.txt
|
||||
if [ -s image-tags.txt ]; then
|
||||
echo "valid=true" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
- name: Log in to Gitea registry
|
||||
if: ${{ steps.tags.outputs.valid == 'true' }}
|
||||
- name: Publish image
|
||||
env:
|
||||
REGISTRY_USERNAME: ${{ secrets.REGISTRY_USERNAME }}
|
||||
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
|
||||
run: |
|
||||
printf '%s' "$REGISTRY_TOKEN" | docker login gitea.lucaswinther.info --username "$REGISTRY_USERNAME" --password-stdin
|
||||
- name: Build and push
|
||||
if: ${{ steps.tags.outputs.valid == 'true' }}
|
||||
run: |
|
||||
set -eu
|
||||
docker build --platform linux/amd64 \
|
||||
--label "org.opencontainers.image.source=https://gitea.lucaswinther.info/lucasw89/recoll-modern" \
|
||||
--label "org.opencontainers.image.description=Self-hosted Recoll search API" \
|
||||
--label "org.opencontainers.image.revision=$GITHUB_SHA" \
|
||||
--label "org.opencontainers.image.version=$GITHUB_REF_NAME" \
|
||||
-t recoll-modern:publish .
|
||||
while IFS= read -r image; do
|
||||
docker tag recoll-modern:publish "$image"
|
||||
docker push "$image"
|
||||
done < image-tags.txt
|
||||
run: bash scripts/publish_image.sh
|
||||
|
||||
@@ -31,21 +31,25 @@ access token with package **Read** permission as the Docker password:
|
||||
docker login gitea.lucaswinther.info
|
||||
```
|
||||
|
||||
Set an absolute host directory for documents and seed the first administrator. `:main` is available
|
||||
after a successful main-branch publish, even before the first version tag:
|
||||
Save the deployment settings in a private `.env` file in the same directory as the Compose file. This
|
||||
keeps the chosen image tag across shells. `:main` is available after a successful main-branch publish,
|
||||
even before the first version tag:
|
||||
|
||||
```sh
|
||||
export RECOLL_MODERN_DOCUMENTS=/srv/documents
|
||||
export RECOLL_MODERN_ADMIN_USERNAME=admin
|
||||
export RECOLL_MODERN_ADMIN_PASSWORD='use-a-long-unique-password'
|
||||
export RECOLL_MODERN_IMAGE=gitea.lucaswinther.info/lucasw89/recoll-modern:main
|
||||
umask 077
|
||||
cat > .env <<'EOF'
|
||||
RECOLL_MODERN_DOCUMENTS=/srv/documents
|
||||
RECOLL_MODERN_ADMIN_USERNAME=admin
|
||||
RECOLL_MODERN_ADMIN_PASSWORD=use-a-long-unique-password
|
||||
RECOLL_MODERN_IMAGE=gitea.lucaswinther.info/lucasw89/recoll-modern:main
|
||||
EOF
|
||||
docker compose -f compose.published.yaml pull
|
||||
docker compose -f compose.published.yaml up -d --no-build
|
||||
docker compose -f compose.published.yaml ps
|
||||
docker compose -f compose.published.yaml logs -f app
|
||||
```
|
||||
|
||||
Once a version appears in Gitea Packages, set `RECOLL_MODERN_IMAGE` to its tag, such as
|
||||
Once a version appears in Gitea Packages, edit `RECOLL_MODERN_IMAGE` in `.env` to its tag, such as
|
||||
`gitea.lucaswinther.info/lucasw89/recoll-modern:v1.2.3`, to pin that release. Omit
|
||||
`RECOLL_MODERN_IMAGE` to use `:latest` after the first version tag is published. The image is published
|
||||
only after the CI checks pass; a tag that has not been published cannot be pulled. To update, set the
|
||||
@@ -53,10 +57,13 @@ desired tag and run `docker compose -f compose.published.yaml pull` followed
|
||||
by `docker compose -f compose.published.yaml up -d --no-build` again. Keep the same Compose project
|
||||
directory so its named volumes are reused.
|
||||
|
||||
For a trusted local HTTP trial with `docker run`, use the same document and administrator variables:
|
||||
For a trusted local HTTP trial with `docker run`, export the settings for that shell:
|
||||
|
||||
```sh
|
||||
export RECOLL_MODERN_IMAGE=${RECOLL_MODERN_IMAGE:-gitea.lucaswinther.info/lucasw89/recoll-modern:main}
|
||||
export RECOLL_MODERN_DOCUMENTS=/srv/documents
|
||||
export RECOLL_MODERN_ADMIN_USERNAME=admin
|
||||
export RECOLL_MODERN_ADMIN_PASSWORD='use-a-long-unique-password'
|
||||
export RECOLL_MODERN_IMAGE=gitea.lucaswinther.info/lucasw89/recoll-modern:main
|
||||
docker pull "$RECOLL_MODERN_IMAGE"
|
||||
docker run -d --name recoll-modern --restart unless-stopped -p 127.0.0.1:8000:8000 \
|
||||
-e RECOLL_MODERN_ADMIN_USERNAME -e RECOLL_MODERN_ADMIN_PASSWORD \
|
||||
@@ -110,7 +117,8 @@ these checks when Compose is not running. Recoll commands run only inside the co
|
||||
## Publishing from Gitea
|
||||
|
||||
Enable repository Actions and use a runner with the `ubuntu-latest` label, Python 3, `uv` setup support,
|
||||
and a working Docker CLI/daemon. In the repository's Actions secrets, set `REGISTRY_USERNAME` to the
|
||||
and a working Docker CLI/daemon. Run jobs in isolated containers so a later job cannot read another
|
||||
job's home directory. In the repository's Actions secrets, set `REGISTRY_USERNAME` to the
|
||||
Gitea account that owns the publishing token and `REGISTRY_TOKEN` to a personal access token with
|
||||
package **Read & Write** permission. Gitea currently [requires a PAT for Actions package
|
||||
publishing](https://docs.gitea.com/usage/actions/comparison/); the built-in job token does not publish
|
||||
|
||||
@@ -32,7 +32,7 @@ web UI and (b) an MCP server, running in Docker, with several independent collec
|
||||
| `helper/` | C reader run by Recoll's `metadatacmds` for each indexed file |
|
||||
| `Dockerfile`, `compose.yaml`, `docker/` | Non-root Recoll runtime, read-only document mount, persistent state/index volumes and config templates |
|
||||
| `compose.published.yaml` | Pull-only deployment with the same runtime settings and persistent mounts |
|
||||
| `.gitea/workflows/image.yaml`, `scripts/image_tags.py` | CI checks and Gitea registry publication; Git refs select published tags |
|
||||
| `.gitea/workflows/image.yaml`, `scripts/image_tags.py`, `scripts/publish_image.sh` | CI checks and Gitea registry publication; Git refs select published tags |
|
||||
| `src/recoll_modern/engine/` | Shared Protocol, deterministic fake, and isolated Recoll Python/indexer adapter |
|
||||
| `src/recoll_modern/domain/auth.py`, `storage/auth.py` | Argon2 password checks, seeded administrator, hashed server-side sessions and CSRF verification |
|
||||
| `src/recoll_modern/api/` | HTTP auth, health, file browsing, collection/membership/rule/job resources, search, document retrieval, prefix/proxy handling and JSON errors |
|
||||
|
||||
@@ -3,6 +3,19 @@
|
||||
Append-only, newest first. Never edit or delete an entry — supersede it with a new one that links back.
|
||||
One entry per decision a future maintainer would otherwise have to reverse-engineer.
|
||||
|
||||
## 2026-09-28 — Isolate registry credentials and tag the built image ID
|
||||
**Status:** accepted
|
||||
**Context:** A persistent Gitea runner can retain Docker login credentials, and jobs sharing a Docker
|
||||
daemon can overwrite a common local image tag while publishing concurrently.
|
||||
**Decision:** the publish script uses a fresh temporary Docker config, removes it on exit, and tags the
|
||||
immutable ID returned by its own build. Document isolated runner jobs as a deployment prerequisite.
|
||||
**Alternatives:** log in to the runner's default Docker config (credential residue); tag a shared local
|
||||
name (revision race); serialize all image jobs (unnecessary when each job tags its own image ID).
|
||||
**Consequences:** routine success and failure paths remove the PAT, and overlapping jobs do not reuse
|
||||
each other's local image tag. Runner isolation still matters for abrupt host failures and untrusted jobs.
|
||||
|
||||
---
|
||||
|
||||
## 2026-09-28 — Publish one amd64 image through Gitea Actions
|
||||
**Status:** accepted
|
||||
**Context:** The Docker deployment exists, but operators must build it locally. [Gitea's documented job
|
||||
|
||||
@@ -86,3 +86,10 @@ spec: docs/specs/006-gitea-image-publishing.md
|
||||
- [x] **Step 4: Update** README with a quick-start using `docker login` when needed, `RECOLL_MODERN_IMAGE`, `docker compose ... pull`, `up -d`, `ps`, `logs`, and an update that retains volumes. Explain first boot, tag semantics, `linux/amd64`, backups, Gitea Actions enablement, runner and PAT secret setup. Keep local-build instructions and revise architecture for the two deployment paths.
|
||||
- [x] **Step 5: Run** all project tests, lint and type checking again; verify Compose config for both files and `git diff --check`; inspect the whole diff against spec 006.
|
||||
- [x] **Step 6: Mark** the plan done after completed local gates, then commit deployment docs/config and checked plan boxes as `docs: explain pulling and running published image`. Keep the spec approved until a real Gitea publish/pull is verified after the branch is merged.
|
||||
|
||||
## Review follow-up
|
||||
|
||||
- [x] Keep Docker registry credentials in a temporary config that is removed after success or failure.
|
||||
- [x] Tag the image ID returned by each build so concurrent jobs cannot swap a shared local tag.
|
||||
- [x] Reject release tags with leading-zero semantic-version components.
|
||||
- [x] Persist the selected image tag in the deployment `.env` example.
|
||||
|
||||
@@ -4,7 +4,7 @@ import re
|
||||
import sys
|
||||
|
||||
IMAGE = "gitea.lucaswinther.info/lucasw89/recoll-modern"
|
||||
RELEASE_REF = re.compile(r"refs/tags/(v[0-9]+\.[0-9]+\.[0-9]+)\Z")
|
||||
RELEASE_REF = re.compile(r"refs/tags/(v(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*)\.(?:0|[1-9][0-9]*))\Z")
|
||||
|
||||
|
||||
def tags_for_ref(ref: str) -> list[str]:
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
tags=$(python3 scripts/image_tags.py "$GITHUB_REF")
|
||||
if [[ -z "$tags" ]]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
export DOCKER_CONFIG
|
||||
DOCKER_CONFIG=$(mktemp -d)
|
||||
trap 'rm -r "$DOCKER_CONFIG"' EXIT
|
||||
|
||||
printf '%s' "$REGISTRY_TOKEN" |
|
||||
docker login gitea.lucaswinther.info --username "$REGISTRY_USERNAME" --password-stdin
|
||||
|
||||
image_id=$(docker build --quiet --platform linux/amd64 \
|
||||
--label "org.opencontainers.image.source=https://gitea.lucaswinther.info/lucasw89/recoll-modern" \
|
||||
--label "org.opencontainers.image.description=Self-hosted Recoll search API" \
|
||||
--label "org.opencontainers.image.revision=$GITHUB_SHA" \
|
||||
--label "org.opencontainers.image.version=$GITHUB_REF_NAME" .)
|
||||
|
||||
while IFS= read -r image; do
|
||||
docker tag "$image_id" "$image"
|
||||
docker push "$image"
|
||||
done <<< "$tags"
|
||||
@@ -13,6 +13,9 @@ import pytest
|
||||
("refs/tags/v1.2.3", ["v1.2.3", "latest"]),
|
||||
("refs/tags/v1.2.3-rc1", []),
|
||||
("refs/tags/vfoo", []),
|
||||
("refs/tags/v01.2.3", []),
|
||||
("refs/tags/v1.02.3", []),
|
||||
("refs/tags/v1.2.03", []),
|
||||
("refs/heads/maintainer", []),
|
||||
("refs/pull/42/merge", []),
|
||||
],
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Exercise the publish script with an isolated Docker command stub."""
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def _fake_docker(tmp_path: Path, *, fail_build: bool = False) -> tuple[Path, Path]:
|
||||
docker = tmp_path / "docker"
|
||||
log = tmp_path / "docker.log"
|
||||
docker.write_text(
|
||||
"#!/bin/sh\n"
|
||||
"printf '%s|%s|%s\\n' \"$1\" \"${2-}\" \"$DOCKER_CONFIG\" >> \"$DOCKER_LOG\"\n"
|
||||
"case \"$1\" in\n"
|
||||
" login) cat >/dev/null ;;\n"
|
||||
f" build) {'exit 42' if fail_build else 'printf sha256:built-revision'} ;;\n"
|
||||
"esac\n"
|
||||
)
|
||||
docker.chmod(0o755)
|
||||
return docker, log
|
||||
|
||||
|
||||
def _run_publish(
|
||||
tmp_path: Path, *, fail_build: bool = False
|
||||
) -> tuple[subprocess.CompletedProcess[str], Path]:
|
||||
_, log = _fake_docker(tmp_path, fail_build=fail_build)
|
||||
env = os.environ.copy()
|
||||
env.update(
|
||||
PATH=f"{tmp_path}:{env['PATH']}",
|
||||
DOCKER_LOG=str(log),
|
||||
GITHUB_REF="refs/tags/v1.2.3",
|
||||
GITHUB_REF_NAME="v1.2.3",
|
||||
GITHUB_SHA="deadbeef",
|
||||
REGISTRY_USERNAME="test-user",
|
||||
REGISTRY_TOKEN="test-token-never-print",
|
||||
)
|
||||
result = subprocess.run(
|
||||
["bash", "scripts/publish_image.sh"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env=env,
|
||||
check=False,
|
||||
)
|
||||
return result, log
|
||||
|
||||
|
||||
def test_publish_uses_immutable_build_id_and_removes_credentials(tmp_path: Path) -> None:
|
||||
result, log = _run_publish(tmp_path)
|
||||
assert result.returncode == 0, result.stderr
|
||||
commands = log.read_text().splitlines()
|
||||
config_dir = Path(commands[0].split("|")[2])
|
||||
assert [line.split("|")[0] for line in commands] == [
|
||||
"login", "build", "tag", "push", "tag", "push"
|
||||
]
|
||||
assert all(line.split("|")[2] == str(config_dir) for line in commands)
|
||||
assert "sha256:built-revision" in commands[2]
|
||||
assert "sha256:built-revision" in commands[4]
|
||||
assert "recoll-modern:publish" not in log.read_text()
|
||||
assert not config_dir.exists()
|
||||
assert "test-token-never-print" not in result.stdout + result.stderr
|
||||
|
||||
|
||||
def test_failed_build_also_removes_registry_credentials(tmp_path: Path) -> None:
|
||||
result, log = _run_publish(tmp_path, fail_build=True)
|
||||
assert result.returncode != 0
|
||||
config_dir = Path(log.read_text().splitlines()[0].split("|")[2])
|
||||
assert not config_dir.exists()
|
||||
Reference in New Issue
Block a user