From ceedbae1f3d4cbf2a8ca29a26b6fd5893f286cd4 Mon Sep 17 00:00:00 2001 From: Lucas Winther Date: Mon, 28 Sep 2026 03:10:28 +0200 Subject: [PATCH] docs: explain pulling and running published image --- README.md | 80 +++++++++++++++++++++++- compose.published.yaml | 28 +++++++++ docs/ARCHITECTURE.md | 1 + docs/TODO.md | 3 + docs/plans/006-gitea-image-publishing.md | 14 ++--- 5 files changed, 116 insertions(+), 10 deletions(-) create mode 100644 compose.published.yaml diff --git a/README.md b/README.md index 7facf8a..f1515d6 100644 --- a/README.md +++ b/README.md @@ -6,10 +6,71 @@ full-text search, with support for multiple independent collections. Status: spec 001's authenticated search API and Docker deployment have completed acceptance and review checks. The web UI and MCP server are later specs. -## Run with Docker +## Published Docker image -Set `RECOLL_MODERN_DOCUMENTS` to an absolute host directory containing documents and provide a first -administrator username and password: +The image is `gitea.lucaswinther.info/lucasw89/recoll-modern` for `linux/amd64`. It currently contains +the authenticated search API and Recoll; the web UI and MCP server are later specs. `:main` follows +the main branch. A release tag such as `:v1.2.3` pins a specific release; `:latest` follows the newest +version tag. Use a version tag when you want a predictable update. The image records its source, +revision and version in OCI labels (`docker image inspect IMAGE`). + +### Run the pulled image with Compose + +Save [compose.published.yaml](compose.published.yaml) alongside this guide, or download it from the +repository after the publishing branch reaches `main`: + +```sh +curl -fL -o compose.published.yaml \ + https://gitea.lucaswinther.info/lucasw89/recoll-modern/raw/branch/main/compose.published.yaml +``` + +If the package is private, log in to the Gitea registry first. Use your Gitea username and a personal +access token with package **Read** permission as the Docker password: + +```sh +docker login gitea.lucaswinther.info +``` + +Set an absolute host directory for documents and seed the first administrator. `:main` is available +after a successful main-branch publish, even before the first version tag: + +```sh +export RECOLL_MODERN_DOCUMENTS=/srv/documents +export RECOLL_MODERN_ADMIN_USERNAME=admin +export RECOLL_MODERN_ADMIN_PASSWORD='use-a-long-unique-password' +export RECOLL_MODERN_IMAGE=gitea.lucaswinther.info/lucasw89/recoll-modern:main +docker compose -f compose.published.yaml pull +docker compose -f compose.published.yaml up -d --no-build +docker compose -f compose.published.yaml ps +docker compose -f compose.published.yaml logs -f app +``` + +Once a version appears in Gitea Packages, set `RECOLL_MODERN_IMAGE` to its tag, such as +`gitea.lucaswinther.info/lucasw89/recoll-modern:v1.2.3`, to pin that release. Omit +`RECOLL_MODERN_IMAGE` to use `:latest` after the first version tag is published. The image is published +only after the CI checks pass; a tag that has not been published cannot be pulled. To update, set the +desired tag and run `docker compose -f compose.published.yaml pull` followed +by `docker compose -f compose.published.yaml up -d --no-build` again. Keep the same Compose project +directory so its named volumes are reused. + +For a trusted local HTTP trial with `docker run`, use the same document and administrator variables: + +```sh +export RECOLL_MODERN_IMAGE=${RECOLL_MODERN_IMAGE:-gitea.lucaswinther.info/lucasw89/recoll-modern:main} +docker pull "$RECOLL_MODERN_IMAGE" +docker run -d --name recoll-modern --restart unless-stopped -p 127.0.0.1:8000:8000 \ + -e RECOLL_MODERN_ADMIN_USERNAME -e RECOLL_MODERN_ADMIN_PASSWORD \ + -e RECOLL_MODERN_HOST=0.0.0.0 -e RECOLL_MODERN_PORT=8000 \ + -e RECOLL_MODERN_DEVELOPMENT_MODE=true \ + --mount "type=bind,src=$RECOLL_MODERN_DOCUMENTS,dst=/documents,readonly" \ + --mount type=volume,src=recoll-modern-service-state,dst=/var/lib/recoll-modern \ + --mount type=volume,src=recoll-modern-index,dst=/var/lib/recoll \ + "$RECOLL_MODERN_IMAGE" +``` + +### Build locally + +The original `compose.yaml` still builds from this repository: ```sh export RECOLL_MODERN_DOCUMENTS=/srv/documents @@ -24,6 +85,9 @@ live in the `service-state` volume; the index lives in the separate `recoll-inde volumes together and keep a separate copy of the documents. `docker compose down` preserves the volumes; never pass `-v` unless you intentionally want to delete the index and service state. +For the direct `docker run` example, the volumes are named `recoll-modern-service-state` and +`recoll-modern-index`. Back up both together. Replacing a container does not delete these volumes. + On first boot, the API is available but `/health` may return 503 until a membership or reindex job has built the first Recoll index. Log in through `POST /auth/login`, create a collection, then add file or directory sources through `POST /collections/{id}/memberships`. Mutations return a durable job ID; @@ -43,5 +107,15 @@ word `zebraplatypus`, set `RECOLL_MODERN_TEST_CORPUS_FILE` to its path relative and run `RECOLL_MODERN_REQUIRE_DOCKER=1 uv run pytest tests/integration`. Ordinary host test runs skip these checks when Compose is not running. Recoll commands run only inside the container. +## Publishing from Gitea + +Enable repository Actions and use a runner with the `ubuntu-latest` label, Python 3, `uv` setup support, +and a working Docker CLI/daemon. In the repository's Actions secrets, set `REGISTRY_USERNAME` to the +Gitea account that owns the publishing token and `REGISTRY_TOKEN` to a personal access token with +package **Read & Write** permission. Gitea currently [requires a PAT for Actions package +publishing](https://docs.gitea.com/usage/actions/comparison/); the built-in job token does not publish +OCI packages. The workflow checks pull requests without registry credentials and publishes only after +checks pass on `main` or a full `vX.Y.Z` tag. No server deployment is performed by CI. + Working here (human or agent): read [`AGENTS.md`](AGENTS.md). It is the single source of truth for commands, workflow, and boundaries. Deeper context lives in [`docs/`](docs/). diff --git a/compose.published.yaml b/compose.published.yaml new file mode 100644 index 0000000..7b41528 --- /dev/null +++ b/compose.published.yaml @@ -0,0 +1,28 @@ +services: + app: + image: ${RECOLL_MODERN_IMAGE:-gitea.lucaswinther.info/lucasw89/recoll-modern:latest} + ports: + - "${RECOLL_MODERN_PUBLISH_PORT:-8000}:8000" + environment: + RECOLL_MODERN_ADMIN_USERNAME: ${RECOLL_MODERN_ADMIN_USERNAME:?Set administrator username} + RECOLL_MODERN_ADMIN_PASSWORD: ${RECOLL_MODERN_ADMIN_PASSWORD:?Set administrator password} + RECOLL_MODERN_HOST: 0.0.0.0 + RECOLL_MODERN_PORT: 8000 + RECOLL_MODERN_ROOT_PATH: ${RECOLL_MODERN_ROOT_PATH:-/} + RECOLL_MODERN_TRUSTED_PROXIES: ${RECOLL_MODERN_TRUSTED_PROXIES:-} + RECOLL_MODERN_DEVELOPMENT_MODE: ${RECOLL_MODERN_DEVELOPMENT_MODE:-false} + RECOLL_MODERN_MEMBERSHIP_MAP: /var/lib/recoll-modern/membership.map + volumes: + - ${RECOLL_MODERN_DOCUMENTS:?Set documents directory}:/documents:ro + - service-state:/var/lib/recoll-modern + - recoll-index:/var/lib/recoll + healthcheck: + test: ["CMD", "python", "-c", "import os,urllib.request; p=os.environ.get('RECOLL_MODERN_ROOT_PATH','/').rstrip('/'); urllib.request.urlopen('http://127.0.0.1:8000'+p+'/health',timeout=5)"] + interval: 30s + timeout: 8s + retries: 3 + start_period: 30s + restart: unless-stopped +volumes: + service-state: + recoll-index: diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 52bf4ef..c5d251a 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -31,6 +31,7 @@ web UI and (b) an MCP server, running in Docker, with several independent collec | `src/recoll_modern/indexing/map_export.py` | Writes a durable atomic map from canonical paths to opaque collection IDs | | `helper/` | C reader run by Recoll's `metadatacmds` for each indexed file | | `Dockerfile`, `compose.yaml`, `docker/` | Non-root Recoll runtime, read-only document mount, persistent state/index volumes and config templates | +| `compose.published.yaml` | Pull-only deployment with the same runtime settings and persistent mounts | | `.gitea/workflows/image.yaml`, `scripts/image_tags.py` | CI checks and Gitea registry publication; Git refs select published tags | | `src/recoll_modern/engine/` | Shared Protocol, deterministic fake, and isolated Recoll Python/indexer adapter | | `src/recoll_modern/domain/auth.py`, `storage/auth.py` | Argon2 password checks, seeded administrator, hashed server-side sessions and CSRF verification | diff --git a/docs/TODO.md b/docs/TODO.md index 7f8ee7f..b5aee38 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -9,6 +9,9 @@ Spec 001 is implemented on `main`. The next product step is spec 002 (web UI). I ## Product and engineering backlog +- [ ] After merging spec 006, configure a Docker-capable Gitea runner and package-write Actions secrets, + then verify a main-branch publish, a version-tag publish and a pull of the published image. Only + then mark spec 006 implemented. - [ ] Decide whether a persistent directory rule may target the document root. Current sources require a named child directory. - [ ] Investigate whether Recoll's Python binding exposes a native numeric relevance score. The API diff --git a/docs/plans/006-gitea-image-publishing.md b/docs/plans/006-gitea-image-publishing.md index f8dee83..a31324e 100644 --- a/docs/plans/006-gitea-image-publishing.md +++ b/docs/plans/006-gitea-image-publishing.md @@ -1,5 +1,5 @@ --- -status: in-progress +status: done spec: docs/specs/006-gitea-image-publishing.md --- @@ -80,9 +80,9 @@ spec: docs/specs/006-gitea-image-publishing.md - `RECOLL_MODERN_IMAGE` selects a complete image reference; default is `gitea.lucaswinther.info/lucasw89/recoll-modern:latest`. - Other environment inputs and volume paths match `compose.yaml`. -- [ ] **Step 1: Run** `docker compose -f compose.published.yaml config` with example document/admin environment values; confirm failure because the file does not exist. -- [ ] **Step 2: Add** a pull-only Compose file that preserves the local file's port, environment, read-only document mount, `service-state` and `recoll-index` volumes, health check and restart policy. Do not include a `build:` key. -- [ ] **Step 3: Run** `docker compose -f compose.published.yaml config --format json` with example inputs. Inspect the resolved image, document `read_only` bind and both named volumes; confirm no build configuration. -- [ ] **Step 4: Update** README with a quick-start using `docker login` when needed, `RECOLL_MODERN_IMAGE`, `docker compose ... pull`, `up -d`, `ps`, `logs`, and an update that retains volumes. Explain first boot, tag semantics, `linux/amd64`, backups, Gitea Actions enablement, runner and PAT secret setup. Keep local-build instructions and revise architecture for the two deployment paths. -- [ ] **Step 5: Run** all project tests, lint and type checking again; verify Compose config for both files and `git diff --check`; inspect the whole diff against spec 006. -- [ ] **Step 6: Mark** the plan done and spec implemented only after completed local gates, then commit deployment docs/config and checked plan boxes as `docs: explain pulling and running published image`. A real Gitea publish/pull remains an external acceptance step until a runner and secret are configured and the branch is merged. +- [x] **Step 1: Run** `docker compose -f compose.published.yaml config` with example document/admin environment values; confirm failure because the file does not exist. +- [x] **Step 2: Add** a pull-only Compose file that preserves the local file's port, environment, read-only document mount, `service-state` and `recoll-index` volumes, health check and restart policy. Do not include a `build:` key. +- [x] **Step 3: Run** `docker compose -f compose.published.yaml config --format json` with example inputs. Inspect the resolved image, document `read_only` bind and both named volumes; confirm no build configuration. +- [x] **Step 4: Update** README with a quick-start using `docker login` when needed, `RECOLL_MODERN_IMAGE`, `docker compose ... pull`, `up -d`, `ps`, `logs`, and an update that retains volumes. Explain first boot, tag semantics, `linux/amd64`, backups, Gitea Actions enablement, runner and PAT secret setup. Keep local-build instructions and revise architecture for the two deployment paths. +- [x] **Step 5: Run** all project tests, lint and type checking again; verify Compose config for both files and `git diff --check`; inspect the whole diff against spec 006. +- [x] **Step 6: Mark** the plan done after completed local gates, then commit deployment docs/config and checked plan boxes as `docs: explain pulling and running published image`. Keep the spec approved until a real Gitea publish/pull is verified after the branch is merged.