diff --git a/.gitignore b/.gitignore index 942d6c8..115f582 100644 --- a/.gitignore +++ b/.gitignore @@ -26,6 +26,7 @@ coverage.xml htmlcov/ # Local runtime state +data/ *.sqlite3 *.sqlite3-shm *.sqlite3-wal diff --git a/README.md b/README.md index 8683c42..1b8ce10 100644 --- a/README.md +++ b/README.md @@ -44,6 +44,7 @@ RECOLL_MODERN_ADMIN_PASSWORD=use-a-long-unique-password RECOLL_MODERN_IMAGE=gitea.lucaswinther.info/lucasw89/recoll-modern:main EOF docker compose -f compose.published.yaml pull +sudo install -d -m 0700 -o 10001 -g 10001 data/service-state data/recoll-index docker compose -f compose.published.yaml up -d --no-build docker compose -f compose.published.yaml ps docker compose -f compose.published.yaml logs -f app @@ -54,8 +55,8 @@ Once a version appears in Gitea Packages, edit `RECOLL_MODERN_IMAGE` in `.env` t `RECOLL_MODERN_IMAGE` to use `:latest` after the first version tag is published. The image is published only after the CI checks pass; a tag that has not been published cannot be pulled. To update, set the desired tag and run `docker compose -f compose.published.yaml pull` followed -by `docker compose -f compose.published.yaml up -d --no-build` again. Keep the same Compose project -directory so its named volumes are reused. +by `docker compose -f compose.published.yaml up -d --no-build` again. Keep the same Compose file +directory so the local data paths are reused. For a trusted local HTTP trial with `docker run`, export the settings for that shell: @@ -65,13 +66,14 @@ export RECOLL_MODERN_ADMIN_USERNAME=admin export RECOLL_MODERN_ADMIN_PASSWORD='use-a-long-unique-password' export RECOLL_MODERN_IMAGE=gitea.lucaswinther.info/lucasw89/recoll-modern:main docker pull "$RECOLL_MODERN_IMAGE" +sudo install -d -m 0700 -o 10001 -g 10001 data/service-state data/recoll-index docker run -d --name recoll-modern --restart unless-stopped -p 127.0.0.1:8000:8000 \ -e RECOLL_MODERN_ADMIN_USERNAME -e RECOLL_MODERN_ADMIN_PASSWORD \ -e RECOLL_MODERN_HOST=0.0.0.0 -e RECOLL_MODERN_PORT=8000 \ -e RECOLL_MODERN_DEVELOPMENT_MODE=true \ --mount "type=bind,src=$RECOLL_MODERN_DOCUMENTS,dst=/documents,readonly" \ - --mount type=volume,src=recoll-modern-service-state,dst=/var/lib/recoll-modern \ - --mount type=volume,src=recoll-modern-index,dst=/var/lib/recoll \ + --mount "type=bind,src=$PWD/data/service-state,dst=/var/lib/recoll-modern" \ + --mount "type=bind,src=$PWD/data/recoll-index,dst=/var/lib/recoll" \ "$RECOLL_MODERN_IMAGE" ``` @@ -88,19 +90,21 @@ docker compose up -d --build The service listens on host port 8000 by default; set `RECOLL_MODERN_PUBLISH_PORT` to change it. The document directory is mounted read-only. SQLite, the membership map and generated Recoll configuration -live in the `service-state` volume; the index lives in the separate `recoll-index` volume. Back up both -volumes together and keep a separate copy of the documents. `docker compose down` preserves the volumes; -never pass `-v` unless you intentionally want to delete the index and service state. +live in `./data/service-state`; the index lives in `./data/recoll-index`, relative to the Compose file +directory. Both directories must be writable by container UID/GID `10001:10001`; create them with the +command above before starting either deployment. Back up both directories together and keep a separate +copy of the documents. Container replacement does not delete these bind-mounted directories. If you +already deployed with the earlier named volumes, copy both volumes' contents into these directories +while the old container is stopped, preserving ownership, before starting the new configuration. Do not +start against empty directories in place of existing data. -For the direct `docker run` example, the volumes are named `recoll-modern-service-state` and -`recoll-modern-index`. Back up both together. Replacing a container does not delete these volumes. - -On first boot, the API is available but `/health` may return 503 until a membership or reindex job has -built the first Recoll index. Log in through `POST /auth/login`, create a collection, then add file or -directory sources through `POST /collections/{id}/memberships`. Mutations return a durable job ID; -inspect `GET /jobs/{id}` for completion or file errors. `POST /jobs/reindex` refreshes the index without -deleting it. The login seed applies only when no administrator exists; changing the environment value -later does not rotate an existing password. Protect and back up the service-state volume. +On first boot, the API is available but `/health` can return 503 with a Recoll database-open error until +a membership or reindex job has built the first index. Log in through `POST /auth/login`, create a +collection, then add file or directory sources through `POST /collections/{id}/memberships`. Mutations +return a durable job ID; inspect `GET /jobs/{id}` for completion or file errors. `POST /jobs/reindex` +refreshes the index without deleting it. The login seed applies only when no administrator exists; +changing the environment value later does not rotate an existing password. Protect and back up +`./data/service-state`. Production sessions use Secure cookies, so serve the API over HTTPS. For a trusted local HTTP trial, set `RECOLL_MODERN_DEVELOPMENT_MODE=true`; do not use that setting on a public endpoint. Set diff --git a/compose.published.yaml b/compose.published.yaml index 7b41528..96e7949 100644 --- a/compose.published.yaml +++ b/compose.published.yaml @@ -14,8 +14,8 @@ services: RECOLL_MODERN_MEMBERSHIP_MAP: /var/lib/recoll-modern/membership.map volumes: - ${RECOLL_MODERN_DOCUMENTS:?Set documents directory}:/documents:ro - - service-state:/var/lib/recoll-modern - - recoll-index:/var/lib/recoll + - ./data/service-state:/var/lib/recoll-modern + - ./data/recoll-index:/var/lib/recoll healthcheck: test: ["CMD", "python", "-c", "import os,urllib.request; p=os.environ.get('RECOLL_MODERN_ROOT_PATH','/').rstrip('/'); urllib.request.urlopen('http://127.0.0.1:8000'+p+'/health',timeout=5)"] interval: 30s @@ -23,6 +23,3 @@ services: retries: 3 start_period: 30s restart: unless-stopped -volumes: - service-state: - recoll-index: diff --git a/compose.yaml b/compose.yaml index 2267a87..815c069 100644 --- a/compose.yaml +++ b/compose.yaml @@ -15,8 +15,8 @@ services: RECOLL_MODERN_MEMBERSHIP_MAP: /var/lib/recoll-modern/membership.map volumes: - ${RECOLL_MODERN_DOCUMENTS:?Set documents directory}:/documents:ro - - service-state:/var/lib/recoll-modern - - recoll-index:/var/lib/recoll + - ./data/service-state:/var/lib/recoll-modern + - ./data/recoll-index:/var/lib/recoll healthcheck: test: ["CMD", "python", "-c", "import os,urllib.request; p=os.environ.get('RECOLL_MODERN_ROOT_PATH','/').rstrip('/'); urllib.request.urlopen('http://127.0.0.1:8000'+p+'/health',timeout=5)"] interval: 30s @@ -24,6 +24,3 @@ services: retries: 3 start_period: 30s restart: unless-stopped -volumes: - service-state: - recoll-index: diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index ff4c8cc..8884867 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -30,7 +30,7 @@ web UI and (b) an MCP server, running in Docker, with several independent collec | `src/recoll_modern/indexing/handlers.py` | Scans membership sources before strict-mode mutations and records file errors | | `src/recoll_modern/indexing/map_export.py` | Writes a durable atomic map from canonical paths to opaque collection IDs | | `helper/` | C reader run by Recoll's `metadatacmds` for each indexed file | -| `Dockerfile`, `compose.yaml`, `docker/` | Non-root Recoll runtime, read-only document mount, persistent state/index volumes and config templates | +| `Dockerfile`, `compose.yaml`, `docker/` | Non-root Recoll runtime, read-only document mount, persistent local state/index directories and config templates | | `compose.published.yaml` | Pull-only deployment with the same runtime settings and persistent mounts | | `.gitea/workflows/image.yaml`, `scripts/image_tags.py`, `scripts/publish_image.sh` | CI checks and Gitea registry publication; Git refs select published tags | | `src/recoll_modern/engine/` | Shared Protocol, deterministic fake, and isolated Recoll Python/indexer adapter | @@ -65,7 +65,7 @@ future distribution track and starts only after the hosted product through 004 i - A Python service owns Recoll through its native Python bindings (`from recoll import recoll`). - SQLite holds collections, membership, indexing jobs, users and sessions. - An HTTP API consumed by both the web UI and the MCP server, so the two never diverge. -- A Docker image containing Recoll itself, plus compose wiring for document volumes and index storage. +- A Docker image containing Recoll itself, plus compose wiring for read-only documents and local index storage. - A packaging seam for a future desktop shell: application logic does not depend on Docker, state paths and listen addresses are configurable, process startup/shutdown is explicit, and every UI operation crosses the same HTTP API. Docker remains the only supported distribution through spec 004. @@ -129,9 +129,9 @@ display is limited to safe media types. The image installs Debian trixie's headless `recollcmd` package and Python binding. It syncs the application into a system-site-packages venv so the binding remains importable. A separate build stage compiles the C helper. The non-root process -copies Recoll config templates into the persistent state volume only when absent; +copies Recoll config templates into the persistent state directory only when absent; Recoll may generate additional support files there. Compose mounts documents -read-only and keeps the SQLite/config and Xapian index volumes separate. Initial +read-only and keeps the SQLite/config and Xapian index bind mounts separate. Initial health can be 503 until an indexing job creates the first index. Gitea Actions tests and builds on pull requests, main pushes and version tags. A dependent job publishes @@ -207,8 +207,8 @@ semantics and test strategy — are closed in the approved `docs/specs/001-searc |---|---| | Recoll (in-container) | All indexing and search. Not installed on the host — see `AGENTS.md` Rules. | | Docker + compose | The only supported way to run the service. | -| Document volumes | Nothing to index; collections resolve to empty. | -| A writable volume for the index and SQLite | All state. The document volume itself may be mounted read-only. | +| Document bind mount | Nothing to index; collections resolve to empty. | +| Writable local directories for the index and SQLite | All state. The document directory itself may be mounted read-only. | Docker is the only supported deployment for specs 001–004, not a permanent application-layer dependency. Spec 005 will design cross-platform packaging of the web UI, API service, Recoll and its diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md index a0cf665..ee200aa 100644 --- a/docs/DECISIONS.md +++ b/docs/DECISIONS.md @@ -442,3 +442,18 @@ layers. A split-package probe still produced a 253 MB compressed layer. **Consequences:** the service remains headless. The full image's largest compressed layer is 39.02 MB, and a local container confirms both Recoll commands and the Python binding are present. Registry publishing and pulling still require a successful Gitea Actions run. + +--- + +## 2026-09-28 — Keep Docker state beside the Compose file +**Status:** accepted +**Context:** named Docker volumes hide the index and service database from operators managing a +deployment from its Compose directory. The non-root container needs writable state owned by UID/GID +`10001:10001`. +**Decision:** both Compose variants bind-mount `./data/service-state` and `./data/recoll-index`. +The direct `docker run` example uses the same local directories. Ignore `data/` in Git and require +operators to create the directories with the container's ownership before startup. +**Alternatives:** retain Docker-managed named volumes and require volume-specific backup commands. +**Consequences:** moving the Compose file changes the resolved data paths. Existing named-volume +deployments must copy both stores into the local directories before switching mounts; neither store +is deleted or rebuilt by this change.