# Decisions Append-only, newest first. Never edit or delete an entry — supersede it with a new one that links back. One entry per decision a future maintainer would otherwise have to reverse-engineer. ## 2026-09-28 — Guard generated API documentation with the service session **Status:** accepted **Context:** FastAPI exposes generated OpenAPI and documentation routes outside ordinary route dependencies. The accepted spec requires authentication for every route except health and login while also requiring OpenAPI to work under a configured mount prefix. **Decision:** enforce the existing session check on the generated routes in prefix middleware after path normalization. Keep OpenAPI and the documentation pages available to logged-in clients. **Alternatives:** leave documentation public (violates R24); disable it (violates the intended OpenAPI surface under R29); add a separate authentication mechanism for documentation (unnecessary state). **Consequences:** generated documentation shares the API's cookie authentication and URL prefix. --- ## 2026-09-27 — Persist replay paths and pin original downloads **Status:** accepted **Context:** Review found that a job could mutate rule or collection state before its indexing step failed, leaving a retry without the old paths. `FileResponse` also reopened a path after containment checking, allowing a concurrent host-side symlink swap. **Decision:** persist affected paths in a running job before rule mutations; enqueue collection cleanup atomically with collection deletion. Open original files beneath the document root with no-follow directory descriptors and stream from that checked descriptor, including byte-range responses. **Alternatives:** recompute old paths on retry (the database may already have lost them); retain `FileResponse` after a path-only check (it reopens the path). **Consequences:** interrupted jobs retain their required reindex paths. File responses are tied to an opened regular file even if directory entries change later. The descriptor traversal relies on Linux `openat` semantics supplied by the current Docker deployment; desktop packaging must revisit it. --- ## 2026-09-18 — Cross-platform desktop packaging is spec 005 **Status:** accepted **Supersedes:** "The product is four specs, built in order" (below, same date), only by appending a fifth delivery after 004; the original order and boundaries of specs 001–004 remain unchanged. **Context:** the hosted Docker service should eventually also be usable as a self-contained installable application on Linux, macOS and Windows, without distracting from delivery of the core product. **Decision:** desktop distribution is spec 005 and starts only after 004. Its intended architecture is a Tauri-style shell that reuses the web UI and launches the existing HTTP API plus Recoll as bundled local sidecars. Tauri is the leading candidate, not a selected dependency; that choice belongs to 005. **Alternatives:** call backend libraries directly from a native shell (creates a second integration path); hide Docker behind a launcher (still requires a container runtime and integrates poorly with the desktop filesystem). **Consequences:** specs 001–004 stay Docker-first, but application logic cannot depend on Docker. State paths and listen addresses are configurable, lifecycle is explicit, and UI behavior crosses the HTTP API. Spec 005 must resolve per-platform Recoll packaging and licensing, filters, loopback security, signing, updates and installer formats before adding dependencies. --- ## 2026-09-18 — Deployment supports a configured URL subpath **Status:** accepted **Context:** the service may sit behind an existing reverse proxy under a path such as `/recoll`, not at the origin root. Hard-coded root URLs break redirects, cookies, OpenAPI and document links there. **Decision:** one configured external mount prefix drives routing, generated URLs and cookie scope. Forwarded scheme, host and prefix values are honored only from explicitly trusted proxies. **Alternatives:** root-only hosting (needlessly constrains deployment); trust every forwarded header (lets a direct client forge external URLs and cookie paths). **Consequences:** API and end-to-end tests run at both `/` and a non-root prefix. Proxy configuration is part of deployment documentation, and all URL generation goes through one boundary. --- ## 2026-09-18 — Search result fields are caller-selected **Status:** accepted **Supersedes:** the fixed result shape in "Search results carry metadata and snippets, not full text" (below, same date). The metadata-plus-snippets shape remains the default, not the only representation. **Context:** browser and MCP consumers have different response-size needs, and users need to choose whether results include metadata, snippets, extracted text or retrieval links. **Decision:** `document_id` is unconditional; callers select all other result fields. Inline extracted text is opt-in, bounded and marked when truncated. Authenticated endpoints return full extracted text and stream the canonical original separately. **Alternatives:** one fixed rich shape (wastes bandwidth and agent context); metadata-only search with a follow-up per hit (too many round trips); unbounded full text inline (a book can dominate the response). **Consequences:** field projection is part of the engine/API contract. Generated retrieval links honor the configured mount prefix, and original-file responses require range and safe-disposition handling. --- ## 2026-09-18 — Directory membership has normalized additive provenance **Status:** accepted **Context:** users need both one-time directory expansion and persistent directory membership, including the ability to omit metadata subdirectories such as `.calibre`. Overlapping sources must not make removal order-dependent. **Decision:** direct file additions and snapshot expansions are durable direct membership. Persistent directory rules materialize their own contributed documents and hold exact recursive exclusions scoped to the rule. Per-file removal creates an override against rule-derived membership; an explicit direct addition still wins. Persistent rules reconcile at startup and explicit reindex, not through watching. **Alternatives:** one polymorphic source ledger (weaker relational constraints); recompute every rule on demand (slow and nondeterministic); persistent rules without provenance (removing one rule can erase membership contributed by another). **Consequences:** the schema has separate direct, rule-derived and exclusion tables. Directory symlinks are never followed; file symlinks are accepted only inside the document root. Jobs default to best-effort, while strict jobs scan first and apply no membership changes if any file fails. --- ## 2026-09-18 — Document identity is a content hash, not a path **Status:** accepted **Context:** the document set may contain files that share a name but differ in content, and files that differ in name but are byte-identical. Keying membership on path merges neither and distinguishes neither correctly. **Decision:** a document is identified by the SHA-256 of its bytes. `documents` is keyed by that hash, `document_paths` maps many paths to one document, and membership attaches to the document. Each document has one **canonical path**, and only that path is reported to Recoll as belonging to any collection. **Alternatives:** path as identity (byte-identical copies appear as separate hits for the same content); relying on Recoll to deduplicate (it cannot — `md5` is empty and `sig` is only size + mtime). **Consequences:** duplicate copies are indexed by Recoll but belong to no collection, so the required collection scope makes them unreachable — verified: a scoped search over a tree with one duplicated file returned it once. Deleting a canonical path must promote another path and force a reindex, or the document silently leaves every collection. --- ## 2026-09-18 — Membership is supplied at index time; documents are never written to **Status:** accepted **Supersedes:** "One index; collection membership is an opaque per-file tag" (below, same date), which carried membership in a `user.collection` extended attribute. That entry's xattr mechanism must not be implemented. **Context:** the service must work with **read-only** access to the documents it indexes. Writing an xattr requires write access, so the superseded design fails outright on a `:ro` mount — verified: `setfattr` returns "Read-only file system". **Decision:** membership reaches Recoll through `metadatacmds` in `recoll.conf` (`metadatacmds = ; collection = %f`), a documented Recoll facility for external metadata. Recoll runs the helper per file at index time; the helper answers from our own store. Everything the superseded entry established about opaque IDs, multi-membership and single-ranked-set OR queries still holds — only the transport changes. **Alternatives:** filtering Recoll's results against SQLite after the query (breaks paging and total counts); one index per collection (cannot express two files in one directory belonging to different collections); requiring a writable document volume (contradicts the requirement). **Consequences:** SQLite becomes the *only* store of membership rather than the authoritative copy of two, so drift between store and index is now impossible and no reconcile path is needed. Extended attributes are no longer a deployment requirement. A membership change needs `recollindex -i -Z ` — verified that neither a plain incremental run nor `recollindex -i` notices it, because Recoll's change signature is size + mtime and a membership change touches neither. The helper is spawned once per file per index run, so it must start fast: over 400 files, indexing took 0.09s with no helper, 0.13s with a shell+awk helper and 0.82s with a `python3` helper. The service exports a flat map from SQLite for a small non-Python helper to read. --- ## 2026-09-18 — Search results carry metadata and snippets, not full text **Status:** accepted **Context:** Recoll can return metadata, keyword-in-context snippets, extracted plain text and the original file. The response shape drives both the web UI and the MCP tool. **Decision:** a search hit carries title, author, path, MIME type, size, date, score, its collections and Recoll's snippets. Extracted text and file download are separate endpoints. **Alternatives:** metadata only (a results list without context is markedly worse, and an MCP client would fetch every candidate to judge it, costing more overall); metadata plus full extracted text (responses become enormous for book-length PDFs and blow an agent's context window). **Consequences:** one search call is enough for an agent to judge relevance. The UI needs a second call to open a document, which is the right trade. --- ## 2026-09-18 — Authentication in 001, user administration in 004 **Status:** accepted **Context:** the service needs real auth, but roles, user CRUD and per-collection permissions are a subsystem in their own right, and `PRODUCT.md` currently describes a single technical operator. **Decision:** 001 ships genuine authentication — argon2 password hashing, server-side sessions, an admin account seeded from the environment at first boot, every endpoint behind an auth dependency. User administration and permissions are deferred to spec 004. **Alternatives:** no auth in the service, trusting a reverse proxy (a misconfigured proxy or an exposed Docker port would leave the destructive endpoints naked); the full identity system in 001 (a large addition to an already-grown spec, building for users who do not exist yet). **Consequences:** 001 is safe to expose behind a proxy. 004 must extend the auth layer without reshaping endpoints, so the auth dependency is designed as a seam from the start. --- ## 2026-09-18 — Search scope is an explicit, required parameter **Status:** accepted **Context:** `docs/ARCHITECTURE.md` proposed the invariant that a query never silently spans collections. A Recoll query with no collection clause returns every document, including untagged ones. **Decision:** every search names one or more collections; omitting the parameter is an error, not an implicit "all". Multiple collections are OR-ed into one Xapian query so ranking stays coherent and each hit reports which collections it belongs to. **Alternatives:** single-collection-only searches (the UI and MCP would be built around a model they later outgrow); a searchable-by-default flag (reintroduces the implicit scope the invariant forbids); per-collection fan-out and application-side merging (scores from separate databases are not comparable, so the merged order would be strictly worse than letting Xapian rank). **Consequences:** the API layer enforces the invariant, because Recoll will not. Every search path needs a test proving an unscoped request is rejected. --- ## 2026-09-18 — Indexing runs as an in-process job queue **Status:** accepted **Context:** collections are created at runtime, so something must index a collection the moment it is created, and report progress back to the UI and to MCP clients. **Decision:** creating a collection, changing membership, or calling the reindex endpoint enqueues a job. A background worker in the same FastAPI process runs `recollindex` and records status, progress and errors in SQLite. One container process, no broker. **Alternatives:** `recollindex -m` real-time monitoring (one supervised process per collection, inotify watch limits, and a new collection still needs a first full index from somewhere — revisit as an opt-in mode later); cron-scheduled full runs (new collections sit empty until the next tick, and there is no way to force a refresh). **Consequences:** jobs die with the process, so the queue must recover interrupted runs at startup. Job state is queryable, so "indexing, 40%" is available to every consumer. --- ## 2026-09-18 — Collections are runtime-managed through the API **Status:** accepted **Context:** collections could be declared in a mounted config file or managed at runtime. **Decision:** collections are created, edited and deleted through the HTTP API, with SQLite as the store of record. The service creates and maintains the index state on demand. **Alternatives:** a declarative mounted `collections.yaml` (reproducible and git-friendly, but no management UI without adding a write path later); one collection per subdirectory by convention (forces the on-disk layout to match the collection layout, which the per-file membership requirement rules out). **Consequences:** 001 grows a persistence layer, validation and migrations. `AGENTS.md` makes deleting or rebuilding an index an ask-first operation, so the delete path needs an explicit guard rather than a bare `DELETE`. --- ## 2026-09-18 — One index; collection membership is an opaque per-file tag **Status:** accepted **Context:** a collection cannot be "a directory", because files sharing a directory must be able to belong to different collections. Verified in a throwaway Recoll 1.43.0 container — details and the exact config in `docs/ARCHITECTURE.md` under "Verified Recoll mechanics". **Decision:** index every document once into a single Recoll index. Membership is a `user.collection` extended attribute mapped to a queryable Recoll field. The tag is an **opaque single-token ID**; the human-readable name lives only in SQLite. **Alternatives:** one index per collection with rule-based selection (a file in two collections is stored and indexed twice and can be returned twice, and hand-picking files in a shared directory is awkward); an explicit file list driving indexing (does not map onto Recoll's topdirs-walking model). **Why opaque IDs:** Recoll tokenises the field value and matches case-insensitively, so a tag of `cook-books` is also matched by `collection:cook` and `collection:books`, and `TechBooks` by `collection:techbooks`. Names-as-tags would let collections silently bleed into each other — exactly what the no-silent-spanning invariant forbids. Opaque IDs tested clean, with no partial matches. **Consequences:** collection names need no character restrictions and renaming never touches the index. Documents must sit on a volume supporting extended attributes, checked at startup — without it every scoped search silently returns nothing. Retagging costs `recollindex -i `, not a rebuild, because a plain incremental run does not notice xattr-only changes. --- ## 2026-09-18 — Stack: Python + FastAPI, managed with uv **Status:** accepted **Context:** the stack was unchosen and is an ask-first decision. Recoll ships first-class Python bindings; every other language must drive `recollq` as a subprocess and parse its output. **Decision:** Python + FastAPI for the engine and HTTP API, with `uv` as the package manager. `uv` 0.12.16 was installed on this host as part of this decision, and `AGENTS.md` updated in the same commit. **Alternatives:** TypeScript on Node/Bun or C# on ASP.NET Core (one language across UI, API and MCP, but both make a fragile output parser load-bearing); a split Python engine behind a TypeScript API (two runtimes and an extra hop inside one container). **Consequences:** structured results with no parsing layer, and OpenAPI for free, which specs 002 and 003 can generate clients from. The web UI will likely be a second language; that boundary is the HTTP API. --- ## 2026-09-18 — The product is four specs, built in order **Status:** accepted **Context:** "a web UI and an MCP server over Recoll" is four subsystems. One spec covering all of them would be too large to plan or review, and `PRODUCT.md` is still an uninterviewed stub, so any UI spec written now would design against fiction. **Decision:** 001 search engine + API, 002 web UI, 003 MCP server, 004 multi-user management. Each gets its own spec, plan and review cycle. The table lives in `docs/ARCHITECTURE.md`. **Alternatives:** one thin vertical slice through all four (would lock in a stack and a UI before `PRODUCT.md` is real); engine plus MCP together (defensible, but it widens 001, which has already grown). **Consequences:** 002 is blocked on `/impeccable init`. 001's HTTP API is the contract 002 and 003 are both built against, so it is the piece that must be got right. --- ## 2026-09-18 — Override the superpowers artefact paths **Status:** accepted **Context:** superpowers 6.3.0 writes specs to `docs/superpowers/specs/YYYY-MM-DD--design.md` and plans to `docs/superpowers/plans/YYYY-MM-DD-.md` (verified in the installed skill files). That nests a second docs tree and dates artefacts that we want numbered and paired. **Decision:** specs -> `docs/specs/NNN-slug.md`, plans -> `docs/plans/NNN-slug.md`, plan number matching its spec number. Recorded in `AGENTS.md` so every session picks it up. **Alternatives:** accept the defaults (two docs trees, no spec/plan pairing); symlink (fragile on Windows and in containers). **Consequences:** the override must be restated if `AGENTS.md` is ever trimmed; a superpowers upgrade that changes its defaults needs this entry re-checked. --- ## 2026-09-18 — Adopt AGENTS.md as single source of truth **Status:** accepted **Context:** The repository was empty — no commits, no stack, no agent context. Agents had nothing durable to read and would re-explore from scratch every session. **Decision:** `AGENTS.md` is canonical; `CLAUDE.md` and `GEMINI.md` are thin pointers with no rules of their own. Enforceable things live in `.claude/settings.json` and `.gitignore` rather than in prose. **Alternatives:** per-tool rule files (drift and synchronisation debt); no rules file (measurably more steps and tokens per task). **Consequences:** rule changes happen in one file; the pointer files must stay thin; `AGENTS.md` is an executable-context and prompt-injection surface, so changes to it get reviewed like CI config. --- ## 2026-09-18 — No DESIGN.md until the first UI exists **Status:** accepted **Context:** The `impeccable` skill reads `PRODUCT.md` and `DESIGN.md`. `PRODUCT.md` could be written from the stated product intent; `DESIGN.md` could not — there is no UI, no tokens, no visual world. **Decision:** write `PRODUCT.md` now; leave `DESIGN.md` absent. The first UI build produces it (`/impeccable document` derives it from shipped code). **Alternatives:** a placeholder `DESIGN.md` — rejected, it would assert a visual system that does not exist and the skill would design against fiction. **Consequences:** the first UI task must end by writing `DESIGN.md`, not skip it. --- ## 2026-09-26 — Retrieve text by Recoll UDI and expose rank-based score **Status:** provisional until the real-container contract runs **Context:** the approved plan proposed an escaped path query for complete text and a relevance score. Recoll's Python API documents `fsudi.fs_udi(path)` and `Db.getDoc()` for exact path lookup, but does not document a numeric query weight on returned documents. **Decision:** derive the UDI from the canonical path for full-text extraction. Set `score` to the reciprocal result rank, preserving Recoll's order across pages while making no claim that the number is Xapian's raw weight. **Alternatives:** a query-language path clause that must safely encode quotes and parentheses; an undocumented Recoll binding attribute for score. **Consequences:** unusual filenames do not enter a query parser. Clients can compare score order within one query, but the value is a rank proxy. Revisit this once the real Recoll container can be tested. --- ## 2026-09-26 — Rotate CSRF tokens on session reads **Status:** accepted **Context:** sessions store only SHA-256 token digests, but `GET /auth/session` must return a usable CSRF token after a page reload. The raw token cannot be recovered from SQLite. **Decision:** authenticate the session, replace its stored CSRF digest, and return a newly generated token. The previous token becomes invalid immediately. **Alternatives:** store the raw CSRF token; derive a repeatable token from the cookie and a server key. **Consequences:** clients should refresh their CSRF token after calling the session endpoint. Session storage contains no raw cookie or CSRF token. --- ## 2026-09-26 — Persist writable Recoll config beside service state **Status:** accepted **Context:** Debian Recoll 1.43 writes generated support files into its config directory. A non-root container cannot write to image-owned `/etc/recoll-modern`; a live first-boot test surfaced this. **Decision:** keep immutable templates under `/etc/recoll-modern`, copy each missing template into `/var/lib/recoll-modern/recoll-config` on startup, and point both the engine and indexer there. Never overwrite an existing runtime config file. Keep the Recoll index in its own named volume. **Alternatives:** chown `/etc/recoll-modern` in the image (runtime changes would not survive container replacement); run the service as root. **Consequences:** service-state backups include generated Recoll configuration, and config changes made by an operator persist across image upgrades. Template updates require an explicit migration if an existing config should adopt them. --- ## 2026-09-26 — Treat an empty Recoll document URL as a missing hit **Status:** accepted **Context:** the live Recoll 1.43 Python binding returns a `Doc` object with an empty `getbinurl()` for a UDI absent from the index; it does not return `None`. Passing that object to `rclextract.Extractor` raises an internal-file error. **Decision:** check the returned URL before extraction and return empty text for an absent UDI. **Alternatives:** catch the generic extraction error, which would hide genuine extraction failures. **Consequences:** exact-path text lookup follows the engine contract for missing documents while real extraction failures still surface as `engine_unavailable`.