Archived
Lucas WintherandClaude Opus 5
883db6a59d
The service must work with read-only access to the documents it indexes. Writing a user.collection extended attribute needs write access, so the previous membership design fails on a :ro mount; setfattr returns "Read-only file system". Verified in the spike container. Membership now reaches Recoll through metadatacmds in recoll.conf, a documented facility for external metadata: Recoll runs a helper per file at index time and the helper answers from our own store. Verified end to end against a read-only bind mount. Everything previously established about opaque collection IDs, multi-membership and single-ranked-set OR queries survives unchanged, because only the transport differs. SQLite becomes the only store of membership rather than the authoritative one of two, so drift is impossible and the reconcile path is gone. Extended attributes are no longer a deployment requirement. Document identity becomes the SHA-256 of file content. Files sharing a name but differing in content are separate documents; one file reachable by several paths is a single document with one canonical path, and only that path is reported as belonging to a collection. Recoll cannot help here: md5 is empty and sig is only size plus mtime. Two measured constraints are recorded. A membership change is invisible to both a plain incremental run and recollindex -i, because the change signature is size plus mtime; recollindex -i -Z on the affected path picks it up in about 15ms. And the helper is spawned once per file per index run, so it must start fast: a 400-file index took 0.09s with no helper, 0.13s with shell and awk, and 0.82s with python3. The superseded entry in docs/DECISIONS.md is left intact, per that file's append-only rule, and the new entry links back to it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2.3 KiB
2.3 KiB
TODO
Durable backlog. Agents append here instead of leaving TODO comments in source.
Small, specific, independently actionable. Remove only when actually done.
Now
- Answer the six open design questions in
docs/ARCHITECTURE.md— closed 2026-09-18, reasoning indocs/DECISIONS.md. - Finish the 001 design: component boundaries and SQLite schema, API surface and error semantics,
test strategy. Then write
docs/specs/001-search-engine-api.md. - Fill the
Commandssection ofAGENTS.mdand delete itsTODO(maintainer)line — must happen in the same commit that scaffolds the app, with every command actually run.
Next
- Scaffold Python + FastAPI with
uv; verify install/dev/test/lint/typecheck/build by running them, then record the verified ones inAGENTS.md. - Dockerfile + compose: Recoll in the image, document volume mounted
:ro, writable volumes for the index and SQLite. - Write the
metadatacmdsmembership helper as a fast-starting non-Python program, fed by a flat map exported from SQLite. Apython3helper made a 400-file index 9x slower; Recoll spawns it once per file per index run. - Re-measure indexing overhead against real documents (PDF, EPUB) rather than small text files — extraction cost should dwarf the helper, but that is an assumption until measured.
- Promote a new canonical path and force a reindex when a canonical path disappears, or the document silently leaves every collection it is in.
- Recover interrupted indexing jobs at startup; they do not survive a process restart on their own.
- Write
PRODUCT.mdproperly via/impeccable init— the current file is a stub from the initial brief, not an interview. Blocks spec 002.
Someday
- Propose a CI check (Gitea Actions): assert
AGENTS.md,CLAUDE.md,GEMINI.mdand thedocs/paths exist, and fail ifAGENTS.mdexceeds 150 lines.
Known issues
- None recorded yet.
Recurring
- Add an
AGENTS.mdrule only in response to a real, observed agent failure — one line, ideally with a code example. Never pre-empt hypothetical mistakes. - Update
AGENTS.mdin the same commit as the convention it describes. A stale rules file misleads. - Prune quarterly: delete rules whose convention changed or whose mistake no longer occurs.