Archived
docs: retire spec 001 checkpoint
This commit is contained in:
Lucas Winther
+6
-5
@@ -12,9 +12,9 @@ web
|
||||
|
||||
## Stack
|
||||
|
||||
The backend stack was confirmed during the 001 design: Python + FastAPI, managed with `uv`, because
|
||||
Recoll provides native Python bindings. The frontend stack remains undecided. No application has been
|
||||
scaffolded yet. See `docs/DECISIONS.md` for the decision and `AGENTS.md` for installed toolchains.
|
||||
The backend stack was confirmed during the 001 design and implemented: Python + FastAPI, managed with
|
||||
`uv`, because Recoll provides native Python bindings. The frontend stack remains undecided, and no web
|
||||
UI has been built. See `docs/DECISIONS.md` for the decision and `AGENTS.md` for installed toolchains.
|
||||
|
||||
## Users
|
||||
|
||||
@@ -50,8 +50,9 @@ Confirmed from the brief:
|
||||
- Web UI, served from a Docker deployment.
|
||||
- MCP server exposing the same search capability.
|
||||
|
||||
Explicitly undecided: authentication, cross-collection search, indexing schedule, result preview depth,
|
||||
supported document types beyond Recoll's own. See `docs/ARCHITECTURE.md` for the full list.
|
||||
Spec 001 settled authentication, cross-collection search, indexing triggers, and API result fields.
|
||||
UI presentation and filter coverage beyond the shipped container remain undecided. See
|
||||
`docs/ARCHITECTURE.md` for the implemented shape.
|
||||
|
||||
## Brand Commitments
|
||||
|
||||
|
||||
+20
-88
@@ -1,96 +1,28 @@
|
||||
# TODO
|
||||
|
||||
Durable backlog. Agents append here instead of leaving `TODO` comments in source.
|
||||
Small, specific, independently actionable. Remove only when actually done.
|
||||
Open work only. Record completed work in its spec, plan, commits, and `docs/DECISIONS.md`; do not
|
||||
leave `TODO` comments in source.
|
||||
|
||||
## Resume here
|
||||
Spec 001 is implemented on `main`. The next product step is spec 002 (web UI). Its first step is the
|
||||
`/impeccable init` interview to replace the `PRODUCT.md` stub; the first UI build then creates
|
||||
`DESIGN.md`. Follow the spec → plan → TDD → review workflow in `AGENTS.md`.
|
||||
|
||||
Last touched 2026-09-28. Spec 001 implementation has completed its review fixes and acceptance matrix.
|
||||
## Product and engineering backlog
|
||||
|
||||
Where the workflow stands (`AGENTS.md` -> Workflow: brainstorm -> spec -> plan -> TDD -> review):
|
||||
|
||||
- Brainstorm for spec 001: **done.** All six design questions in `docs/ARCHITECTURE.md` are closed, with
|
||||
reasoning in `docs/DECISIONS.md`.
|
||||
- Spec `docs/specs/001-search-engine-api.md`: **approved 2026-09-18.** It includes directory rules,
|
||||
selectable search fields, subpath hosting and the packaging seam for deferred desktop spec 005.
|
||||
- Plan `docs/plans/001-search-engine-api.md`: **approved 2026-09-18.** Its listed Python runtime and
|
||||
development dependencies are approved. Execution follows the plan task by task.
|
||||
- Plan Tasks 1–3 are complete and independently reviewed. Tasks 4–14 add discovery, content-hash
|
||||
identity, additive membership, durable jobs, the atomic membership map, the engine boundary,
|
||||
session authentication, the HTTP shell, membership/rule/job routes, scoped search and document
|
||||
retrieval, Docker packaging and acceptance coverage. Their checkboxes are current in the plan.
|
||||
The first code review found replay, reconciliation, and download races. A follow-up review found
|
||||
generated documentation exposed without a session, incomplete unfiltered reindexing, stale tags
|
||||
after an earlier duplicate became canonical, and lost membership on best-effort rule resubmission.
|
||||
These are fixed on `feat/001-document-discovery`; branch handoff remains.
|
||||
|
||||
The next action is branch integration by the maintainer's choice. Do not repeat Tasks 1–14. The review
|
||||
fixes include idempotent rule creation, persisted replay paths, atomic collection cleanup enqueue,
|
||||
unreadable-file preservation, departed-path reindexing, directory-rule pagination, recovery request
|
||||
validation, descriptor-pinned downloads, authenticated API documentation, complete unfiltered
|
||||
reindexing, canonical-path retagging, and best-effort rule resubmission. The generated real-container
|
||||
lifecycle test covers the major flows through Compose. On 2026-09-28, the host suite passed with 160
|
||||
passed and 8 container-dependent skips, and the required-container suite passed all 9 checks. Lint,
|
||||
typecheck and build passed. The isolated acceptance Compose project was stopped with
|
||||
`docker compose down` and its named volumes were preserved.
|
||||
|
||||
Things a resuming agent will otherwise get wrong:
|
||||
|
||||
- The design carried collection membership in a `user.collection` extended attribute for part of this
|
||||
session. **That is superseded** — the documents are read-only. `docs/DECISIONS.md` has both entries,
|
||||
newest first; the older one is left intact by that file's append-only rule, so read the newer one.
|
||||
- The Recoll facts in `docs/ARCHITECTURE.md` -> "Verified Recoll mechanics" were measured in a container,
|
||||
not recalled. Trust them over intuition, and re-verify only if Recoll's major version moves.
|
||||
- The throwaway spike container is not part of the repo and does not survive the session. Everything
|
||||
worth keeping from it is already in `docs/`.
|
||||
- `uv` was installed on this host during this session and `AGENTS.md` was corrected accordingly. Task 1
|
||||
replaced its command stub in the scaffold commit after running every listed command.
|
||||
|
||||
## Now
|
||||
- [x] Answer the six open design questions in `docs/ARCHITECTURE.md` — closed 2026-09-18, reasoning in
|
||||
`docs/DECISIONS.md`.
|
||||
- [x] Finish the 001 design: component boundaries and SQLite schema, API surface and error semantics,
|
||||
test strategy. Then write `docs/specs/001-search-engine-api.md`.
|
||||
- [x] Fill the `Commands` section of `AGENTS.md` and delete its `TODO(maintainer)` line — must happen in
|
||||
the same commit that scaffolds the app, with every command actually run.
|
||||
|
||||
## Next
|
||||
- [ ] Verify the Recoll adapter against a generated corpus in the Compose container, including
|
||||
exact-path text retrieval and whether the Python API exposes a native numeric relevance score.
|
||||
- [x] Scaffold Python + FastAPI with `uv`; verify install/dev/test/lint/typecheck/build by running them,
|
||||
then record the verified ones in `AGENTS.md`.
|
||||
- [x] Dockerfile + compose: Recoll in the image, document volume mounted `:ro`, writable volumes for
|
||||
the index and SQLite.
|
||||
- [x] Write the `metadatacmds` membership helper as a fast-starting non-Python program, fed by a flat
|
||||
map exported from SQLite. A `python3` helper made a 400-file index 9x slower; Recoll spawns it
|
||||
once per file per index run.
|
||||
- [ ] Re-measure indexing overhead against real documents (PDF, EPUB) rather than small text files —
|
||||
extraction cost should dwarf the helper, but that is an assumption until measured.
|
||||
- [x] Promote a new canonical path and force a reindex when a canonical path disappears, or the
|
||||
document silently leaves every collection it is in.
|
||||
- [x] Recover interrupted indexing jobs at startup; Task 6 requeues valid running jobs and fails
|
||||
unreadable requests.
|
||||
- [ ] Write `PRODUCT.md` properly via `/impeccable init` — the current file is a stub from the initial
|
||||
brief, not an interview. Blocks spec 002.
|
||||
|
||||
## Someday
|
||||
- [ ] After delivery 004, write spec 005 for a cross-platform installable desktop distribution. Start
|
||||
from a Tauri-style shell that reuses the web UI and bundles the API and Recoll as sidecars; decide
|
||||
the actual framework, licensing, signing, updates and per-platform filter coverage in that spec.
|
||||
- [ ] Propose a CI check (Gitea Actions): assert `AGENTS.md`, `CLAUDE.md`, `GEMINI.md` and the `docs/`
|
||||
paths exist, and fail if `AGENTS.md` exceeds 150 lines.
|
||||
- [ ] Decide whether a persistent directory rule may target the document root. Current sources require
|
||||
a named child directory.
|
||||
- [ ] Investigate whether Recoll's Python binding exposes a native numeric relevance score. The API
|
||||
currently returns a rank-based proxy; exact-path text retrieval was verified in the container.
|
||||
- [ ] Measure membership-helper indexing overhead on representative PDF and EPUB documents. Current
|
||||
measurements cover small text files only.
|
||||
- [ ] After spec 004, design spec 005 for an installable desktop distribution, including framework,
|
||||
Recoll packaging and licensing, signing, updates, and filter coverage.
|
||||
- [ ] Consider a Gitea Actions check for required agent/docs files and the `AGENTS.md` length limit.
|
||||
|
||||
## Known issues
|
||||
- The API test suite reports deprecation warnings from the installed FastAPI/Starlette TestClient
|
||||
dependency. Replace its client integration when the dependency stack supports the successor API.
|
||||
- Directory sources currently require a named child of the document root; decide whether the root
|
||||
itself should be accepted as a persistent rule before finishing membership endpoints.
|
||||
- Task 3's repository ID test checks the generated collection ID prefix and length rather than asserting
|
||||
the full `^c[0-9a-f]{10}$` regex. The implementation uses `secrets.token_hex(5)` and is correct; add
|
||||
the stronger regression assertion when work resumes.
|
||||
|
||||
## Recurring
|
||||
- [ ] Add an `AGENTS.md` rule only in response to a real, observed agent failure — one line, ideally with
|
||||
a code example. Never pre-empt hypothetical mistakes.
|
||||
- [ ] Update `AGENTS.md` in the same commit as the convention it describes. A stale rules file misleads.
|
||||
- [ ] Prune quarterly: delete rules whose convention changed or whose mistake no longer occurs.
|
||||
- The API tests report deprecation warnings from FastAPI/Starlette's TestClient dependencies. Update
|
||||
the client integration when a supported replacement is available.
|
||||
- The repository ID test checks the collection ID prefix and length, rather than the full
|
||||
`^c[0-9a-f]{10}$` pattern. The implementation uses `secrets.token_hex(5)`; strengthen this
|
||||
regression assertion when touching that test.
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Status:** approved 2026-09-18
|
||||
**Status:** done 2026-09-28 (approved 2026-09-18)
|
||||
|
||||
**Goal:** Build the authenticated, collection-scoped Recoll service defined by spec 001, including durable membership/indexing jobs, document retrieval, and its Docker deployment.
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
status: approved
|
||||
status: implemented
|
||||
plan: docs/plans/001-search-engine-api.md
|
||||
supersedes: none
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user