feat: add membership provenance and rules

docs/ARCHITECTURE.md

@@ -4,9 +4,9 @@ How `recoll-modern` is built today. Update in the same commit as any structural
 
 ## Overview
 
-The Python service scaffold, SQLite schema, collection repository, and document discovery/identity
-modules are built. The API, job queue, and Recoll adapter remain intended structure. Recoll behaviour
-below was verified against a real container.
+The Python service scaffold, SQLite schema, collection and membership repositories, and document
+discovery/identity modules are built. The API, job queue, and Recoll adapter remain intended structure.
+Recoll behaviour below was verified against a real container.
 
 The goal: a self-hosted service that wraps Recoll's indexing and full-text search behind (a) a modern
 web UI and (b) an MCP server, running in Docker, with several independent collections such as
@@ -22,6 +22,8 @@ web UI and (b) an MCP server, running in Docker, with several independent collec
 | `src/recoll_modern/` | Python service package; configuration, application factory and CLI boundary |
 | `src/recoll_modern/indexing/` | Root-relative path policy, read-only discovery, stable file hashing |
 | `src/recoll_modern/storage/` | SQLite migrations, collection metadata and content-hash identity |
+| `src/recoll_modern/domain/membership.py` | Validates sources and applies direct additions and rule changes |
+| `src/recoll_modern/indexing/reconcile.py` | Refreshes persistent directory rules while retaining excluded or unreadable files as present |
 | `tests/` | Automated service tests |
 
 Discovery produces content hashes and root-relative paths without writing to the document volume.

docs/TODO.md

@@ -15,12 +15,11 @@ Where the workflow stands (`AGENTS.md` -> Workflow: brainstorm -> spec -> plan -
   selectable search fields, subpath hosting and the packaging seam for deferred desktop spec 005.
 - Plan `docs/plans/001-search-engine-api.md`: **approved 2026-09-18.** Its listed Python runtime and
   development dependencies are approved. Execution follows the plan task by task.
-- Plan Tasks 1–3 are complete and independently reviewed. Task 4 adds read-only discovery, stable
-  hashing and content-hash identity; its checkboxes are current in the plan. The next task is additive
-  membership and directory rules.
+- Plan Tasks 1–3 are complete and independently reviewed. Tasks 4–5 add read-only discovery,
+  content-hash identity, additive membership and directory rules. Their checkboxes are current in the
+  plan. The next task is the durable job queue.
 
-The next action is Task 5, "Implement additive membership and directory rules." Do not repeat
-Tasks 1–4.
+The next action is Task 6, "Add the durable single-worker job queue." Do not repeat Tasks 1–5.
 
 Things a resuming agent will otherwise get wrong:
 

docs/plans/001-search-engine-api.md

@@ -412,7 +412,7 @@ git commit -m "feat: add document discovery and identity"
 - Produces: `MembershipService.plan_add`, `plan_remove_document`, `plan_patch_rule`, `plan_delete_rule`
 - Produces: `reconcile_rule(rule_id, policy, repositories) -> ReconcileResult`
 
-- [ ] **Step 1: Write failing provenance tests**
+- [x] **Step 1: Write failing provenance tests**
 
 ```python
 def test_direct_membership_wins_over_rule_exclusions(repo: MembershipRepository) -> None:
@@ -432,11 +432,11 @@ def test_rule_reconciliation_removes_departed_file_without_touching_direct(repo:
     assert repo.has_direct(COLLECTION_ID, DEPARTED_HASH)
 ```
 
-- [ ] **Step 2: Run tests and observe failure**
+- [x] **Step 2: Run tests and observe failure**
 
 Run: `uv run pytest tests/domain/test_membership.py tests/indexing/test_reconcile.py tests/storage/test_membership_repository.py -v`
 
-- [ ] **Step 3: Implement desired-state operations**
+- [x] **Step 3: Implement desired-state operations**
 
 Snapshot directory results and file sources call `add_direct`; persistent directories create `r` plus ten hex digits, validate exclusions as exact descendants, then atomically replace that rule's materialized membership. Effective membership is this SQL shape:
 
@@ -455,7 +455,7 @@ ORDER BY collection_id;
 
 Removing a document deletes its direct row and upserts a collection-level rule override. Explicitly adding that file again deletes the override and adds direct membership. Rule deletion returns canonical paths whose effective collection set changed.
 
-- [ ] **Step 4: Verify and commit**
+- [x] **Step 4: Verify and commit**
 
 Run: `uv run pytest tests/domain/test_membership.py tests/indexing/test_reconcile.py tests/storage/test_membership_repository.py -v && uv run ruff check . && uv run mypy src tests`