diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 249345d..80bab88 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -14,8 +14,8 @@ user layer or files a Changes-inbox event. ## Map | Path | Responsibility | State | |---|---|---| -| `apps/server` | `Bun.serve`: `/api/v1`, `/ingest/`, job worker, static web | planned | -| `apps/web` | React app (HTML imports) | planned | +| `apps/server` | `main.ts` boots (config → `createDb` → `migrateToLatest` → `loadPlugins` → worker + scheduler + daily backup → `Bun.serve`). `app.ts` `createApp(deps)` = `/api/v1` route table + `/ingest/*` mount (404 in Phase 1), testable via `app.fetch`. `http.ts` = `route()`, auth (session cookie / Bearer app token), CSRF header, body caps, rate limit, error mapping. `routes/.ts` one file per area; `openapi.ts` builds `/api/v1/openapi.json` from the table | exists | +| `apps/web` | React app (HTML imports); `index.html` is a placeholder until Task 17 | placeholder | | `packages/core` | domain services, one module per area (import by path, no barrel): `auth`, `crypto`, `works`, `matching`, `entries` (entries, library listing, quick add, per-user moves, match suggestions), `engagements`, `history` (`user_history`, restore), `lists`, `copies`, `custom-fields`, `attachments`, `export` (per-user bundle, CSV, restore), `backup` (scheduled bundles + SQLite `VACUUM INTO`), `sync/` (the change engine: `snapshot` hashing and versions, `apply` per-record diff and apply, `run` = `runSync`, `changes` = Changes inbox `listChanges`/`resolveChange`, `listens` = `ingestListens`), `jobs` (queue, worker, scheduler), `registry` (`loadPlugins`), `connections` (CRUD, plugin context, `syncHandler`) | exists (partial) | | `packages/db` | Kysely over one Bun.SQL dialect (SQLite + Postgres), ids, JSON helpers, migrations, `forEachDialect` test harness | exists (schema pending) | | `packages/plugin-sdk` | plugin contract, `SourceRecord`, http/rate-limit helpers, fixture test harness | planned | @@ -23,6 +23,15 @@ user layer or files a Changes-inbox event. | `docs/design/mockups` | approved UI mockups (reference only) | exists | ## Data flow +HTTP request (`apps/server`): +1. `Bun.serve` routes `/api/*` and `/ingest/*` to `app.fetch`; everything else gets the web app (HTML import). +2. `apiHandler` matches method + path in the route table, applies the per-IP rate limit (login/register), authenticates + (`session` cookie `stt_session`, Bearer app token with a scope, or either), checks `X-Requested-With: sttracker` on + cookie-authenticated writes, reads the body (JSON ≤ 1 MB, multipart ≤ 50 MB) and validates it with zod. +3. The handler calls one core function with the caller's `userId` and actor `"user"`. Core enforces ownership + (`NotFoundError` → 404). Errors become `{ error: { code, message } }`; unknown errors are logged and return 500 `internal`. +4. `POST sources/:id/run|import` only enqueue a `sync` job (lockKey `sync:`, 409 when held); the worker runs it. + Sync lifecycle (`packages/core/src/sync/`; steps 1–2 are `jobs.ts` and `connections.ts`): 1. The scheduler enqueues a `sync` job (lockKey `sync:`) for each non-paused connection that is due. 2. The worker claims the job; `syncHandler` loads the connection and plugin and calls `runSync(deps, plugin, connectionId, mode)`, injecting diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md index c6ec997..2fb29c6 100644 --- a/docs/DECISIONS.md +++ b/docs/DECISIONS.md @@ -4,6 +4,33 @@ Append-only. Never edit or delete an entry. Supersede it with a new one that lin One entry per decision a future maintainer would otherwise have to reverse-engineer. No index: `grep -n '^## ' docs/DECISIONS.md` lists every entry (date, title, line number). +## 2026-10-30: HTTP API: auth modes, CSRF header, error mapping, actor +**Status:** accepted +**Context:** Task 16 (`apps/server`). +**Decision:** +- **Auth modes per route:** `none`, `session` (cookie only: tokens, sources create/edit/delete, device auth, bundle import, + admin), `token:`, `any:` (cookie or Bearer). A Bearer token valid for another scope gets 403, an unknown or + revoked one 401. App tokens cannot mint tokens or handle source credentials. +- **CSRF:** cookie-authenticated non-GET requests must send `X-Requested-With: sttracker` (403 otherwise). Browsers cannot + add it cross-site without a CORS preflight, which the server never grants. Bearer requests are exempt. Cookie is + `stt_session`, HttpOnly, SameSite=Lax, Secure when `PUBLIC_URL` is https, 30 days. +- **Error mapping:** zod and `ValidationError` → 400 `validation` (`field: reason`); `AuthError` → + 401 (`invalid_credentials`, `unauthenticated`), 403 (`forbidden`), 400 with its own code (`invite_required`, + `username_taken`, `invalid_username`); `NotFoundError` → 404 `not_found`; `ImportError` → 400 with its code (409 for + `account_not_empty`, `conflict`); unique violations → 409 `conflict`; body over the cap → 413 `validation`; + rate limit → 429 `rate_limited`; anything else → 500 `internal` with a fixed message, detail logged only. +- **Limits:** JSON bodies 1 MB, multipart 50 MB (counted while streaming, not trusted from Content-Length). + Login and register: 10 per minute per client IP, in process (resets on restart; not shared across instances). +- **Actor:** every API write uses actor `"user"`, including token-authenticated ones, because the change engine only + protects fields whose `field_sources` is `"user"`. A `token:` actor would let the next sync overwrite a script's edit. +- **Daily backup:** queued for the next 03:00 local with lockKey `"backup"` at boot, 5 s after each run (the finishing job + still holds the lock while it runs), and hourly as a safety net. All three are the same idempotent `enqueue`. +- **Deleted connections** (`paused` with no credentials) are hidden from `GET sources` and 404 on every `sources/:id` route. +- **File imports** travel in the job payload as base64 (`{ connectionId, file: { name, base64 } }`); `syncHandler` decodes + them into `runSync`'s `{ file }` mode, recorded as run mode `file`. +- **OpenAPI** 3.1 is generated from the route table (`z.toJSONSchema`, input side, unrepresentable → any): every route + with method, path, summary, security and request schema. Responses are not typed yet. + ## 2026-10-30: Job failure semantics and plugin reload limits **Status:** accepted **Context:** Task 9 fix round 1. diff --git a/docs/TODO.md b/docs/TODO.md index b237630..0f8ce4f 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -7,7 +7,14 @@ Small, specific, independently actionable. Remove only when actually done. - [ ] Choose a linter/formatter (e.g. Biome), then add `lint` to `package.json`, `AGENTS.md` and `docs/PERMISSIONS.md`. - [ ] Add CI on Gitea Actions: install, typecheck, and test on SQLite and Postgres. Add a guard that fails if `AGENTS.md` is over 150 lines. +- [ ] Agent permissions for Task 16 commands: add `bun run typecheck`, `bun apps/server/src/main.ts` and `bun --hot apps/server/src/main.ts` (all verified 2026-10-30) to `docs/PERMISSIONS.md`, `.claude/settings.json`, `.gemini/settings.json`, then fill Typecheck/Dev/Start in `AGENTS.md` Commands. Needs a human: agents may not edit their own permission files. + ## Next +- [ ] API: response schemas in the OpenAPI document (requests only today). +- [ ] API: login/register rate limit is per process and keyed on the socket IP; behind a reverse proxy every client shares one key. Add a trusted-proxy setting for `X-Forwarded-For`. +- [ ] API: file imports are base64 in the `jobs.payload` row (up to ~67 MB of text). Move uploads to `DATA_DIR/uploads/` and pass a path. +- [ ] API: `PATCH list-items/:id` only moves items; editing an item's note needs a core function. +- [ ] Connections: a user-paused connection that never had credentials looks deleted (`paused` + no credentials) and disappears from `GET sources`. Add an explicit `deleted_at`. - [ ] Phase 2 and Phase 3 plans (spec §12). - [ ] Generate image thumbnails for attachments (Phase 1 sizes images with CSS).