# Decisions Append-only. Never edit or delete an entry. Supersede it with a new one that links back. One entry per decision a future maintainer would otherwise have to reverse-engineer. No index: `grep -n '^## ' docs/DECISIONS.md` lists every entry (date, title, line number). ## 2026-10-30: Token actor in history, connection deleted_at, CSRF on every browser write **Status:** accepted. Supersedes the **Actor**, **CSRF** and **Deleted connections** bullets of "HTTP API: auth modes, CSRF header, error mapping, actor" below. **Context:** Task 16 fix round 1 (controller rulings). **Decision:** - **Actor vs. owner.** API writes pass actor `token:` for Bearer requests and `"user"` for sessions, so history shows which script changed what (spec §6.9). `field_sources` stores `fieldOwner(actor)`: every actor that does not start with `plugin:` is stored as `"user"`. A token edit is the user's edit, and syncs file a pending change instead of overwriting it. List ownership, which is read from history, applies the same mapping. - **CSRF.** `X-Requested-With: sttracker` is required on every non-GET request that has no Bearer token, including the `auth: "none"` routes login and register (login CSRF). JSON-body routes also require `Content-Type: application/json` (415 `unsupported_media_type` otherwise), so a cross-site simple form post cannot pass as JSON. - **Deleted connections** carry `source_connections.deleted_at` (migration 0003). `deleteConnection` sets it and still pauses the connection and wipes its credentials. Listing, lookups, updates, the scheduler and the sync handler ignore deleted rows. A user-paused connection without credentials is a normal, visible, resumable connection. - **Dev mode** (`Bun.serve({ development })`) is on only with `NODE_ENV=development` or `bun --hot` (detected through `process.execArgv`; `Bun.argv` does not carry runtime flags). Everything else runs in production mode. ## 2026-10-30: HTTP API: auth modes, CSRF header, error mapping, actor **Status:** accepted **Context:** Task 16 (`apps/server`). **Decision:** - **Auth modes per route:** `none`, `session` (cookie only: tokens, sources create/edit/delete, device auth, bundle import, admin), `token:`, `any:` (cookie or Bearer). A Bearer token valid for another scope gets 403, an unknown or revoked one 401. App tokens cannot mint tokens or handle source credentials. - **CSRF:** cookie-authenticated non-GET requests must send `X-Requested-With: sttracker` (403 otherwise). Browsers cannot add it cross-site without a CORS preflight, which the server never grants. Bearer requests are exempt. Cookie is `stt_session`, HttpOnly, SameSite=Lax, Secure when `PUBLIC_URL` is https, 30 days. - **Error mapping:** zod and `ValidationError` → 400 `validation` (`field: reason`); `AuthError` → 401 (`invalid_credentials`, `unauthenticated`), 403 (`forbidden`), 400 with its own code (`invite_required`, `username_taken`, `invalid_username`); `NotFoundError` → 404 `not_found`; `ImportError` → 400 with its code (409 for `account_not_empty`, `conflict`); unique violations → 409 `conflict`; body over the cap → 413 `validation`; rate limit → 429 `rate_limited`; anything else → 500 `internal` with a fixed message, detail logged only. - **Limits:** JSON bodies 1 MB, multipart 50 MB (counted while streaming, not trusted from Content-Length). Login and register: 10 per minute per client IP, in process (resets on restart; not shared across instances). - **Actor:** every API write uses actor `"user"`, including token-authenticated ones, because the change engine only protects fields whose `field_sources` is `"user"`. A `token:` actor would let the next sync overwrite a script's edit. - **Daily backup:** queued for the next 03:00 local with lockKey `"backup"` at boot, 5 s after each run (the finishing job still holds the lock while it runs), and hourly as a safety net. All three are the same idempotent `enqueue`. - **Deleted connections** (`paused` with no credentials) are hidden from `GET sources` and 404 on every `sources/:id` route. - **File imports** travel in the job payload as base64 (`{ connectionId, file: { name, base64 } }`); `syncHandler` decodes them into `runSync`'s `{ file }` mode, recorded as run mode `file`. - **OpenAPI** 3.1 is generated from the route table (`z.toJSONSchema`, input side, unrepresentable → any): every route with method, path, summary, security and request schema. Responses are not typed yet. ## 2026-10-30: Job failure semantics and plugin reload limits **Status:** accepted **Context:** Task 9 fix round 1. **Decision:** - **Failures before `runSync` never retry.** A missing/paused connection drops the job (`DropJobError`: `done`, message in `last_error`). A missing plugin or undecryptable credentials (`CryptoError`) insert a failed `source_runs` row, set the connection to `error`, advance `last_run_at` and throw `NonRetryableError` (job `failed`, no backoff). The scheduler still picks `error` connections, one interval later, so fixing the cause heals them. - **Stale locks.** Every worker tick re-queues `running` jobs whose `locked_at` is over an hour old (attempts unchanged), so a crash cannot hold a lockKey forever. Job bookkeeping errors are logged (`[jobs]`), never unhandled; `stop()` never rejects and aborts the `signal` passed to handlers (`handler(payload, { signal })`). - **Reload** evicts the entry file from `require.cache` and imports with `?v=` (Bun ignores the query alone for cached `.ts`). Only entry files reload: changes in a plugin's other modules need a server restart. - `credentials.save` and `updateConnection` read-merge-write in one transaction (row lock on Postgres). `ctx.log` is buffered and appended to the run's `source_runs.log` when the run ends. ## 2026-10-30: Jobs, plugin registry, connections: rate limit in the manifest, secret split, per-connection lock **Status:** accepted **Context:** Task 9 (`packages/core/src/{jobs,registry,connections}.ts`). **Decision:** - **Rate limit lives in the manifest** (`PluginManifest.rateLimit?: { perMinute, burst? }`), not in plugin code. The host builds `ctx.http` from it (default 60/min, burst 5, UA `STTracker/0.1 (+self-hosted)`), so every plugin is throttled the same way and the per-plugin `RATE_PER_MINUTE` constants are gone. - **Secrets are split off config.** Config fields typed `"secret"` in the manifest are stored only in the encrypted `credentials_enc` blob (with OAuth tokens a plugin saves), never in `config`. `makePluginContext` merges them back into `ctx.config`; `listConnections` returns plain config and `hasCredentials` only. `updateConnection` keeps secrets that are omitted and replaces plain config wholesale. `credentials.save` merges and persists immediately, so a token refreshed mid-pull survives a crash. - **Constructor deviation.** `createConnection`/`updateConnection` take the plugin `manifest` as a trailing argument (needed for the split), and `makePluginContext` is async (it decrypts) and takes the manifest before an optional `http`. - **One sync per connection.** Sync jobs carry `lockKey = "sync:"`; `enqueue` returns null while a `queued|running` job holds the key. On Postgres the check and insert run under `pg_advisory_xact_lock(hashtext(key))`. Claiming is `update … where status = 'queued'` checked on affected rows. Retries back off `2^attempts` minutes, 5 attempts, then `failed`. - **Mode choice.** The sync handler runs `full` when asked, when the plugin lacks `pullChanges`, or when no `ok` full run started within 7 days; else `changes`. A failed run is rethrown so the worker retries it. **Consequences:** Reload re-imports plugin modules with `?v=`; old module instances are not freed by Bun. ## 2026-10-30: Sync engine: hash composition, anti-flip-flop, no revive, root ancestor, dedupe **Status:** accepted **Context:** Task 8 built the change engine (`packages/core/src/sync/`). Spec §8.3 fixes the outcomes; these are the mechanics a maintainer would otherwise reverse-engineer. **Decision:** - **Hash composition.** sha256 of the stable-key-ordered JSON of `{kind, work, entry|engagement|listen|list, deleted, mergedIntoKey}`. `key` and `raw` are excluded; `deleted` is always a boolean and `mergedIntoKey` always present (null), so an omitted flag and an explicit `false` hash alike. A version row stores base64(`Bun.gzipSync`) of `{normalized, raw}`: the normalised record is what the next change is diffed against (field-level diff = previous version vs. incoming), the raw payload is kept for the record. - **Per-field decision.** Only fields that differ from the previous version are considered, and of those only values that differ from the current STTracker value. `field_sources[f]` unset or this connection's actor → applied; `"user"` → `pending changed {field, from: current, to: incoming}`; another source → `pending conflict {field, current, currentSource, incoming, incomingSource}`. A new record landing on an entry that already exists goes through the same per-field decision, so a first sync never overwrites what the user or another source wrote. Unchanged values are never sent, so a plugin never takes ownership of a field it did not change. - **Anti-flip-flop and superseding.** A pending `changed`/`conflict` for the same (record, field, incoming value) is never filed twice. A different newer value for the same (record, field) resolves the older pending event as `ignored` with `payload.supersededBy = `, so at most one is pending per field. If the source comes round to the value STTracker already holds, its pending events for that field are resolved `ignored` with `payload.reason = "source_agrees"`. `keep_mine` re-stamps `field_sources[f] = "user"`, so later values from any source become a user-decided `changed`, never another source-vs-source conflict. - **No revive.** If the mapped entry/engagement is soft-deleted, the record is still snapshotted but nothing is applied and no event is filed; a new record on a work whose entry the user deleted adds nothing. Only the user revives. - **Root ancestor.** Creating an entry on a child work also creates a `status "none"` entry (plugin actor, empty `field_sources`) on the root ancestor work, unless the user has one, live or deleted. - **Mapped records are never re-matched.** Only new records go through `matchWork`. Its result is committed at once as the record's row (`current_hash ""`, `target_entity "work"`), so a failed apply is retried against the same work and an ID-less hint never leaks a second draft. A changed work hint refreshes the mapped work via `updateWorkFromHint` (shared works only). `moveEntryToWork` re-points `source_records` from a merged-away entry to the survivor in the same transaction, so a user's merge never orphans a source. - **Engagement dedupe.** Among live engagements on the entry not already sourced by this connection: finish dates match when both are datetimes ≤ 24 h apart as instants, or when both are day precision and their dates (first 10 chars, computed with `Date.UTC`, no timezone shift) are ≤ 1 day apart; or start datetimes are the same instant. A match attaches an `engagement_sources` row and fills fields that are empty and that no source has set. Every other differing field goes through the per-field decision (so a disagreeing rating is a `conflict`), except the date that identified the play, whose difference is within the tolerance by definition. - **Removal.** Absence counts only after a `full` pull whose iteration finished and whose records all applied without error; any record error fails the run and skips removal. A removed record that comes back clears `deleted_at` and retires its still-pending `removed` event as `ignored`. A merged-away record (`merged_into_key` set, or a pending `merged` event) is expected to vanish: no removal, ever. Accepting `removed` on an engagement other sources still report only deletes this connection's `engagement_sources` row; the engagement is soft-deleted when no source remains. Run counts include only records whose transaction committed. - **Listens skip snapshots.** Listen records are not stored as `source_records` (they are high-volume and append-only). `listens` insert-or-ignore is the idempotence; they are never removed. Per batch of 500: one lookup of existing keys, each distinct recording resolved once, one multi-row insert. - **Lists.** A new `list` record creates a user list (kind `list`, description `Imported from `), items matched outside the transaction (an ID-less item first reuses the user's own private work with the same media type and title). Lists have no `field_sources`; ownership is derived from `user_history`: `name`/`description` belong to whoever last changed them, `items` to the user once any user edit exists. A user-owned field becomes a pending `changed` (for items, the whole incoming item set); otherwise the fields are applied and items replaced (`replaceListItems`, one history row). - **Merge accept.** The target work is the merge target record's mapped entry's work, else `matchWork` on its latest hint. Admin-merge events (`toWorkId`, no connection) resolve through the same `resolveChange`. - **Gone targets.** Accept/keep_mine on an event whose target is deleted (or merged away with no live survivor, found through the user's work alias) resolves it `ignored` with `payload.reason = "target_gone"`. **Alternatives:** - Diffing against the live entry instead of the previous version would re-file decided conflicts each run. - Hashing `raw` would turn every cosmetic API change into a new version. - Keeping one run "ok" despite failing records risks marking those records removed. **Consequences:** Concurrent runs of one connection are not locked here (Task 9's worker; `docs/TODO.md`). ## 2026-10-30: Export format `sttracker-export/v1`: per-user zip, camelCase JSONL, deleted rows included, no secrets **Status:** accepted **Context:** Task 15 (`packages/core/src/{export,backup}.ts`) fixed the on-disk bundle format that backups and restore share. **Decision:** - **Files.** A zip (`fflate`) holding `manifest.json` (`{ format, exportedAt, user: { username, displayName } }`), `works.jsonl`, `entries.jsonl`, `engagements.jsonl`, `listens.jsonl`, `lists.jsonl` (items nested), `copies.jsonl`, `custom_fields.jsonl` (field definitions), `history.jsonl`, `attachments.jsonl` (links nested) and `attachments/` bytes. One JSON object per line, `\n` separated. - **Keys.** camelCase; JSON columns are parsed into real JSON; `favorite` is a boolean. `user_id` columns are omitted (the importing user owns everything). `works.jsonl` adds `private` and `externalIds`. - **Deleted rows are included** with their `deletedAt`, and `fieldSources`/timestamps are kept, so a restore is lossless. - **No secrets.** No users row beyond username/display name, no sessions, tokens, connections, credentials or source snapshots. - **Merges and aliases.** `works.jsonl` rows carry `mergedIntoId` (the survivor and ancestors are exported too) and `work_aliases.jsonl` holds the user's `{ fromWorkId, toWorkId }` rows. Both are additive to v1. Restore re-creates tombstones (for works it creates) and aliases, so a later write to the old work still resolves to the moved entry. - **Export is one snapshot:** all reads run in a single transaction (repeatable read on Postgres). - **Restore** (`importUserBundle`) needs an account with no entries (`account_not_empty`) and `format === "sttracker-export/v1"` (`unsupported_format`). Works with external IDs are re-matched (else created shared); works without become private. Ids are remapped (UUIDs inside history `before`/`after` and custom-field value keys too). Rows are inserted directly, so no new history is written. The import validates first (attachment mime/size/sha256, rating 0-100, status, visibility, 2 GB uncompressed cap) and writes all rows, attachments included, in one transaction; errors are `invalid_attachment`, `corrupt_bundle`, `bundle_too_large`, `conflict`. Works merged away at export time are exported as the tombstone row without `mergedIntoId`. - **Backups** write `/backups/'-'>/.zip` per user plus `sttracker.db` (`VACUUM INTO`, SQLite only), are built in `.tmp` then renamed (modes 0700/0600), and prune to the newest `keep` (default 14). **Consequences:** Any change to these files after release needs `v2` and a migration path for old bundles. ## 2026-10-30: User layer: per-user moves merge entries, keep aliases, and roll the library up by hierarchy **Status:** accepted **Context:** Task 6 built the user layer (`packages/core/src/{entries,engagements,history,lists,copies}.ts`). Several behaviours are not obvious from the spec and would otherwise have to be reverse-engineered. **Decision:** - **Merge on move.** `moveEntryToWork` onto a work where the user already has an entry merges the two: - Live target: the target's values win. Empty target fields are filled from the moved entry, carrying the moved entry's `field_sources`. Tags are unioned and custom fields and overrides merged; their `field_sources` is `"user"` if either side's was, otherwise the target's. - Differing non-empty `review`/`private_notes` are concatenated (target, `\n\n---\n\n`, moved). - A conflicting rating keeps the target's; the moved rating is recorded in the history row's `before`. - Engagements are re-parented, each with an `entryId` history row (restorable), and the moved entry is soft-deleted. - Soft-deleted target: revived with the moved entry's values and `field_sources` as-is. - **Alias bookkeeping.** A move writes `work_aliases (user, old → target)`, re-points aliases that led to the old work, drops any alias leaving the target (no cycles), tombstones the user's own private draft, and re-points this user's copies (with history) and list items. Writes to an aliased or merged work (`upsertEntry`, copies, list items) resolve to the target. **Undoing a move** via `restoreVersion` deletes the forward alias and writes no reverse alias, so later syncs onto the target land on the target. - **Library cursor.** `listEntries` cursors are base64 JSON `{ sort, sortValue, id }`. The sort is carried so a cursor cannot be replayed under another sort. For the title sort, `sortValue` is the SQL `lower(w.title)` of the last row, never a JS lowercase, so SQL and cursor agree on non-ASCII titles. - **Custom fields patches replace the whole map** (`customFields` in an entry, engagement or copy patch). This makes restore exact. `field_sources` tracks the key `customFields` as a whole. Restore does not re-check values against today's definitions, because it is the user's own past data. - **List items are hard-deleted.** `list_items` has no `deleted_at`. The list's `user_history` snapshot holds the ordered items, so `restoreVersion` brings removed items back. - **Hierarchy visibility.** `childSummary` counts the user's live entries up to three levels below the entry's work (child, grandchild, great-grandchild). The default library view (`parentId: null`) lists entries with no ancestor, up to three levels, on which the user has a live entry. An episode whose show is untracked therefore stays visible, and is rolled up once the show gets an entry. Drilling in (`parentId: X`) applies the same rule relative to X: descendants up to three levels with no live-entry work strictly between them and X. `childSummary` still counts every descendant entry. - **Concurrent creates.** `upsertEntry` inserts with `ON CONFLICT (user_id, work_id) DO NOTHING`, then patches through the update path, so a concurrent creator never aborts the caller's transaction. **Alternatives:** - Refusing a move onto an existing entry pushes the merge onto the user. - Recursive CTEs for hierarchy are portable but heavier; three levels cover show → season → episode. - A JS-lowercased cursor breaks on SQLite, whose `lower()` is ASCII-only. **Consequences:** Undoing a move does not move copies or list items back, and hierarchy deeper than three levels is not rolled up (both in `docs/TODO.md`). ## 2026-10-30: A single Kysely dialect over Bun.SQL for SQLite and Postgres **Status:** accepted **Context:** Kysely ships no driver for Bun. `Bun.SQL` speaks both `sqlite://` and `postgres://` behind one API. **Decision:** - `packages/db/src/dialect.ts` is one Kysely `Dialect` whose driver wraps `new SQL(url)`, choosing Kysely's SQLite or Postgres adapter, compiler and introspector by URL scheme. - Postgres connections come from `sql.reserve()`. Bun's SQLite adapter does not support `reserve()`, so SQLite uses the single `SQL` instance as its only connection, serialised by a mutex so transactions never interleave. - Transactions are plain `begin`/`commit`/`rollback` on the held connection. **Alternatives:** - Separate dialects over `bun:sqlite` and `Bun.sql`: two drivers to maintain for the same job. - Community Bun dialect packages: an extra dependency, and none covers both databases. **Consequences:** SQLite is effectively single-connection (fine for a self-hosted app). Savepoints and real streaming are not implemented; `streamQuery` returns the whole result as one chunk. ## 2026-10-30: JSON columns are stored as text on both dialects **Status:** accepted **Context:** Spec §11 calls for `jsonb`. SQLite has no JSON type, so `jsonb` on Postgres would need per-dialect read/write code in every service that touches a JSON column. **Decision:** JSON columns are `text` everywhere, written with `toJson()` and read with `fromJson()` from `@sttracker/db/json`. `fromJson` returns a caller-supplied fallback for null or invalid text. **Alternatives:** `jsonb` on Postgres with per-dialect read/write code. **Consequences:** No JSON operators or indexes in SQL; filtering on JSON content happens in app code. Can be superseded later by a migration to `jsonb` if a query needs it. ## 2026-10-30: UUIDv7 instead of ULID for IDs **Status:** accepted **Context:** Spec and `AGENTS.md` say IDs are ULIDs generated in app code. Bun has `Bun.randomUUIDv7()` built in. **Decision:** `newId()` in `@sttracker/db/ids` returns `Bun.randomUUIDv7()`. IDs are stored as `text`. **Alternatives:** ULID, which needs a dependency or hand-written encoder. **Consequences:** IDs are equally time-sortable as strings with no extra dependency. They are 36 characters instead of 26. Where `AGENTS.md` or the spec say "ULID", read "UUIDv7". ## 2026-10-30: Product design decisions recorded in spec 001 **Status:** accepted **Context:** Brainstorming with the owner produced the v1 design. **Decision:** See `docs/specs/001-sttracker-v1-design.md` §13 for the designer's calls. Owner decisions: - Phase 1 is read-only. - Write-back is opt-in per connection. - Conflicts between sources go to the Changes inbox. - Backloggd waits for its official export. - The app UI is Hardcover-style; the public page is scrapbook + river. **Alternatives:** Recorded in the spec. **Consequences:** Phase 1 has no `push` implementations. ## 2026-10-30: Bun workspaces monorepo with Kysely for both databases **Status:** accepted **Context:** The owner requires Bun with SQLite and Postgres options. **Decision:** - Bun workspaces (`apps/*`, `packages/*`, `plugins/*`). - Kysely is the single query layer, with portable migrations. - SQLite runs through `bun:sqlite` and Postgres through `Bun.sql`. **Alternatives:** - Drizzle needs a separate schema per dialect. - Prisma does not fit Bun-native drivers. - Raw SQL would duplicate every query per dialect. **Consequences:** We may need a small custom Kysely dialect for `Bun.sql` or `bun:sqlite`. Every DB test runs on both databases. ## 2026-10-30: Adopt AGENTS.md as single source of truth **Status:** accepted **Context:** Agents had no durable project context and no consistent workflow between sessions. **Decision:** - `AGENTS.md` is canonical; `CLAUDE.md` and `GEMINI.md` are pointers. - Specs are permanent in `docs/specs/`. - Plans live in `docs/plans/` only while work is in progress and are removed before the merge request. - Both locations override the superpowers default paths. **Alternatives:** - Per-tool rule files: they drift and create synchronisation debt. - No rules file: agents re-explore the repo every session, which is measurably slower and costs more tokens. **Consequences:** Rule changes happen in one file, and the pointer files must stay thin. Plans are working artefacts: kept after merge, they go stale and contradict the code, so the spec is the record. ## 2026-10-30: Catalogue matching is race-safe through unique keys plus whole-transaction retry **Status:** accepted **Context:** Two syncs matching the same new work at once both miss the lookup and both insert. On Postgres the loser hit the `external_ids (namespace, value)` unique key and crashed the sync; seasons and episodes had no key at all and were duplicated. **Decision:** - Migration `0002_works_position_unique` adds a unique index on `works (parent_id, position)`. NULLs are distinct on both dialects, so top-level works and position-less children are unaffected. - `matchWork` re-runs the whole resolve in a fresh transaction on a unique violation (SQLite "UNIQUE constraint failed", Postgres SQLSTATE 23505), up to 3 attempts. Postgres aborts the transaction on the error, so a statement-level retry is not possible. - Adding missing external IDs to an existing work uses `ON CONFLICT DO NOTHING`. - An ID match only counts if the work is shared or owned by the matching user, and private works can never carry external IDs (`createWork` refuses; global merges accept only shared works). - A global merge repoints children of `from` to `to` unless `to` already has a child at that position. **Alternatives:** - Advisory or table locks: dialect-specific, and SQLite already serialises writes. - Upserts throughout `createWork`: would hide genuine conflicts and need per-dialect `RETURNING` handling. **Consequences:** Parents resolved without IDs or position (title/year reuse) still have no unique key and can be duplicated under concurrency; tracked in `docs/TODO.md`.