GET log (engagements and listens, keyset over at/kind/id), listChanges title and intoTitle, listCopies workTitle, all via batched lookups. Co-Authored-By: Claude Opus 5.5 <[email protected]>
371 lines
30 KiB
Markdown
371 lines
30 KiB
Markdown
# Decisions
|
||
|
||
Append-only. Never edit or delete an entry. Supersede it with a new one that links back.
|
||
One entry per decision a future maintainer would otherwise have to reverse-engineer.
|
||
No index: `grep -n '^## ' docs/DECISIONS.md` lists every entry (date, title, line number).
|
||
|
||
## 2026-10-30: Log feed, titles on Changes, work titles on copies (Task 18 API additions)
|
||
**Status:** accepted
|
||
**Context:** Task 18's Log and Changes screens had nothing to read from (controller ruling).
|
||
**Decision:**
|
||
- **`GET log?cursor=&limit=`** (any:read) → `{ items, nextCursor }` from `packages/core/src/log.ts` `listLog`: the user's
|
||
live engagements on live entries plus their listens, newest first. Each item's `at` is a listen's `listened_at`, or an
|
||
engagement's finish date, else start date, else `created_at`. Partial dates sort as strings, never through `Date`.
|
||
Keyset cursor over (`at`, `kind`, `id`) all descending, base64url JSON; default 50, max 200. Two queries (one per
|
||
kind, each `limit + 1` past the cursor) are merged in code, so no dialect-specific UNION is needed.
|
||
- **`listChanges` items carry `title`** (an entry's title with the user's override; an engagement resolves to its entry; a
|
||
list to its name; null when gone) **and, on merged events, `intoTitle`**: the title of the entry that the same
|
||
connection's `intoKey` record maps to, else null. Titles come from a fixed number of batched queries
|
||
(`packages/core/src/titles.ts`), not one per event.
|
||
- **`listCopies` items carry `workTitle`** (override-aware, null while unmatched) so Owned needs no per-row work lookups.
|
||
The export bundle is unchanged.
|
||
|
||
## 2026-10-30: Web app (Task 17): preferences, first run, optimistic edits, AA on orange, self-hosted fonts
|
||
**Status:** accepted
|
||
**Context:** Task 17 and its fix round 1 (controller rulings).
|
||
**Decision:**
|
||
- **Three small API routes for the web app.** `GET auth/status` (no auth) → `{ hasUsers }`: `/login` redirects a fresh
|
||
instance to `/register`, which shows "Create the admin account", or invite mode when users exist (also on
|
||
`invite_required`, and with `?invite=` prefilled). `GET attachments?entity=&entityId=` lists an owned record's
|
||
attachments with their links (404 when the record is not the caller's). `GET/PATCH me/preferences` →
|
||
`{ ratingScale: "5half"|"5"|"10"|"100", defaultVisibility }`, stored in `users.preferences` under the same keys
|
||
`defaultVisibility()` already reads (`packages/core/src/preferences.ts`); unknown stored keys are kept.
|
||
- **Rating scale** comes from preferences (default five stars with halves). Stored ratings stay integers 0–100; the web
|
||
formats them with `formatRating(value, scale)`.
|
||
- **Entry edits are optimistic and serial.** `useEntryPatch` updates the cached entry in `onMutate` (rollback on error),
|
||
runs one entry's PATCHes in order through a TanStack mutation `scope`, and `edit(cur => patch)` builds from the latest
|
||
cache, so two quick tag adds keep both. Only the last queued edit writes the server's answer back.
|
||
- **Keyboard rating saves once:** arrow steps update locally and save 500 ms after the last key, on blur, or on unmount,
|
||
so stepping does not write a history row per keypress.
|
||
- **Errors in the product's voice:** every error display goes through `friendlyError()` (`apps/web/src/format.ts`), which maps
|
||
`validation` field/reason pairs and 413 to plain copy; other API errors keep the server's message.
|
||
- **AA contrast on the approved palette:** dark text on `--warn`, `--text-faint` group labels, `--text-muted` placeholders on
|
||
`--surface-3` (recorded in DESIGN.md).
|
||
- **Fonts are vendored** (Inter woff2 from fontsource; Gelasio as the official variable TTFs from github.com/google/fonts
|
||
`ofl/gelasio`, unmodified, because no woff2 tooling was available offline) with their OFL texts.
|
||
- **The web app mirrors wire types** in `apps/web/src/api.ts` instead of importing `@sttracker/core` (apps/web has no
|
||
workspace dependency on it, and the browser bundle must not pull server code).
|
||
|
||
## 2026-10-30: Token actor in history, connection deleted_at, CSRF on every browser write
|
||
**Status:** accepted. Supersedes the **Actor**, **CSRF** and **Deleted connections** bullets of
|
||
"HTTP API: auth modes, CSRF header, error mapping, actor" below.
|
||
**Context:** Task 16 fix round 1 (controller rulings).
|
||
**Decision:**
|
||
- **Actor vs. owner.** API writes pass actor `token:<name>` for Bearer requests and `"user"` for sessions, so history
|
||
shows which script changed what (spec §6.9). `field_sources` stores `fieldOwner(actor)`: every actor that does not start
|
||
with `plugin:` is stored as `"user"`. A token edit is the user's edit, and syncs file a pending change instead of
|
||
overwriting it. List ownership, which is read from history, applies the same mapping.
|
||
- **CSRF.** `X-Requested-With: sttracker` is required on every non-GET request that has no Bearer token, including the
|
||
`auth: "none"` routes login and register (login CSRF). JSON-body routes also require `Content-Type: application/json`
|
||
(415 `unsupported_media_type` otherwise), so a cross-site simple form post cannot pass as JSON.
|
||
- **Deleted connections** carry `source_connections.deleted_at` (migration 0003). `deleteConnection` sets it and still pauses
|
||
the connection and wipes its credentials. Listing, lookups, updates, the scheduler and the sync handler ignore deleted rows.
|
||
A user-paused connection without credentials is a normal, visible, resumable connection.
|
||
- **Dev mode** (`Bun.serve({ development })`) is on only with `NODE_ENV=development` or `bun --hot` (detected through
|
||
`process.execArgv`; `Bun.argv` does not carry runtime flags). Everything else runs in production mode.
|
||
|
||
## 2026-10-30: HTTP API: auth modes, CSRF header, error mapping, actor
|
||
**Status:** accepted
|
||
**Context:** Task 16 (`apps/server`).
|
||
**Decision:**
|
||
- **Auth modes per route:** `none`, `session` (cookie only: tokens, sources create/edit/delete, device auth, bundle import,
|
||
admin), `token:<scope>`, `any:<scope>` (cookie or Bearer). A Bearer token valid for another scope gets 403, an unknown or
|
||
revoked one 401. App tokens cannot mint tokens or handle source credentials.
|
||
- **CSRF:** cookie-authenticated non-GET requests must send `X-Requested-With: sttracker` (403 otherwise). Browsers cannot
|
||
add it cross-site without a CORS preflight, which the server never grants. Bearer requests are exempt. Cookie is
|
||
`stt_session`, HttpOnly, SameSite=Lax, Secure when `PUBLIC_URL` is https, 30 days.
|
||
- **Error mapping:** zod and `ValidationError` → 400 `validation` (`field: reason`); `AuthError` →
|
||
401 (`invalid_credentials`, `unauthenticated`), 403 (`forbidden`), 400 with its own code (`invite_required`,
|
||
`username_taken`, `invalid_username`); `NotFoundError` → 404 `not_found`; `ImportError` → 400 with its code (409 for
|
||
`account_not_empty`, `conflict`); unique violations → 409 `conflict`; body over the cap → 413 `validation`;
|
||
rate limit → 429 `rate_limited`; anything else → 500 `internal` with a fixed message, detail logged only.
|
||
- **Limits:** JSON bodies 1 MB, multipart 50 MB (counted while streaming, not trusted from Content-Length).
|
||
Login and register: 10 per minute per client IP, in process (resets on restart; not shared across instances).
|
||
- **Actor:** every API write uses actor `"user"`, including token-authenticated ones, because the change engine only
|
||
protects fields whose `field_sources` is `"user"`. A `token:<name>` actor would let the next sync overwrite a script's edit.
|
||
- **Daily backup:** queued for the next 03:00 local with lockKey `"backup"` at boot, 5 s after each run (the finishing job
|
||
still holds the lock while it runs), and hourly as a safety net. All three are the same idempotent `enqueue`.
|
||
- **Deleted connections** (`paused` with no credentials) are hidden from `GET sources` and 404 on every `sources/:id` route.
|
||
- **File imports** travel in the job payload as base64 (`{ connectionId, file: { name, base64 } }`); `syncHandler` decodes
|
||
them into `runSync`'s `{ file }` mode, recorded as run mode `file`.
|
||
- **OpenAPI** 3.1 is generated from the route table (`z.toJSONSchema`, input side, unrepresentable → any): every route
|
||
with method, path, summary, security and request schema. Responses are not typed yet.
|
||
|
||
## 2026-10-30: Job failure semantics and plugin reload limits
|
||
**Status:** accepted
|
||
**Context:** Task 9 fix round 1.
|
||
**Decision:**
|
||
- **Failures before `runSync` never retry.** A missing/paused connection drops the job (`DropJobError`: `done`, message in
|
||
`last_error`). A missing plugin or undecryptable credentials (`CryptoError`) insert a failed `source_runs` row, set the
|
||
connection to `error`, advance `last_run_at` and throw `NonRetryableError` (job `failed`, no backoff). The scheduler still
|
||
picks `error` connections, one interval later, so fixing the cause heals them.
|
||
- **Stale locks.** Every worker tick re-queues `running` jobs whose `locked_at` is over an hour old (attempts unchanged), so a
|
||
crash cannot hold a lockKey forever. Job bookkeeping errors are logged (`[jobs]`), never unhandled; `stop()` never rejects
|
||
and aborts the `signal` passed to handlers (`handler(payload, { signal })`).
|
||
- **Reload** evicts the entry file from `require.cache` and imports with `?v=<Date.now()>` (Bun ignores the query alone for
|
||
cached `.ts`). Only entry files reload: changes in a plugin's other modules need a server restart.
|
||
- `credentials.save` and `updateConnection` read-merge-write in one transaction (row lock on Postgres). `ctx.log` is buffered and
|
||
appended to the run's `source_runs.log` when the run ends.
|
||
|
||
## 2026-10-30: Jobs, plugin registry, connections: rate limit in the manifest, secret split, per-connection lock
|
||
**Status:** accepted
|
||
**Context:** Task 9 (`packages/core/src/{jobs,registry,connections}.ts`).
|
||
**Decision:**
|
||
- **Rate limit lives in the manifest** (`PluginManifest.rateLimit?: { perMinute, burst? }`), not in plugin code. The host builds
|
||
`ctx.http` from it (default 60/min, burst 5, UA `STTracker/0.1 (+self-hosted)`), so every plugin is throttled the same way
|
||
and the per-plugin `RATE_PER_MINUTE` constants are gone.
|
||
- **Secrets are split off config.** Config fields typed `"secret"` in the manifest are stored only in the encrypted
|
||
`credentials_enc` blob (with OAuth tokens a plugin saves), never in `config`. `makePluginContext` merges them back into
|
||
`ctx.config`; `listConnections` returns plain config and `hasCredentials` only. `updateConnection` keeps secrets that are
|
||
omitted and replaces plain config wholesale. `credentials.save` merges and persists immediately, so a token refreshed
|
||
mid-pull survives a crash.
|
||
- **Constructor deviation.** `createConnection`/`updateConnection` take the plugin `manifest` as a trailing argument (needed
|
||
for the split), and `makePluginContext` is async (it decrypts) and takes the manifest before an optional `http`.
|
||
- **One sync per connection.** Sync jobs carry `lockKey = "sync:<connectionId>"`; `enqueue` returns null while a
|
||
`queued|running` job holds the key. On Postgres the check and insert run under `pg_advisory_xact_lock(hashtext(key))`.
|
||
Claiming is `update … where status = 'queued'` checked on affected rows. Retries back off `2^attempts` minutes, 5 attempts, then `failed`.
|
||
- **Mode choice.** The sync handler runs `full` when asked, when the plugin lacks `pullChanges`, or when no `ok` full run started
|
||
within 7 days; else `changes`. A failed run is rethrown so the worker retries it.
|
||
**Consequences:** Reload re-imports plugin modules with `?v=<Date.now()>`; old module instances are not freed by Bun.
|
||
|
||
|
||
## 2026-10-30: Sync engine: hash composition, anti-flip-flop, no revive, root ancestor, dedupe
|
||
**Status:** accepted
|
||
**Context:** Task 8 built the change engine (`packages/core/src/sync/`). Spec §8.3 fixes the outcomes; these
|
||
are the mechanics a maintainer would otherwise reverse-engineer.
|
||
**Decision:**
|
||
- **Hash composition.** sha256 of the stable-key-ordered JSON of `{kind, work, entry|engagement|listen|list,
|
||
deleted, mergedIntoKey}`. `key` and `raw` are excluded; `deleted` is always a boolean and `mergedIntoKey`
|
||
always present (null), so an omitted flag and an explicit `false` hash alike. A version row stores
|
||
base64(`Bun.gzipSync`) of `{normalized, raw}`: the normalised record is what the next change is diffed
|
||
against (field-level diff = previous version vs. incoming), the raw payload is kept for the record.
|
||
- **Per-field decision.** Only fields that differ from the previous version are considered, and of those
|
||
only values that differ from the current STTracker value. `field_sources[f]` unset or this connection's
|
||
actor → applied; `"user"` → `pending changed {field, from: current, to: incoming}`; another source →
|
||
`pending conflict {field, current, currentSource, incoming, incomingSource}`. A new record landing on an
|
||
entry that already exists goes through the same per-field decision, so a first sync never overwrites
|
||
what the user or another source wrote. Unchanged values are never sent, so a plugin never takes
|
||
ownership of a field it did not change.
|
||
- **Anti-flip-flop and superseding.** A pending `changed`/`conflict` for the same (record, field, incoming
|
||
value) is never filed twice. A different newer value for the same (record, field) resolves the older
|
||
pending event as `ignored` with `payload.supersededBy = <new event id>`, so at most one is pending per
|
||
field. If the source comes round to the value STTracker already holds, its pending events for that field
|
||
are resolved `ignored` with `payload.reason = "source_agrees"`. `keep_mine` re-stamps
|
||
`field_sources[f] = "user"`, so later values from any source become a user-decided `changed`, never
|
||
another source-vs-source conflict.
|
||
- **No revive.** If the mapped entry/engagement is soft-deleted, the record is still snapshotted but nothing
|
||
is applied and no event is filed; a new record on a work whose entry the user deleted adds nothing.
|
||
Only the user revives.
|
||
- **Root ancestor.** Creating an entry on a child work also creates a `status "none"` entry (plugin actor,
|
||
empty `field_sources`) on the root ancestor work, unless the user has one, live or deleted.
|
||
- **Mapped records are never re-matched.** Only new records go through `matchWork`. Its result is committed
|
||
at once as the record's row (`current_hash ""`, `target_entity "work"`), so a failed apply is retried
|
||
against the same work and an ID-less hint never leaks a second draft. A changed work hint refreshes the
|
||
mapped work via `updateWorkFromHint` (shared works only). `moveEntryToWork` re-points `source_records`
|
||
from a merged-away entry to the survivor in the same transaction, so a user's merge never orphans a source.
|
||
- **Engagement dedupe.** Among live engagements on the entry not already sourced by this connection:
|
||
finish dates match when both are datetimes ≤ 24 h apart as instants, or when both are day precision and
|
||
their dates (first 10 chars, computed with `Date.UTC`, no timezone shift) are ≤ 1 day apart; or start
|
||
datetimes are the same instant. A match attaches an `engagement_sources` row and fills fields that are
|
||
empty and that no source has set. Every other differing field goes through the per-field decision
|
||
(so a disagreeing rating is a `conflict`), except the date that identified the play, whose difference is
|
||
within the tolerance by definition.
|
||
- **Removal.** Absence counts only after a `full` pull whose iteration finished and whose records all
|
||
applied without error; any record error fails the run and skips removal. A removed record that comes
|
||
back clears `deleted_at` and retires its still-pending `removed` event as `ignored`. A merged-away record
|
||
(`merged_into_key` set, or a pending `merged` event) is expected to vanish: no removal, ever. Accepting
|
||
`removed` on an engagement other sources still report only deletes this connection's
|
||
`engagement_sources` row; the engagement is soft-deleted when no source remains. Run counts include only
|
||
records whose transaction committed.
|
||
- **Listens skip snapshots.** Listen records are not stored as `source_records` (they are high-volume and
|
||
append-only). `listens` insert-or-ignore is the idempotence; they are never removed. Per batch of 500:
|
||
one lookup of existing keys, each distinct recording resolved once, one multi-row insert.
|
||
- **Lists.** A new `list` record creates a user list (kind `list`, description `Imported from <plugin id>`),
|
||
items matched outside the transaction (an ID-less item first reuses the user's own private work with the
|
||
same media type and title). Lists have no `field_sources`; ownership is derived from `user_history`:
|
||
`name`/`description` belong to whoever last changed them, `items` to the user once any user edit exists.
|
||
A user-owned field becomes a pending `changed` (for items, the whole incoming item set); otherwise the
|
||
fields are applied and items replaced (`replaceListItems`, one history row).
|
||
- **Merge accept.** The target work is the merge target record's mapped entry's work, else `matchWork` on
|
||
its latest hint. Admin-merge events (`toWorkId`, no connection) resolve through the same `resolveChange`.
|
||
- **Gone targets.** Accept/keep_mine on an event whose target is deleted (or merged away with no live
|
||
survivor, found through the user's work alias) resolves it `ignored` with `payload.reason = "target_gone"`.
|
||
|
||
**Alternatives:**
|
||
- Diffing against the live entry instead of the previous version would re-file decided conflicts each run.
|
||
- Hashing `raw` would turn every cosmetic API change into a new version.
|
||
- Keeping one run "ok" despite failing records risks marking those records removed.
|
||
|
||
**Consequences:** Concurrent runs of one connection are not locked here (Task 9's worker; `docs/TODO.md`).
|
||
|
||
## 2026-10-30: Export format `sttracker-export/v1`: per-user zip, camelCase JSONL, deleted rows included, no secrets
|
||
**Status:** accepted
|
||
**Context:** Task 15 (`packages/core/src/{export,backup}.ts`) fixed the on-disk bundle format that backups and restore share.
|
||
**Decision:**
|
||
- **Files.** A zip (`fflate`) holding `manifest.json` (`{ format, exportedAt, user: { username, displayName } }`),
|
||
`works.jsonl`, `entries.jsonl`, `engagements.jsonl`, `listens.jsonl`, `lists.jsonl` (items nested),
|
||
`copies.jsonl`, `custom_fields.jsonl` (field definitions), `history.jsonl`, `attachments.jsonl` (links nested)
|
||
and `attachments/<sha256>` bytes. One JSON object per line, `\n` separated.
|
||
- **Keys.** camelCase; JSON columns are parsed into real JSON; `favorite` is a boolean. `user_id` columns are omitted
|
||
(the importing user owns everything). `works.jsonl` adds `private` and `externalIds`.
|
||
- **Deleted rows are included** with their `deletedAt`, and `fieldSources`/timestamps are kept, so a restore is lossless.
|
||
- **No secrets.** No users row beyond username/display name, no sessions, tokens, connections, credentials or source snapshots.
|
||
- **Merges and aliases.** `works.jsonl` rows carry `mergedIntoId` (the survivor and ancestors are exported too) and
|
||
`work_aliases.jsonl` holds the user's `{ fromWorkId, toWorkId }` rows. Both are additive to v1. Restore re-creates
|
||
tombstones (for works it creates) and aliases, so a later write to the old work still resolves to the moved entry.
|
||
- **Export is one snapshot:** all reads run in a single transaction (repeatable read on Postgres).
|
||
- **Restore** (`importUserBundle`) needs an account with no entries (`account_not_empty`) and `format === "sttracker-export/v1"`
|
||
(`unsupported_format`). Works with external IDs are re-matched (else created shared); works without become private.
|
||
Ids are remapped (UUIDs inside history `before`/`after` and custom-field value keys too). Rows are inserted directly, so no
|
||
new history is written. The import validates first (attachment mime/size/sha256, rating 0-100, status, visibility, 2 GB
|
||
uncompressed cap) and writes all rows, attachments included, in one transaction; errors are
|
||
`invalid_attachment`, `corrupt_bundle`, `bundle_too_large`, `conflict`. Works merged away at export time are exported as the tombstone row without `mergedIntoId`.
|
||
- **Backups** write `<dataDir>/backups/<ISO with ':'->'-'>/<username>.zip` per user plus `sttracker.db` (`VACUUM INTO`, SQLite only),
|
||
are built in `<stamp>.tmp` then renamed (modes 0700/0600), and prune to the newest `keep` (default 14).
|
||
**Consequences:** Any change to these files after release needs `v2` and a migration path for old bundles.
|
||
|
||
## 2026-10-30: User layer: per-user moves merge entries, keep aliases, and roll the library up by hierarchy
|
||
**Status:** accepted
|
||
**Context:** Task 6 built the user layer (`packages/core/src/{entries,engagements,history,lists,copies}.ts`).
|
||
Several behaviours are not obvious from the spec and would otherwise have to be reverse-engineered.
|
||
**Decision:**
|
||
- **Merge on move.** `moveEntryToWork` onto a work where the user already has an entry merges the two:
|
||
- Live target: the target's values win. Empty target fields are filled from the moved entry, carrying
|
||
the moved entry's `field_sources`. Tags are unioned and custom fields and overrides merged; their
|
||
`field_sources` is `"user"` if either side's was, otherwise the target's.
|
||
- Differing non-empty `review`/`private_notes` are concatenated (target, `\n\n---\n\n`, moved).
|
||
- A conflicting rating keeps the target's; the moved rating is recorded in the history row's `before`.
|
||
- Engagements are re-parented, each with an `entryId` history row (restorable), and the moved entry is
|
||
soft-deleted.
|
||
- Soft-deleted target: revived with the moved entry's values and `field_sources` as-is.
|
||
- **Alias bookkeeping.** A move writes `work_aliases (user, old → target)`, re-points aliases that led to
|
||
the old work, drops any alias leaving the target (no cycles), tombstones the user's own private draft,
|
||
and re-points this user's copies (with history) and list items. Writes to an aliased or merged work
|
||
(`upsertEntry`, copies, list items) resolve to the target. **Undoing a move** via `restoreVersion` deletes
|
||
the forward alias and writes no reverse alias, so later syncs onto the target land on the target.
|
||
- **Library cursor.** `listEntries` cursors are base64 JSON `{ sort, sortValue, id }`. The sort is carried
|
||
so a cursor cannot be replayed under another sort. For the title sort, `sortValue` is the SQL
|
||
`lower(w.title)` of the last row, never a JS lowercase, so SQL and cursor agree on non-ASCII titles.
|
||
- **Custom fields patches replace the whole map** (`customFields` in an entry, engagement or copy patch).
|
||
This makes restore exact. `field_sources` tracks the key `customFields` as a whole. Restore does not
|
||
re-check values against today's definitions, because it is the user's own past data.
|
||
- **List items are hard-deleted.** `list_items` has no `deleted_at`. The list's `user_history` snapshot
|
||
holds the ordered items, so `restoreVersion` brings removed items back.
|
||
- **Hierarchy visibility.** `childSummary` counts the user's live entries up to three levels below the
|
||
entry's work (child, grandchild, great-grandchild). The default library view (`parentId: null`) lists
|
||
entries with no ancestor, up to three levels, on which the user has a live entry. An episode whose show
|
||
is untracked therefore stays visible, and is rolled up once the show gets an entry. Drilling in
|
||
(`parentId: X`) applies the same rule relative to X: descendants up to three levels with no live-entry
|
||
work strictly between them and X. `childSummary` still counts every descendant entry.
|
||
- **Concurrent creates.** `upsertEntry` inserts with `ON CONFLICT (user_id, work_id) DO NOTHING`, then
|
||
patches through the update path, so a concurrent creator never aborts the caller's transaction.
|
||
|
||
**Alternatives:**
|
||
- Refusing a move onto an existing entry pushes the merge onto the user.
|
||
- Recursive CTEs for hierarchy are portable but heavier; three levels cover show → season → episode.
|
||
- A JS-lowercased cursor breaks on SQLite, whose `lower()` is ASCII-only.
|
||
|
||
**Consequences:** Undoing a move does not move copies or list items back, and hierarchy deeper than three
|
||
levels is not rolled up (both in `docs/TODO.md`).
|
||
|
||
## 2026-10-30: A single Kysely dialect over Bun.SQL for SQLite and Postgres
|
||
**Status:** accepted
|
||
**Context:** Kysely ships no driver for Bun. `Bun.SQL` speaks both `sqlite://` and `postgres://` behind one API.
|
||
**Decision:**
|
||
- `packages/db/src/dialect.ts` is one Kysely `Dialect` whose driver wraps `new SQL(url)`, choosing Kysely's
|
||
SQLite or Postgres adapter, compiler and introspector by URL scheme.
|
||
- Postgres connections come from `sql.reserve()`. Bun's SQLite adapter does not support `reserve()`, so SQLite
|
||
uses the single `SQL` instance as its only connection, serialised by a mutex so transactions never interleave.
|
||
- Transactions are plain `begin`/`commit`/`rollback` on the held connection.
|
||
|
||
**Alternatives:**
|
||
- Separate dialects over `bun:sqlite` and `Bun.sql`: two drivers to maintain for the same job.
|
||
- Community Bun dialect packages: an extra dependency, and none covers both databases.
|
||
|
||
**Consequences:** SQLite is effectively single-connection (fine for a self-hosted app). Savepoints and real
|
||
streaming are not implemented; `streamQuery` returns the whole result as one chunk.
|
||
|
||
## 2026-10-30: JSON columns are stored as text on both dialects
|
||
**Status:** accepted
|
||
**Context:** Spec §11 calls for `jsonb`. SQLite has no JSON type, so `jsonb` on Postgres would need
|
||
per-dialect read/write code in every service that touches a JSON column.
|
||
**Decision:** JSON columns are `text` everywhere, written with `toJson()` and read with `fromJson()` from
|
||
`@sttracker/db/json`. `fromJson` returns a caller-supplied fallback for null or invalid text.
|
||
**Alternatives:** `jsonb` on Postgres with per-dialect read/write code.
|
||
**Consequences:** No JSON operators or indexes in SQL; filtering on JSON content happens in app code. Can be
|
||
superseded later by a migration to `jsonb` if a query needs it.
|
||
|
||
## 2026-10-30: UUIDv7 instead of ULID for IDs
|
||
**Status:** accepted
|
||
**Context:** Spec and `AGENTS.md` say IDs are ULIDs generated in app code. Bun has `Bun.randomUUIDv7()` built in.
|
||
**Decision:** `newId()` in `@sttracker/db/ids` returns `Bun.randomUUIDv7()`. IDs are stored as `text`.
|
||
**Alternatives:** ULID, which needs a dependency or hand-written encoder.
|
||
**Consequences:** IDs are equally time-sortable as strings with no extra dependency. They are 36 characters
|
||
instead of 26. Where `AGENTS.md` or the spec say "ULID", read "UUIDv7".
|
||
|
||
## 2026-10-30: Product design decisions recorded in spec 001
|
||
**Status:** accepted
|
||
**Context:** Brainstorming with the owner produced the v1 design.
|
||
**Decision:** See `docs/specs/001-sttracker-v1-design.md` §13 for the designer's calls. Owner decisions:
|
||
- Phase 1 is read-only.
|
||
- Write-back is opt-in per connection.
|
||
- Conflicts between sources go to the Changes inbox.
|
||
- Backloggd waits for its official export.
|
||
- The app UI is Hardcover-style; the public page is scrapbook + river.
|
||
**Alternatives:** Recorded in the spec.
|
||
**Consequences:** Phase 1 has no `push` implementations.
|
||
|
||
## 2026-10-30: Bun workspaces monorepo with Kysely for both databases
|
||
**Status:** accepted
|
||
**Context:** The owner requires Bun with SQLite and Postgres options.
|
||
**Decision:**
|
||
- Bun workspaces (`apps/*`, `packages/*`, `plugins/*`).
|
||
- Kysely is the single query layer, with portable migrations.
|
||
- SQLite runs through `bun:sqlite` and Postgres through `Bun.sql`.
|
||
|
||
**Alternatives:**
|
||
- Drizzle needs a separate schema per dialect.
|
||
- Prisma does not fit Bun-native drivers.
|
||
- Raw SQL would duplicate every query per dialect.
|
||
|
||
**Consequences:** We may need a small custom Kysely dialect for `Bun.sql` or `bun:sqlite`. Every DB test runs on both databases.
|
||
|
||
## 2026-10-30: Adopt AGENTS.md as single source of truth
|
||
**Status:** accepted
|
||
**Context:** Agents had no durable project context and no consistent workflow between sessions.
|
||
**Decision:**
|
||
- `AGENTS.md` is canonical; `CLAUDE.md` and `GEMINI.md` are pointers.
|
||
- Specs are permanent in `docs/specs/`.
|
||
- Plans live in `docs/plans/` only while work is in progress and are removed before the merge request.
|
||
- Both locations override the superpowers default paths.
|
||
|
||
**Alternatives:**
|
||
- Per-tool rule files: they drift and create synchronisation debt.
|
||
- No rules file: agents re-explore the repo every session, which is measurably slower and costs more tokens.
|
||
|
||
**Consequences:** Rule changes happen in one file, and the pointer files must stay thin. Plans are working
|
||
artefacts: kept after merge, they go stale and contradict the code, so the spec is the record.
|
||
|
||
## 2026-10-30: Catalogue matching is race-safe through unique keys plus whole-transaction retry
|
||
**Status:** accepted
|
||
**Context:** Two syncs matching the same new work at once both miss the lookup and both insert. On
|
||
Postgres the loser hit the `external_ids (namespace, value)` unique key and crashed the sync; seasons and
|
||
episodes had no key at all and were duplicated.
|
||
**Decision:**
|
||
- Migration `0002_works_position_unique` adds a unique index on `works (parent_id, position)`. NULLs are
|
||
distinct on both dialects, so top-level works and position-less children are unaffected.
|
||
- `matchWork` re-runs the whole resolve in a fresh transaction on a unique violation (SQLite
|
||
"UNIQUE constraint failed", Postgres SQLSTATE 23505), up to 3 attempts. Postgres aborts the transaction
|
||
on the error, so a statement-level retry is not possible.
|
||
- Adding missing external IDs to an existing work uses `ON CONFLICT DO NOTHING`.
|
||
- An ID match only counts if the work is shared or owned by the matching user, and private works can
|
||
never carry external IDs (`createWork` refuses; global merges accept only shared works).
|
||
- A global merge repoints children of `from` to `to` unless `to` already has a child at that position.
|
||
|
||
**Alternatives:**
|
||
- Advisory or table locks: dialect-specific, and SQLite already serialises writes.
|
||
- Upserts throughout `createWork`: would hide genuine conflicts and need per-dialect `RETURNING` handling.
|
||
|
||
**Consequences:** Parents resolved without IDs or position (title/year reuse) still have no unique key
|
||
and can be duplicated under concurrency; tracked in `docs/TODO.md`.
|