STTracker/docs/DECISIONS.md

Decisions

Append-only. Never edit or delete an entry. Supersede it with a new one that links back. One entry per decision a future maintainer would otherwise have to reverse-engineer. No index: grep -n '^## ' docs/DECISIONS.md lists every entry (date, title, line number).

2026-10-30: Export format sttracker-export/v1: per-user zip, camelCase JSONL, deleted rows included, no secrets

Status: accepted Context: Task 15 (packages/core/src/{export,backup}.ts) fixed the on-disk bundle format that backups and restore share. Decision:

• Files. A zip (fflate) holding manifest.json ({ format, exportedAt, user: { username, displayName } }), works.jsonl, entries.jsonl, engagements.jsonl, listens.jsonl, lists.jsonl (items nested), copies.jsonl, custom_fields.jsonl (field definitions), history.jsonl, attachments.jsonl (links nested) and attachments/<sha256> bytes. One JSON object per line, \n separated.
• Keys. camelCase; JSON columns are parsed into real JSON; favorite is a boolean. user_id columns are omitted (the importing user owns everything). works.jsonl adds private and externalIds.
• Deleted rows are included with their deletedAt, and fieldSources/timestamps are kept, so a restore is lossless.
• No secrets. No users row beyond username/display name, no sessions, tokens, connections, credentials or source snapshots.
• Merges and aliases. works.jsonl rows carry mergedIntoId (the survivor and ancestors are exported too) and work_aliases.jsonl holds the user's { fromWorkId, toWorkId } rows. Both are additive to v1. Restore re-creates tombstones (for works it creates) and aliases, so a later write to the old work still resolves to the moved entry.
• Export is one snapshot: all reads run in a single transaction (repeatable read on Postgres).
• Restore (importUserBundle) needs an account with no entries (account_not_empty) and format === "sttracker-export/v1" (unsupported_format). Works with external IDs are re-matched (else created shared); works without become private. Ids are remapped (UUIDs inside history before/after and custom-field value keys too). Rows are inserted directly, so no new history is written. The import validates first (attachment mime/size/sha256, rating 0-100, status, visibility, 2 GB uncompressed cap) and writes all rows, attachments included, in one transaction; errors are invalid_attachment, corrupt_bundle, bundle_too_large, conflict. Works merged away at export time are exported as the tombstone row without mergedIntoId.
• Backups write <dataDir>/backups/<ISO with ':'->'-'>/<username>.zip per user plus sttracker.db (VACUUM INTO, SQLite only), are built in <stamp>.tmp then renamed (modes 0700/0600), and prune to the newest keep (default 14). Consequences: Any change to these files after release needs v2 and a migration path for old bundles.

2026-10-30: Sync engine: hash composition, anti-flip-flop, no revive, root ancestor, dedupe

Status: accepted Context: Task 8 built the change engine (packages/core/src/sync/). Spec §8.3 fixes the outcomes; these are the mechanics a maintainer would otherwise reverse-engineer. Decision:

• Hash composition. sha256 of the stable-key-ordered JSON of {kind, work, entry|engagement|listen|list, deleted, mergedIntoKey}. key and raw are excluded; deleted is always a boolean and mergedIntoKey always present (null), so an omitted flag and an explicit false hash alike. A version row stores base64(Bun.gzipSync) of {normalized, raw}: the normalised record is what the next change is diffed against (field-level diff = previous version vs. incoming), the raw payload is kept for the record.
• Per-field decision. Only fields that differ from the previous version are considered, and of those only values that differ from the current STTracker value. field_sources[f] unset or this connection's actor → applied; "user" → pending changed {field, from: current, to: incoming}; another source → pending conflict {field, current, currentSource, incoming, incomingSource}. A new record landing on an entry that already exists goes through the same per-field decision, so a first sync never overwrites what the user or another source wrote. Unchanged values are never sent, so a plugin never takes ownership of a field it did not change.
• Anti-flip-flop. A pending changed/conflict for the same (record, field, incoming value) is never filed twice. keep_mine re-stamps field_sources[f] = "user", so later values from any source become a user-decided changed, never another source-vs-source conflict.
• No revive. If the mapped entry/engagement is soft-deleted, the record is still snapshotted but nothing is applied and no event is filed; a new record on a work whose entry the user deleted adds nothing. Only the user revives.
• Root ancestor. Creating an entry on a child work also creates a status "none" entry (plugin actor, empty field_sources) on the root ancestor work, unless the user has one, live or deleted.
• Mapped records are never re-matched. Only new records go through matchWork. A changed work hint refreshes the mapped work via updateWorkFromHint (shared works only).
• Engagement dedupe. Among live engagements on the entry not already sourced by this connection: finish dates match when both are datetimes ≤ 24 h apart as instants, or when both are day precision and their dates (first 10 chars, computed with Date.UTC, no timezone shift) are ≤ 1 day apart; or start datetimes are the same instant. A match attaches an engagement_sources row and fills only fields that are empty and that no source has set; the first source's values otherwise stand.
• Removal. Absence counts only after a full pull whose iteration finished and whose records all applied without error; any record error fails the run and skips removal. A removed record that comes back clears deleted_at and retires its still-pending removed event as ignored.
• Listens skip snapshots. Listen records are not stored as source_records (they are high-volume and append-only). listens insert-or-ignore is the idempotence; they are never removed.
• Merge accept. The target work is the merge target record's mapped entry's work, else matchWork on its latest hint. Admin-merge events (toWorkId, no connection) resolve through the same resolveChange.

Alternatives:

• Diffing against the live entry instead of the previous version would re-file decided conflicts each run.
• Hashing raw would turn every cosmetic API change into a new version.
• Keeping one run "ok" despite failing records risks marking those records removed.

Consequences: list records are snapshotted but not applied, and removal of a de-duplicated engagement deletes it even if another source still reports it (both in docs/TODO.md).

2026-10-30: User layer: per-user moves merge entries, keep aliases, and roll the library up by hierarchy

Status: accepted Context: Task 6 built the user layer (packages/core/src/{entries,engagements,history,lists,copies}.ts). Several behaviours are not obvious from the spec and would otherwise have to be reverse-engineered. Decision:

• Merge on move. moveEntryToWork onto a work where the user already has an entry merges the two:
  • Live target: the target's values win. Empty target fields are filled from the moved entry, carrying the moved entry's field_sources. Tags are unioned and custom fields and overrides merged; their field_sources is "user" if either side's was, otherwise the target's.
  • Differing non-empty review/private_notes are concatenated (target, \n\n---\n\n, moved).
  • A conflicting rating keeps the target's; the moved rating is recorded in the history row's before.
  • Engagements are re-parented, each with an entryId history row (restorable), and the moved entry is soft-deleted.
  • Soft-deleted target: revived with the moved entry's values and field_sources as-is.
• Alias bookkeeping. A move writes work_aliases (user, old → target), re-points aliases that led to the old work, drops any alias leaving the target (no cycles), tombstones the user's own private draft, and re-points this user's copies (with history) and list items. Writes to an aliased or merged work (upsertEntry, copies, list items) resolve to the target. Undoing a move via restoreVersion deletes the forward alias and writes no reverse alias, so later syncs onto the target land on the target.
• Library cursor. listEntries cursors are base64 JSON { sort, sortValue, id }. The sort is carried so a cursor cannot be replayed under another sort. For the title sort, sortValue is the SQL lower(w.title) of the last row, never a JS lowercase, so SQL and cursor agree on non-ASCII titles.
• Custom fields patches replace the whole map (customFields in an entry, engagement or copy patch). This makes restore exact. field_sources tracks the key customFields as a whole. Restore does not re-check values against today's definitions, because it is the user's own past data.
• List items are hard-deleted. list_items has no deleted_at. The list's user_history snapshot holds the ordered items, so restoreVersion brings removed items back.
• Hierarchy visibility. childSummary counts the user's live entries up to three levels below the entry's work (child, grandchild, great-grandchild). The default library view (parentId: null) lists entries with no ancestor, up to three levels, on which the user has a live entry. An episode whose show is untracked therefore stays visible, and is rolled up once the show gets an entry. Drilling in (parentId: X) applies the same rule relative to X: descendants up to three levels with no live-entry work strictly between them and X. childSummary still counts every descendant entry.
• Concurrent creates. upsertEntry inserts with ON CONFLICT (user_id, work_id) DO NOTHING, then patches through the update path, so a concurrent creator never aborts the caller's transaction.

Alternatives:

• Refusing a move onto an existing entry pushes the merge onto the user.
• Recursive CTEs for hierarchy are portable but heavier; three levels cover show → season → episode.
• A JS-lowercased cursor breaks on SQLite, whose lower() is ASCII-only.

Consequences: Undoing a move does not move copies or list items back, and hierarchy deeper than three levels is not rolled up (both in docs/TODO.md).

2026-10-30: A single Kysely dialect over Bun.SQL for SQLite and Postgres

Status: accepted Context: Kysely ships no driver for Bun. Bun.SQL speaks both sqlite:// and postgres:// behind one API. Decision:

• packages/db/src/dialect.ts is one Kysely Dialect whose driver wraps new SQL(url), choosing Kysely's SQLite or Postgres adapter, compiler and introspector by URL scheme.
• Postgres connections come from sql.reserve(). Bun's SQLite adapter does not support reserve(), so SQLite uses the single SQL instance as its only connection, serialised by a mutex so transactions never interleave.
• Transactions are plain begin/commit/rollback on the held connection.

Alternatives:

• Separate dialects over bun:sqlite and Bun.sql: two drivers to maintain for the same job.
• Community Bun dialect packages: an extra dependency, and none covers both databases.

Consequences: SQLite is effectively single-connection (fine for a self-hosted app). Savepoints and real streaming are not implemented; streamQuery returns the whole result as one chunk.

2026-10-30: JSON columns are stored as text on both dialects

Status: accepted Context: Spec §11 calls for jsonb. SQLite has no JSON type, so jsonb on Postgres would need per-dialect read/write code in every service that touches a JSON column. Decision: JSON columns are text everywhere, written with toJson() and read with fromJson() from @sttracker/db/json. fromJson returns a caller-supplied fallback for null or invalid text. Alternatives: jsonb on Postgres with per-dialect read/write code. Consequences: No JSON operators or indexes in SQL; filtering on JSON content happens in app code. Can be superseded later by a migration to jsonb if a query needs it.

2026-10-30: UUIDv7 instead of ULID for IDs

Status: accepted Context: Spec and AGENTS.md say IDs are ULIDs generated in app code. Bun has Bun.randomUUIDv7() built in. Decision: newId() in @sttracker/db/ids returns Bun.randomUUIDv7(). IDs are stored as text. Alternatives: ULID, which needs a dependency or hand-written encoder. Consequences: IDs are equally time-sortable as strings with no extra dependency. They are 36 characters instead of 26. Where AGENTS.md or the spec say "ULID", read "UUIDv7".

2026-10-30: Product design decisions recorded in spec 001

Status: accepted Context: Brainstorming with the owner produced the v1 design. Decision: See docs/specs/001-sttracker-v1-design.md §13 for the designer's calls. Owner decisions:

• Phase 1 is read-only.
• Write-back is opt-in per connection.
• Conflicts between sources go to the Changes inbox.
• Backloggd waits for its official export.
• The app UI is Hardcover-style; the public page is scrapbook + river. Alternatives: Recorded in the spec. Consequences: Phase 1 has no push implementations.

2026-10-30: Bun workspaces monorepo with Kysely for both databases

Status: accepted Context: The owner requires Bun with SQLite and Postgres options. Decision:

• Bun workspaces (apps/*, packages/*, plugins/*).
• Kysely is the single query layer, with portable migrations.
• SQLite runs through bun:sqlite and Postgres through Bun.sql.

Alternatives:

• Drizzle needs a separate schema per dialect.
• Prisma does not fit Bun-native drivers.
• Raw SQL would duplicate every query per dialect.

Consequences: We may need a small custom Kysely dialect for Bun.sql or bun:sqlite. Every DB test runs on both databases.

2026-10-30: Adopt AGENTS.md as single source of truth

Status: accepted Context: Agents had no durable project context and no consistent workflow between sessions. Decision:

• AGENTS.md is canonical; CLAUDE.md and GEMINI.md are pointers.
• Specs are permanent in docs/specs/.
• Plans live in docs/plans/ only while work is in progress and are removed before the merge request.
• Both locations override the superpowers default paths.

Alternatives:

• Per-tool rule files: they drift and create synchronisation debt.
• No rules file: agents re-explore the repo every session, which is measurably slower and costs more tokens.

Consequences: Rule changes happen in one file, and the pointer files must stay thin. Plans are working artefacts: kept after merge, they go stale and contradict the code, so the spec is the record.

2026-10-30: Catalogue matching is race-safe through unique keys plus whole-transaction retry

Status: accepted Context: Two syncs matching the same new work at once both miss the lookup and both insert. On Postgres the loser hit the external_ids (namespace, value) unique key and crashed the sync; seasons and episodes had no key at all and were duplicated. Decision:

• Migration 0002_works_position_unique adds a unique index on works (parent_id, position). NULLs are distinct on both dialects, so top-level works and position-less children are unaffected.
• matchWork re-runs the whole resolve in a fresh transaction on a unique violation (SQLite "UNIQUE constraint failed", Postgres SQLSTATE 23505), up to 3 attempts. Postgres aborts the transaction on the error, so a statement-level retry is not possible.
• Adding missing external IDs to an existing work uses ON CONFLICT DO NOTHING.
• An ID match only counts if the work is shared or owned by the matching user, and private works can never carry external IDs (createWork refuses; global merges accept only shared works).
• A global merge repoints children of from to to unless to already has a child at that position.

Alternatives:

• Advisory or table locks: dialect-specific, and SQLite already serialises writes.
• Upserts throughout createWork: would hide genuine conflicts and need per-dialect RETURNING handling.

Consequences: Parents resolved without IDs or position (title/year reuse) still have no unique key and can be duplicated under concurrency; tracked in docs/TODO.md.