--- status: in-progress --- # Phase 1, "Own your data": implementation plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task by task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** A self-hosted Bun app where users connect Hardcover, Trakt, MangaBaka, ListenBrainz, Last.fm and CSV/JSON sources. STTracker pulls and snapshots their data, detects platform-side changes into a Changes inbox, and never overwrites what the user wrote. Users can log anything without IDs and export or back up everything. SQLite or Postgres, shipped in Docker. **Architecture:** Bun workspaces. - `packages/db`: Kysely over a single custom dialect wrapping `Bun.SQL`, which speaks both `sqlite://` and `postgres://`. - `packages/plugin-sdk`: the plugin contract. - `packages/core`: domain services and the sync/change engine. - `plugins/*`: source integrations. They return `SourceRecord`s and never touch the DB. - `apps/server`: `Bun.serve` API, job worker, and serving the web app. - `apps/web`: React app via HTML imports. **Tech stack:** Bun 1.3.14, TypeScript 5.9, Kysely 0.28+, zod 4, fflate, React 19, wouter, @tanstack/react-query. **Spec:** `docs/specs/001-sttracker-v1-design.md` (read §6–§11 before any task). Design: `DESIGN.md`, `docs/design/mockups/round2-app-scrapbook-river.html`. ## Global constraints - Follow `AGENTS.md` rules. In particular: Bun-native APIs only; plugins import only `@sttracker/plugin-sdk`; syncs never overwrite user-edited fields. - **Runtime deps are approved for this plan only:** - `kysely`, `zod`, `fflate` (root); - `react`, `react-dom`, `wouter`, `@tanstack/react-query` (apps/web); - `@types/react`, `@types/react-dom` (dev). - Ask before adding anything else. - **IDs:** `newId()` from `@sttracker/db/ids`. It returns `Bun.randomUUIDv7()`. (UUIDv7 replaces the spec's ULID: it is equally time-sortable and built in. This is recorded in DECISIONS by Task 1.) - **Timestamps:** ISO-8601 UTC strings (`new Date().toISOString()`) stored as `text` on both dialects. - **JSON columns:** `text` on both dialects. Use `toJson()`/`fromJson()` from `@sttracker/db/json`. (A simplification of spec §11's jsonb; recorded in DECISIONS by Task 1.) - **Ratings:** integer 0–100 or null. - **Partial dates:** `{ value: string; precision: "day" | "month" | "year" | "unknown" }`. `value` is `YYYY-MM-DD`, `YYYY-MM`, `YYYY` or `""`, or a full ISO datetime when the source gives a time. Partial dates are never converted through `Date`, which would shift them by timezone. - **Status enum:** `"planned" | "in_progress" | "paused" | "completed" | "dropped" | "none"`. - **MediaType enum:** `book comic manga game movie show season episode podcast podcast_episode audio_drama artist album track video other`. - **Package exports:** each package's `package.json` has `"exports": { "./*": "./src/*.ts" }`. Import by path (e.g. `@sttracker/core/entries`). There are no barrel `index.ts` files, which avoids merge conflicts between parallel tasks. - **DB tests** use `forEachDialect()` from `@sttracker/db/testing` and must pass on SQLite always, and on Postgres when `TEST_POSTGRES_URL` is set. Local value: `TEST_POSTGRES_URL=postgres://postgres:postgres@127.0.0.1:55432/postgres` (container `sttracker-pg-test`). - **Tests never hit the network.** Plugins are tested with fixtures in `plugins//test/fixtures/*.json`, written by hand from the documented API shapes, and played through `createFixtureHttp()`. - **Attribution strings** are shown on the About page (exact text): - "STTracker uses Trakt but is not endorsed or certified by Trakt." - "Manga data from MangaBaka (CC BY-NC-SA 4.0)." - "Music data from ListenBrainz and Last.fm." - "Book data from Hardcover." - **Commits:** Conventional Commits, ending with the line `Co-Authored-By: Claude Opus 5.5 `. ## Review focus 1. **A full pull that fails part-way** (network error, 429s exhausted, thrown parser error) must mark *nothing* as removed. Only a pull that finished emits `removed` events. Test lives in Task 8. 2. **Running the same sync twice with unchanged source data** creates zero new `source_record_versions`, `change_events`, `engagements` or `listens`. Test lives in Task 8. 3. **A user edits a field, then the source changes it:** the user's value stays and a pending change appears. The same goes for two sources disagreeing: a pending `conflict`, with no flip-flopping on the next run. Test lives in Task 8 on both dialects. 4. **Messy CSV input** must round-trip without data loss: UTF-8 BOM, `;` delimiter, quoted newlines and quotes, empty rating, "2019" as a date, emoji titles. Test lives in Task 13. 5. **Rotating single-use refresh tokens (Trakt):** refreshed credentials are saved before any further request, and two syncs of the same connection never run at once. Tests live in Task 10 (save-before-continue) and Task 9 (connection lock). --- ## Dependency graph and waves ``` Wave A (parallel): T1 db-core T3 plugin-sdk Wave B: T2 schema (needs T1) Wave C (parallel): T4 auth T5 catalogue+matching T6 user-layer T7 attachments+crypto T10 trakt T11 hardcover T12 mangabaka T13 csv/json T14 listenbrainz+lastfm (plugins need only T3) Wave D (parallel): T8 sync engine (T3,T5,T6) T9 jobs+registry (T2,T3,T7) T15 export/backup (T5,T6,T7) Wave E: T16 server API (all core) Wave F (parallel): T17 web shell+library+detail T18 web changes/sources/lists/owned/settings/export T19 docker+CI+docs ``` Parallel tasks run in separate git worktrees branched from `feat/phase1` and are merged back into it after review. --- ### Task 1: `packages/db`: dialect, ids, json, migrations runner, test harness **Files:** - Create: `packages/db/package.json` (`@sttracker/db`, deps `kysely`), `packages/db/src/dialect.ts`, `packages/db/src/db.ts`, `packages/db/src/ids.ts`, `packages/db/src/json.ts`, `packages/db/src/migrate.ts`, `packages/db/src/testing.ts` - Create: workspace `package.json` stubs with `exports` for `packages/core` (`@sttracker/core`), `packages/plugin-sdk` (`@sttracker/plugin-sdk`), `apps/server` (`@sttracker/server`), `apps/web` (`@sttracker/web`). Add all approved deps to the stubs now (`bun add` in each), so later parallel tasks never touch `bun.lock`. - Modify: `docs/DECISIONS.md` (UUIDv7, JSON as text, single Bun.SQL dialect) - Test: `packages/db/test/db.test.ts` **Interfaces (produces):** - `type Dialect = "sqlite" | "postgres"` - `createDb(url: string): Promise`, where `interface Db { k: Kysely; dialect: Dialect; close(): Promise }`. `url` is `sqlite://`, `sqlite://:memory:` or `postgres://…`. `Database` is imported from `./schema` (Task 2). Until Task 2 lands, use a placeholder `export interface Database {}` in `src/schema.ts`. - `newId(): string`, `nowIso(): string` - `toJson(v: unknown): string`, `fromJson(s: string | null | undefined, fallback: T): T` - `migrateToLatest(db: Db): Promise` uses Kysely `Migrator` with migrations imported statically from `src/migrations/index.ts` (`export const migrations: Record`). - `forEachDialect(name: string, fn: (getDb: () => Db) => void): void` registers a `describe` per dialect. - Before each test it builds a fresh migrated DB: SQLite `:memory:`, or on Postgres a fresh `create database st_` derived from `TEST_POSTGRES_URL`. - It closes the DB after each test, and drops the Postgres DB in `afterAll`. - [ ] **Step 1: Write failing tests** in `db.test.ts` using `forEachDialect("db basics", …)`: - `"round-trips a row"`: create a temp table via `sql`, insert `{id: newId(), j: toJson({a: 1})}`, select, `expect(fromJson(row.j, {})).toEqual({a: 1})`. - `"transactions roll back"`: `await expect(db.k.transaction().execute(async tx => { insert…; throw new Error("x") })).rejects.toThrow("x")`, then count is 0. - `"newId is time-sortable"`: 1000 ids are already sorted. - [ ] **Step 2:** Run `bun test packages/db` → FAIL (module not found). - [ ] **Step 3: Implement.** - `dialect.ts`: a Kysely `Dialect` whose `Driver` wraps `new SQL(url)`. `acquireConnection` returns a connection from `sql.reserve()`. `executeQuery` calls `conn.unsafe(compiled.sql, compiled.parameters)` and maps it to `{ rows, numAffectedRows }`. Transactions use `BEGIN`/`COMMIT`/`ROLLBACK` on the reserved connection. - Adapter/compiler/introspector: `SqliteAdapter`/`SqliteQueryCompiler` for sqlite, and `PostgresAdapter`/`PostgresQueryCompiler` for postgres. - For sqlite, run `PRAGMA foreign_keys = ON; PRAGMA journal_mode = WAL;` on create (skip WAL for `:memory:`). - [ ] **Step 4:** `bun test packages/db` → PASS. Also run `TEST_POSTGRES_URL=… bun test packages/db` → PASS on both. - [ ] **Step 5:** Commit `feat(db): Bun.SQL Kysely dialect for sqlite and postgres`. ### Task 2: Schema migration and `Database` types **Files:** Create `packages/db/src/schema.ts` (Kysely table interfaces) and `packages/db/src/migrations/0001_initial.ts`; modify `packages/db/src/migrations/index.ts`. Test: `packages/db/test/schema.test.ts`. **Interfaces (produces):** the `Database` interface with these tables. Column names are snake_case. Every table has `id text pk` unless noted. `ts` means ISO text. JSON is text. - `users`(username unique, display_name, password_hash, role `admin|member`, preferences json, created_at, updated_at) - `sessions`(user_id fk, token_hash unique, expires_at, created_at) - `invites`(code_hash unique, created_by fk, used_by nullable, expires_at, created_at) - `app_tokens`(user_id, name, token_hash unique, scopes json `("read"|"write"|"ingest")[]`, last_used_at, revoked_at, created_at) - `works`(media_type, title, original_title, year int null, release_date null, creators json, description, cover_url, cover_attachment_id null, metadata json, parent_id null fk works, position json null, owner_user_id null fk users, merged_into_id null, created_at, updated_at) - `external_ids`(work_id fk, namespace, value). Unique (namespace, value). - `work_aliases`(user_id, from_work_id, to_work_id). Unique (user_id, from_work_id). - `work_history`(work_id, before json, after json, actor, created_at) - `entries`(user_id, work_id, status, rating int null, review null, private_notes null, favorite int 0/1, tags json, custom_fields json, visibility `private|friends|public`, overrides json, media_type_override null, field_sources json, created_at, updated_at, deleted_at null). Unique (user_id, work_id). - `engagements`(entry_id, started_at null, started_precision, finished_at null, finished_precision, progress json, platform null, rating null, notes null, custom_fields json, field_sources json, created_at, updated_at, deleted_at) - `engagement_sources`(engagement_id, connection_id, record_key). Unique (connection_id, record_key). - `listens`(user_id, listened_at, artist, track, album null, duration_ms null, mbids json, work_id null, source, source_key, created_at). Unique (user_id, source, source_key). Index (user_id, listened_at). - `lists`(user_id, name, description, kind `list|grouping|thread`, visibility, rating null, review null, created_at, updated_at, deleted_at) - `list_items`(list_id, work_id, position int, note null) - `custom_field_definitions`(user_id, name, type, options json, applies_to json `MediaType[] | "all"`, level `entry|engagement|copy`, created_at) - `copies`(user_id, work_id null, title_hint null, format null, acquired_from null, acquired_at null, acquired_precision, price_cents int null, currency null, notes null, custom_fields json, source_ref null, created_at, updated_at, deleted_at) - `attachments`(user_id, sha256, mime, size int, filename, caption null, taken_at null, visibility null, created_at) - `attachment_links`(attachment_id, entity `entry|engagement|copy|list|work`, entity_id) - `user_history`(user_id, entity, entity_id, before json, after json, actor, created_at) - `match_suggestions`(user_id, work_id, candidate_work_id, score real, status `pending|accepted|rejected`, created_at) - `source_connections`(user_id, plugin_id, name, config json, credentials_enc null, schedule_minutes int null, cursor null, status `active|paused|error`, last_run_at null, created_at) - `source_runs`(connection_id, mode `full|changes|file`, started_at, finished_at null, status `running|ok|failed`, counts json, log text, error null) - `source_records`(connection_id, key, kind, current_hash, first_seen, last_seen, deleted_at null, merged_into_key null, target_entity null, target_id null). Unique (connection_id, key). - `source_record_versions`(record_id, hash, raw_gz text (base64 of gzip), seen_at) - `change_events`(user_id, connection_id null, run_id null, record_id null, type `added|changed|removed|merged|conflict`, target_entity, target_id null, payload json, status `pending|auto_applied|accepted|ignored|kept_mine`, created_at, resolved_at null) - `jobs`(type, payload json, run_at, status `queued|running|done|failed`, attempts int, last_error null, locked_at null, lock_key null, created_at). Index (status, run_at). Unique partial behaviour for lock_key is enforced in code (Task 9). - [ ] **Step 1: Failing test** `forEachDialect("schema")`, `"all tables exist and unique constraints hold"`: - insert a user, a work, and two `external_ids` rows with the same (namespace, value); - expect the second insert to reject; - insert duplicate `entries` (user, work) and expect a reject. - [ ] **Step 2:** `bun test packages/db/test/schema.test.ts` → FAIL. - [ ] **Step 3:** Write the migration using only the Kysely schema builder with portable types (`text`, `integer`, `real`), plus indexes for every fk column. - [ ] **Step 4:** PASS on both dialects. - [ ] **Step 5:** Commit `feat(db): initial schema`. ### Task 3: `packages/plugin-sdk`: contract, http client, fixture harness **Files:** Create `packages/plugin-sdk/src/{types.ts,http.ts,testing.ts,partial-date.ts}`. Tests: `packages/plugin-sdk/test/{http.test.ts,partial-date.test.ts}`. **Interfaces (produces):** `types.ts`, exactly these names: ```ts export type MediaType = /* enum from Global constraints */; export type Status = "planned" | "in_progress" | "paused" | "completed" | "dropped" | "none"; export type Precision = "day" | "month" | "year" | "unknown"; export interface PartialDate { value: string; precision: Precision } export type Capability = "pull" | "pullChanges" | "push" | "ownedLibrary" | "ingest" | "lookup" | "importFile" | "exportFile"; export interface ConfigField { key: string; label: string; type: "string" | "secret" | "number" | "boolean" | "select"; required?: boolean; options?: string[]; help?: string } export interface PluginManifest { id: string; name: string; version: string; sdkVersion: 1; mediaTypes: MediaType[]; capabilities: Capability[]; config: ConfigField[]; attribution?: string } export interface WorkHint { title: string; mediaType: MediaType; year?: number; creators?: { name: string; role?: string }[]; externalIds: Record; coverUrl?: string; description?: string; parent?: WorkHint; position?: { season?: number; episode?: number; index?: number } } export interface EntryFields { status?: Status; rating?: number | null; review?: string | null; notes?: string | null; favorite?: boolean; tags?: string[] } export interface EngagementFields { startedAt?: PartialDate | null; finishedAt?: PartialDate | null; progress?: Record; platform?: string | null; rating?: number | null; notes?: string | null } export interface ListenFields { listenedAt: string; artist: string; track: string; album?: string; durationMs?: number; mbids?: { recording?: string; release?: string; artist?: string } } interface Base { key: string; raw: unknown; deleted?: boolean; mergedIntoKey?: string } export type SourceRecord = | (Base & { kind: "entry"; work: WorkHint; entry: EntryFields }) | (Base & { kind: "engagement"; work: WorkHint; engagement: EngagementFields }) | (Base & { kind: "listen"; listen: ListenFields }) | (Base & { kind: "list"; list: { name: string; description?: string; items: { work: WorkHint; note?: string }[] } }); export interface HttpClient { json(url: string, init?: RequestInit & { query?: Record }): Promise; raw(url: string, init?: RequestInit): Promise } export interface PluginContext { http: HttpClient; log: (msg: string) => void; config: Record; credentials: { get(): Promise | null>; save(c: Record): Promise }; cursor: { get(): Promise; set(c: string): Promise }; now(): Date; signal: AbortSignal; } export interface DeviceAuth { start(ctx: PluginContext): Promise<{ userCode: string; verificationUrl: string; deviceCode: string; intervalSec: number; expiresInSec: number }>; poll(ctx: PluginContext, deviceCode: string): Promise<"pending" | Record> } export interface SourcePlugin { manifest: PluginManifest; deviceAuth?: DeviceAuth; pull?(ctx: PluginContext): AsyncIterable; pullChanges?(ctx: PluginContext): AsyncIterable; lookup?(ctx: PluginContext, q: { title: string; mediaType?: MediaType; year?: number }): Promise; importFile?(ctx: PluginContext, file: { name: string; bytes: Uint8Array }): AsyncIterable; } export function definePlugin(p: SourcePlugin): SourcePlugin; ``` - `http.ts`: `createHttpClient(opts: { ratePerMinute?: number; burst?: number; maxRetries?: number; userAgent: string; fetchImpl?: typeof fetch; sleep?: (ms: number) => Promise }): HttpClient`. - Token-bucket rate limiting. - Retries on 429/5xx with exponential backoff, honouring `Retry-After` (seconds or HTTP date) and `X-RateLimit-Reset-In`. - Throws `HttpError { status, body }` after the retries run out. - `json` sends `Accept: application/json` and appends `query`. - `testing.ts`: `createFixtureHttp(routes: Record unknown)>): { fetch: typeof fetch; calls: Request[] }`. The key is `"METHOD url-without-query"` or `"METHOD full-url"`. An unknown route throws. `createTestContext(partial?: Partial & { fetch?: typeof fetch }): PluginContext & { saved: { credentials: Record[]; cursor: string[] } }` provides in-memory credentials and cursor. `collect(it: AsyncIterable): Promise`. - `partial-date.ts`: `parsePartialDate(input: string | null | undefined): PartialDate | null` accepts `2019`, `2019-03`, `2019-03-04`, full ISO datetime, `03/04/2019`→null (ambiguous), and `""`→null. `comparePartialDates(a, b): number`. - [ ] **Step 1: Failing tests.** - `http.test.ts`: - `"retries 429 honouring Retry-After"`: fetch returns 429 `Retry-After: 2` then 200; `sleep` was called with 2000; the result parsed. - `"gives up after maxRetries with HttpError"`. - `"rate limits to ratePerMinute"`: with a fake clock, 3 calls at `ratePerMinute: 60, burst: 1` sleep ~1000ms between calls. - `partial-date.test.ts`: a table of the inputs above to the expected `{value, precision}`. - [ ] **Step 2:** FAIL. - [ ] **Step 3:** Implement. - [ ] **Step 4:** `bun test packages/plugin-sdk` → PASS. - [ ] **Step 5:** Commit `feat(sdk): plugin contract, http client, fixture harness`. ### Task 4: Auth: users, sessions, invites, app tokens **Files:** Create `packages/core/src/auth.ts`. Test: `packages/core/test/auth.test.ts`. **Interfaces (produces):** - `registerUser(db: Db, input: { username: string; password: string; displayName?: string; inviteCode?: string }): Promise`. The first user becomes `admin` and needs no invite. Later users need a valid, unused, unexpired invite or it throws `AuthError("invite_required")`. Usernames match `/^[a-z0-9_]{2,32}$/`, lowercased, unique. - `login(db, username, password): Promise<{ user: User; sessionToken: string }>`. Sessions last 30 days. The token is 32 random bytes in base64url; only its SHA-256 is stored. - `getSessionUser(db, token): Promise`, `logout(db, token)`. - `createInvite(db, adminId, ttlDays = 7): Promise` (admins only, else `AuthError("forbidden")`). - `createAppToken(db, userId, name, scopes): Promise<{ id: string; token: string }>`. The token is formatted `stt_` and only shown once. `verifyAppToken(db, token, scope): Promise` updates `last_used_at`. `revokeAppToken(db, userId, id)`. - `type User = { id: string; username: string; displayName: string; role: "admin" | "member" }`. Passwords use `Bun.password.hash` (argon2id). - [ ] **Step 1: Failing tests** (`forEachDialect`): - first user is admin; - second registration without invite → `invite_required`; - with invite → member, and the invite cannot be reused; - wrong password → `AuthError("invalid_credentials")`; - session round-trip and logout; - app token: `verifyAppToken` with a missing scope → null; revoked → null. - [ ] **Steps 2–4:** FAIL, then implement, then PASS on both dialects. - [ ] **Step 5:** Commit `feat(core): users, sessions, invites, app tokens`. ### Task 5: Catalogue: works, external IDs, matching **Files:** Create `packages/core/src/works.ts` and `packages/core/src/matching.ts`. Tests: `packages/core/test/{works,matching}.test.ts`. **Interfaces (produces):** - `createWork(db, input: { hint: WorkHint; ownerUserId?: string | null; parentId?: string | null }, actor: string): Promise` inserts the work and its external_ids. - `getWork(db, id): Promise` follows `merged_into_id` chains. - `updateWorkFromHint(db, workId, hint, actor)` fills or updates shared-work display fields, writes `work_history`, adds missing external IDs, and never touches private works. - `findWorkByExternalIds(db, ids: Record): Promise`. - `searchWorks(db, userId, q: string, opts?: { mediaType?: MediaType; limit?: number }): Promise` covers shared works plus the user's private ones, with a case-insensitive `LIKE` on title. - `matchWork(db, userId, hint: WorkHint, actor: string): Promise<{ workId: string; created: boolean; draft: boolean }>` follows spec §8.4: 1. By external ID: link, and update display fields. 2. If `hint.parent` + `hint.position`: resolve the parent recursively, then find the child by (parent_id, position). If not found, create a shared child. 3. Hint has external IDs: create a shared work. 4. Otherwise: create a private draft (`owner_user_id = userId`) and store up to 5 `match_suggestions`, where `fuzzyScore ≥ 0.6`. - Shows, seasons and episodes create the whole chain. - `fuzzyScore(a: {title, year?, creators?}, b: same): number` returns 0..1. Normalise case, punctuation, leading "the/a/an" and diacritics. Score is title token-set Jaccard × 0.7, + year match 0.2 (±1 year counts as 0.1), + any creator match 0.1. - `resolveAlias(db, userId, workId): Promise`. - `acceptSuggestion(db, userId, suggestionId)` moves the user's entry from the draft work to the candidate (via `moveEntryToWork` from T6, imported lazily to avoid a cycle). `rejectSuggestion(db, userId, suggestionId)`. - `adminMergeWorks(db, adminUserId, fromId, toId)` (admin only, else `AuthError("forbidden")`) sets `merged_into_id`, moves the external_ids, writes `work_history`, and repoints every user's entries unless the user already has an entry on `toId`. In that case the user's `from` entry is kept and they get a `pending` `merged` change event. - `type Work` mirrors the row with JSON fields parsed. - [ ] **Step 1: Failing tests:** - `"match by isbn13 links existing"`; - `"episode hint creates show→season→episode chain once"` (calling it twice → same ids, 3 works total); - `"hint without ids creates private draft with suggestions"`; - `"private works invisible to other users in search"`; - `fuzzyScore("The Expanse: Leviathan Wakes", "Leviathan Wakes")` > 0.6; - `fuzzyScore("Dune","Dune Messiah")` < 0.6; - `"Pokémon"` vs `"Pokemon"` = 1 when year and creators are equal or absent. - [ ] **Steps 2–4:** FAIL, then implement, then PASS on both dialects. - [ ] **Step 5:** Commit `feat(core): works catalogue and matching`. ### Task 6: User layer: entries, engagements, history, lists, copies, custom fields **Files:** Create `packages/core/src/{history.ts,entries.ts,engagements.ts,lists.ts,copies.ts,custom-fields.ts}`. Tests: one test file per source file in `packages/core/test/`. **Interfaces (produces):** - `recordHistory(tx, { userId, entity, entityId, before, after, actor })` stores only the changed keys. - `getHistory(db, userId, entity, entityId)` and `restoreVersion(db, userId, historyId)` (re-applies `before` as actor `user`). - `type Actor = "user" | \`plugin:${string}:${string}\` | \`token:${string}\``. Plugin actors are `plugin::`. - `upsertEntry(db, userId, workId, patch: Partial, actor: Actor): Promise`. `EntryPatch` keys: `status rating review privateNotes favorite tags customFields visibility overrides mediaTypeOverride`. - It sets `field_sources[key] = actor` for each patched key. - It writes history. - It validates rating 0–100 and custom fields against definitions. - Defaults: `status "none"`, `visibility` from user prefs (`"private"`). - `getEntry(db, userId, entryId)`. - `listEntries(db, userId, filter: { mediaType?; status?; tag?; draft?: boolean; q?; parentId?: string | null; sort?: "updated" | "title" | "rating"; limit?; cursor? }): Promise<{ items: EntryView[]; nextCursor: string | null }>`. - `EntryView` = entry + work (with overrides applied) + `draft` (the work has no external IDs) + `childSummary: { total: number; completed: number } | null`. - The default `parentId: null` lists top-level only. - `softDeleteEntry`, `moveEntryToWork(db, userId, entryId, targetWorkId, actor)` (writes a `work_aliases` row). - `quickAdd(db, userId, { title, mediaType? = "other" }): Promise` creates a private draft work, then the entry. - `addEngagement` / `updateEngagement` / `softDeleteEngagement` (the same field_sources and history rules), plus `listEngagements(db, userId, entryId)`. - Lists: - `createList`, `updateList`, `addListItem`, `moveListItem`, `removeListItem`, `getList`, `listLists`; - `kind` is `list|grouping|thread`, and a grouping may carry `rating`/`review`. - Copies: `createCopy`, `updateCopy`, `softDeleteCopy`, `listCopies(db, userId, { workId? })`. Price is stored in cents. - Custom fields: `defineCustomField`, `listCustomFields(db, userId, mediaType?)`, `validateCustomValues(defs, values)`, which throws `ValidationError(field, reason)`. - [ ] **Step 1: Failing tests:** - `"upsertEntry records field_sources and history"`; - `"restoreVersion brings back previous review"`; - `"rating 101 rejected"`; - `"quickAdd creates draft entry"` (`draft === true`); - `"listEntries rolls up children"`: a show with 3 episode entries, 2 completed → `childSummary {total:3, completed:2}`, and episodes are not listed at the top level; - `"engagement with year precision keeps '2019'"`; - `"moveEntryToWork leaves alias"`; - `"custom select field rejects unknown option"`; - `"grouping list can be rated"`. - [ ] **Steps 2–4:** FAIL, then implement, then PASS on both dialects. - [ ] **Step 5:** Commit `feat(core): entries, engagements, history, lists, copies, custom fields`. ### Task 7: Attachments and secret encryption **Files:** Create `packages/core/src/attachments.ts` and `packages/core/src/crypto.ts`. Tests: `packages/core/test/{attachments,crypto}.test.ts`. **Interfaces (produces):** - `encryptJson(secret: string, value: unknown): Promise` and `decryptJson(secret: string, blob: string): Promise`. Use AES-256-GCM via WebCrypto, with the key = HKDF-SHA256 of `secret` and info `"sttracker-credentials"`. The blob is `v1..`. A wrong secret throws `CryptoError`. - `storeAttachment(db, dataDir, userId, file: { bytes: Uint8Array; mime: string; filename: string; caption?: string; takenAt?: string }, link: { entity; entityId }): Promise`. Content is addressed by sha256 at `${dataDir}/attachments/ab/cd/` and deduplicated. Allowed mimes are `image/*`, `application/pdf` and `text/plain`, with a max of 25 MB (else `ValidationError`). - `readAttachment(db, dataDir, userId, id): Promise<{ file: BunFile; mime }>` checks the owner. - `listAttachments(db, userId, entity, entityId)`, `unlinkAttachment(...)`. - Thumbnails are out of Phase 1. The UI sizes images with CSS (recorded in TODO). - [ ] **Step 1: Failing tests:** encrypt/decrypt round-trip; wrong secret throws; the same bytes stored twice produce one file and two links; `.exe` mime rejected; another user's read → null. - [ ] **Steps 2–4:** FAIL, then implement, then PASS. - [ ] **Step 5:** Commit `feat(core): attachments and credential encryption`. ### Task 8: Sync engine: snapshots, diff, apply, Changes inbox **Files:** Create `packages/core/src/sync/{snapshot.ts,apply.ts,run.ts,changes.ts,listens.ts}`. Tests: `packages/core/test/sync/*.test.ts`. **Interfaces:** - **Consumes:** `SourcePlugin`, `SourceRecord`, `PluginContext` (T3); `matchWork`, `updateWorkFromHint` (T5); `upsertEntry`, `addEngagement`, `moveEntryToWork`, `softDeleteEntry`, `softDeleteEngagement` (T6). - **Produces:** - `runSync(deps: { db: Db; makeContext: (conn: Connection, runId: string) => PluginContext }, plugin: SourcePlugin, connectionId: string, mode: "full" | "changes" | { file: { name: string; bytes: Uint8Array } }): Promise`. - `RunSummary = { runId; status: "ok" | "failed"; counts: { seen; added; changed; removed; merged; conflicts; autoApplied; pending; listens }; error?: string }`. - `listChanges(db, userId, { status?: "pending" | "all"; connectionId?; limit?; cursor? })`. - `resolveChange(db, userId, changeId, action: "accept" | "ignore" | "keep_mine"): Promise`. - `ingestListens(db, userId, source: string, records: SourceRecord[]): Promise<{ inserted: number }>`. **Rules (spec §8.3; implement exactly):** 1. **Hash** = sha256 of a stable-key-ordered JSON of `{kind, work, entry|engagement|listen|list, deleted, mergedIntoKey}`. The raw payload is excluded from the hash, but stored in a version gzipped with `Bun.gzipSync` and base64-encoded. A new `source_record_versions` row is written only when the hash differs from `current_hash`. 2. **New record (`added`):** match the work. For `entry`, `upsertEntry` with only the fields present, under the plugin actor. For `engagement`, ensure an entry exists, then de-duplicate: if any non-deleted engagement on that entry has `finished_at` within 24 h of the incoming one (day-precision values compare as dates) or an equal `started_at` datetime, attach an `engagement_sources` row instead of inserting. Write a `change_events` row with status `auto_applied`. 3. **Changed record:** for each field that differs from the previous normalized version: - `field_sources[f]` is unset or equals this connection's actor → apply, `auto_applied`; - it is `"user"` → `pending` `changed` event with `{field, from, to}`, and the current value is kept; - it is another connection's actor and the value differs → `pending` `conflict` event with `{field, current, currentSource, incoming, incomingSource}`. - A pending event for the same (record, field, incoming value) is never duplicated. That is the anti-flip-flop rule. 4. **Removed:** only when the mode is `"full"`, the plugin iteration completed without throwing, and the record was not seen in this run, or when `deleted: true` arrives in any mode. Set `source_records.deleted_at` and write a `pending` `removed` event. Accepting it soft-deletes the target entry or engagement. 5. **Merged:** when `mergedIntoKey` is set, write a `pending` `merged` event. Accepting it calls `moveEntryToWork` to the work matched for the target record's hint. 6. **Listens:** go to `ingestListens`. Insert-or-ignore on (user, source, source_key), and match a work only via `mbids.recording` (→ an existing work, or a shared track work created with a `musicbrainz_recording` id). There is no change event per listen; counts go in the run summary. 7. **`resolveChange`:** `accept` applies the incoming value with actor `plugin:…` and sets `field_sources` to that actor. `keep_mine` sets `field_sources[f] = "user"` for conflicts. `ignore` does nothing to data. All three set `status` and `resolved_at`. 8. **Run bookkeeping:** set `source_runs.status`, `counts`, `log` and `error`; `source_connections.last_run_at`; and set `status = "error"` on failure. - [ ] **Step 1: Failing tests** using a fake in-test plugin whose `pull` yields records from an array: - `"first full pull creates entries and auto_applied events"`; - `"second identical pull creates nothing new"` (Review focus 2: counts versions, events, engagements and listens before/after); - `"failed pull marks nothing removed"` (Review focus 1: the plugin yields 1 of 2 records, then throws); - `"record missing from completed full pull → pending removed; accept soft-deletes entry"`; - `"user-edited rating not overwritten; pending changed event"` (Review focus 3); - `"two connections disagree → one pending conflict, not duplicated on rerun; keep_mine stops further conflicts for that value"`; - `"same play from two sources dedupes into one engagement with two engagement_sources"`; - `"merged record → pending merged; accept moves entry"`; - `"listens insert-or-ignore and mbid matching"`. - All of these run on both dialects. - [ ] **Steps 2–4:** FAIL, then implement, then PASS on both dialects. - [ ] **Step 5:** Commit `feat(core): sync engine and changes inbox`. ### Task 9: Jobs, scheduler, plugin registry, connections **Files:** Create `packages/core/src/{jobs.ts,registry.ts,connections.ts}`. Tests: `packages/core/test/{jobs,registry,connections}.test.ts`, plus fixture plugin `packages/core/test/fixtures/plugins/echo/index.ts`. **Interfaces (produces):** - `enqueue(db, type: string, payload: unknown, opts?: { runAt?: string; lockKey?: string }): Promise` returns null when a `queued|running` job with the same `lockKey` exists. Sync jobs use `lockKey = "sync:"` (Review focus 5). - `startWorker(db, handlers: Record Promise>, opts?: { pollMs?: number; concurrency?: number }): { stop(): Promise }`. - A job is claimed atomically (`update … where id = ? and status = 'queued'` and check affected rows). - Retry with backoff `2^attempts` minutes up to 5 attempts, then `failed`. - `startScheduler(db, opts?: { tickMs?: number }): { stop() }` enqueues `sync` for each active connection whose `last_run_at + schedule_minutes` has passed. - `loadPlugins(dirs: string[]): Promise`, where `PluginRegistry = { get(id): SourcePlugin | undefined; list(): PluginManifest[]; reload(): Promise }`. - Each subdirectory with `index.ts` whose default export passes `definePlugin` shape checks is registered (`sdkVersion === 1`, unique id). - `reload` re-imports with `?v=`. - A bad plugin is logged and skipped, never fatal. - Connections: - `createConnection(db, secret, userId, { pluginId, name, config, credentials?, scheduleMinutes? })`, `updateConnection`, `deleteConnection` (soft: `status "paused"` + credentials cleared), `listConnections(db, userId)`; - `makePluginContext(db, secret, conn, runId, http: HttpClient): PluginContext`. Its credentials `save` encrypts and persists *immediately* (Review focus 5), and its cursor is persisted on `set`. - `syncHandler(deps)` is the `"sync"` job handler: it loads the connection and plugin, calls `runSync` with `mode` from the payload (default `pullChanges` if the plugin supports it, otherwise `full`), and runs a weekly `full` regardless. - [ ] **Step 1: Failing tests:** - `"enqueue with same lockKey returns null"`; - `"worker retries then fails"`; - `"scheduler enqueues due connections only"`; - `"registry loads echo plugin and skips broken one"`; - `"credentials.save persists encrypted before pull continues"` (the echo plugin saves creds, then throws; the DB holds the new encrypted creds). - [ ] **Steps 2–4:** FAIL, then implement, then PASS on both dialects. - [ ] **Step 5:** Commit `feat(core): jobs, scheduler, plugin registry, connections`. ### Task 10: Trakt plugin **Files:** `plugins/trakt/{package.json,index.ts,src/*.ts,test/trakt.test.ts,test/fixtures/*.json}`. **Interfaces:** `definePlugin` with: - manifest `id "trakt"`, capabilities `["pull","pullChanges","lookup"]`, mediaTypes `movie show season episode`; - config: `clientId` (string, required), `clientSecret` (secret, required); - `attribution` = the exact Trakt line from Global constraints; - `deviceAuth` via `POST /oauth/device/code` and `/oauth/device/token`. Requests use base `https://api.trakt.tv` with headers `trakt-api-version: 2` and `trakt-api-key: clientId`, at `ratePerMinute: 100`. Before each run, refresh the access token if it expires within 24 h (`POST /oauth/token`, `grant_type=refresh_token`), then **`ctx.credentials.save` before any other request**. **pull** yields: - `GET /sync/history?page=&limit=250` (paged by `X-Pagination-Page-Count`): each history item → an `engagement` record with key `history:`, and `finishedAt` = `watched_at` at day precision but keeping the full datetime in `value`. Movies: hint `movie` with ids `{trakt, tmdb_movie, imdb}`. Episodes: hint `episode` with `parent` season → `parent` show (`tmdb_show`, `tvdb`, `imdb`, `trakt`) and `position {season, episode}`. - `GET /sync/ratings`: `entry` records with key `rating::`, `rating = rating × 10`. - `GET /sync/watchlist`: `entry` with status `planned`, key `watchlist::`. - `GET /users/me/lists` + `/users/me/lists/:id/items`: `list` records. **pullChanges:** `GET /sync/last_activities`, compared with the cursor (JSON of the last activity timestamps). For the categories that changed, re-fetch: - history with `start_at = previous cursor time`; - ratings and watchlist in full. - Set the new cursor at the end. **lookup:** `GET /search/movie,show?query=`. - [ ] **Step 1: Failing tests with fixtures:** history paging over 2 pages; episode hint chain and position; rating 8 → 80; watchlist → planned; a token near expiry is refreshed and saved before the first `/sync` call (assert the order of `fixture.calls` and `ctx.saved.credentials`); `pullChanges` with an unchanged `last_activities` yields nothing. - [ ] **Steps 2–4:** FAIL, then implement, then PASS (`bun test plugins/trakt`). - [ ] **Step 5:** Commit `feat(plugins): trakt`. ### Task 11: Hardcover plugin **Files:** `plugins/hardcover/...` (same layout as Task 10). **Interfaces:** manifest `id "hardcover"`, capabilities `["pull","pullChanges","lookup"]`, mediaTypes `book`, config `token` (secret, required: personal access token, sent as `Authorization: Bearer`), and attribution "Book data from Hardcover." Requests are `POST https://api.hardcover.app/v1/graphql` at `ratePerMinute: 50`. GraphQL errors throw `HttpError`. **pull** pages `me { user_books(limit: 100, offset) { id status_id rating review_raw private_notes starred updated_at merged_at original_book_id book { id title release_year contributions { author { name } } book_series { position series { id name } } } edition { id isbn_13 isbn_10 asin } user_book_reads { id started_at finished_at progress_pages } } }` and yields: - `entry` with key `ub:`: - status map: 1→planned, 2→in_progress, 3→completed, 4→paused, 5→dropped, 6→none; - `rating` ×20 (Hardcover is 0–5 with halves); - `review_raw` → review, `private_notes` → notes, starred → favorite; - hint ids `{hardcover_book, isbn13, isbn10, asin, hardcover_edition}`; if there is a series, `parent` = series hint (`hardcover_series`) with `position {index}`. - `engagement` for each read with key `read:`, day-precision dates. - When `merged_at` is set and `original_book_id ≠ book.id`, also set `mergedIntoKey` on a synthetic record with key `ub:@book:`, so the engine sees the old identity merged into the new one. - Lists via `me { lists { id name description list_books { book {…} } } }`. **pullChanges:** the same query filtered by `updated_at > cursor`. Deletions are not reported, so the weekly full pull detects them. - [ ] **Step 1: Failing tests:** status and rating mapping (4.5 → 90); series parent hint; re-reads → 2 engagements; merge produces `mergedIntoKey`; GraphQL `errors` → throws; offset paging stops on a short page. - [ ] **Steps 2–4:** FAIL, then implement, then PASS. - [ ] **Step 5:** Commit `feat(plugins): hardcover`. ### Task 12: MangaBaka plugin **Files:** `plugins/mangabaka/...`. **Interfaces:** manifest `id "mangabaka"`, capabilities `["pull","pullChanges","lookup"]`, mediaTypes `manga comic`, config `token` (secret, required, header `x-api-key`), and attribution "Manga data from MangaBaka (CC BY-NC-SA 4.0)." Base `https://api.mangabaka.org`, `ratePerMinute: 150`. **pull** pages `GET /v1/my/library?page=` and yields `entry` records keyed `series:`: - state map: considering|plan_to_read→planned, reading|rereading→in_progress, paused→paused, dropped→dropped, completed→completed; - rating is already 0–100; - `note` → notes; - progress → an `engagement` record keyed `progress:` with `progress {chapter, volume}` and start/finish dates; - hint ids `{mangabaka, anilist, mal, mangaupdates, kitsu}` from `source.*`. - If the series `state === "merged"`, set `mergedIntoKey = "series:"`. **pullChanges:** `GET /v2/my/library/changes?cursor=`. Upserts yield records, deletes yield `deleted: true`, and the new cursor is saved. - [ ] **Step 1: Failing tests:** state mapping table; a merged series sets `mergedIntoKey`; the changes feed yields a deletion; cursor saved. - [ ] **Steps 2–4:** FAIL, then implement, then PASS. - [ ] **Step 5:** Commit `feat(plugins): mangabaka`. ### Task 13: CSV/JSON import plugin and CSV codec **Files:** `packages/plugin-sdk/src/csv.ts` (the codec lives in the SDK so core can reuse it), `plugins/csv/{index.ts,src/mapping.ts,test/*.test.ts,test/fixtures/*}`. **Interfaces (produces):** - `parseCsv(text: string, opts?: { delimiter?: "," | ";" | "\t" }): string[][]` (RFC 4180). It strips a BOM and auto-detects the delimiter from the header line. - `toCsv(rows: string[][]): string` quotes as needed and uses `\r\n`. This is reused by Task 15. - The manifest is `id "csv"`, capabilities `["importFile"]`, and config `mapping` (string: JSON `{ column: field }`, where the fields are `title mediaType status rating ratingScale(5|10|100) review finishedAt startedAt tags isbn13 imdb tmdb_movie igdb notes`). - `importFile`: - **`.csv`:** maps rows into `entry` records (plus an `engagement` when dates are present), keyed `row:`. - **`.json`:** accepts an array of objects with the same field names. - **Without a mapping**, it guesses from header names, case-insensitively, including the Goodreads export columns (`Title`, `Author`, `My Rating`, `Date Read`, `ISBN13`, `Exclusive Shelf`, `My Review`). - [ ] **Step 1: Failing tests (Review focus 4):** a fixture `messy.csv` with BOM, `;`, quoted newline and `""` quotes, an empty rating, `Date Read` = `2019`, and an emoji title; `parseCsv(toCsv(x))` deep-equals `x`; Goodreads fixture headers are auto-mapped; `Exclusive Shelf` `to-read` → planned and `read` → completed; `My Rating` 4 at scale 5 → 80. - [ ] **Steps 2–4:** FAIL, then implement, then PASS. - [ ] **Step 5:** Commit `feat(plugins): csv and json import`. ### Task 14: ListenBrainz and Last.fm plugins **Files:** `plugins/listenbrainz/...` and `plugins/lastfm/...`. **ListenBrainz:** manifest `id "listenbrainz"`, capabilities `["pull","pullChanges"]`, mediaTypes `track album artist`, config `username` (string) + `token` (secret, `Authorization: Token`). Base `https://api.listenbrainz.org`. Respect `X-RateLimit-Reset-In` (already handled in T3). - **pull (full backfill):** 1. `POST /1/export/` (if the response is 400 "already pending", fall through to listing). 2. Poll `GET /1/export/list` every 30 s via `ctx.now`/an injected sleep until the newest export's status is `complete`. 3. Download `GET /1/export/{id}/download`. 4. Unzip with `fflate.unzipSync`. 5. Stream `listens/**/*.jsonl` → `listen` records keyed `:`, with `mbids` from `track_metadata.mbid_mapping`. 6. `feedback.jsonl` with score 1 → an `entry` with `favorite: true`, track hint ids `{musicbrainz_recording}`. - **pullChanges:** `GET /1/user/{u}/listens?min_ts=&count=1000` loops until empty, and the cursor becomes the max `listened_at`. **Last.fm:** manifest `id "lastfm"`, capabilities `["pull","pullChanges"]`, config `username` + `apiKey` (secret). Base `https://ws.audioscrobbler.com/2.0/`, `format=json`, `ratePerMinute: 240`. - **pull:** pages `user.getRecentTracks&limit=200&page=`, skipping the `@attr.nowplaying` track, yielding `listen` records keyed `::` with MBIDs when non-empty. Also `user.getLovedTracks` → favorite `entry` records. - **pullChanges:** `from=`. - Error code 29 → throw `HttpError` with status 429 so the client retries. - [ ] **Step 1: Failing tests:** - LB: export flow over fixtures (a pending then complete list, plus a zip built in-test with `fflate.zipSync`); a feedback love → favorite entry; the incremental cursor advances. - Last.fm: the nowplaying track is skipped; empty MBIDs are omitted; error 29 is retried (via the fixture fetch returning 29 once). - [ ] **Steps 2–4:** FAIL, then implement, then PASS. - [ ] **Step 5:** Commit `feat(plugins): listenbrainz and lastfm`. ### Task 15: Export bundle, restore, scheduled backups **Files:** Create `packages/core/src/export.ts` and `packages/core/src/backup.ts`. Tests: `packages/core/test/{export,backup}.test.ts`. **Interfaces (produces):** - `exportUserBundle(db, dataDir, userId): Promise` returns a zip (fflate) containing: - `manifest.json` `{ format: "sttracker-export/v1", exportedAt, user: { username, displayName } }`; - `works.jsonl` (every referenced work, plus ancestors, with `externalIds`); - `entries.jsonl`, `engagements.jsonl`, `listens.jsonl`, `lists.jsonl` (with items), `copies.jsonl`, `custom_fields.jsonl`, `history.jsonl`, `attachments.jsonl`; - `attachments/` file bytes. - It excludes credentials, sessions and tokens. - In all files, camelCase keys mirror the `Entry`/`Work` view types. - `exportUserCsv(db, userId): Promise>` returns `{ "entries.csv": …, "engagements.csv": …, "listens.csv": … }` using `toCsv` from `@sttracker/plugin-sdk/csv` (Task 13). - `importUserBundle(db, dataDir, userId, zip: Uint8Array): Promise<{ works: number; entries: number }>` refuses unless the user has no entries (`ImportError("account_not_empty")`). It re-matches works by external IDs; unmatched works become private works; old IDs are remapped to new ones. - `runBackup(db, dataDir, opts: { keep: number }): Promise` writes `${dataDir}/backups//` containing one bundle per user, plus on SQLite `VACUUM INTO 'sttracker.db'`. It prunes to the newest `keep` (default 14). - `"backup"` job handler: `backupHandler(deps)`. - [ ] **Step 1: Failing tests:** - `"export then import into fresh user reproduces entries, ratings, engagements with precision, lists and attachments"`; - `"bundle has no credential fields"` (grep the zip text for `password`, `token`, `credentials`); - `"import into non-empty account refused"`; - `"backup prunes to keep"`. - All on both dialects (VACUUM is SQLite only). - [ ] **Steps 2–4:** FAIL, then implement, then PASS. - [ ] **Step 5:** Commit `feat(core): export bundle, restore, backups`. ### Task 16: `apps/server`: API, auth middleware, ingest mount, OpenAPI, boot **Files:** Create `apps/server/src/{main.ts,config.ts,http.ts,routes/*.ts,openapi.ts}` and `apps/server/test/api.test.ts`. **Interfaces:** - **Config** (`config.ts`) reads env: - `DATABASE_URL` (default `sqlite://${DATA_DIR}/sttracker.db`); - `DATA_DIR` (default `./data`); - `STTRACKER_SECRET` (required; refuse to boot without it, with the message "Set STTRACKER_SECRET to a long random string. It encrypts source credentials."); - `PUBLIC_URL`, `PORT` (default 3000); - `PLUGIN_DIRS` (default `plugins,${DATA_DIR}/plugins`). - `createApp(deps: { db; registry; secret; dataDir }): { fetch(req: Request): Promise; routes }` is testable without listening. `main.ts` migrates, loads plugins, starts the worker and scheduler, schedules a daily `backup` job, and calls `Bun.serve({ routes: { "/*": index.html }, fetch: app.fetch })`. - `http.ts`: `route({ method, path, auth: "none" | "session" | "token:read" | "token:write" | "any:read" | "any:write", body?: ZodType, query?: ZodType, handler })`. - JSON errors have the shape `{ error: { code, message } }`, with codes `400 validation`, `401 unauthenticated`, `403 forbidden`, `404 not_found`, `409 conflict`. - Session cookie: `stt_session`, HttpOnly, SameSite=Lax, Secure when `PUBLIC_URL` is https. - `any:*` accepts a session or a Bearer app token. - **Routes under `/api/v1`** (bodies validated with zod; each calls the core function named in Tasks 4–15): - auth: `POST auth/register`, `POST auth/login`, `POST auth/logout`, `GET me`, `GET/POST/DELETE me/tokens`, `POST admin/invites`; - entries: `GET entries` (the `listEntries` filter as query), `POST entries/quick`, `GET/PATCH/DELETE entries/:id`, `GET entries/:id/history`, `POST history/:id/restore`, `POST entries/:id/move`; - engagements: `GET/POST entries/:id/engagements`, `PATCH/DELETE engagements/:id`; - works: `GET works/search`, `GET works/:id`, `GET works/:id/suggestions`, `POST suggestions/:id/accept|reject`; - lists: `GET/POST lists`, `GET/PATCH/DELETE lists/:id`, `POST lists/:id/items`, `PATCH/DELETE list-items/:id`; - copies: `GET/POST copies`, `PATCH/DELETE copies/:id`; - custom fields: `GET/POST custom-fields`; - attachments: `POST attachments` (multipart: `file`, `entity`, `entityId`, `caption`), `GET attachments/:id`, `DELETE attachments/:id/links/:linkId`; - plugins and sources: `GET plugins` (manifests), `GET/POST sources`, `PATCH/DELETE sources/:id`, `POST sources/:id/run` (enqueue, 409 if locked), `POST sources/:id/import` (multipart file → enqueue a file-mode run), `POST sources/:id/device/start`, `POST sources/:id/device/poll`, `GET sources/:id/runs`; - changes: `GET changes`, `POST changes/:id/resolve` `{action}`; - export: `GET export/bundle` (zip download), `GET export/csv?file=entries.csv`, `POST import/bundle`; - `GET openapi.json` (generated from the route table with `z.toJSONSchema`); - `GET about` (attributions from plugin manifests). - `/ingest//*` → 404 in Phase 1 (the mount point exists, and no plugin has `ingest`). - `POST admin/plugins/reload` (admin) → `registry.reload()`. - [ ] **Step 1: Failing tests** (`forEachDialect`, calling `app.fetch` directly): - register → login cookie → `GET me`; - unauthenticated `GET entries` → 401 JSON; - quick add, then list shows a draft; - PATCH rating 150 → 400 `validation`; - an app token with `read` cannot PATCH (403); - another user's entry → 404; - `POST sources` with the echo plugin, then `POST sources/:id/run` twice → the second returns 409; - export bundle content-type is `application/zip`; - `openapi.json` lists `/api/v1/entries`; - booting without `STTRACKER_SECRET` throws the exact message. - [ ] **Steps 2–4:** FAIL, then implement, then PASS. - [ ] Update `AGENTS.md` Commands with Dev (`bun --hot apps/server/src/main.ts`), verified by booting and hitting `/api/v1/openapi.json`. Update `docs/PERMISSIONS.md` and the tool files accordingly. - [ ] **Step 5:** Commit `feat(server): REST API, auth, sources, changes, export`. ### Task 17: Web app: shell, auth, library, quick add, entry detail **Files:** Create `apps/web/{index.html,src/main.tsx,src/api.ts,src/theme.css,src/routes/*.tsx,src/components/*.tsx}` and `apps/web/test/*.test.tsx` (logic only: `api.ts` and formatters). Follow `DESIGN.md` tokens exactly and use impeccable `craft`. Routes (wouter): - `/login`, `/register` (the first-run screen when no users exist: "Create the admin account"); - `/` (library); - `/entries/:id`. Library: - The three-pane layout from the round 2 mockup: sidebar nav (Library, Log, Lists, Owned, Changes with a pending-count badge in `--warn`, a Sources list, Settings), the list pane, and the detail pane. - Search, filter chips by media type and status, and a "Draft" filter. - The Quick add box, with the placeholder text "Quick add: type a title, sort it out later". Enter creates the entry and selects it. - Rows show cover (or the striped draft cover), title, `media type · meta`, and stars or the amber `Draft` badge. Children roll up as "season 3 of 5"-style meta from `childSummary`. Detail: - Cover, title, editable rating (stars in the user's scale; the setting defaults to 5 with halves), status select, tags, and custom fields. - The orange change alert when pending changes target this entry, with "See what changed" → `/changes?entry=:id`. - Boxes for Reads/Plays (engagements with partial dates rendered by precision: "Mar 2025 (no exact date)"), Your copy, Attachments (upload, thumbnails via CSS), Linked sources, and Review (Gelasio, Markdown textarea). - Children with their own ratings, a History drawer with restore, and "Link to catalogue" for drafts (suggestions plus search). Formatters: `formatRating(value0to100, scale)` and `formatPartialDate(pd)`. - [ ] **Step 1: Failing tests:** `formatRating(90, "5half")` = `"★★★★½"`, `formatRating(null, …)` = `""`, `formatPartialDate({value:"2025-03",precision:"month"})` = `"Mar 2025"`, `{value:"2019",precision:"year"}` = `"2019"`; `api.ts` throws an `ApiError` carrying `code` on a non-2xx JSON error. - [ ] **Steps 2–4:** FAIL, then implement, then PASS. - [ ] **Step 5:** Manual check: run the server, register, quick add, and open the detail pane. Take a screenshot and compare it with the mockup. Fix glaring deviations. - [ ] **Step 6:** Commit `feat(web): app shell, library, entry detail`. ### Task 18: Web app: Changes, Sources, Log, Lists, Owned, Settings, Export, About **Files:** `apps/web/src/routes/{changes,sources,log,lists,owned,settings,about}.tsx` plus components. This task depends on Task 17's `api.ts`, `theme.css` and layout components; create new components only. - **Changes:** - Grouped by run. - Each card shows `type`, the entry title, a field-level before/after (or both sources' values for conflicts), and buttons "Use the new value" (accept), "Keep mine" (keep_mine) and "Ignore". - Removed events say " no longer has this. Your copy here is safe." with buttons "Delete here too" (accept) and "Keep it" (ignore). - Merged events say " merged this into ." with "Follow the merge". - **Sources:** - Plugin cards from `GET plugins`, and a connect form rendered from the `config` fields (secrets as password inputs). - The device-code flow modal shows the `userCode` and the verification link, and polls. - Per connection: status, last run, "Sync now", schedule select (off/hourly/6h/daily), runs log, and file upload for `importFile` plugins. - **Log:** a chronological feed of engagements and listens (paged). - **Lists:** create lists of kind list, grouping or thread; add works via search; drag-to-reorder (up/down buttons are acceptable); a grouping can be rated. - **Owned:** a copies table with add/edit (bought where, when, price, format, notes). - **Settings:** profile, rating scale, default visibility, custom fields editor, app tokens (create shows the token once with "Copy"; revoke), invites (admin), "Reload plugins" (admin), and Export (download bundle and CSVs, import a bundle into an empty account). - **About:** the attributions. - [ ] **Step 1: Failing tests** for pure helpers: `describeChange(event)` returns the exact strings above for the removed/merged/conflict fixtures; `scheduleLabel(minutes)`. - [ ] **Steps 2–4:** FAIL, then implement, then PASS. - [ ] **Step 5:** Manual check of the Changes flow against a fake connection (echo plugin) in the running app. - [ ] **Step 6:** Commit `feat(web): changes inbox, sources, log, lists, owned, settings`. ### Task 19: Docker, compose, CI, docs **Files:** Create `Dockerfile`, `docker-compose.yml`, `.dockerignore`, `.env.example` and `.gitea/workflows/ci.yml`. Modify `AGENTS.md` (Commands), `docs/ARCHITECTURE.md` (replace "planned" with reality), `docs/PERMISSIONS.md` and the tool files, `docs/TODO.md` and `README.md` (Quick start). - **Dockerfile:** `oven/bun:1.3` base; `bun install --frozen-lockfile --production`; copy the source; `VOLUME /data`; `ENV DATA_DIR=/data`; `EXPOSE 3000`; `CMD ["bun","apps/server/src/main.ts"]`; run as a non-root `bun` user. - **Compose:** the `sttracker` service with `./data:/data`, env from `.env`; plus a `postgres:17-alpine` service under `profiles: ["postgres"]`, with a comment showing the `DATABASE_URL` for it. - **CI:** on push/PR, a postgres:17-alpine service, then `bun install --frozen-lockfile`, `bun run typecheck`, `bun test` with `TEST_POSTGRES_URL` set, and `test $(wc -l < AGENTS.md) -le 150`. - [ ] **Step 1:** `docker build -t sttracker:dev .` → succeeds. - [ ] **Step 2:** `docker run --rm -e STTRACKER_SECRET=test -p 3999:3000 sttracker:dev` → `curl -s localhost:3999/api/v1/openapi.json` returns JSON. - [ ] **Step 3:** Repeat with the compose postgres profile and `DATABASE_URL` → boots and migrates. - [ ] **Step 4:** Update the docs. Every command in `AGENTS.md` must be verified in this session (typecheck now passes; `bun test` passes on both DBs). - [ ] **Step 5:** Commit `chore: docker, compose, CI, docs`. --- ## Completion - [ ] `bun run typecheck` clean. `bun test` passes with and without `TEST_POSTGRES_URL`. - [ ] Whole-branch review (superpowers:requesting-code-review). Fix the findings. - [ ] `git rm docs/plans/001-phase1-own-your-data.md`. Set the spec front matter to `status: implemented`. Merge `feat/phase1` → `main` via superpowers:finishing-a-development-branch.