80 Commits
Author SHA1 Message Date
Lucas WintherandClaude Opus 5.5 9546fa13f2 fix(web): settle every review save, not just the latest
Per-call mutate() callbacks fire only for an observer's latest call, so
with overlapping saves the in-flight count could stay above zero and the
review would ignore every later server value, including a Restore.

Saves now settle through useEntryPatch's hook-level onSettled (fires for
every mutation), in FIFO order (the entry-patch scope is serial). The
draft counts pending saves, remembers the latest incoming value, skips our
own optimistic/rollback echoes, and applies an outside change once the
saves settle.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 22:19:15 +01:00
Lucas WintherandClaude Opus 5.5 4608c977b7 fix(web): keep review text on a failed save; review round 2
- Review draft sync is a pure helper (format.ts): incoming values are
  ignored while a save is in flight, and a failed save restores the
  previous synced value, so the typed text is kept whichever of the
  rollback and the error callback lands first. Tests cover both orders.
- A failed background refetch keeps showing the cached entry instead of
  the error screen (it unmounted the review box mid-outage).
- History restore joins the entry-patch mutation scope.
- updatePreferences: read-modify-write in a transaction, row locked on
  Postgres.
- Invalidate auth-status after register; out-of-range rating errors speak
  in the user's scale.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 22:14:02 +01:00
Lucas WintherandClaude Opus 5.5 5fa59310f3 fix(web): review round 1: optimistic edits, friendly errors, preferences, Gelasio
- Tags and custom fields update the cached entry optimistically and build
  from the latest cache; one entry's PATCHes run in order (mutation scope).
- The review textarea follows server changes (Restore) unless edited.
- friendlyError() maps validation reasons and 413 to plain copy everywhere.
- Rating scale from me/preferences; /login and /register use auth/status;
  Attachments lists from the API.
- Gelasio vendored (official OFL variable TTFs); minors from the review.
- Docs: DESIGN (AA on orange, faint labels), DECISIONS, TODO, ARCHITECTURE.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 22:03:51 +01:00
Lucas WintherandClaude Opus 5.5 d204775235 feat(server): auth status, attachments list, user preferences
GET auth/status ({ hasUsers }), GET attachments?entity=&entityId= (owner
scoped, 404 for foreign targets) and GET/PATCH me/preferences (ratingScale,
defaultVisibility in users.preferences).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 21:57:08 +01:00
Lucas Winther d547355e5c Merge branch 'feat/phase1' into task/17 2026-10-30 21:55:33 +01:00
Lucas WintherandClaude Opus 5.5 42fea7d54e merge: task 16 server API
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 21:40:38 +01:00
Lucas WintherandClaude Opus 5.5 439d138967 feat(web): app shell, library, entry detail
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 21:37:56 +01:00
Lucas WintherandClaude Opus 5.5 70bc6c36d3 feat(web): theme tokens, self-hosted Inter, HTML entry
DESIGN.md tokens as CSS custom properties. Inter (OFL) is vendored so the
app makes no third-party font requests.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 21:37:56 +01:00
Lucas WintherandClaude Opus 5.5 0b34ede62e docs: token actor, connection deleted_at and CSRF decisions; follow-ups
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 21:30:59 +01:00
Lucas WintherandClaude Opus 5.5 5f7d54b805 fix(server): token actor, login CSRF, 415, cookie decoding, admin invites, production default
- API writes pass token:<name> for Bearer requests and "user" for sessions.
- X-Requested-With: sttracker on every non-Bearer write, login and register included.
- JSON-body routes reject non-JSON content types with 415.
- A malformed stt_session cookie counts as no cookie (401), not 500.
- DELETE me/tokens/:id is 404 for unknown or foreign ids; admin/invites is admin-only.
- Bun.serve development mode only with NODE_ENV=development or --hot.
- OpenAPI documents multipart fields.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 21:30:43 +01:00
Lucas WintherandClaude Opus 5.5 aa8e4db070 fix(core): token writes keep their actor in history but are user-owned in field_sources
fieldOwner(actor) maps every non-plugin actor to "user" wherever field_sources is written
(entries, engagements, list ownership from history), so syncs never overwrite a field an API
token edited. authenticateAppToken also returns the token name; revokeAppToken reports whether
it revoked; listSuggestions queries the canonical work id.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 21:29:38 +01:00
Lucas WintherandClaude Opus 5.5 e3a759d768 fix(core): soft-delete source connections with deleted_at
Migration 0003 adds source_connections.deleted_at. deleteConnection sets it; listing, lookups,
updates, the scheduler and the sync handler ignore deleted connections, so a paused
credential-less connection stays visible and can be resumed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 21:25:59 +01:00
Lucas WintherandClaude Opus 5.5 0bb871cd19 feat(web): api client and display formatters
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 21:25:41 +01:00
Lucas WintherandClaude Opus 5.5 0a540208e1 test(server): silence the broken fixture plugin warning
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 21:19:08 +01:00
Lucas WintherandClaude Opus 5.5 348b04e928 docs: server architecture, API decisions, follow-ups
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 21:17:18 +01:00
Lucas WintherandClaude Opus 5.5 02b7eed5ec feat(server): boot: migrate, load plugins, worker, scheduler, daily backup, web placeholder
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 21:16:15 +01:00
Lucas WintherandClaude Opus 5.5 6963e297b0 feat(server): REST API, auth, sources, changes, export
createApp() serves /api/v1 from a typed route table (zod bodies and queries), with
session-cookie and Bearer app-token auth, a CSRF header for cookie writes, login/register
rate limiting, body caps, JSON error mapping, OpenAPI generation and a 404 /ingest mount.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 21:15:01 +01:00
Lucas WintherandClaude Opus 5.5 7b0a6738dd feat(core): API support helpers: token list, suggestions, attachment links, connection runs, device auth, file-mode sync
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 21:10:39 +01:00
Lucas WintherandClaude Opus 5.5 7f3d0d5c44 merge: task 9 jobs, scheduler, registry, connections
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 21:02:52 +01:00
Lucas WintherandClaude Sonnet 5.5 f0d41f23c8 fix(core): reaper skips own in-flight jobs, heartbeat, no attempt burned on shutdown
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 20:57:56 +01:00
Lucas Winther 839d9697b0 merge: feat/phase1 into task/9 2026-10-30 20:53:22 +01:00
Lucas WintherandClaude Sonnet 5.5 6f628c4693 fix(core): job failure semantics, stale-lock reaper, run-scoped context, reload
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 20:52:51 +01:00
Lucas WintherandClaude Opus 5.5 cea4afd0de merge: task 8 sync engine
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 20:48:48 +01:00
Lucas WintherandClaude Opus 5.5 a003c3eb7d fix(core): sync review round 1: lists, supersede, dedupe conflicts, retries
- dedupe files conflicts for disagreeing fields instead of dropping them
- merged-away records never file removals
- matched work committed before apply so retries reuse it (no draft leaks)
- stale pending events superseded by newer values
- multi-source engagement removal only detaches the source
- list records applied with history-derived ownership
- gone targets resolve as ignored; counts only for committed records

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 20:43:08 +01:00
Lucas WintherandClaude Opus 5.5 0604c323c3 fix(core): re-point source records when a per-user move merges entries
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 20:43:08 +01:00
Lucas WintherandClaude Opus 5.5 6efb97eb51 fix(core): batch listen ingestion per 500
One existing-key lookup, one resolve per distinct recording, and one
multi-row insert-or-ignore per batch instead of per-listen queries.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 20:43:08 +01:00
Lucas WintherandClaude Sonnet 5.5 166b8ecb5e feat(core): jobs, scheduler, plugin registry, connections
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 20:39:17 +01:00
Lucas Winther 207c92bb4d merge: task 8 sync engine 2026-10-30 20:35:37 +01:00
Lucas Winther b2e6e1948b Merge branch 'feat/phase1' into task/8
# Conflicts:
#	docs/ARCHITECTURE.md
#	docs/DECISIONS.md
#	docs/TODO.md
2026-10-30 20:35:25 +01:00
Lucas WintherandClaude Opus 5.5 c8ce107f73 merge: task 15 export, restore, backups
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 20:26:55 +01:00
Lucas WintherandClaude Opus 5.5 1975e971f9 feat(core): sync engine and changes inbox
runSync snapshots and diffs source records, applies new records and
never-edited fields, files pending changed/conflict/removed/merged
events, de-duplicates plays across sources and ingests listens.
listChanges/resolveChange drive the Changes inbox.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 20:26:03 +01:00
Lucas WintherandClaude Opus 5.5 37a0b0ba94 feat(core): source record snapshots with stable hash and gzipped versions
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 20:26:03 +01:00
Lucas WintherandClaude Sonnet 5.5 66b155876f fix(core): atomic restore with aliases, snapshot export, hardened backups
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 20:24:10 +01:00
Lucas WintherandClaude Sonnet 5.5 a047020d54 feat(core): export bundle, restore, backups
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 20:17:25 +01:00
Lucas WintherandClaude Opus 5.5 33631aef3c merge: task 6 user layer
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 20:13:03 +01:00
Lucas WintherandClaude Opus 5.5 c6865b2522 fix(core): drill-down lists descendants under untracked intermediates
listEntries({ parentId: X }) now applies the top-level rule relative to X: this user's
live entries on descendants of X (up to three levels) with no live-entry work strictly
between them and X, so it agrees with childSummary. Restoring an engagement's re-parenting
onto a soft-deleted entry throws ValidationError("entryId", "target entry is deleted").

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 20:05:12 +01:00
Lucas WintherandClaude Opus 5.5 00fc0180b0 docs: record user-layer decisions, core module map and known gaps
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 20:00:14 +01:00
Lucas WintherandClaude Opus 5.5 7154ca0a61 fix(core): race-safe entry upsert and lossless per-user moves
- upsertEntry inserts with ON CONFLICT DO NOTHING and patches through the update path,
  so a concurrent creator never aborts the caller's transaction on Postgres.
- Undoing a move deletes the forward alias and writes no reverse alias.
- Merging into an existing entry keeps 'user' field_sources on combined fields, carries
  the moved entry's field_sources as-is when reviving a deleted target, concatenates
  differing review/private notes, records a dropped rating in history, and writes history
  for re-parented engagements (restorable).
- Moves re-point this user's copies (with history) and list items to the target work.
- rejectSuggestion requires a pending suggestion; engagements of a deleted entry are not
  editable; restoring an entry does not re-check custom fields against current definitions.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 19:59:54 +01:00
Lucas WintherandClaude Opus 5.5 512e6e9464 fix(core): roll up descendants and surface orphaned entries; SQL-derived title cursor
- childSummary counts the user's live entries up to three levels below the work.
- The default library view (parentId null) lists entries with no ancestor (up to three
  levels) carrying a live entry of this user, so episodes of an untracked show still show.
- The title keyset cursor carries the SQL lower(w.title) value, so SQLite no longer skips
  non-ASCII titles.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 19:59:09 +01:00
Lucas Winther 4c65939d61 merge: feat/phase1 into task/6 2026-10-30 19:54:33 +01:00
Lucas WintherandClaude Opus 5.5 d98b2b9909 chore: ignore agent worktree directory
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 19:37:04 +01:00
Lucas WintherandClaude Opus 5.5 a13d9cd476 merge: task 5 works and matching
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 19:35:57 +01:00
Lucas WintherandClaude Opus 5.5 c7471f7761 feat(core): engagements, lists and copies with history restore
Engagements store partial dates as value + precision and track field_sources;
lists (list/grouping/thread, grouping-only rating/review) keep dense item
order and snapshot items in history; copies store price in cents. All three
restore through restoreVersion.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 19:35:15 +01:00
Lucas WintherandClaude Opus 5.5 13b81b1cd5 feat(core): entries, user history and match suggestion accept/reject
Entries carry per-field field_sources, write user_history diffs, resolve
aliased/merged works, and support quick add, per-user moves (with alias and
draft tombstone) and keyset-paginated library listing with child rollups.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 19:34:47 +01:00
Lucas WintherandClaude Opus 5.5 50c115b4c7 fix(core): follow merged tombstones in the position lookup
A merged child still holds its (parent_id, position) slot under the unique
index, so skipping tombstones made every retry collide. Follow the tombstone
to the surviving work instead.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 19:32:27 +01:00
Lucas WintherandClaude Opus 5.5 437ca143e5 fix(core): race-safe matching, visibility-checked ID matches, child-preserving merges
- migration 0002: unique index on works (parent_id, position)
- matchWork retries the whole transaction on a unique violation (3 attempts)
- addMissingExternalIds uses ON CONFLICT DO NOTHING
- ID matches only return shared works or the user's own; private works cannot carry external ids
- adminMergeWorks requires shared works and repoints non-colliding children
- draft is computed for the aliased work id

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 19:29:48 +01:00
Lucas WintherandClaude Opus 5.5 60876c2930 merge: task 7 attachments and crypto
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 19:27:30 +01:00
Lucas WintherandClaude Sonnet 5.5 4acb368c87 fix(core): serialise attachment file mutations, atomic writes, reject svg
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 19:26:16 +01:00
Lucas WintherandClaude Opus 5.5 c7effb5f0a merge: task 4 auth
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:59:31 +01:00
Lucas WintherandClaude Opus 5.5 56b86e27da feat(core): custom field definitions and value validation
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:56:51 +01:00
Lucas WintherandClaude Sonnet 5.5 fe3c9e4881 fix(core): serialise first-user registration, equalise login timing
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 15:56:36 +01:00
Lucas WintherandClaude Sonnet 5.5 a46fb60eee feat(core): works catalogue and matching
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 15:56:05 +01:00
Lucas WintherandClaude Sonnet 5.5 e1faa7433b feat(core): users, sessions, invites, app tokens
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 15:54:47 +01:00
Lucas WintherandClaude Opus 5.5 83940a3b31 merge: task 14
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:54:39 +01:00
Lucas WintherandClaude Opus 5.5 92f7c9a461 merge: task 13
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:54:39 +01:00
Lucas WintherandClaude Opus 5.5 94877c1e6e merge: task 10
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:54:39 +01:00
Lucas WintherandClaude Opus 5.5 8f58ba4cdc feat(core): attachments and credential encryption
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:54:33 +01:00
Lucas WintherandClaude Opus 5.5 a02fd808a4 merge: task 12
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:53:35 +01:00
Lucas WintherandClaude Opus 5.5 370275090f merge: task 11
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:53:35 +01:00
Lucas WintherandClaude Opus 5.5 d392a65aa7 merge: task 2 schema
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:52:10 +01:00
Lucas WintherandClaude Sonnet 5.5 00742937c9 fix(plugins): walk listenbrainz listens backwards with max_ts
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 15:51:58 +01:00
Lucas WintherandClaude Sonnet 5.5 9708fb5e70 fix(plugins): type-specific trakt external id namespaces
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 15:51:58 +01:00
Lucas WintherandClaude Sonnet 5.5 4c08c16e48 fix(plugins): unique csv row keys, per-file default mediaType, scale-aware zero rating
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 15:51:22 +01:00
Lucas WintherandClaude Opus 5.5 2573c83f70 fix(plugins): mangabaka type-specific external id namespaces
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:50:51 +01:00
Lucas WintherandClaude Sonnet 5.5 26d021af48 fix(plugins): test hardcover lookup and all status mappings
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 15:50:44 +01:00
Lucas WintherandClaude Opus 5.5 d9f935d8a8 test(plugins): fix mangabaka upsert-without-entry fixture
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:50:44 +01:00
Lucas WintherandClaude Opus 5.5 f30c0db2b1 fix(plugins): mangabaka never deletes on upsert without entry
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:50:32 +01:00
Lucas WintherandClaude Sonnet 5.5 59ccc90bc1 feat(plugins): trakt
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 15:49:47 +01:00
Lucas WintherandClaude Sonnet 5.5 3eec2f2184 feat(plugins): listenbrainz and lastfm
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 15:49:31 +01:00
Lucas WintherandClaude Sonnet 5.5 ff595c1c5f fix(db): add down migration, template-DB test harness, EntryStatus type
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 15:49:06 +01:00
Lucas WintherandClaude Sonnet 5.5 a540c7d34d feat(plugins): csv and json import
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 15:48:59 +01:00
Lucas WintherandClaude Sonnet 5.5 c09968086f feat(plugins): hardcover
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 15:48:46 +01:00
Lucas WintherandClaude Opus 5.5 ddd938935e feat(plugins): mangabaka
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:48:39 +01:00
Lucas WintherandClaude Sonnet 5.5 da04adb286 feat(db): initial schema
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
2026-10-30 15:39:14 +01:00
Lucas WintherandClaude Opus 5.5 06a59e1b5c fix(sdk): block real network in test context, strip query from HttpError message
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:38:45 +01:00
Lucas WintherandClaude Opus 5.5 a14c4e7747 feat(sdk): plugin contract, http client, fixture harness
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:36:55 +01:00
Lucas WintherandClaude Opus 5.5 27ec24eb4b feat(core): shared domain error classes
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:30:49 +01:00
Lucas WintherandClaude Opus 5.5 8248a2ac27 feat(db): Bun.SQL Kysely dialect for sqlite and postgres
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:30:49 +01:00
Lucas WintherandClaude Opus 5.5 6c5a9fbad9 chore: workspace package stubs with approved dependencies
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:30:49 +01:00
Lucas WintherandClaude Opus 5.5 fb4ff36602 docs: phase 1 implementation plan
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 15:24:48 +01:00
183 changed files with 20312 additions and 12 deletions
+1
View File
@@ -23,3 +23,4 @@ CLAUDE.local.md
.DS_Store
.idea/
.vscode/
.claude/worktrees/
+1 -1
View File
@@ -47,7 +47,7 @@ Use impeccable for user-facing UI: `craft` for new UI, `audit`/`polish` on exist
- Plugins (`plugins/*`) import only `@sttracker/plugin-sdk`. They never import `packages/db` or core
internals, and they return `SourceRecord`s instead of writing.
- `packages/*` and `plugins/*` never import from `apps/*`.
- IDs are ULIDs generated in app code. Ratings are integers 0–100. Dates carry a precision
- IDs are UUIDv7 (`newId()`) generated in app code. Ratings are integers 0–100. Dates carry a precision
(`day | month | year | unknown`).
- Tests never call live APIs. Plugin tests use recorded fixtures through the SDK test harness.
- Write-back (`push`) is opt-in per connection and off by default. No plugin pushes in Phase 1.
+4
View File
@@ -23,6 +23,10 @@ Dark, calm, dense. Three panes on desktop (nav · library list · entry detail);
- Type: Inter for the UI; Gelasio for long-form review text.
- Radii: 8px for controls, 10px for boxes, 999px for pills, 3–5px for covers.
- Orange means "a platform changed something". Never use it decoratively.
- Text on `--warn` (the Changes count, the "See what changed" button) is dark `#1c0f04` (6.4:1), never white: white on `#f97316` is 2.8:1 and fails WCAG AA.
- Sidebar group labels ("Sources") use `--text-faint`. The mockup's `#6f69a3` is 3.6:1 on `--surface-0` and fails AA.
- Placeholders on `--surface-3` inputs use `--text-muted`: `--text-faint` there is 4.15:1.
- Fonts are self-hosted in `apps/web/src/fonts/` (Inter, Gelasio; both OFL). The app makes no third-party font requests.
- Drafts (entries with no catalogue link) get a small amber outlined "Draft" badge and a striped placeholder cover.
## Media type colours (shared by both surfaces)
+15
View File
@@ -0,0 +1,15 @@
{
"name": "@sttracker/server",
"private": true,
"type": "module",
"exports": {
"./*": "./src/*.ts"
},
"dependencies": {
"@sttracker/core": "workspace:*",
"@sttracker/db": "workspace:*",
"@sttracker/plugin-sdk": "workspace:*",
"kysely": "^0.29.7",
"zod": "^4.6.7"
}
}
+49
View File
@@ -0,0 +1,49 @@
import { apiHandler, errorResponse, type AppDeps, type RouteDef } from "./http.ts";
import { attachmentRoutes } from "./routes/attachments.ts";
import { authRoutes } from "./routes/auth.ts";
import { changeRoutes } from "./routes/changes.ts";
import { copyRoutes } from "./routes/copies.ts";
import { customFieldRoutes } from "./routes/custom-fields.ts";
import { engagementRoutes } from "./routes/engagements.ts";
import { entryRoutes } from "./routes/entries.ts";
import { exportRoutes } from "./routes/export.ts";
import { listRoutes } from "./routes/lists.ts";
import { metaRoutes } from "./routes/meta.ts";
import { sourceRoutes } from "./routes/sources.ts";
import { workRoutes } from "./routes/works.ts";
export interface App {
/** Handle `/api/v1/*` and `/ingest/*`. Anything else is a JSON 404 (the web app is served by `Bun.serve` routes). */
fetch(req: Request, server?: { requestIP(req: Request): { address: string } | null }): Promise<Response>;
routes: RouteDef[];
}
/** The HTTP API, testable without a listening socket. */
export function createApp(deps: AppDeps): App {
const routes: RouteDef[] = [
...authRoutes,
...entryRoutes,
...engagementRoutes,
...workRoutes,
...listRoutes,
...copyRoutes,
...customFieldRoutes,
...attachmentRoutes,
...sourceRoutes,
...changeRoutes,
...exportRoutes,
];
routes.push(...metaRoutes(() => routes));
const api = apiHandler(routes, deps);
return {
routes,
async fetch(req, server) {
const { pathname } = new URL(req.url);
if (pathname.startsWith("/api/v1/")) return api(req, server?.requestIP(req)?.address ?? "unknown");
// Phase 1: the ingest mount exists but no plugin has the `ingest` capability.
if (pathname.startsWith("/ingest/")) return errorResponse(404, "not_found", "No ingest endpoint for this plugin.");
return errorResponse(404, "not_found", "No such endpoint.");
},
};
}
+18
View File
@@ -0,0 +1,18 @@
import { enqueue } from "@sttracker/core/jobs";
import type { Db } from "@sttracker/db/db";
/** The next 03:00 local time strictly after `now`. */
export function nextBackupAt(now: Date = new Date()): Date {
const d = new Date(now);
d.setHours(3, 0, 0, 0);
if (d.getTime() <= now.getTime()) d.setDate(d.getDate() + 1);
return d;
}
/**
* Queue the daily `backup` job for the next 03:00 unless one is already queued or running (lockKey "backup").
* Returns the job id, or null when one is pending.
*/
export function ensureBackupScheduled(db: Db, now: Date = new Date()): Promise<string | null> {
return enqueue(db, "backup", {}, { lockKey: "backup", runAt: nextBackupAt(now).toISOString() });
}
+30
View File
@@ -0,0 +1,30 @@
export interface Config {
databaseUrl: string;
dataDir: string;
secret: string;
publicUrl: string | undefined;
port: number;
pluginDirs: string[];
}
export const MISSING_SECRET = "Set STTRACKER_SECRET to a long random string. It encrypts source credentials.";
/** Read the server config from the environment. Throws when `STTRACKER_SECRET` is missing. */
export function loadConfig(env: Record<string, string | undefined> = process.env): Config {
const secret = env.STTRACKER_SECRET;
if (!secret) throw new Error(MISSING_SECRET);
const dataDir = env.DATA_DIR || "./data";
const port = Number(env.PORT || 3000);
if (!Number.isInteger(port) || port < 0 || port > 65535) throw new Error(`PORT must be a port number, got "${env.PORT}"`);
return {
databaseUrl: env.DATABASE_URL || `sqlite://${dataDir}/sttracker.db`,
dataDir,
secret,
publicUrl: env.PUBLIC_URL || undefined,
port,
pluginDirs: (env.PLUGIN_DIRS || `plugins,${dataDir}/plugins`)
.split(",")
.map((d) => d.trim())
.filter((d) => d !== ""),
};
}
+336
View File
@@ -0,0 +1,336 @@
import { authenticateAppToken, getSessionUser, verifyAppToken, type User } from "@sttracker/core/auth";
import { AuthError, ImportError, NotFoundError, ValidationError } from "@sttracker/core/errors";
import type { Actor } from "@sttracker/core/history";
import { isUniqueViolation } from "@sttracker/core/matching";
import type { PluginRegistry } from "@sttracker/core/registry";
import type { Db } from "@sttracker/db/db";
import type { TokenScope } from "@sttracker/db/schema";
import { z } from "zod";
export type AuthMode = "none" | "session" | "token:read" | "token:write" | "any:read" | "any:write";
export type Method = "GET" | "POST" | "PATCH" | "DELETE";
export interface AppDeps {
db: Db;
registry: PluginRegistry;
secret: string;
dataDir: string;
/** When it starts with `https:`, the session cookie is `Secure`. */
publicUrl?: string;
}
export const SESSION_COOKIE = "stt_session";
export const CSRF_HEADER = "x-requested-with";
export const CSRF_VALUE = "sttracker";
export const MAX_JSON_BYTES = 1024 * 1024;
export const MAX_UPLOAD_BYTES = 50 * 1024 * 1024;
const SESSION_MAX_AGE_S = 30 * 24 * 60 * 60;
export interface Ctx<B, Q> {
req: Request;
url: URL;
params: Record<string, string>;
body: B;
query: Q;
/** The authenticated user. Throws on `auth: "none"` routes, which have none. */
user: User;
/** Who writes: `token:<name>` for Bearer requests, `"user"` for sessions. Pass it to every core write. */
actor: Actor;
/** The session token, when the request was authenticated by cookie. */
sessionToken: string | null;
/** Parsed multipart body, for `multipart: true` routes. */
form: FormData;
deps: AppDeps;
ip: string;
}
export interface RouteDef<B = any, Q = any> {
method: Method;
/** Relative to `/api/v1`, with `:param` segments. */
path: string;
summary: string;
auth: AuthMode;
body?: z.ZodType<B>;
query?: z.ZodType<Q>;
/** Parse the body as multipart form data (upload limit) instead of JSON. */
multipart?: boolean;
/** Non-file multipart fields, for the OpenAPI document (handlers validate them). Every multipart route also takes `file`. */
formFields?: z.ZodObject;
/** Rate limit per client IP (login, register). */
rateLimited?: boolean;
/** Only admins may call it (403 otherwise). */
admin?: boolean;
handler(ctx: Ctx<B, Q>): Promise<unknown>;
}
/** Declare a route; returns it unchanged with the handler's body and query types inferred from the schemas. */
export function route<BS extends z.ZodType = z.ZodUnknown, QS extends z.ZodType = z.ZodUnknown>(
def: Omit<RouteDef<z.output<BS>, z.output<QS>>, "body" | "query"> & { body?: BS; query?: QS },
): RouteDef {
return def as RouteDef;
}
/** Throw from a handler to answer with a specific status and error code. */
export class HttpError extends Error {
constructor(
readonly status: number,
readonly code: string,
message: string,
) {
super(message);
}
}
export function json(data: unknown, status = 200, headers: Record<string, string> = {}): Response {
return new Response(JSON.stringify(data), { status, headers: { "content-type": "application/json; charset=utf-8", ...headers } });
}
export function errorResponse(status: number, code: string, message: string, headers: Record<string, string> = {}): Response {
return json({ error: { code, message } }, status, headers);
}
export const noContent = (): Response => new Response(null, { status: 204 });
const AUTH_STATUS: Record<string, [number, string, string]> = {
invalid_credentials: [401, "unauthenticated", "Wrong username or password."],
unauthenticated: [401, "unauthenticated", "Sign in first."],
forbidden: [403, "forbidden", "You are not allowed to do that."],
invite_required: [400, "invite_required", "Registration needs a valid invite code."],
username_taken: [400, "username_taken", "That username is taken."],
invalid_username: [400, "invalid_username", "Usernames are 2–32 characters: lowercase letters, digits and underscores."],
};
/** Map a thrown error to the JSON error response (ruling 5). Unknown errors are logged and hidden. */
export function toErrorResponse(e: unknown): Response {
if (e instanceof HttpError) return errorResponse(e.status, e.code, e.message);
if (e instanceof z.ZodError) {
const msg = e.issues.map((i) => `${i.path.length ? i.path.join(".") : "body"}: ${i.message}`).join("; ");
return errorResponse(400, "validation", msg);
}
if (e instanceof ValidationError) return errorResponse(400, "validation", `${e.field}: ${e.reason}`);
if (e instanceof NotFoundError) return errorResponse(404, "not_found", e.message);
if (e instanceof AuthError) {
const [status, code, message] = AUTH_STATUS[e.code] ?? [403, "forbidden", "You are not allowed to do that."];
return errorResponse(status, code, message);
}
if (e instanceof ImportError) {
const status = e.code === "account_not_empty" || e.code === "conflict" ? 409 : 400;
return errorResponse(status, e.code, `Import failed: ${e.code}.`);
}
if (isUniqueViolation(e)) return errorResponse(409, "conflict", "That already exists.");
console.error("[api]", e);
return errorResponse(500, "internal", "Something went wrong on the server.");
}
export function sessionCookie(token: string, publicUrl: string | undefined): string {
const secure = publicUrl?.startsWith("https:") ? "; Secure" : "";
return `${SESSION_COOKIE}=${token}; Path=/; HttpOnly; SameSite=Lax; Max-Age=${SESSION_MAX_AGE_S}${secure}`;
}
export function clearSessionCookie(publicUrl: string | undefined): string {
const secure = publicUrl?.startsWith("https:") ? "; Secure" : "";
return `${SESSION_COOKIE}=; Path=/; HttpOnly; SameSite=Lax; Max-Age=0${secure}`;
}
function readCookie(req: Request, name: string): string | null {
const header = req.headers.get("cookie");
if (!header) return null;
for (const part of header.split(";")) {
const i = part.indexOf("=");
if (i > 0 && part.slice(0, i).trim() === name) {
try {
return decodeURIComponent(part.slice(i + 1).trim()) || null;
} catch {
return null; // malformed percent-encoding: treat as no cookie
}
}
}
return null;
}
/** Read a request body, failing with 413 once it exceeds `max` bytes (whatever Content-Length claims). */
async function readLimited(req: Request, max: number): Promise<Uint8Array<ArrayBuffer>> {
const tooLarge = () => new HttpError(413, "validation", `Request body is too large (max ${max / (1024 * 1024)} MB).`);
const declared = Number(req.headers.get("content-length") ?? NaN);
if (declared > max) throw tooLarge();
if (!req.body) return new Uint8Array();
const reader = req.body.getReader();
const chunks: Uint8Array[] = [];
let total = 0;
for (;;) {
const { done, value } = await reader.read();
if (done) break;
total += value.byteLength;
if (total > max) {
await reader.cancel();
throw tooLarge();
}
chunks.push(value);
}
const out = new Uint8Array(total);
let at = 0;
for (const c of chunks) {
out.set(c, at);
at += c.byteLength;
}
return out;
}
/** Fixed-window-ish limiter: at most `limit` hits per `windowMs` per key. */
export function rateLimiter(limit: number, windowMs: number): (key: string) => boolean {
const hits = new Map<string, number[]>();
return (key) => {
const now = Date.now();
if (hits.size > 10_000) for (const [k, v] of hits) if (v.every((t) => now - t >= windowMs)) hits.delete(k);
const recent = (hits.get(key) ?? []).filter((t) => now - t < windowMs);
if (recent.length >= limit) {
hits.set(key, recent);
return false;
}
recent.push(now);
hits.set(key, recent);
return true;
};
}
const SCOPES: TokenScope[] = ["read", "write", "ingest"];
/** Resolve the caller for `mode`. Returns the user and the session token used, if any. */
async function authenticate(
req: Request,
mode: AuthMode,
db: Db,
): Promise<{ user: User | null; sessionToken: string | null; actor: Actor }> {
if (mode === "none") return { user: null, sessionToken: null, actor: "user" };
const authz = req.headers.get("authorization");
const bearer = authz?.startsWith("Bearer ") ? authz.slice(7).trim() : null;
const [kind, scopeName] = mode.split(":") as ["session" | "token" | "any", TokenScope | undefined];
if (kind !== "session" && bearer) {
const scope = scopeName!;
const auth = await authenticateAppToken(db, bearer, scope);
if (auth) return { user: auth.user, sessionToken: null, actor: `token:${auth.tokenName}` };
for (const other of SCOPES) {
if (other !== scope && (await verifyAppToken(db, bearer, other))) {
throw new HttpError(403, "forbidden", `This app token lacks the '${scope}' scope.`);
}
}
throw new HttpError(401, "unauthenticated", "Invalid or revoked app token.");
}
if (kind === "token") throw new HttpError(401, "unauthenticated", "Send an app token: Authorization: Bearer <token>.");
const token = readCookie(req, SESSION_COOKIE);
const user = token ? await getSessionUser(db, token) : null;
if (!user) throw new HttpError(401, "unauthenticated", "Sign in first.");
return { user, sessionToken: token, actor: "user" };
}
interface Compiled {
def: RouteDef;
segments: string[];
}
function matchSegments(pattern: string[], parts: string[]): Record<string, string> | null {
if (pattern.length !== parts.length) return null;
const params: Record<string, string> = {};
for (let i = 0; i < pattern.length; i++) {
const p = pattern[i]!;
const v = parts[i]!;
if (p.startsWith(":")) {
if (v === "") return null;
try {
params[p.slice(1)] = decodeURIComponent(v);
} catch {
return null;
}
} else if (p !== v) return null;
}
return params;
}
/** Build the `/api/v1/*` request handler over a route table. */
export function apiHandler(routes: RouteDef[], deps: AppDeps, prefix = "/api/v1/"): (req: Request, ip: string) => Promise<Response> {
const compiled: Compiled[] = routes.map((def) => ({ def, segments: def.path.split("/") }));
const limit = rateLimiter(10, 60_000);
return async (req, ip) => {
try {
const url = new URL(req.url);
const parts = url.pathname.slice(prefix.length).replace(/\/$/, "").split("/");
let found: { def: RouteDef; params: Record<string, string> } | null = null;
for (const c of compiled) {
if (c.def.method !== req.method) continue;
const params = matchSegments(c.segments, parts);
if (params) {
found = { def: c.def, params };
break;
}
}
if (!found) return errorResponse(404, "not_found", "No such endpoint.");
const { def, params } = found;
if (def.rateLimited && !limit(`${def.path}|${ip}`)) {
return errorResponse(429, "rate_limited", "Too many attempts. Wait a minute and try again.", { "retry-after": "60" });
}
// CSRF: every non-Bearer write (login and register included) must carry this header. Browsers cannot add it
// cross-site without a CORS preflight, which the server never grants. Bearer requests are not ambient, so exempt.
const bearer = req.headers.get("authorization")?.startsWith("Bearer ") ?? false;
if (req.method !== "GET" && req.method !== "HEAD" && !bearer && req.headers.get(CSRF_HEADER) !== CSRF_VALUE) {
throw new HttpError(403, "forbidden", `Browser requests that change data must send the header X-Requested-With: ${CSRF_VALUE}.`);
}
const { user, sessionToken, actor } = await authenticate(req, def.auth, deps.db);
if (def.admin && user?.role !== "admin") throw new HttpError(403, "forbidden", "Only an admin can do that.");
let body: unknown;
let form: FormData | undefined;
if (def.multipart) {
const type = req.headers.get("content-type") ?? "";
if (!type.startsWith("multipart/form-data")) throw new HttpError(400, "validation", "body: expected multipart/form-data");
const bytes = await readLimited(req, MAX_UPLOAD_BYTES);
try {
form = await new Response(bytes, { headers: { "content-type": type } }).formData();
} catch {
throw new HttpError(400, "validation", "body: malformed multipart form data");
}
} else if (def.body) {
const bytes = await readLimited(req, MAX_JSON_BYTES);
let raw: unknown;
if (bytes.byteLength > 0) {
const type = (req.headers.get("content-type") ?? "").split(";")[0]!.trim().toLowerCase();
if (type !== "application/json") throw new HttpError(415, "unsupported_media_type", "Send the body as Content-Type: application/json.");
try {
raw = JSON.parse(new TextDecoder().decode(bytes));
} catch {
throw new HttpError(400, "validation", "body: invalid JSON");
}
}
body = def.body.parse(raw);
}
const query = def.query ? def.query.parse(Object.fromEntries(url.searchParams)) : undefined;
const ctx: Ctx<unknown, unknown> = {
req,
url,
params,
body,
query,
get user(): User {
if (!user) throw new Error(`route ${def.method} ${def.path} has no authenticated user`);
return user;
},
actor,
sessionToken,
get form(): FormData {
if (!form) throw new Error(`route ${def.method} ${def.path} is not multipart`);
return form;
},
deps,
ip,
};
const out = await def.handler(ctx);
return out instanceof Response ? out : json(out);
} catch (e) {
return toErrorResponse(e);
}
};
}
+68
View File
@@ -0,0 +1,68 @@
import { mkdir } from "node:fs/promises";
import { join } from "node:path";
import { backupHandler } from "@sttracker/core/backup";
import { syncHandler } from "@sttracker/core/connections";
import { startScheduler, startWorker } from "@sttracker/core/jobs";
import { loadPlugins } from "@sttracker/core/registry";
import { createDb } from "@sttracker/db/db";
import { migrateToLatest } from "@sttracker/db/migrate";
import index from "../../web/index.html";
import { createApp } from "./app.ts";
import { ensureBackupScheduled } from "./boot.ts";
import { loadConfig } from "./config.ts";
const BACKUP_CHECK_MS = 60 * 60 * 1000;
const config = loadConfig();
await mkdir(join(config.dataDir, "plugins"), { recursive: true });
const db = await createDb(config.databaseUrl);
await migrateToLatest(db);
const registry = await loadPlugins(config.pluginDirs);
const ensureBackup = () => ensureBackupScheduled(db).catch((e) => console.error("[backup] scheduling failed", e));
const runBackup = backupHandler({ db, dataDir: config.dataDir });
const worker = startWorker(db, {
sync: syncHandler({ db, secret: config.secret, registry }),
backup: async () => {
try {
await runBackup();
} finally {
// The finishing job still holds lockKey "backup" until the worker marks it done; queue the next one shortly after.
setTimeout(() => void ensureBackup(), 5_000);
}
},
});
const scheduler = startScheduler(db);
await ensureBackup();
// Safety net: re-queue the daily backup if a run's follow-up was missed (e.g. it failed for good).
const backupTimer = setInterval(() => void ensureBackup(), BACKUP_CHECK_MS);
const app = createApp({ db, registry, secret: config.secret, dataDir: config.dataDir, publicUrl: config.publicUrl });
const server = Bun.serve({
port: config.port,
// Production unless asked: dev mode adds HMR and verbose error pages.
// `bun --hot` puts the flag in execArgv, not Bun.argv.
development: process.env.NODE_ENV === "development" || process.execArgv.includes("--hot"),
routes: {
"/api/*": (req, srv) => app.fetch(req, srv),
"/ingest/*": (req, srv) => app.fetch(req, srv),
"/*": index,
},
fetch: (req, srv) => app.fetch(req, srv),
});
console.log(`STTracker listening on ${server.url} (${db.dialect}, ${registry.list().length} plugin(s))`);
let stopping = false;
async function shutdown(signal: string): Promise<void> {
if (stopping) return;
stopping = true;
console.log(`${signal}: shutting down`);
clearInterval(backupTimer);
scheduler.stop();
await server.stop();
await worker.stop();
await db.close();
process.exit(0);
}
process.on("SIGTERM", () => void shutdown("SIGTERM"));
process.on("SIGINT", () => void shutdown("SIGINT"));
+72
View File
@@ -0,0 +1,72 @@
import { z } from "zod";
import type { RouteDef } from "./http.ts";
type JsonSchema = Record<string, unknown>;
const toSchema = (s: z.ZodType): JsonSchema => {
const { $schema: _, ...rest } = z.toJSONSchema(s, { io: "input", unrepresentable: "any" }) as JsonSchema;
return rest;
};
const SECURITY: Record<RouteDef["auth"], { [k: string]: string[] }[]> = {
none: [],
session: [{ session: [] }],
"token:read": [{ appToken: ["read"] }],
"token:write": [{ appToken: ["write"] }],
"any:read": [{ session: [] }, { appToken: ["read"] }],
"any:write": [{ session: [] }, { appToken: ["write"] }],
};
/** OpenAPI 3.1 document generated from the route table (paths, summaries, request schemas). */
export function buildOpenApi(routes: RouteDef[], prefix = "/api/v1"): JsonSchema {
const paths: Record<string, Record<string, JsonSchema>> = {};
for (const r of routes) {
const path = `${prefix}/${r.path.replace(/:(\w+)/g, "{$1}")}`;
const parameters: JsonSchema[] = [...r.path.matchAll(/:(\w+)/g)].map((m) => ({
name: m[1],
in: "path",
required: true,
schema: { type: "string" },
}));
if (r.query) {
const q = toSchema(r.query) as { properties?: Record<string, JsonSchema>; required?: string[] };
for (const [name, schema] of Object.entries(q.properties ?? {})) {
parameters.push({ name, in: "query", required: q.required?.includes(name) ?? false, schema });
}
}
const op: JsonSchema = {
operationId: `${r.method.toLowerCase()}_${r.path.replace(/[^\w]+/g, "_")}`,
summary: r.summary,
security: SECURITY[r.auth],
responses: {
default: {
description: "Success, or an error: `{ error: { code, message } }`.",
},
},
};
if (parameters.length > 0) op.parameters = parameters;
if (r.body) op.requestBody = { required: !r.body.safeParse(undefined).success, content: { "application/json": { schema: toSchema(r.body) } } };
if (r.multipart) {
const fields = r.formFields ? (toSchema(r.formFields) as { properties?: JsonSchema; required?: string[] }) : {};
const schema = {
type: "object",
properties: { file: { type: "string", format: "binary" }, ...fields.properties },
required: ["file", ...(fields.required ?? [])],
};
op.requestBody = { required: true, content: { "multipart/form-data": { schema } } };
}
(paths[path] ??= {})[r.method.toLowerCase()] = op;
}
return {
openapi: "3.1.0",
info: { title: "STTracker API", version: "1" },
servers: [{ url: "/" }],
components: {
securitySchemes: {
session: { type: "apiKey", in: "cookie", name: "stt_session", description: "Non-GET requests also need X-Requested-With: sttracker." },
appToken: { type: "http", scheme: "bearer", description: "App token with scopes read, write or ingest." },
},
},
paths,
};
}
+89
View File
@@ -0,0 +1,89 @@
import { listAttachmentLinks, listAttachments, readAttachment, requireLinkTarget, storeAttachment, unlinkAttachment } from "@sttracker/core/attachments";
import { NotFoundError } from "@sttracker/core/errors";
import type { AttachmentEntity } from "@sttracker/db/schema";
import { z } from "zod";
import { json, noContent, route } from "../http.ts";
import { formFile } from "./form.ts";
const fields = z.object({
entity: z.enum(["entry", "engagement", "copy", "list", "work"]),
entityId: z.string().min(1),
caption: z.string().max(10_000).optional(),
});
/** A filename safe inside a quoted Content-Disposition value. */
export function safeFilename(name: string): string {
const cleaned = name.replace(/[^\w.\- ]/g, "_").slice(0, 200).trim();
return cleaned === "" ? "file" : cleaned;
}
export const attachmentRoutes = [
route({
method: "GET",
path: "attachments",
summary: "Attachments linked to one of your records (oldest first), each with its links.",
auth: "any:read",
query: z.object({ entity: fields.shape.entity, entityId: z.string().min(1) }),
handler: async ({ deps, user, query }) => {
await requireLinkTarget(deps.db, user.id, query.entity as AttachmentEntity, query.entityId);
const atts = await listAttachments(deps.db, user.id, query.entity as AttachmentEntity, query.entityId);
return Promise.all(atts.map(async (a) => ({ ...a, links: await listAttachmentLinks(deps.db, user.id, a.id) })));
},
}),
route({
method: "POST",
path: "attachments",
summary: "Upload a file (multipart: file, entity, entityId, caption) and link it to one of your records.",
auth: "any:write",
multipart: true,
formFields: fields,
handler: async ({ deps, user, form }) => {
const file = formFile(form);
const f = fields.parse({
entity: form.get("entity") ?? undefined,
entityId: form.get("entityId") ?? undefined,
caption: form.get("caption") ?? undefined,
});
await requireLinkTarget(deps.db, user.id, f.entity as AttachmentEntity, f.entityId);
const att = await storeAttachment(
deps.db,
deps.dataDir,
user.id,
{ bytes: new Uint8Array(await file.arrayBuffer()), mime: file.type || "application/octet-stream", filename: file.name, caption: f.caption },
{ entity: f.entity as AttachmentEntity, entityId: f.entityId },
);
return json({ ...att, links: await listAttachmentLinks(deps.db, user.id, att.id) }, 201);
},
}),
route({
method: "GET",
path: "attachments/:id",
summary: "Download an attachment's file.",
auth: "any:read",
handler: async ({ deps, user, params }) => {
const a = await readAttachment(deps.db, deps.dataDir, user.id, params.id!);
if (!a || !(await a.file.exists())) throw new NotFoundError("attachment");
return new Response(a.file, {
headers: {
"content-type": a.mime,
"x-content-type-options": "nosniff",
"content-disposition": `inline; filename="${safeFilename(a.filename)}"`,
"content-security-policy": "default-src 'none'; sandbox",
"cache-control": "private, max-age=3600",
},
});
},
}),
route({
method: "DELETE",
path: "attachments/:id/links/:linkId",
summary: "Unlink an attachment; it is deleted when no links remain.",
auth: "any:write",
handler: async ({ deps, user, params }) => {
const link = (await listAttachmentLinks(deps.db, user.id, params.id!)).find((l) => l.id === params.linkId);
if (!link) throw new NotFoundError("attachment link");
await unlinkAttachment(deps.db, deps.dataDir, user.id, params.id!, link.entity, link.entityId);
return noContent();
},
}),
];
+102
View File
@@ -0,0 +1,102 @@
import { createAppToken, createInvite, hasUsers, listAppTokens, login, logout, registerUser, revokeAppToken } from "@sttracker/core/auth";
import { getPreferences, RATING_SCALES, updatePreferences } from "@sttracker/core/preferences";
import { NotFoundError } from "@sttracker/core/errors";
import { z } from "zod";
import { clearSessionCookie, json, noContent, route, sessionCookie } from "../http.ts";
import { visibility } from "./schemas.ts";
const credentials = z.object({
username: z.string().min(1).max(64),
password: z.string().min(8).max(1024),
});
export const authRoutes = [
route({
method: "GET",
path: "auth/status",
summary: "Whether any account exists yet (the web app shows first-run registration when not).",
auth: "none",
handler: async ({ deps }) => ({ hasUsers: await hasUsers(deps.db) }),
}),
route({
method: "POST",
path: "auth/register",
summary: "Create an account. The first user becomes admin; later ones need an invite code.",
auth: "none",
rateLimited: true,
body: credentials.extend({ displayName: z.string().min(1).max(100).optional(), inviteCode: z.string().max(200).optional() }).strict(),
handler: async ({ deps, body }) => json(await registerUser(deps.db, body), 201),
}),
route({
method: "POST",
path: "auth/login",
summary: "Sign in; sets the stt_session cookie.",
auth: "none",
rateLimited: true,
body: z.object({ username: z.string().max(64), password: z.string().max(1024) }).strict(),
handler: async ({ deps, body }) => {
const { user, sessionToken } = await login(deps.db, body.username, body.password);
return json(user, 200, { "set-cookie": sessionCookie(sessionToken, deps.publicUrl) });
},
}),
route({
method: "POST",
path: "auth/logout",
summary: "End the current session and clear the cookie.",
auth: "session",
handler: async ({ deps, sessionToken }) => {
if (sessionToken) await logout(deps.db, sessionToken);
return new Response(null, { status: 204, headers: { "set-cookie": clearSessionCookie(deps.publicUrl) } });
},
}),
route({ method: "GET", path: "me", summary: "The signed-in user.", auth: "any:read", handler: async ({ user }) => user }),
route({
method: "GET",
path: "me/preferences",
summary: "Your preferences: rating scale and default visibility.",
auth: "any:read",
handler: async ({ deps, user }) => getPreferences(deps.db, user.id),
}),
route({
method: "PATCH",
path: "me/preferences",
summary: "Update your preferences (partial).",
auth: "any:write",
body: z.object({ ratingScale: z.enum(RATING_SCALES), defaultVisibility: visibility }).partial().strict(),
handler: async ({ deps, user, body }) => updatePreferences(deps.db, user.id, body as Parameters<typeof updatePreferences>[2]),
}),
route({
method: "GET",
path: "me/tokens",
summary: "List your app tokens (metadata only).",
auth: "session",
handler: async ({ deps, user }) => listAppTokens(deps.db, user.id),
}),
route({
method: "POST",
path: "me/tokens",
summary: "Create an app token. The token is returned once.",
auth: "session",
body: z.object({ name: z.string().min(1).max(100), scopes: z.array(z.enum(["read", "write", "ingest"])).min(1) }).strict(),
handler: async ({ deps, user, body }) => json(await createAppToken(deps.db, user.id, body.name, [...new Set(body.scopes)]), 201),
}),
route({
method: "DELETE",
path: "me/tokens/:id",
summary: "Revoke an app token.",
auth: "session",
handler: async ({ deps, user, params }) => {
if (!(await revokeAppToken(deps.db, user.id, params.id!))) throw new NotFoundError("app token");
return noContent();
},
}),
route({
method: "POST",
path: "admin/invites",
summary: "Create an invite code (admin). The code is returned once.",
auth: "session",
admin: true,
body: z.object({ ttlDays: z.number().int().min(1).max(365).optional() }).strict().optional(),
handler: async ({ deps, user, body }) => json({ code: await createInvite(deps.db, user.id, body?.ttlDays) }, 201),
}),
];
+31
View File
@@ -0,0 +1,31 @@
import { listChanges, resolveChange } from "@sttracker/core/sync/changes";
import { z } from "zod";
import { noContent, route } from "../http.ts";
import { qInt } from "./schemas.ts";
export const changeRoutes = [
route({
method: "GET",
path: "changes",
summary: "The Changes inbox (pending by default; status=all for everything).",
auth: "any:read",
query: z.object({
status: z.enum(["pending", "all"]).optional(),
connectionId: z.string().optional(),
limit: qInt.optional(),
cursor: z.string().optional(),
}),
handler: async ({ deps, user, query }) => listChanges(deps.db, user.id, query),
}),
route({
method: "POST",
path: "changes/:id/resolve",
summary: "Resolve a pending change: accept, ignore or keep_mine.",
auth: "any:write",
body: z.object({ action: z.enum(["accept", "ignore", "keep_mine"]) }).strict(),
handler: async ({ deps, user, params, body }) => {
await resolveChange(deps.db, user.id, params.id!, body.action);
return noContent();
},
}),
];
+58
View File
@@ -0,0 +1,58 @@
import { createCopy, listCopies, softDeleteCopy, updateCopy } from "@sttracker/core/copies";
import { z } from "zod";
import { json, noContent, route } from "../http.ts";
import { customFields, partialDate, text } from "./schemas.ts";
const copyPatch = z
.object({
workId: z.string().min(1).nullable(),
titleHint: z.string().max(1000).nullable(),
format: z.string().max(200).nullable(),
acquiredFrom: z.string().max(500).nullable(),
acquiredAt: partialDate.nullable(),
priceCents: z.number().int().nullable(),
currency: z.string().length(3).nullable(),
notes: text().nullable(),
customFields,
sourceRef: z.string().max(500).nullable(),
})
.partial()
.strict();
type CopyArg = Parameters<typeof updateCopy>[3];
export const copyRoutes = [
route({
method: "GET",
path: "copies",
summary: "Your owned copies (optionally of one work).",
auth: "any:read",
query: z.object({ workId: z.string().optional() }),
handler: async ({ deps, user, query }) => listCopies(deps.db, user.id, query),
}),
route({
method: "POST",
path: "copies",
summary: "Record an owned copy.",
auth: "any:write",
body: copyPatch,
handler: async ({ deps, user, body, actor }) => json(await createCopy(deps.db, user.id, body as CopyArg, actor), 201),
}),
route({
method: "PATCH",
path: "copies/:id",
summary: "Edit an owned copy.",
auth: "any:write",
body: copyPatch,
handler: async ({ deps, user, params, body, actor }) => updateCopy(deps.db, user.id, params.id!, body as CopyArg, actor),
}),
route({
method: "DELETE",
path: "copies/:id",
summary: "Soft-delete an owned copy.",
auth: "any:write",
handler: async ({ deps, user, params, actor }) => {
await softDeleteCopy(deps.db, user.id, params.id!, actor);
return noContent();
},
}),
];
+32
View File
@@ -0,0 +1,32 @@
import { CUSTOM_FIELD_TYPES, defineCustomField, listCustomFields } from "@sttracker/core/custom-fields";
import { z } from "zod";
import { json, route } from "../http.ts";
const level = z.enum(["entry", "engagement", "copy"]);
export const customFieldRoutes = [
route({
method: "GET",
path: "custom-fields",
summary: "Your custom field definitions.",
auth: "any:read",
query: z.object({ mediaType: z.string().optional(), level: level.optional() }),
handler: async ({ deps, user, query }) => listCustomFields(deps.db, user.id, query.mediaType, query.level),
}),
route({
method: "POST",
path: "custom-fields",
summary: "Define a custom field.",
auth: "any:write",
body: z
.object({
name: z.string().min(1).max(200),
type: z.enum(CUSTOM_FIELD_TYPES),
options: z.array(z.string().max(200)).max(500).optional(),
appliesTo: z.union([z.literal("all"), z.array(z.string().max(50))]).optional(),
level: level.optional(),
})
.strict(),
handler: async ({ deps, user, body }) => json(await defineCustomField(deps.db, user.id, body), 201),
}),
];
+55
View File
@@ -0,0 +1,55 @@
import { addEngagement, listEngagements, softDeleteEngagement, updateEngagement } from "@sttracker/core/engagements";
import { z } from "zod";
import { json, noContent, route } from "../http.ts";
import { customFields, partialDate, rating, text } from "./schemas.ts";
const engagementPatch = z
.object({
startedAt: partialDate.nullable(),
finishedAt: partialDate.nullable(),
progress: z.record(z.string(), z.union([z.number(), z.string().max(200)])),
platform: z.string().max(200).nullable(),
rating,
notes: text().nullable(),
customFields,
})
.partial()
.strict();
export const engagementRoutes = [
route({
method: "GET",
path: "entries/:id/engagements",
summary: "Reads, watches or plays of an entry.",
auth: "any:read",
handler: async ({ deps, user, params }) => listEngagements(deps.db, user.id, params.id!),
}),
route({
method: "POST",
path: "entries/:id/engagements",
summary: "Log a read, watch or play.",
auth: "any:write",
body: engagementPatch,
handler: async ({ deps, user, params, body, actor }) =>
json(await addEngagement(deps.db, user.id, params.id!, body as Parameters<typeof addEngagement>[3], actor), 201),
}),
route({
method: "PATCH",
path: "engagements/:id",
summary: "Edit an engagement.",
auth: "any:write",
body: engagementPatch,
handler: async ({ deps, user, params, body, actor }) =>
updateEngagement(deps.db, user.id, params.id!, body as Parameters<typeof updateEngagement>[3], actor),
}),
route({
method: "DELETE",
path: "engagements/:id",
summary: "Soft-delete an engagement.",
auth: "any:write",
handler: async ({ deps, user, params, actor }) => {
await softDeleteEngagement(deps.db, user.id, params.id!, actor);
return noContent();
},
}),
];
+107
View File
@@ -0,0 +1,107 @@
import { getEntry, listEntries, moveEntryToWork, quickAdd, softDeleteEntry, upsertEntry } from "@sttracker/core/entries";
import { getHistory, restoreVersion } from "@sttracker/core/history";
import { z } from "zod";
import { json, noContent, route } from "../http.ts";
import { customFields, mediaType, qBool, qInt, rating, status, text, visibility } from "./schemas.ts";
const entryPatch = z
.object({
status,
rating,
review: text().nullable(),
privateNotes: text().nullable(),
favorite: z.boolean(),
tags: z.array(z.string().max(100)).max(200),
customFields,
visibility,
overrides: z.object({ title: z.string().max(1000), coverUrl: z.string().max(2000), year: z.number().int().nullable() }).partial().strict(),
mediaTypeOverride: mediaType.nullable(),
})
.partial()
.strict();
export const entryRoutes = [
route({
method: "GET",
path: "entries",
summary: "List your library entries (filter, sort, cursor pagination).",
auth: "any:read",
query: z.object({
mediaType: z.string().optional(),
status: status.optional(),
tag: z.string().optional(),
draft: qBool.optional(),
q: z.string().max(500).optional(),
parentId: z.string().optional(),
sort: z.enum(["updated", "title", "rating"]).optional(),
limit: qInt.optional(),
cursor: z.string().optional(),
}),
handler: async ({ deps, user, query }) => listEntries(deps.db, user.id, query as Parameters<typeof listEntries>[2]),
}),
route({
method: "POST",
path: "entries/quick",
summary: "Quick-add an entry by title (creates a private draft work).",
auth: "any:write",
body: z.object({ title: z.string().min(1).max(1000), mediaType: mediaType.optional() }).strict(),
handler: async ({ deps, user, body }) => json(await quickAdd(deps.db, user.id, body), 201),
}),
route({
method: "GET",
path: "entries/:id",
summary: "One entry with its work.",
auth: "any:read",
handler: async ({ deps, user, params }) => getEntry(deps.db, user.id, params.id!),
}),
route({
method: "PATCH",
path: "entries/:id",
summary: "Edit an entry. Edited fields become user-owned (syncs stop overwriting them).",
auth: "any:write",
body: entryPatch,
handler: async ({ deps, user, params, body, actor }) => {
const cur = await getEntry(deps.db, user.id, params.id!);
await upsertEntry(deps.db, user.id, cur.workId, body as Parameters<typeof upsertEntry>[3], actor);
return getEntry(deps.db, user.id, params.id!);
},
}),
route({
method: "DELETE",
path: "entries/:id",
summary: "Soft-delete an entry.",
auth: "any:write",
handler: async ({ deps, user, params, actor }) => {
await softDeleteEntry(deps.db, user.id, params.id!, actor);
return noContent();
},
}),
route({
method: "GET",
path: "entries/:id/history",
summary: "Change history of an entry, newest first.",
auth: "any:read",
handler: async ({ deps, user, params }) => getHistory(deps.db, user.id, "entry", params.id!),
}),
route({
method: "POST",
path: "history/:id/restore",
summary: "Restore the values a history row replaced.",
auth: "any:write",
handler: async ({ deps, user, params }) => {
await restoreVersion(deps.db, user.id, params.id!);
return noContent();
},
}),
route({
method: "POST",
path: "entries/:id/move",
summary: "Move an entry to another work (for you only).",
auth: "any:write",
body: z.object({ workId: z.string().min(1) }).strict(),
handler: async ({ deps, user, params, body, actor }) => {
const moved = await moveEntryToWork(deps.db, user.id, params.id!, body.workId, actor);
return getEntry(deps.db, user.id, moved.id);
},
}),
];
+49
View File
@@ -0,0 +1,49 @@
import { NotFoundError } from "@sttracker/core/errors";
import { exportUserBundle, exportUserCsv, importUserBundle } from "@sttracker/core/export";
import { z } from "zod";
import { route } from "../http.ts";
import { formFile } from "./form.ts";
const stamp = () => new Date().toISOString().slice(0, 10);
export const exportRoutes = [
route({
method: "GET",
path: "export/bundle",
summary: "Download your data as an sttracker-export/v1 zip bundle.",
auth: "any:read",
handler: async ({ deps, user }) =>
new Response((await exportUserBundle(deps.db, deps.dataDir, user.id)) as Uint8Array<ArrayBuffer>, {
headers: {
"content-type": "application/zip",
"content-disposition": `attachment; filename="sttracker-${user.username}-${stamp()}.zip"`,
},
}),
}),
route({
method: "GET",
path: "export/csv",
summary: "Download one CSV file of your data (e.g. file=entries.csv).",
auth: "any:read",
query: z.object({ file: z.string().regex(/^[a-z_]+\.csv$/) }),
handler: async ({ deps, user, query }) => {
const files = await exportUserCsv(deps.db, user.id);
const body = files[query.file];
if (body === undefined) throw new NotFoundError("csv file");
return new Response(body, {
headers: { "content-type": "text/csv; charset=utf-8", "content-disposition": `attachment; filename="${query.file}"` },
});
},
}),
route({
method: "POST",
path: "import/bundle",
summary: "Restore an export bundle (multipart: file) into your empty account.",
auth: "session",
multipart: true,
handler: async ({ deps, user, form }) => {
const file = formFile(form);
return importUserBundle(deps.db, deps.dataDir, user.id, new Uint8Array(await file.arrayBuffer()));
},
}),
];
+8
View File
@@ -0,0 +1,8 @@
import { HttpError } from "../http.ts";
/** The `file` part of a multipart body, or a 400. */
export function formFile(form: FormData, field = "file"): File {
const f = form.get(field);
if (!(f instanceof File)) throw new HttpError(400, "validation", `${field}: a file is required`);
return f;
}
+84
View File
@@ -0,0 +1,84 @@
import { addListItem, createList, getList, listLists, moveListItem, removeListItem, softDeleteList, updateList } from "@sttracker/core/lists";
import { z } from "zod";
import { json, noContent, route } from "../http.ts";
import { rating, text, visibility } from "./schemas.ts";
const kind = z.enum(["list", "grouping", "thread"]);
const listPatch = z
.object({ name: z.string().min(1).max(500), description: text(), kind, visibility, rating, review: text().nullable() })
.partial()
.strict();
type ListPatchArg = Parameters<typeof updateList>[3];
export const listRoutes = [
route({
method: "GET",
path: "lists",
summary: "Your lists.",
auth: "any:read",
query: z.object({ kind: kind.optional() }),
handler: async ({ deps, user, query }) => listLists(deps.db, user.id, query),
}),
route({
method: "POST",
path: "lists",
summary: "Create a list.",
auth: "any:write",
body: listPatch.required({ name: true }),
handler: async ({ deps, user, body, actor }) => json(await createList(deps.db, user.id, body as ListPatchArg & { name: string }, actor), 201),
}),
route({
method: "GET",
path: "lists/:id",
summary: "A list with its items.",
auth: "any:read",
handler: async ({ deps, user, params }) => getList(deps.db, user.id, params.id!),
}),
route({
method: "PATCH",
path: "lists/:id",
summary: "Edit a list.",
auth: "any:write",
body: listPatch,
handler: async ({ deps, user, params, body, actor }) => updateList(deps.db, user.id, params.id!, body as ListPatchArg, actor),
}),
route({
method: "DELETE",
path: "lists/:id",
summary: "Soft-delete a list.",
auth: "any:write",
handler: async ({ deps, user, params, actor }) => {
await softDeleteList(deps.db, user.id, params.id!, actor);
return noContent();
},
}),
route({
method: "POST",
path: "lists/:id/items",
summary: "Add a work to a list.",
auth: "any:write",
body: z.object({ workId: z.string().min(1), note: text(10_000).nullable().optional(), position: z.number().int().min(0).optional() }).strict(),
handler: async ({ deps, user, params, body, actor }) => json(await addListItem(deps.db, user.id, params.id!, body, actor), 201),
}),
route({
method: "PATCH",
path: "list-items/:id",
summary: "Move a list item to a new 0-based position.",
auth: "any:write",
body: z.object({ position: z.number().int().min(0) }).strict(),
handler: async ({ deps, user, params, body, actor }) => {
await moveListItem(deps.db, user.id, params.id!, body.position, actor);
return noContent();
},
}),
route({
method: "DELETE",
path: "list-items/:id",
summary: "Remove an item from a list.",
auth: "any:write",
handler: async ({ deps, user, params, actor }) => {
await removeListItem(deps.db, user.id, params.id!, actor);
return noContent();
},
}),
];
+40
View File
@@ -0,0 +1,40 @@
import { route, type RouteDef } from "../http.ts";
import { buildOpenApi } from "../openapi.ts";
/** Routes that describe the API and instance. `table` returns the full route table (for OpenAPI). */
export function metaRoutes(table: () => RouteDef[]): RouteDef[] {
let doc: unknown;
return [
route({
method: "GET",
path: "openapi.json",
summary: "This API's OpenAPI document.",
auth: "none",
handler: async () => (doc ??= buildOpenApi(table())),
}),
route({
method: "GET",
path: "about",
summary: "Instance information and data-source attributions.",
auth: "none",
handler: async ({ deps }) => ({
name: "STTracker",
attributions: deps.registry
.list()
.filter((m) => m.attribution)
.map((m) => ({ pluginId: m.id, name: m.name, attribution: m.attribution })),
}),
}),
route({
method: "POST",
path: "admin/plugins/reload",
summary: "Rescan the plugin directories (admin).",
auth: "session",
admin: true,
handler: async ({ deps }) => {
await deps.registry.reload();
return deps.registry.list();
},
}),
];
}
+19
View File
@@ -0,0 +1,19 @@
import { MEDIA_TYPES, STATUSES, VISIBILITIES } from "@sttracker/core/entries";
import { z } from "zod";
/** Shared request schemas. Core functions re-validate; these give early, documented 400s. */
export const rating = z.number().int().min(0).max(100).nullable();
export const mediaType = z.enum(MEDIA_TYPES);
export const status = z.enum(STATUSES as [string, ...string[]]);
export const visibility = z.enum(VISIBILITIES as [string, ...string[]]);
export const partialDate = z.object({
value: z.string().max(40),
precision: z.enum(["day", "month", "year", "unknown"]),
});
export const customFields = z.record(z.string(), z.unknown());
export const text = (max = 100_000) => z.string().max(max);
/** Query-string boolean: "true" / "false". */
export const qBool = z.enum(["true", "false"]).transform((v) => v === "true");
/** Query-string positive integer. */
export const qInt = z.coerce.number().int().min(1).max(1000);
+142
View File
@@ -0,0 +1,142 @@
import {
createConnection,
deleteConnection,
devicePoll,
deviceStart,
getConnection,
listConnections,
listRuns,
updateConnection,
} from "@sttracker/core/connections";
import { ValidationError } from "@sttracker/core/errors";
import { enqueue } from "@sttracker/core/jobs";
import type { PluginRegistry } from "@sttracker/core/registry";
import { z } from "zod";
import { HttpError, json, noContent, route } from "../http.ts";
import { formFile } from "./form.ts";
import { qInt } from "./schemas.ts";
const ALREADY_RUNNING = "A sync for this source is already queued or running.";
function pluginOf(registry: PluginRegistry, pluginId: string) {
const plugin = registry.get(pluginId);
if (!plugin) throw new ValidationError("pluginId", "not_installed");
return plugin;
}
async function enqueueSync(db: Parameters<typeof enqueue>[0], connectionId: string, payload: Record<string, unknown>): Promise<Response> {
const jobId = await enqueue(db, "sync", { connectionId, ...payload }, { lockKey: `sync:${connectionId}` });
if (jobId === null) throw new HttpError(409, "conflict", ALREADY_RUNNING);
return json({ jobId }, 202);
}
const schedule = z.number().int().min(1).nullable();
const config = z.record(z.string(), z.unknown());
const creds = z.record(z.string(), z.string());
export const sourceRoutes = [
route({
method: "GET",
path: "plugins",
summary: "Installed source plugins (manifests).",
auth: "any:read",
handler: async ({ deps }) => deps.registry.list(),
}),
route({
method: "GET",
path: "sources",
summary: "Your source connections (never includes secrets).",
auth: "any:read",
handler: async ({ deps, user }) => listConnections(deps.db, user.id),
}),
route({
method: "POST",
path: "sources",
summary: "Connect a source. Config fields typed secret are stored encrypted.",
auth: "session",
body: z
.object({ pluginId: z.string().min(1), name: z.string().min(1).max(200), config: config.optional(), credentials: creds.optional(), scheduleMinutes: schedule.optional() })
.strict(),
handler: async ({ deps, user, body }) => {
const plugin = pluginOf(deps.registry, body.pluginId);
const id = await createConnection(deps.db, deps.secret, user.id, { ...body, config: body.config ?? {} }, plugin.manifest);
return json(await getConnection(deps.db, user.id, id), 201);
},
}),
route({
method: "PATCH",
path: "sources/:id",
summary: "Edit a source connection. Omitted secrets are kept.",
auth: "session",
body: z
.object({ name: z.string().min(1).max(200), config, credentials: creds, scheduleMinutes: schedule, status: z.enum(["active", "paused"]) })
.partial()
.strict(),
handler: async ({ deps, user, params, body }) => {
const conn = await getConnection(deps.db, user.id, params.id!);
await updateConnection(deps.db, deps.secret, user.id, conn.id, body, pluginOf(deps.registry, conn.pluginId).manifest);
return getConnection(deps.db, user.id, conn.id);
},
}),
route({
method: "DELETE",
path: "sources/:id",
summary: "Disconnect a source: pauses it and wipes its credentials. Imported data stays.",
auth: "session",
handler: async ({ deps, user, params }) => {
await getConnection(deps.db, user.id, params.id!);
await deleteConnection(deps.db, user.id, params.id!);
return noContent();
},
}),
route({
method: "POST",
path: "sources/:id/run",
summary: "Queue a sync now. 409 if one is already queued or running.",
auth: "any:write",
body: z.object({ mode: z.enum(["full", "changes"]).optional() }).strict().optional(),
handler: async ({ deps, user, params, body }) => {
const conn = await getConnection(deps.db, user.id, params.id!);
return enqueueSync(deps.db, conn.id, body?.mode ? { mode: body.mode } : {});
},
}),
route({
method: "POST",
path: "sources/:id/import",
summary: "Upload an export file (multipart: file) and queue a file-mode sync.",
auth: "any:write",
multipart: true,
handler: async ({ deps, user, params, form }) => {
const conn = await getConnection(deps.db, user.id, params.id!);
if (!pluginOf(deps.registry, conn.pluginId).importFile) throw new ValidationError("pluginId", "cannot_import_files");
const file = formFile(form);
const base64 = Buffer.from(await file.arrayBuffer()).toString("base64");
return enqueueSync(deps.db, conn.id, { file: { name: file.name, base64 } });
},
}),
route({
method: "POST",
path: "sources/:id/device/start",
summary: "Start a device-code sign-in with the source.",
auth: "session",
handler: async ({ deps, user, params }) => deviceStart(deps.db, deps.secret, deps.registry, user.id, params.id!),
}),
route({
method: "POST",
path: "sources/:id/device/poll",
summary: "Poll a device-code sign-in: pending or connected.",
auth: "session",
body: z.object({ deviceCode: z.string().min(1).max(2000) }).strict(),
handler: async ({ deps, user, params, body }) => ({
status: await devicePoll(deps.db, deps.secret, deps.registry, user.id, params.id!, body.deviceCode),
}),
}),
route({
method: "GET",
path: "sources/:id/runs",
summary: "Recent sync runs of a source, newest first.",
auth: "any:read",
query: z.object({ limit: qInt.optional() }),
handler: async ({ deps, user, params, query }) => listRuns(deps.db, user.id, params.id!, query.limit),
}),
];
+56
View File
@@ -0,0 +1,56 @@
import { acceptSuggestion, listSuggestions, rejectSuggestion, resolveVisibleWork } from "@sttracker/core/entries";
import { NotFoundError } from "@sttracker/core/errors";
import { getWork, searchWorks } from "@sttracker/core/works";
import { z } from "zod";
import { noContent, route } from "../http.ts";
import { mediaType, qInt } from "./schemas.ts";
export const workRoutes = [
route({
method: "GET",
path: "works/search",
summary: "Search works visible to you by title.",
auth: "any:read",
query: z.object({ q: z.string().min(1).max(500), mediaType: mediaType.optional(), limit: qInt.optional() }),
handler: async ({ deps, user, query }) => searchWorks(deps.db, user.id, query.q, { mediaType: query.mediaType, limit: query.limit }),
}),
route({
method: "GET",
path: "works/:id",
summary: "One work (follows merges and your aliases).",
auth: "any:read",
handler: async ({ deps, user, params }) => {
const visible = await resolveVisibleWork(deps.db.k, user.id, params.id!);
const work = await getWork(deps.db, visible.id);
if (!work) throw new NotFoundError("work");
return work;
},
}),
route({
method: "GET",
path: "works/:id/suggestions",
summary: "Pending match suggestions for a work.",
auth: "any:read",
handler: async ({ deps, user, params }) => listSuggestions(deps.db, user.id, params.id!),
}),
route({
method: "POST",
path: "suggestions/:id/accept",
summary: "Accept a match suggestion (moves your entry to the candidate work).",
auth: "any:write",
handler: async ({ deps, user, params }) => {
await acceptSuggestion(deps.db, user.id, params.id!);
return noContent();
},
}),
route({
method: "POST",
path: "suggestions/:id/reject",
summary: "Reject a match suggestion.",
auth: "any:write",
handler: async ({ deps, user, params }) => {
await rejectSuggestion(deps.db, user.id, params.id!);
return noContent();
},
}),
];
+447
View File
@@ -0,0 +1,447 @@
import { afterEach, beforeEach, describe, expect, spyOn, test } from "bun:test";
import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { loadPlugins, type PluginRegistry } from "@sttracker/core/registry";
import { forEachDialect } from "@sttracker/db/testing";
import { createApp } from "../src/app.ts";
import { loadConfig } from "../src/config.ts";
import { sessionCookie as buildSessionCookie } from "../src/http.ts";
import { safeFilename } from "../src/routes/attachments.ts";
const ECHO_DIR = join(import.meta.dir, "../../../packages/core/test/fixtures/plugins");
// The fixture dir also holds a deliberately broken plugin; silence its load warning.
const warn = spyOn(console, "warn").mockImplementation(() => {});
const registry: PluginRegistry = await loadPlugins([ECHO_DIR]);
warn.mockRestore();
const BASE = "http://localhost/api/v1";
type App = ReturnType<typeof createApp>;
interface CallOpts {
body?: unknown;
cookie?: string;
token?: string;
headers?: Record<string, string>;
form?: FormData;
/** Omit the X-Requested-With header that browsers (the web app) send on writes. */
noCsrf?: boolean;
}
async function call(app: App, method: string, path: string, o: CallOpts = {}): Promise<Response> {
const headers: Record<string, string> = { ...o.headers };
if (o.cookie) headers.cookie = o.cookie;
if (o.token) headers.authorization = `Bearer ${o.token}`;
else if (!o.noCsrf) headers["x-requested-with"] ??= "sttracker";
let body: BodyInit | undefined;
if (o.form) body = o.form;
else if (o.body !== undefined) {
headers["content-type"] = "application/json";
body = JSON.stringify(o.body);
}
const url = path.startsWith("http") ? path : `${BASE}/${path}`;
return app.fetch(new Request(url, { method, headers, body }));
}
function sessionCookie(res: Response): string {
const set = res.headers.get("set-cookie") ?? "";
const m = /stt_session=([^;]*)/.exec(set);
if (!m) throw new Error(`no session cookie in ${set}`);
return `stt_session=${m[1]}`;
}
async function signUp(app: App, username: string, inviteCode?: string): Promise<string> {
const reg = await call(app, "POST", "auth/register", { body: { username, password: "correct horse", inviteCode } });
expect(reg.status).toBe(201);
const res = await call(app, "POST", "auth/login", { body: { username, password: "correct horse" } });
expect(res.status).toBe(200);
return sessionCookie(res);
}
test("booting without STTRACKER_SECRET throws the exact message", () => {
expect(() => loadConfig({})).toThrow("Set STTRACKER_SECRET to a long random string. It encrypts source credentials.");
});
test("safeFilename strips quotes, newlines and path characters", () => {
expect(safeFilename('a"b\r\nc/../d.png')).toBe("a_b__c_.._d.png");
expect(safeFilename("")).toBe("file");
});
test("session cookie is Secure only behind an https PUBLIC_URL", () => {
expect(buildSessionCookie("t", "https://st.example")).toContain("; Secure");
expect(buildSessionCookie("t", "http://localhost:3000")).not.toContain("Secure");
expect(buildSessionCookie("t", undefined)).toContain("HttpOnly; SameSite=Lax");
});
test("config defaults", () => {
const c = loadConfig({ STTRACKER_SECRET: "s" });
expect(c).toMatchObject({
dataDir: "./data",
databaseUrl: "sqlite://./data/sttracker.db",
port: 3000,
pluginDirs: ["plugins", "./data/plugins"],
});
const d = loadConfig({ STTRACKER_SECRET: "s", DATA_DIR: "/srv/st", PORT: "4000", PLUGIN_DIRS: "a, b" });
expect(d).toMatchObject({ databaseUrl: "sqlite:///srv/st/sttracker.db", port: 4000, pluginDirs: ["a", "b"] });
});
forEachDialect("api", (getDb) => {
let dataDir: string;
let app: App;
beforeEach(async () => {
dataDir = await mkdtemp(join(tmpdir(), "stt-api-"));
app = createApp({ db: getDb(), registry, secret: "test-secret", dataDir });
});
afterEach(async () => {
await rm(dataDir, { recursive: true, force: true });
});
test("register, login cookie, GET me", async () => {
const reg = await call(app, "POST", "auth/register", { body: { username: "ana", password: "correct horse" } });
expect(reg.status).toBe(201);
const login = await call(app, "POST", "auth/login", { body: { username: "ana", password: "correct horse" } });
expect(login.status).toBe(200);
const set = login.headers.get("set-cookie")!;
expect(set).toContain("HttpOnly");
expect(set).toContain("SameSite=Lax");
expect(set).not.toContain("Secure");
const me = await call(app, "GET", "me", { cookie: sessionCookie(login) });
expect(me.status).toBe(200);
const body = (await me.json()) as Record<string, unknown>;
expect(body).toMatchObject({ username: "ana", role: "admin" });
expect(JSON.stringify(body)).not.toContain("password");
});
test("bad login is 401 JSON", async () => {
await call(app, "POST", "auth/register", { body: { username: "ana", password: "correct horse" } });
const res = await call(app, "POST", "auth/login", { body: { username: "ana", password: "nope nope" } });
expect(res.status).toBe(401);
expect(((await res.json()) as { error: { code: string } }).error.code).toBe("unauthenticated");
});
test("unauthenticated GET entries is 401 JSON", async () => {
const res = await call(app, "GET", "entries");
expect(res.status).toBe(401);
expect(res.headers.get("content-type")).toContain("application/json");
const body = (await res.json()) as { error: { code: string; message: string } };
expect(body.error.code).toBe("unauthenticated");
expect(typeof body.error.message).toBe("string");
});
test("quick add, then the list shows a draft", async () => {
const cookie = await signUp(app, "ana");
const add = await call(app, "POST", "entries/quick", { cookie, body: { title: "Dune", mediaType: "book" } });
expect(add.status).toBe(201);
const list = await call(app, "GET", "entries?draft=true", { cookie });
expect(list.status).toBe(200);
const body = (await list.json()) as { items: { draft: boolean; work: { title: string } }[] };
expect(body.items).toHaveLength(1);
expect(body.items[0]).toMatchObject({ draft: true, work: { title: "Dune" } });
});
test("PATCH rating 150 is 400 validation; 80 works", async () => {
const cookie = await signUp(app, "ana");
const add = (await (await call(app, "POST", "entries/quick", { cookie, body: { title: "Dune" } })).json()) as { id: string };
const bad = await call(app, "PATCH", `entries/${add.id}`, { cookie, body: { rating: 150 } });
expect(bad.status).toBe(400);
const err = (await bad.json()) as { error: { code: string; message: string } };
expect(err.error.code).toBe("validation");
expect(err.error.message).toContain("rating");
const ok = await call(app, "PATCH", `entries/${add.id}`, { cookie, body: { rating: 80 } });
expect(ok.status).toBe(200);
expect(((await ok.json()) as { rating: number }).rating).toBe(80);
});
test("an app token with read can GET but cannot PATCH (403)", async () => {
const cookie = await signUp(app, "ana");
const add = (await (await call(app, "POST", "entries/quick", { cookie, body: { title: "Dune" } })).json()) as { id: string };
const mk = await call(app, "POST", "me/tokens", { cookie, body: { name: "script", scopes: ["read"] } });
expect(mk.status).toBe(201);
const { token } = (await mk.json()) as { token: string };
expect((await call(app, "GET", `entries/${add.id}`, { token })).status).toBe(200);
const res = await call(app, "PATCH", `entries/${add.id}`, { token, body: { rating: 10 } });
expect(res.status).toBe(403);
expect(((await res.json()) as { error: { code: string } }).error.code).toBe("forbidden");
const list = (await (await call(app, "GET", "me/tokens", { cookie })).json()) as Record<string, unknown>[];
expect(list).toHaveLength(1);
expect(JSON.stringify(list)).not.toContain("hash");
expect((await call(app, "GET", "entries", { token: "stt_bogus" })).status).toBe(401);
});
test("another user's entry is 404", async () => {
const ana = await signUp(app, "ana");
const inv = await call(app, "POST", "admin/invites", { cookie: ana });
expect(inv.status).toBe(201);
const { code } = (await inv.json()) as { code: string };
const bob = await signUp(app, "bob", code);
const add = (await (await call(app, "POST", "entries/quick", { cookie: ana, body: { title: "Dune" } })).json()) as { id: string };
expect((await call(app, "GET", `entries/${add.id}`, { cookie: bob })).status).toBe(404);
const patch = await call(app, "PATCH", `entries/${add.id}`, { cookie: bob, body: { rating: 10 } });
expect(patch.status).toBe(404);
expect(((await patch.json()) as { error: { code: string } }).error.code).toBe("not_found");
expect((await call(app, "POST", "admin/invites", { cookie: bob })).status).toBe(403);
});
test("cookie-authenticated writes need the CSRF header", async () => {
const cookie = await signUp(app, "ana");
const res = await call(app, "POST", "entries/quick", { cookie, body: { title: "Dune" }, noCsrf: true });
expect(res.status).toBe(403);
// Login and register too (login CSRF).
const login = await call(app, "POST", "auth/login", { body: { username: "ana", password: "correct horse" }, noCsrf: true });
expect(login.status).toBe(403);
expect(login.headers.get("set-cookie")).toBeNull();
expect((await call(app, "POST", "auth/register", { body: { username: "zed", password: "correct horse" }, noCsrf: true })).status).toBe(403);
// Bearer requests are exempt.
const { token } = (await (await call(app, "POST", "me/tokens", { cookie, body: { name: "w", scopes: ["write"] } })).json()) as { token: string };
expect((await call(app, "POST", "entries/quick", { token, body: { title: "Dune" } })).status).toBe(201);
});
test("login is rate limited to 10 per minute", async () => {
let last = 0;
for (let i = 0; i < 11; i++) {
last = (await call(app, "POST", "auth/login", { body: { username: "x", password: "y" } })).status;
}
expect(last).toBe(429);
});
test("JSON bodies over 1 MB are rejected", async () => {
const cookie = await signUp(app, "ana");
const res = await call(app, "POST", "entries/quick", { cookie, body: { title: "x".repeat(1_100_000) } });
expect(res.status).toBe(413);
expect(((await res.json()) as { error: { code: string } }).error.code).toBe("validation");
});
test("sources: create with echo, run twice gives 409, secrets never returned", async () => {
const cookie = await signUp(app, "ana");
const plugins = (await (await call(app, "GET", "plugins", { cookie })).json()) as { id: string }[];
expect(plugins.map((p) => p.id)).toContain("echo");
const mk = await call(app, "POST", "sources", { cookie, body: { pluginId: "echo", name: "E", config: { label: "x", token: "sekrit" } } });
expect(mk.status).toBe(201);
const { id } = (await mk.json()) as { id: string };
const listed = await (await call(app, "GET", "sources", { cookie })).text();
expect(listed).not.toContain("sekrit");
expect(listed).toContain('"hasCredentials":true');
expect((await call(app, "POST", `sources/${id}/run`, { cookie })).status).toBe(202);
const again = await call(app, "POST", `sources/${id}/run`, { cookie });
expect(again.status).toBe(409);
expect(await again.json()).toEqual({
error: { code: "conflict", message: "A sync for this source is already queued or running." },
});
expect((await call(app, "GET", `sources/${id}/runs`, { cookie })).status).toBe(200);
expect((await call(app, "DELETE", `sources/${id}`, { cookie })).status).toBe(204);
expect((await (await call(app, "GET", "sources", { cookie })).json()) as unknown[]).toHaveLength(0);
expect((await call(app, "POST", `sources/${id}/run`, { cookie })).status).toBe(404);
});
test("sources: unknown plugin is 400; file import without importFile is 400", async () => {
const cookie = await signUp(app, "ana");
const bad = await call(app, "POST", "sources", { cookie, body: { pluginId: "nope", name: "E" } });
expect(bad.status).toBe(400);
const { id } = (await (await call(app, "POST", "sources", { cookie, body: { pluginId: "echo", name: "E" } })).json()) as { id: string };
const form = new FormData();
form.set("file", new File(["a,b"], "x.csv", { type: "text/csv" }));
expect((await call(app, "POST", `sources/${id}/import`, { cookie, form })).status).toBe(400);
});
test("export bundle is a zip; csv picks a file", async () => {
const cookie = await signUp(app, "ana");
const res = await call(app, "GET", "export/bundle", { cookie });
expect(res.status).toBe(200);
expect(res.headers.get("content-type")).toBe("application/zip");
const bytes = new Uint8Array(await res.arrayBuffer());
expect([bytes[0], bytes[1]]).toEqual([0x50, 0x4b]);
const csv = await call(app, "GET", "export/csv?file=entries.csv", { cookie });
expect(csv.status).toBe(200);
expect(csv.headers.get("content-type")).toContain("text/csv");
expect((await call(app, "GET", "export/csv?file=nope.csv", { cookie })).status).toBe(404);
});
test("attachments: upload to own entry, serve with safe headers; another user's entry is 404", async () => {
const ana = await signUp(app, "ana");
const add = (await (await call(app, "POST", "entries/quick", { cookie: ana, body: { title: "Dune" } })).json()) as { id: string };
const form = new FormData();
form.set("file", new File([new Uint8Array([0x89, 0x50, 0x4e, 0x47])], 'we<ird>na;me?.png', { type: "image/png" }));
form.set("entity", "entry");
form.set("entityId", add.id);
const up = await call(app, "POST", "attachments", { cookie: ana, form });
expect(up.status).toBe(201);
const att = (await up.json()) as { id: string; links: { id: string }[] };
const got = await call(app, "GET", `attachments/${att.id}`, { cookie: ana });
expect(got.status).toBe(200);
expect(got.headers.get("content-type")).toBe("image/png");
expect(got.headers.get("x-content-type-options")).toBe("nosniff");
expect(got.headers.get("content-disposition")).toBe('inline; filename="we_ird_na_me_.png"');
const { code } = (await (await call(app, "POST", "admin/invites", { cookie: ana })).json()) as { code: string };
const bob = await signUp(app, "bob", code);
const bobs = (await (await call(app, "POST", "entries/quick", { cookie: bob, body: { title: "Bob's" } })).json()) as { id: string };
const form2 = new FormData();
form2.set("file", new File([new Uint8Array([1])], "a.png", { type: "image/png" }));
form2.set("entity", "entry");
form2.set("entityId", bobs.id);
expect((await call(app, "POST", "attachments", { cookie: ana, form: form2 })).status).toBe(404);
expect((await call(app, "GET", `attachments/${att.id}`, { cookie: bob })).status).toBe(404);
expect((await call(app, "DELETE", `attachments/${att.id}/links/${att.links[0]!.id}`, { cookie: ana })).status).toBe(204);
expect((await call(app, "GET", `attachments/${att.id}`, { cookie: ana })).status).toBe(404);
});
test("auth/status says whether any user exists, without auth", async () => {
expect(await (await call(app, "GET", "auth/status")).json()).toEqual({ hasUsers: false });
await signUp(app, "ana");
expect(await (await call(app, "GET", "auth/status")).json()).toEqual({ hasUsers: true });
});
test("attachments list: owner-scoped per entity; a foreign or unknown entity is 404", async () => {
const ana = await signUp(app, "ana");
const add = (await (await call(app, "POST", "entries/quick", { cookie: ana, body: { title: "Dune" } })).json()) as { id: string };
const q = `attachments?entity=entry&entityId=${add.id}`;
expect(await (await call(app, "GET", q, { cookie: ana })).json()).toEqual([]);
const form = new FormData();
form.set("file", new File([new Uint8Array([0x89, 0x50])], "shot.png", { type: "image/png" }));
form.set("entity", "entry");
form.set("entityId", add.id);
const up = (await (await call(app, "POST", "attachments", { cookie: ana, form })).json()) as { id: string };
const list = (await (await call(app, "GET", q, { cookie: ana })).json()) as { id: string; mime: string; filename: string; links: { id: string }[] }[];
expect(list.map((a) => [a.id, a.mime, a.filename])).toEqual([[up.id, "image/png", "shot.png"]]);
expect(list[0]!.links.length).toBe(1);
const { code } = (await (await call(app, "POST", "admin/invites", { cookie: ana })).json()) as { code: string };
const bob = await signUp(app, "bob", code);
expect((await call(app, "GET", q, { cookie: bob })).status).toBe(404);
expect((await call(app, "GET", "attachments?entity=entry&entityId=nope", { cookie: ana })).status).toBe(404);
expect((await call(app, "GET", `attachments?entity=bogus&entityId=${add.id}`, { cookie: ana })).status).toBe(400);
expect((await call(app, "GET", q)).status).toBe(401);
});
test("me/preferences: defaults, validated PATCH, and quick add uses defaultVisibility", async () => {
const ana = await signUp(app, "ana");
expect(await (await call(app, "GET", "me/preferences", { cookie: ana })).json()).toEqual({ ratingScale: "5half", defaultVisibility: "private" });
const bad = await call(app, "PATCH", "me/preferences", { cookie: ana, body: { ratingScale: "7" } });
expect(bad.status).toBe(400);
expect((await call(app, "PATCH", "me/preferences", { cookie: ana, body: { extra: 1 } })).status).toBe(400);
const ok = await call(app, "PATCH", "me/preferences", { cookie: ana, body: { ratingScale: "10" } });
expect(await ok.json()).toEqual({ ratingScale: "10", defaultVisibility: "private" });
await call(app, "PATCH", "me/preferences", { cookie: ana, body: { defaultVisibility: "friends" } });
expect(await (await call(app, "GET", "me/preferences", { cookie: ana })).json()).toEqual({ ratingScale: "10", defaultVisibility: "friends" });
const e = (await (await call(app, "POST", "entries/quick", { cookie: ana, body: { title: "Dune" } })).json()) as { visibility: string };
expect(e.visibility).toBe("friends");
expect((await call(app, "PATCH", "me/preferences", { cookie: ana, body: { ratingScale: "5" }, noCsrf: true })).status).toBe(403);
});
test("JSON bodies need Content-Type application/json (415)", async () => {
const res = await app.fetch(
new Request(`${BASE}/auth/register`, {
method: "POST",
headers: { "content-type": "text/plain", "x-requested-with": "sttracker" },
body: JSON.stringify({ username: "ana", password: "correct horse" }),
}),
);
expect(res.status).toBe(415);
expect(((await res.json()) as { error: { code: string } }).error.code).toBe("unsupported_media_type");
});
test("a malformed session cookie is 401, not 500", async () => {
expect((await call(app, "GET", "me", { headers: { cookie: "stt_session=%" } })).status).toBe(401);
});
test("cookie plus a bogus Bearer on a write is 401", async () => {
const cookie = await signUp(app, "ana");
const res = await call(app, "POST", "entries/quick", { cookie, token: "stt_bogus", body: { title: "Dune" } });
expect(res.status).toBe(401);
});
test("a write-scope token can write; history records the token, field_sources the user", async () => {
const cookie = await signUp(app, "ana");
const add = (await (await call(app, "POST", "entries/quick", { cookie, body: { title: "Dune" } })).json()) as { id: string };
const { token } = (await (await call(app, "POST", "me/tokens", { cookie, body: { name: "script", scopes: ["write", "read"] } })).json()) as { token: string };
const res = await call(app, "PATCH", `entries/${add.id}`, { token, body: { rating: 70 } });
expect(res.status).toBe(200);
expect(((await res.json()) as { fieldSources: Record<string, string> }).fieldSources.rating).toBe("user");
const hist = (await (await call(app, "GET", `entries/${add.id}/history`, { token })).json()) as { actor: string }[];
expect(hist[0]!.actor).toBe("token:script");
});
test("DELETE me/tokens/:id is 404 for unknown or foreign tokens", async () => {
const ana = await signUp(app, "ana");
const { code } = (await (await call(app, "POST", "admin/invites", { cookie: ana })).json()) as { code: string };
const bob = await signUp(app, "bob", code);
const { id } = (await (await call(app, "POST", "me/tokens", { cookie: ana, body: { name: "t", scopes: ["read"] } })).json()) as { id: string };
expect((await call(app, "DELETE", `me/tokens/${id}`, { cookie: bob })).status).toBe(404);
expect((await call(app, "DELETE", "me/tokens/nope", { cookie: ana })).status).toBe(404);
expect((await call(app, "DELETE", `me/tokens/${id}`, { cookie: ana })).status).toBe(204);
});
test("admin-only routes are 403 for members", async () => {
const ana = await signUp(app, "ana");
const { code } = (await (await call(app, "POST", "admin/invites", { cookie: ana })).json()) as { code: string };
const bob = await signUp(app, "bob", code);
expect((await call(app, "POST", "admin/plugins/reload", { cookie: bob })).status).toBe(403);
expect((await call(app, "POST", "admin/invites", { cookie: bob })).status).toBe(403);
const warn = spyOn(console, "warn").mockImplementation(() => {});
try {
expect((await call(app, "POST", "admin/plugins/reload", { cookie: ana })).status).toBe(200);
} finally {
warn.mockRestore();
}
});
test("a paused source without credentials stays listed and can be resumed", async () => {
const cookie = await signUp(app, "ana");
const { id } = (await (await call(app, "POST", "sources", { cookie, body: { pluginId: "echo", name: "E" } })).json()) as { id: string };
expect((await call(app, "PATCH", `sources/${id}`, { cookie, body: { status: "paused" } })).status).toBe(200);
const listed = (await (await call(app, "GET", "sources", { cookie })).json()) as { id: string; status: string }[];
expect(listed).toMatchObject([{ id, status: "paused" }]);
const resumed = await call(app, "PATCH", `sources/${id}`, { cookie, body: { status: "active" } });
expect(((await resumed.json()) as { status: string }).status).toBe("active");
});
test("a multipart upload over 50 MB is 413", async () => {
const cookie = await signUp(app, "ana");
const { id } = (await (await call(app, "POST", "sources", { cookie, body: { pluginId: "echo", name: "E" } })).json()) as { id: string };
const form = new FormData();
form.set("file", new File([new Uint8Array(50 * 1024 * 1024 + 1)], "big.csv", { type: "text/csv" }));
const res = await call(app, "POST", `sources/${id}/import`, { cookie, form });
expect(res.status).toBe(413);
expect(((await res.json()) as { error: { code: string } }).error.code).toBe("validation");
});
test("openapi.json lists /api/v1/entries", async () => {
const res = await call(app, "GET", "openapi.json");
expect(res.status).toBe(200);
const doc = (await res.json()) as { openapi: string; paths: Record<string, Record<string, { summary: string }>> };
expect(doc.openapi).toStartWith("3.");
expect(doc.paths["/api/v1/entries"]?.get?.summary).toBeTruthy();
expect(doc.paths["/api/v1/entries/{id}"]?.patch).toBeTruthy();
const up = doc.paths["/api/v1/attachments"]?.post as unknown as {
requestBody: { content: Record<string, { schema: { properties: Record<string, unknown>; required: string[] } }> };
};
const mp = up.requestBody.content["multipart/form-data"]!.schema;
expect(Object.keys(mp.properties).sort()).toEqual(["caption", "entity", "entityId", "file"]);
expect(mp.required).toEqual(["file", "entity", "entityId"]);
});
test("about lists attributions; ingest and unknown routes are 404 JSON", async () => {
expect((await call(app, "GET", "about")).status).toBe(200);
const ing = await call(app, "POST", "http://localhost/ingest/echo/hook");
expect(ing.status).toBe(404);
expect(((await ing.json()) as { error: { code: string } }).error.code).toBe("not_found");
expect((await call(app, "GET", "nope")).status).toBe(404);
});
});
describe("internal errors", () => {
test("are 500 with a generic message", async () => {
const broken = { k: null, dialect: "sqlite", close: async () => {} } as never;
const app = createApp({ db: broken, registry, secret: "s", dataDir: "/nonexistent" });
const orig = console.error;
console.error = () => {};
try {
const res = await call(app, "POST", "auth/login", { body: { username: "a", password: "b" } });
expect(res.status).toBe(500);
expect(await res.json()).toEqual({ error: { code: "internal", message: "Something went wrong on the server." } });
} finally {
console.error = orig;
}
});
});
+22
View File
@@ -0,0 +1,22 @@
import { expect, test } from "bun:test";
import { forEachDialect } from "@sttracker/db/testing";
import { ensureBackupScheduled, nextBackupAt } from "../src/boot.ts";
test("nextBackupAt is the next 03:00 local time", () => {
const before = new Date(2026, 9, 30, 2, 59);
expect(nextBackupAt(before).getTime()).toBe(new Date(2026, 9, 30, 3, 0).getTime());
const after = new Date(2026, 9, 30, 3, 0);
expect(nextBackupAt(after).getTime()).toBe(new Date(2026, 9, 31, 3, 0).getTime());
});
forEachDialect("boot", (getDb) => {
test("ensureBackupScheduled queues one backup job under lockKey backup", async () => {
const db = getDb();
expect(await ensureBackupScheduled(db)).not.toBeNull();
expect(await ensureBackupScheduled(db)).toBeNull();
const jobs = await db.k.selectFrom("jobs").selectAll().where("type", "=", "backup").execute();
expect(jobs).toHaveLength(1);
expect(jobs[0]).toMatchObject({ lock_key: "backup", status: "queued" });
expect(new Date(jobs[0]!.run_at).getHours()).toBe(3);
});
});
+15
View File
@@ -0,0 +1,15 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="color-scheme" content="dark" />
<meta name="theme-color" content="#17152b" />
<title>STTracker</title>
<link rel="stylesheet" href="./src/theme.css" />
</head>
<body>
<div id="root"></div>
<script type="module" src="./src/main.tsx"></script>
</body>
</html>
+18
View File
@@ -0,0 +1,18 @@
{
"name": "@sttracker/web",
"private": true,
"type": "module",
"exports": {
"./*": "./src/*.ts"
},
"dependencies": {
"@tanstack/react-query": "^5.105.0",
"react": "^19.3.1",
"react-dom": "^19.3.1",
"wouter": "^3.13.0"
},
"devDependencies": {
"@types/react": "^19.3.0",
"@types/react-dom": "^19.3.0"
}
}
+258
View File
@@ -0,0 +1,258 @@
/**
* The `/api/v1` client. Cookie auth (`credentials: "same-origin"`); every non-GET request sends
* `X-Requested-With: sttracker` (the server's CSRF rule) and, unless it is multipart, a JSON content type.
* Wire types mirror the server's responses; the web app cannot import `@sttracker/core`.
*/
import type { PartialDate, RatingScale, Status } from "./format.ts";
export class ApiError extends Error {
constructor(
readonly status: number,
readonly code: string,
message: string,
) {
super(message);
this.name = "ApiError";
}
}
type Method = "GET" | "POST" | "PATCH" | "DELETE";
type QueryValue = string | number | boolean | null | undefined;
export interface RequestOpts {
body?: unknown;
form?: FormData;
query?: Record<string, QueryValue>;
fetchImpl?: typeof fetch;
}
export async function request<T = unknown>(method: Method, path: string, opts: RequestOpts = {}): Promise<T> {
const qs = new URLSearchParams();
for (const [k, v] of Object.entries(opts.query ?? {})) if (v !== undefined && v !== null && v !== "") qs.set(k, String(v));
const url = `/api/v1/${path}${qs.size ? `?${qs}` : ""}`;
const headers: Record<string, string> = { accept: "application/json" };
let body: BodyInit | undefined;
if (method !== "GET") {
headers["x-requested-with"] = "sttracker";
if (opts.form) body = opts.form;
else {
headers["content-type"] = "application/json";
if (opts.body !== undefined) body = JSON.stringify(opts.body);
}
}
let res: Response;
try {
res = await (opts.fetchImpl ?? fetch)(url, { method, headers, body, credentials: "same-origin" });
} catch {
throw new ApiError(0, "network", "Can't reach the server. Check your connection and try again.");
}
if (res.status === 204) return undefined as T;
const text = await res.text();
let data: unknown;
try {
data = text ? JSON.parse(text) : undefined;
} catch {
data = undefined;
}
if (!res.ok) {
const err = (data as { error?: { code?: string; message?: string } } | undefined)?.error;
if (err?.code) throw new ApiError(res.status, err.code, err.message ?? err.code);
throw new ApiError(res.status, `http_${res.status}`, `The server answered ${res.status}. Try again in a moment.`);
}
return data as T;
}
// ---------------------------------------------------------------------------------------------
// Wire types
// ---------------------------------------------------------------------------------------------
export interface User {
id: string;
username: string;
displayName: string;
role: "admin" | "member";
}
export interface EntryView {
id: string;
workId: string;
status: Status;
rating: number | null;
review: string | null;
privateNotes: string | null;
favorite: boolean;
tags: string[];
customFields: Record<string, unknown>;
visibility: "private" | "friends" | "public";
overrides: { title?: string; coverUrl?: string; year?: number | null };
mediaTypeOverride: string | null;
fieldSources: Record<string, string>;
createdAt: string;
updatedAt: string;
work: { id: string; mediaType: string; title: string; originalTitle: string; year: number | null; coverUrl: string; parentId: string | null };
draft: boolean;
childSummary: { total: number; completed: number } | null;
}
export type EntryPatch = Partial<
Pick<EntryView, "status" | "rating" | "review" | "privateNotes" | "favorite" | "tags" | "customFields" | "visibility" | "overrides">
>;
export interface Page<T> {
items: T[];
nextCursor: string | null;
}
export interface Engagement {
id: string;
entryId: string;
startedAt: PartialDate | null;
finishedAt: PartialDate | null;
progress: Record<string, number | string>;
platform: string | null;
rating: number | null;
notes: string | null;
fieldSources: Record<string, string>;
createdAt: string;
}
export interface Copy {
id: string;
workId: string | null;
titleHint: string | null;
format: string | null;
acquiredFrom: string | null;
acquiredAt: PartialDate | null;
priceCents: number | null;
currency: string | null;
notes: string | null;
}
export interface Attachment {
id: string;
mime: string;
filename: string;
caption: string | null;
size: number;
}
export interface Preferences {
ratingScale: RatingScale;
defaultVisibility: "private" | "friends" | "public";
}
export interface Connection {
id: string;
pluginId: string;
name: string;
status: string;
lastRunAt: string | null;
hasCredentials: boolean;
}
export interface ChangeEvent {
id: string;
type: string;
status: string;
connectionId: string | null;
targetEntity: string;
targetId: string | null;
payload: Record<string, unknown>;
createdAt: string;
}
export interface HistoryRecord {
id: string;
entity: string;
entityId: string;
before: Record<string, unknown>;
after: Record<string, unknown>;
actor: string;
createdAt: string;
}
export interface CustomFieldDef {
id: string;
name: string;
type: "text" | "long_text" | "number" | "date" | "boolean" | "single_select" | "multi_select" | "url" | "rating";
options: string[];
}
export interface Work {
id: string;
media_type: string;
title: string;
year: number | null;
cover_url: string;
creators: { name: string; role?: string }[];
}
export interface Suggestion {
id: string;
workId: string;
candidateWorkId: string;
score: number;
}
// ---------------------------------------------------------------------------------------------
// Endpoints
// ---------------------------------------------------------------------------------------------
export interface EntryQuery {
q?: string;
mediaType?: string;
status?: Status;
draft?: boolean;
parentId?: string;
cursor?: string;
limit?: number;
}
export const api = {
authStatus: () => request<{ hasUsers: boolean }>("GET", "auth/status"),
me: () => request<User>("GET", "me"),
preferences: () => request<Preferences>("GET", "me/preferences"),
updatePreferences: (patch: Partial<Preferences>) => request<Preferences>("PATCH", "me/preferences", { body: patch }),
login: (username: string, password: string) => request<User>("POST", "auth/login", { body: { username, password } }),
register: (body: { username: string; password: string; displayName?: string; inviteCode?: string }) =>
request<User>("POST", "auth/register", { body }),
logout: () => request<void>("POST", "auth/logout"),
entries: (query: EntryQuery) => request<Page<EntryView>>("GET", "entries", { query: { ...query } }),
entry: (id: string) => request<EntryView>("GET", `entries/${id}`),
quickAdd: (title: string) => request<EntryView>("POST", "entries/quick", { body: { title } }),
patchEntry: (id: string, patch: EntryPatch) => request<EntryView>("PATCH", `entries/${id}`, { body: patch }),
moveEntry: (id: string, workId: string) => request<EntryView>("POST", `entries/${id}/move`, { body: { workId } }),
history: (id: string) => request<HistoryRecord[]>("GET", `entries/${id}/history`),
restore: (historyId: string) => request<void>("POST", `history/${historyId}/restore`),
engagements: (entryId: string) => request<Engagement[]>("GET", `entries/${entryId}/engagements`),
addEngagement: (entryId: string, body: { startedAt?: PartialDate | null; finishedAt?: PartialDate | null }) =>
request<Engagement>("POST", `entries/${entryId}/engagements`, { body }),
deleteEngagement: (id: string) => request<void>("DELETE", `engagements/${id}`),
copies: (workId: string) => request<Copy[]>("GET", "copies", { query: { workId } }),
addCopy: (body: Partial<Omit<Copy, "id">>) => request<Copy>("POST", "copies", { body }),
attachments: (entity: "entry" | "engagement" | "copy", entityId: string) =>
request<Attachment[]>("GET", "attachments", { query: { entity, entityId } }),
upload: (file: File, entityId: string) => {
const form = new FormData();
form.set("file", file);
form.set("entity", "entry");
form.set("entityId", entityId);
return request<Attachment>("POST", "attachments", { form });
},
sources: () => request<Connection[]>("GET", "sources"),
changes: () => request<Page<ChangeEvent>>("GET", "changes", { query: { limit: 200 } }),
customFields: (mediaType: string) => request<CustomFieldDef[]>("GET", "custom-fields", { query: { mediaType, level: "entry" } }),
work: (id: string) => request<Work>("GET", `works/${id}`),
searchWorks: (q: string, mediaType?: string) => request<Work[]>("GET", "works/search", { query: { q, mediaType, limit: 8 } }),
suggestions: (workId: string) => request<Suggestion[]>("GET", `works/${workId}/suggestions`),
acceptSuggestion: (id: string) => request<void>("POST", `suggestions/${id}/accept`),
rejectSuggestion: (id: string) => request<void>("POST", `suggestions/${id}/reject`),
};
+18
View File
@@ -0,0 +1,18 @@
import { useState } from "react";
import { mediaColour } from "../format.ts";
/** A cover image, the media-type gradient when there is none, or the striped draft placeholder. */
export function Cover({ url, mediaType, draft, size }: { url: string; mediaType: string; draft: boolean; size?: "large" | "small" }) {
const [failed, setFailed] = useState(false);
const cls = `cover${size ? ` ${size}` : ""}`;
if (url && !failed) {
return (
<div className={cls}>
<img src={url} alt="" loading="lazy" onError={() => setFailed(true)} />
</div>
);
}
if (draft) return <div className={`${cls} is-draft`} aria-hidden="true" />;
const c = mediaColour(mediaType);
return <div className={cls} aria-hidden="true" style={{ background: `linear-gradient(160deg, ${c}, color-mix(in srgb, ${c} 25%, #0b0716))` }} />;
}
+551
View File
@@ -0,0 +1,551 @@
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { useEffect, useRef, useState, type FormEvent, type ReactNode } from "react";
import { Link } from "wouter";
import { api, type Copy, type CustomFieldDef, type Engagement, type EntryView } from "../api.ts";
import { orderEngagements, setCustomValue } from "../edits.ts";
import {
actorConnectionId,
describeRating,
draftIncoming,
draftSettled,
draftStartSave,
engagementNouns,
formatDateRange,
formatPartialDate,
formatRating,
friendlyError,
newDraft,
ordinal,
parsePartialDate,
rollupLabel,
statusLabel,
timeAgo,
type Draft,
type PartialDate,
} from "../format.ts";
import { useEntryPatch, useSources } from "../hooks.ts";
import { useRatingScale } from "../settings.ts";
import { Icon } from "./Icon.tsx";
import { StarRating } from "./StarRating.tsx";
function Box({ title, id, action, wide, className, children }: { title: string; id: string; action?: ReactNode; wide?: boolean; className?: string; children: ReactNode }) {
return (
<section className={`box${wide ? " wide" : ""}${className ? ` ${className}` : ""}`} aria-labelledby={id}>
<div className="box-head">
<h2 id={id}>{title}</h2>
{action}
</div>
{children}
</section>
);
}
// ---------------------------------------------------------------------------------------------
// Reads / plays
// ---------------------------------------------------------------------------------------------
export function EngagementsBox({
entry,
nouns,
engagements,
loading,
}: {
entry: EntryView;
nouns: ReturnType<typeof engagementNouns>;
engagements: Engagement[] | undefined;
loading: boolean;
}) {
const qc = useQueryClient();
const [adding, setAdding] = useState(false);
const [formError, setFormError] = useState<string | null>(null);
const refresh = () => {
void qc.invalidateQueries({ queryKey: ["engagements", entry.id] });
void qc.invalidateQueries({ queryKey: ["history", entry.id] });
};
const add = useMutation({
mutationFn: (b: { startedAt: PartialDate | null; finishedAt: PartialDate | null }) => api.addEngagement(entry.id, b),
onSuccess: () => {
setAdding(false);
refresh();
},
onError: (e) => setFormError(friendlyError(e)),
});
const [removeError, setRemoveError] = useState<string | null>(null);
const remove = useMutation({
mutationFn: api.deleteEngagement,
onSuccess: () => (setRemoveError(null), refresh()),
onError: (e) => setRemoveError(friendlyError(e)),
});
function submit(e: FormEvent<HTMLFormElement>) {
e.preventDefault();
const f = new FormData(e.currentTarget);
const read = (k: string) => {
const raw = String(f.get(k) ?? "").trim();
if (!raw) return { ok: true as const, v: null };
const pd = parsePartialDate(raw);
return pd ? { ok: true as const, v: pd } : { ok: false as const, v: null };
};
const s = read("startedAt");
const d = read("finishedAt");
if (!s.ok || !d.ok) return setFormError("Use a year, a month or a day: 2025, 2025-03 or 2025-03-14.");
setFormError(null);
add.mutate({ startedAt: s.v, finishedAt: d.v });
}
// Oldest is the 1st; show newest first, like the mockup.
const ordered = orderEngagements(engagements ?? []);
const rows = ordered.map((g, i) => ({ g, n: i + 1 })).reverse();
return (
<Box
title={nouns.box}
id="box-engagements"
action={
!adding && (
<button type="button" className="btn ghost small" onClick={() => setAdding(true)}>
<Icon name="plus" /> {nouns.log}
</button>
)
}
>
{loading ? null : rows.length === 0 && !adding ? (
<p className="empty-line">Nothing logged yet.</p>
) : (
rows.map(({ g, n }) => (
<div className="r" key={g.id}>
<span>{ordinal(n)} {nouns.one}</span>
<span style={{ display: "inline-flex", gap: 6, alignItems: "center" }}>
<span className="tabular">{formatDateRange(g.startedAt, g.finishedAt)}</span>
<button
type="button"
className="btn ghost small x"
aria-label={`Remove ${ordinal(n)} ${nouns.one}`}
disabled={remove.isPending}
onClick={() => remove.mutate(g.id)}
>
<Icon name="x" />
</button>
</span>
</div>
))
)}
{removeError && <p className="error-text" role="alert">{removeError}</p>}
{adding && (
<form onSubmit={submit} aria-label={nouns.log}>
<div className="two">
<label className="field">
<span>Started</span>
<input className="input tabular" name="startedAt" placeholder="2025-03-14" inputMode="numeric" autoFocus aria-describedby="date-hint" />
</label>
<label className="field">
<span>Finished</span>
<input className="input tabular" name="finishedAt" placeholder="2025-03" inputMode="numeric" aria-describedby="date-hint" />
</label>
</div>
<small className="hint" id="date-hint">A year, month or day is fine. Leave blank if you don't know.</small>
{formError && <p className="error-text" role="alert">{formError}</p>}
<div className="actions">
<button className="btn primary small" disabled={add.isPending}>Save</button>
<button type="button" className="btn ghost small" onClick={() => (setAdding(false), setFormError(null))}>Cancel</button>
</div>
</form>
)}
</Box>
);
}
// ---------------------------------------------------------------------------------------------
// Your copy
// ---------------------------------------------------------------------------------------------
function formatPrice(c: Copy): string {
if (c.priceCents === null) return "";
const amount = c.priceCents / 100;
if (!c.currency) return amount.toFixed(2);
try {
return new Intl.NumberFormat(undefined, { style: "currency", currency: c.currency, minimumFractionDigits: c.priceCents % 100 ? 2 : 0 }).format(amount);
} catch {
return `${amount.toFixed(2)} ${c.currency}`;
}
}
export function CopyBox({ entry }: { entry: EntryView }) {
const qc = useQueryClient();
const copies = useQuery({ queryKey: ["copies", entry.workId], queryFn: () => api.copies(entry.workId) });
const [adding, setAdding] = useState(false);
const [formError, setFormError] = useState<string | null>(null);
const add = useMutation({
mutationFn: api.addCopy,
onSuccess: () => {
setAdding(false);
void qc.invalidateQueries({ queryKey: ["copies", entry.workId] });
},
onError: (e) => setFormError(friendlyError(e)),
});
function submit(e: FormEvent<HTMLFormElement>) {
e.preventDefault();
const f = new FormData(e.currentTarget);
const s = (k: string) => String(f.get(k) ?? "").trim() || null;
const when = s("acquiredAt");
const acquiredAt = when ? parsePartialDate(when) : null;
if (when && !acquiredAt) return setFormError("Use a year, a month or a day for when: 2025, 2025-02 or 2025-02-14.");
const price = s("price");
const priceNum = price ? Number(price.replace(",", ".")) : null;
if (price && (priceNum === null || Number.isNaN(priceNum))) return setFormError("Price should be a number, like 14 or 14.99.");
const currency = s("currency")?.toUpperCase() ?? null;
if (currency && !/^[A-Z]{3}$/.test(currency)) return setFormError("Currency is a three-letter code, like GBP or EUR.");
setFormError(null);
add.mutate({
workId: entry.workId,
format: s("format"),
acquiredFrom: s("acquiredFrom"),
acquiredAt,
priceCents: priceNum === null ? null : Math.round(priceNum * 100),
currency,
});
}
const list = copies.data ?? [];
return (
<Box
title="Your copy"
id="box-copy"
action={
!adding && list.length > 0 && (
<button type="button" className="btn ghost small" onClick={() => setAdding(true)}>
<Icon name="plus" /> Another
</button>
)
}
>
{copies.isPending ? null : list.length === 0 && !adding ? (
<div className="r">
<span>You haven't recorded a copy.</span>
<button type="button" className="btn ghost small" onClick={() => setAdding(true)}>Add your copy</button>
</div>
) : (
list.map((c) => {
const when = [formatPartialDate(c.acquiredAt), formatPrice(c)].filter(Boolean).join(" · ");
return (
<div className="copy" key={c.id}>
{c.acquiredFrom && <div className="r"><span>Bought</span><span>{c.acquiredFrom}</span></div>}
{when && <div className="r"><span>When / price</span><span className="tabular">{when}</span></div>}
{c.format && <div className="r"><span>Format</span><span>{c.format}</span></div>}
{!c.acquiredFrom && !when && !c.format && <div className="r"><span>Owned</span><span>No details yet</span></div>}
</div>
);
})
)}
{adding && (
<form onSubmit={submit} aria-label="Add your copy">
<div className="two">
<label className="field"><span>Bought at</span><input className="input" name="acquiredFrom" placeholder="Waterstones, Leeds" autoFocus /></label>
<label className="field"><span>Format</span><input className="input" name="format" placeholder="Hardback" /></label>
</div>
<div className="two">
<label className="field"><span>When</span><input className="input tabular" name="acquiredAt" placeholder="2025-02" /></label>
<div className="two" style={{ gridTemplateColumns: "1fr 72px" }}>
<label className="field"><span>Price</span><input className="input tabular" name="price" inputMode="decimal" placeholder="14" /></label>
<label className="field"><span>Currency</span><input className="input" name="currency" placeholder="GBP" maxLength={3} /></label>
</div>
</div>
{formError && <p className="error-text" role="alert">{formError}</p>}
<div className="actions">
<button className="btn primary small" disabled={add.isPending}>Save</button>
<button type="button" className="btn ghost small" onClick={() => (setAdding(false), setFormError(null))}>Cancel</button>
</div>
</form>
)}
</Box>
);
}
// ---------------------------------------------------------------------------------------------
// Attachments
// ---------------------------------------------------------------------------------------------
export function AttachmentsBox({ entry }: { entry: EntryView }) {
const qc = useQueryClient();
const list = useQuery({ queryKey: ["attachments", entry.id], queryFn: () => api.attachments("entry", entry.id) });
const items = list.data ?? [];
const [error, setError] = useState<string | null>(null);
const upload = useMutation({
mutationFn: (file: File) => api.upload(file, entry.id),
onSuccess: () => {
setError(null);
void qc.invalidateQueries({ queryKey: ["attachments", entry.id] });
},
onError: (e) => setError(friendlyError(e)),
});
return (
<Box title="Attachments" id="box-attachments">
<div className="thumbs">
{items.map((a) => {
const url = `/api/v1/attachments/${a.id}`;
const image = a.mime.startsWith("image/");
return (
<a
key={a.id}
className="thumb"
href={url}
target="_blank"
rel="noreferrer"
title={a.filename}
aria-label={`Open ${a.filename}`}
style={image ? { backgroundImage: `url("${url}")` } : undefined}
>
{!image && <Icon name="file" />}
</a>
);
})}
<label className="thumb upload" title="Add screenshots, photos or receipts">
<span className="sr-only">Upload an attachment</span>
{upload.isPending ? "…" : <Icon name="upload" />}
<input
type="file"
className="sr-only"
accept="image/*,application/pdf,text/plain"
disabled={upload.isPending}
onChange={(e) => {
const f = e.currentTarget.files?.[0];
if (f) upload.mutate(f);
e.currentTarget.value = "";
}}
/>
</label>
</div>
{error && <p className="error-text" role="alert">{error}</p>}
{list.error && <p className="error-text" role="alert">{friendlyError(list.error)}</p>}
{items.length === 0 && !error && !list.isPending && <p className="hint">Screenshots, photos of your copy, receipts.</p>}
</Box>
);
}
// ---------------------------------------------------------------------------------------------
// Linked sources
// ---------------------------------------------------------------------------------------------
export function SourcesBox({ entry, engagements }: { entry: EntryView; engagements: Engagement[] }) {
const sources = useSources();
// A source is linked when it has written any field of the entry or its reads.
const ids = new Set<string>();
for (const fs of [entry.fieldSources, ...engagements.map((g) => g.fieldSources)]) {
for (const actor of Object.values(fs ?? {})) {
const id = actorConnectionId(actor);
if (id) ids.add(id);
}
}
const linked = (sources.data ?? []).filter((s) => ids.has(s.id));
return (
<Box title="Linked sources" id="box-sources">
{linked.length === 0 ? (
<p className="empty-line">No source syncs this entry. Everything here is yours.</p>
) : (
linked.map((s) => (
<div className="src" key={s.id}>
<i className={s.status === "active" ? "" : "paused"} aria-hidden="true" />
{s.name} · {s.status !== "active" ? "paused" : s.lastRunAt ? `synced ${timeAgo(s.lastRunAt)}` : "not synced yet"}
</div>
))
)}
</Box>
);
}
// ---------------------------------------------------------------------------------------------
// Custom fields
// ---------------------------------------------------------------------------------------------
export function CustomFieldsBox({ entry, type }: { entry: EntryView; type: string }) {
const defs = useQuery({ queryKey: ["custom-fields", type], queryFn: () => api.customFields(type), staleTime: 60_000 });
const patch = useEntryPatch(entry.id);
const scale = useRatingScale();
if (!defs.data?.length) return null;
const save = (def: CustomFieldDef, value: unknown) =>
patch.edit((cur) => {
const next = setCustomValue(cur.customFields, def.id, value);
return JSON.stringify(next) === JSON.stringify(cur.customFields) ? null : { customFields: next };
});
return (
<Box title="Details" id="box-custom" wide>
<div className="custom">
{defs.data.map((d) => (
<CustomField key={d.id} def={d} value={entry.customFields[d.id]} onSave={(v) => save(d, v)} />
))}
</div>
{patch.error && <p className="error-text" role="alert">{friendlyError(patch.error, { ratingScale: scale })}</p>}
</Box>
);
}
function CustomField({ def, value, onSave }: { def: CustomFieldDef; value: unknown; onSave: (v: unknown) => void }) {
const scale = useRatingScale();
const str = value === undefined || value === null ? "" : String(value);
const text = (multiline: boolean, inputType = "text") => (
<label className="field">
<span>{def.name}</span>
{multiline ? (
<textarea className="textarea" defaultValue={str} key={str} rows={3} onBlur={(e) => onSave(e.target.value.trim())} />
) : (
<input
className="input"
type={inputType}
defaultValue={str}
key={str}
placeholder={def.type === "date" ? "2025-03-14" : undefined}
onBlur={(e) => {
const v = e.target.value.trim();
if (def.type === "number") onSave(v === "" ? null : Number(v));
else if (def.type === "date") onSave(v === "" || parsePartialDate(v) ? v : str);
else onSave(v);
}}
/>
)}
</label>
);
switch (def.type) {
case "long_text":
return text(true);
case "number":
return text(false, "number");
case "url":
return text(false, "url");
case "boolean":
return (
<label className="inline">
<input type="checkbox" checked={value === true} onChange={(e) => onSave(e.target.checked ? true : null)} />
{def.name}
</label>
);
case "single_select":
return (
<label className="field">
<span>{def.name}</span>
<select className="select" value={str} onChange={(e) => onSave(e.target.value || null)}>
<option value="">—</option>
{def.options.map((o) => <option key={o} value={o}>{o}</option>)}
</select>
</label>
);
case "multi_select": {
const cur = Array.isArray(value) ? (value as string[]) : [];
return (
<fieldset className="field" style={{ border: 0 }}>
<legend className="hint" style={{ color: "var(--text-muted)", fontWeight: 500, marginBottom: 5 }}>{def.name}</legend>
<div className="checks">
{def.options.map((o) => (
<label key={o} className="inline">
<input type="checkbox" checked={cur.includes(o)} onChange={(e) => onSave(e.target.checked ? [...cur, o] : cur.filter((x) => x !== o))} />
{o}
</label>
))}
</div>
</fieldset>
);
}
case "rating":
return (
<div className="field">
<span>{def.name}</span>
<StarRating value={typeof value === "number" ? value : null} scale={scale} label={def.name} onChange={onSave} />
</div>
);
default:
return text(false);
}
}
// ---------------------------------------------------------------------------------------------
// Children (seasons, episodes, volumes)
// ---------------------------------------------------------------------------------------------
export function ChildrenBox({ entry, type }: { entry: EntryView; type: string }) {
const scale = useRatingScale();
const children = useQuery({
queryKey: ["entries", { parentId: entry.workId }],
queryFn: () => api.entries({ parentId: entry.workId, limit: 200 }),
});
const label = rollupLabel(type, entry.childSummary ? { ...entry.childSummary, completed: 0 } : null) ?? "Parts";
const title = label.replace(/^\d+\s+/, "");
return (
<Box title={title.charAt(0).toUpperCase() + title.slice(1)} id="box-children" wide>
<ul className="children">
{(children.data?.items ?? []).map((c) => (
<li key={c.id}>
<Link href={`/entries/${c.id}`}>
<span className="t">
{c.work.title}
<span className="s">{statusLabel(c.status, c.mediaTypeOverride ?? c.work.mediaType)}</span>
</span>
{c.rating !== null ? (
<span className="stars" role="img" aria-label={describeRating(c.rating, scale)}>{formatRating(c.rating, scale)}</span>
) : (
<span className="s">Not rated</span>
)}
</Link>
</li>
))}
</ul>
</Box>
);
}
// ---------------------------------------------------------------------------------------------
// Review
// ---------------------------------------------------------------------------------------------
export function ReviewBox({ entry }: { entry: EntryView }) {
const [saved, setSaved] = useState(false);
// Local text plus the server value it last agreed with (see `Draft` in format.ts). A Restore shows
// up here unless the user has typed since; a failed save keeps what they typed.
const [draft, setDraft] = useState<Draft>(() => newDraft(entry.review ?? ""));
// Saves in flight, oldest first. The entry-patch scope runs them serially, so they settle in order.
const saves = useRef<{ was: string; stored: string }[]>([]);
const patch = useEntryPatch(entry.id, {
onSettled: (_p, error) => {
const save = saves.current.shift();
if (!save) return;
setDraft((d) => draftSettled(d, save, error === null));
setSaved(error === null && saves.current.length === 0);
},
});
useEffect(() => setDraft((d) => draftIncoming(d, entry.review ?? "")), [entry.review]);
const save = () => {
const next = draft.text.trim() === "" ? null : draft.text;
const stored = next ?? "";
if (stored === draft.synced && stored === (entry.review ?? "")) return;
const s = { was: draft.synced, stored };
saves.current.push(s);
setDraft((d) => draftStartSave(d, s));
patch.mutate({ review: next });
};
return (
<Box
title="Your review"
id="box-review"
wide
className="review"
action={<span className="saved" aria-live="polite">{patch.isPending ? "Saving…" : saved ? "Saved" : ""}</span>}
>
<label htmlFor="review-text" className="sr-only">Your review (Markdown)</label>
<textarea
id="review-text"
value={draft.text}
placeholder="What stayed with you? Markdown works here."
onChange={(e) => {
const text = e.target.value;
setDraft((d) => ({ ...d, text }));
setSaved(false);
}}
onBlur={save}
onKeyDown={(e) => {
if (e.key === "Enter" && (e.metaKey || e.ctrlKey)) save();
}}
/>
{patch.error && <p className="error-text" role="alert">{friendlyError(patch.error)}</p>}
</Box>
);
}
+187
View File
@@ -0,0 +1,187 @@
import { useQuery } from "@tanstack/react-query";
import { useRef, useState } from "react";
import { Link, useLocation } from "wouter";
import { ApiError, api, type EntryView } from "../api.ts";
import { addTag, removeTag } from "../edits.ts";
import { engagementNouns, formatPartialDate, friendlyError, mediaTypeLabel, shortDate, STATUSES, statusLabel } from "../format.ts";
import { changesFor, useEntryPatch, usePendingChanges, useSources } from "../hooks.ts";
import { useRatingScale } from "../settings.ts";
import { Cover } from "./Cover.tsx";
import { AttachmentsBox, ChildrenBox, CopyBox, CustomFieldsBox, EngagementsBox, ReviewBox, SourcesBox } from "./DetailBoxes.tsx";
import { HistoryDrawer } from "./HistoryDrawer.tsx";
import { Icon } from "./Icon.tsx";
import { LinkToCatalogue } from "./LinkToCatalogue.tsx";
import { StarRating } from "./StarRating.tsx";
const TIMES = ["", "once", "twice", "three times"];
export function EntryDetail({ id }: { id: string }) {
const entry = useQuery({ queryKey: ["entry", id], queryFn: () => api.entry(id) });
if (entry.isPending) return <div className="detail" aria-busy="true" />;
// A failed background refetch (server briefly down) keeps showing the cached entry, so nothing the
// user is typing gets unmounted; only a first load that fails shows the error screen.
if (!entry.data) {
const gone = entry.error instanceof ApiError && entry.error.status === 404;
return (
<div className="detail-empty" role="alert">
<p>
<strong>{gone ? "This entry isn't here" : "Couldn't load this entry"}</strong>
{gone ? "It may have been deleted or moved. Pick another from your library." : friendlyError(entry.error)}
</p>
</div>
);
}
return <Detail entry={entry.data} />;
}
function Detail({ entry }: { entry: EntryView }) {
const scale = useRatingScale();
const patch = useEntryPatch(entry.id);
const [, navigate] = useLocation();
const [historyOpen, setHistoryOpen] = useState(false);
const historyBtn = useRef<HTMLButtonElement>(null);
const type = entry.mediaTypeOverride ?? entry.work.mediaType;
const nouns = engagementNouns(type);
const engagements = useQuery({ queryKey: ["engagements", entry.id], queryFn: () => api.engagements(entry.id) });
const changes = usePendingChanges();
const sources = useSources();
const engagementIds = new Set((engagements.data ?? []).map((g) => g.id));
const pending = changesFor(changes.data?.items, entry.id, engagementIds);
// "Book · Read twice · last finished 14 Sep 2026"
const finished = (engagements.data ?? []).filter((g) => g.finishedAt && g.finishedAt.precision !== "unknown");
const meta = [mediaTypeLabel(type)];
if (entry.work.year) meta.push(String(entry.work.year));
if (finished.length) {
meta.push(`${nouns.past} ${TIMES[finished.length] ?? `${finished.length} times`}`);
const last = finished.map((g) => g.finishedAt!).sort((a, b) => (a.value < b.value ? 1 : -1))[0]!;
meta.push(`last finished ${formatPartialDate(last)}`);
}
return (
<article className="detail" aria-labelledby="entry-title">
<Link href="/" className="btn ghost back"><Icon name="back" /> Library</Link>
<header className="det-head">
<Cover url={entry.work.coverUrl} mediaType={type} draft={entry.draft} size="large" />
<div style={{ minWidth: 0 }}>
<h1 id="entry-title">
{entry.work.title}
{entry.draft && <span className="draft">Draft</span>}
</h1>
<p className="m">{meta.join(" · ")}</p>
<div className="det-controls">
<StarRating value={entry.rating} scale={scale} label="Your rating" onChange={(rating) => patch.mutate({ rating })} />
<label>
<span className="sr-only">Status</span>
<select className="select" value={entry.status} onChange={(e) => patch.mutate({ status: e.target.value as EntryView["status"] })}>
{STATUSES.map((s) => (
<option key={s} value={s}>{statusLabel(s, type)}</option>
))}
</select>
</label>
<div className="det-tools">
<button
ref={historyBtn}
type="button"
className="btn ghost small"
aria-expanded={historyOpen}
aria-controls="history-drawer"
onClick={() => setHistoryOpen((o) => !o)}
>
<Icon name="history" /> History
</button>
</div>
</div>
<Tags
tags={entry.tags}
onAdd={(t) => patch.edit((cur) => ({ tags: addTag(cur.tags, t) }))}
onRemove={(t) => patch.edit((cur) => ({ tags: removeTag(cur.tags, t) }))}
/>
{patch.error && <p className="error-text" role="alert">{friendlyError(patch.error, { ratingScale: scale })}</p>}
</div>
</header>
{pending.length > 0 && (
<div className="alert" role="status">
<p>
<b>{whoChanged(pending, sources.data)} changed this {mediaTypeLabel(type).toLowerCase()}.</b>{" "}
{pending.length === 1 ? `On ${shortDate(pending[0]!.createdAt)}.` : `${pending.length} changes are waiting for you.`} Your rating,{" "}
{nouns.box.toLowerCase()} and review here are unchanged.
</p>
<button type="button" className="btn" onClick={() => navigate(`/changes?entry=${entry.id}`)}>See what changed</button>
</div>
)}
{entry.draft && <LinkToCatalogue entry={entry} />}
<div className="det-grid">
<EngagementsBox entry={entry} nouns={nouns} engagements={engagements.data} loading={engagements.isPending} />
<CopyBox entry={entry} />
<AttachmentsBox entry={entry} />
<SourcesBox entry={entry} engagements={engagements.data ?? []} />
<CustomFieldsBox entry={entry} type={type} />
{entry.childSummary && <ChildrenBox entry={entry} type={type} />}
<ReviewBox entry={entry} />
</div>
{historyOpen && (
<HistoryDrawer
entry={entry}
onClose={() => {
setHistoryOpen(false);
historyBtn.current?.focus();
}}
/>
)}
</article>
);
}
function whoChanged(pending: { connectionId: string | null }[], sources: { id: string; name: string }[] | undefined): string {
const ids = new Set(pending.map((p) => p.connectionId));
if (ids.size === 1) {
const name = sources?.find((s) => s.id === [...ids][0])?.name;
if (name) return name;
}
return ids.size > 1 ? "Your sources" : "A source";
}
function Tags({ tags, onAdd, onRemove }: { tags: string[]; onAdd: (t: string) => void; onRemove: (t: string) => void }) {
const [draft, setDraft] = useState("");
function add() {
if (draft.trim()) onAdd(draft);
setDraft("");
}
return (
<ul className="pills" aria-label="Tags" style={{ listStyle: "none" }}>
{tags.map((t) => (
<li key={t} className="pill">
{t}
<button type="button" aria-label={`Remove tag ${t}`} onClick={() => onRemove(t)}>
<Icon name="x" />
</button>
</li>
))}
<li>
<label>
<span className="sr-only">Add a tag</span>
<input
className="tag-input"
placeholder="+ tag"
value={draft}
maxLength={100}
onChange={(e) => setDraft(e.target.value)}
onKeyDown={(e) => {
if (e.key === "Enter" || e.key === ",") {
e.preventDefault();
add();
}
}}
onBlur={add}
/>
</label>
</li>
</ul>
);
}
+117
View File
@@ -0,0 +1,117 @@
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { useEffect, useRef } from "react";
import { api, type EntryView, type HistoryRecord } from "../api.ts";
import { actorConnectionId, formatPartialDate, formatRating, friendlyError, statusLabel, timeAgo, type PartialDate, type Status } from "../format.ts";
import { useSources } from "../hooks.ts";
import { useRatingScale } from "../settings.ts";
import { Icon } from "./Icon.tsx";
const LABELS: Record<string, string> = {
status: "Status",
rating: "Rating",
review: "Review",
privateNotes: "Private notes",
favorite: "Favourite",
tags: "Tags",
customFields: "Details",
visibility: "Visibility",
overrides: "Title and cover",
mediaTypeOverride: "Media type",
workId: "Catalogue link",
deletedAt: "Removed",
};
const SKIP = new Set(["id", "userId", "fieldSources", "createdAt", "updatedAt"]);
/** History of an entry, newest first, with "Restore" to put back what a change replaced. */
export function HistoryDrawer({ entry, onClose }: { entry: EntryView; onClose: () => void }) {
const qc = useQueryClient();
const scale = useRatingScale();
const sources = useSources();
const closeBtn = useRef<HTMLButtonElement>(null);
const history = useQuery({ queryKey: ["history", entry.id], queryFn: () => api.history(entry.id) });
const restore = useMutation({
// Same scope as the entry's PATCHes (useEntryPatch), so a restore never races a pending blur-save.
scope: { id: `entry-patch:${entry.id}` },
mutationFn: api.restore,
onSuccess: () => {
for (const key of [["entry", entry.id], ["history", entry.id], ["engagements", entry.id], ["entries"]]) void qc.invalidateQueries({ queryKey: key });
},
});
useEffect(() => {
closeBtn.current?.focus();
const onKey = (e: KeyboardEvent) => e.key === "Escape" && onClose();
window.addEventListener("keydown", onKey);
return () => window.removeEventListener("keydown", onKey);
}, [onClose]);
const who = (actor: string) => {
if (actor === "user") return "You";
if (actor.startsWith("token:")) return `App token “${actor.slice(6)}”`;
const id = actorConnectionId(actor);
return sources.data?.find((s) => s.id === id)?.name ?? actor.split(":")[1] ?? "A source";
};
const show = (field: string, v: unknown): string => {
if (v === null || v === undefined || v === "") return "empty";
if (field === "rating") return formatRating(v as number, scale);
if (field === "status") return statusLabel(v as Status, entry.mediaTypeOverride ?? entry.work.mediaType);
if (Array.isArray(v)) return v.length ? v.join(", ") : "none";
if (typeof v === "object" && "precision" in (v as object)) return formatPartialDate(v as PartialDate) || "unknown";
if (typeof v === "object") return Object.keys(v as object).length ? "edited" : "empty";
if (typeof v === "boolean") return v ? "yes" : "no";
const s = String(v);
return s.length > 80 ? `${s.slice(0, 80)}…` : s;
};
const diffs = (h: HistoryRecord) =>
[...new Set([...Object.keys(h.before ?? {}), ...Object.keys(h.after ?? {})])]
.filter((k) => !SKIP.has(k) && JSON.stringify(h.before?.[k]) !== JSON.stringify(h.after?.[k]))
.map((k) => ({ k, before: show(k, h.before?.[k]), after: show(k, h.after?.[k]) }));
return (
<aside className="drawer" id="history-drawer" role="dialog" aria-modal="false" aria-labelledby="history-title">
<header>
<h2 id="history-title">History</h2>
<button ref={closeBtn} type="button" className="btn ghost small" onClick={onClose} aria-label="Close history">
<Icon name="x" />
</button>
</header>
{history.isPending ? null : history.error ? (
<p className="error-text" style={{ padding: "0 20px" }} role="alert">{friendlyError(history.error)}</p>
) : (
<ol>
{history.data.map((h) => {
const d = diffs(h);
const created = Object.keys(h.before ?? {}).length === 0;
return (
<li key={h.id}>
<div className="when">
<span><span className="who">{who(h.actor)}</span> · {timeAgo(h.createdAt)}</span>
{!created && d.length > 0 && (
<button type="button" className="btn small" disabled={restore.isPending} onClick={() => restore.mutate(h.id)}>
Restore
</button>
)}
</div>
{created ? (
<p className="hint">Added to your library.</p>
) : d.length === 0 ? (
<p className="hint">No visible change.</p>
) : (
<dl>
{d.map(({ k, before, after }) => (
<div key={k} style={{ display: "contents" }}>
<dt>{LABELS[k] ?? k}</dt>
<dd><s>{before}</s><span className="arrow" aria-label="changed to">→</span>{after}</dd>
</div>
))}
</dl>
)}
</li>
);
})}
</ol>
)}
{restore.error && <p className="error-text" role="alert" style={{ padding: "0 20px 16px" }}>{friendlyError(restore.error)}</p>}
</aside>
);
}
+21
View File
@@ -0,0 +1,21 @@
/** A small authored icon set: 24px grid, 1.75 stroke, round caps. */
const PATHS = {
plus: "M12 5v14M5 12h14",
search: "M11 18a7 7 0 1 0 0-14 7 7 0 0 0 0 14zM20 20l-4-4",
x: "M6 6l12 12M18 6L6 18",
history: "M3 12a9 9 0 1 0 3-6.7L3 8M3 3v5h5M12 7v5l3 2",
back: "M15 18l-6-6 6-6",
upload: "M12 16V4M7 9l5-5 5 5M5 20h14",
link: "M10 14a4 4 0 0 0 5.7 0l3-3a4 4 0 0 0-5.7-5.7l-1 1M14 10a4 4 0 0 0-5.7 0l-3 3a4 4 0 0 0 5.7 5.7l1-1",
file: "M14 3H7a2 2 0 0 0-2 2v14a2 2 0 0 0 2 2h10a2 2 0 0 0 2-2V8zM14 3v5h5",
} as const;
export type IconName = keyof typeof PATHS;
export function Icon({ name }: { name: IconName }) {
return (
<svg className="icon" viewBox="0 0 24 24" aria-hidden="true" focusable="false">
<path d={PATHS[name]} />
</svg>
);
}
+102
View File
@@ -0,0 +1,102 @@
import { useMutation, useQueries, useQuery, useQueryClient } from "@tanstack/react-query";
import { useState, type ReactNode } from "react";
import { useLocation } from "wouter";
import { api, type EntryView, type Work } from "../api.ts";
import { friendlyError, mediaTypeLabel } from "../format.ts";
import { useDebounced } from "../hooks.ts";
import { Cover } from "./Cover.tsx";
import { Icon } from "./Icon.tsx";
/** Drafts only: suggested matches and a catalogue search. Linking keeps the user's rating, reads and review. */
export function LinkToCatalogue({ entry }: { entry: EntryView }) {
const qc = useQueryClient();
const [, navigate] = useLocation();
const [q, setQ] = useState("");
const term = useDebounced(q.trim());
const suggestions = useQuery({ queryKey: ["suggestions", entry.workId], queryFn: () => api.suggestions(entry.workId) });
const candidates = useQueries({
queries: (suggestions.data ?? []).slice(0, 5).map((s) => ({ queryKey: ["work", s.candidateWorkId], queryFn: () => api.work(s.candidateWorkId), staleTime: 300_000 })),
});
const search = useQuery({
queryKey: ["works-search", term],
queryFn: () => api.searchWorks(term),
enabled: term.length >= 2,
});
const done = (newId?: string) => {
void qc.invalidateQueries({ queryKey: ["entries"] });
void qc.invalidateQueries({ queryKey: ["entry", entry.id] });
void qc.invalidateQueries({ queryKey: ["suggestions", entry.workId] });
if (newId && newId !== entry.id) navigate(`/entries/${newId}`, { replace: true });
};
const move = useMutation({ mutationFn: (workId: string) => api.moveEntry(entry.id, workId), onSuccess: (e) => done(e.id) });
const accept = useMutation({ mutationFn: api.acceptSuggestion, onSuccess: () => done() });
const reject = useMutation({ mutationFn: api.rejectSuggestion, onSuccess: () => done() });
const busy = move.isPending || accept.isPending || reject.isPending;
const error = move.error ?? accept.error ?? reject.error;
const row = (w: Work, actions: ReactNode) => (
<li key={w.id}>
<Cover url={w.cover_url} mediaType={w.media_type} draft={false} size="small" />
<div style={{ minWidth: 0 }}>
<div>{w.title}</div>
<div className="m">{[mediaTypeLabel(w.media_type), w.year, w.creators?.[0]?.name].filter(Boolean).join(" · ")}</div>
</div>
<div className="acts">{actions}</div>
</li>
);
const results = (search.data ?? []).filter((w) => w.id !== entry.workId);
return (
<section className="box link-box" aria-labelledby="link-title">
<div className="box-head">
<h2 id="link-title"><Icon name="link" /> Link to catalogue</h2>
</div>
<p>This is a draft. Link it to a catalogue entry to fill in the cover and details. Your rating, reads and review stay as they are.</p>
{candidates.length > 0 && (
<>
<h3 className="sub">Suggested</h3>
<ul className="results">
{suggestions.data!.slice(0, 5).map((s, i) => {
const w = candidates[i]?.data;
if (!w) return null;
return row(
w,
<>
<button type="button" className="btn primary small" disabled={busy} onClick={() => accept.mutate(s.id)}>Link</button>
<button type="button" className="btn ghost small" disabled={busy} onClick={() => reject.mutate(s.id)}>Not this</button>
</>,
);
})}
</ul>
</>
)}
<div className="search">
<Icon name="search" />
<label htmlFor="catalogue-search" className="sr-only">Search the catalogue</label>
<input
id="catalogue-search"
className="input"
type="search"
placeholder="Search the catalogue…"
value={q}
onChange={(e) => setQ(e.target.value)}
/>
</div>
{term.length >= 2 && !search.isPending && (
results.length === 0 ? (
<p className="hint">Nothing in the catalogue matches “{term}”. Connect a source to bring in more.</p>
) : (
<ul className="results" aria-label="Search results">
{results.map((w) =>
row(w, <button type="button" className="btn primary small" disabled={busy} onClick={() => move.mutate(w.id)}>Link</button>),
)}
</ul>
)
)}
{error && <p className="error-text" role="alert">{friendlyError(error)}</p>}
</section>
);
}
+79
View File
@@ -0,0 +1,79 @@
import { useQueryClient } from "@tanstack/react-query";
import type { ReactNode } from "react";
import { Link, useLocation } from "wouter";
import { api } from "../api.ts";
import { useMe, usePendingChanges, useSources } from "../hooks.ts";
import { resetLibraryState } from "../routes/library.tsx";
function NavLink({ href, active, children, className }: { href: string; active: boolean; children: ReactNode; className?: string }) {
return (
<Link href={href} className={`nav${className ? ` ${className}` : ""}`} aria-current={active ? "page" : undefined}>
{children}
</Link>
);
}
/** Sidebar navigation plus the main area. Stacks into a top bar on narrow screens. */
export function Shell({ children }: { children: ReactNode }) {
const [path, navigate] = useLocation();
const qc = useQueryClient();
const me = useMe();
const changes = usePendingChanges();
const sources = useSources();
const pending = changes.data?.items.length ?? 0;
const at = (p: string) => path === p || path.startsWith(`${p}/`);
async function signOut() {
try {
await api.logout();
} catch {
// The session may already be gone; signing out locally is still right.
}
qc.clear();
resetLibraryState();
navigate("/login", { replace: true });
}
return (
<div className="shell">
<aside className="side">
<Link href="/" className="logo">STTracker</Link>
<nav aria-label="Main">
<NavLink href="/" active={path === "/" || at("/entries")}>Library</NavLink>
<NavLink href="/log" active={at("/log")}>Log</NavLink>
<NavLink href="/lists" active={at("/lists")}>Lists</NavLink>
<NavLink href="/owned" active={at("/owned")}>Owned</NavLink>
<NavLink href="/changes" active={at("/changes")}>
Changes
{pending > 0 && (
<span className="count">
{changes.data?.nextCursor ? `${pending}+` : pending}
<span className="sr-only"> waiting</span>
</span>
)}
</NavLink>
</nav>
<h2 className="grp">Sources</h2>
<nav aria-label="Sources" className="src-nav">
{(sources.data ?? []).map((s) => (
<NavLink key={s.id} href={`/sources/${s.id}`} active={at(`/sources/${s.id}`)}>
<span className="name">{s.name}</span>
<span className={`dot${s.status === "active" ? "" : " paused"}`} role="img" aria-label={s.status === "active" ? "Active" : "Paused"} />
</NavLink>
))}
<NavLink href="/sources" active={path === "/sources"}>
{sources.data?.length ? "All sources" : "Connect a source"}
</NavLink>
</nav>
<nav aria-label="Account" style={{ marginTop: 14 }}>
<NavLink href="/settings" active={at("/settings")}>Settings</NavLink>
</nav>
<div className="side-foot">
{me.data && <span className="who">{me.data.displayName || me.data.username}</span>}
<button type="button" onClick={() => void signOut()}>Sign out</button>
</div>
</aside>
<main className="main">{children}</main>
</div>
);
}
+148
View File
@@ -0,0 +1,148 @@
import { useEffect, useId, useRef, useState, type KeyboardEvent, type PointerEvent } from "react";
import { describeRating, ratingStep, snapRating, type RatingScale } from "../format.ts";
const STAR = "M12 2.6l2.9 5.9 6.5.9-4.7 4.6 1.1 6.5L12 17.4l-5.8 3.1 1.1-6.5L2.6 9.4l6.5-.9z";
/**
* Editable rating in the user's scale. Stars are a slider: arrows step (half a star on "5half"),
* Home/End jump, Delete clears. The 10 and 100 point scales use a number field.
*/
export function StarRating({
value,
scale,
label,
onChange,
}: {
value: number | null;
scale: RatingScale;
label: string;
onChange: (v: number | null) => void;
}) {
const [hover, setHover] = useState<number | null>(null);
// Arrow keys step a local value; it is saved once the user pauses (or leaves), not on every step.
const [stepping, setStepping] = useState<{ v: number | null } | null>(null);
const timer = useRef<ReturnType<typeof setTimeout> | null>(null);
const clipBase = useId();
const step = ratingStep(scale);
// Leaving mid-step (navigating to another entry) still saves the stepped value.
const latest = useRef({ stepping, value, onChange });
latest.current = { stepping, value, onChange };
useEffect(
() => () => {
if (timer.current) clearTimeout(timer.current);
const { stepping: st, value: v, onChange: save } = latest.current;
if (st && st.v !== v) save(st.v);
},
[],
);
if (scale === "10" || scale === "100") {
const max = scale === "10" ? 10 : 100;
const shown = value === null ? "" : String(scale === "10" ? snapRating(value, scale) / 10 : value);
return (
<label className="rating-number">
<span className="sr-only">{label}</span>
<input
className="input tabular"
type="number"
min={0}
max={max}
step={scale === "10" ? 0.5 : 1}
defaultValue={shown}
key={shown}
onBlur={(e) => {
const raw = e.currentTarget.value.trim();
if (raw === "") return value !== null && onChange(null);
const n = Number(raw);
if (Number.isNaN(n)) return;
const v = snapRating(scale === "10" ? n * 10 : n, scale);
if (v !== value) onChange(v);
}}
/>
<span>/ {max}</span>
</label>
);
}
const set = (v: number | null) => {
if (timer.current) clearTimeout(timer.current);
timer.current = null;
setStepping(null);
if (v !== value) onChange(v === null ? null : snapRating(v, scale));
};
const flush = () => stepping && set(stepping.v);
const current = stepping ? stepping.v : value;
const onKey = (e: KeyboardEvent) => {
const cur = current ?? 0;
const map: Record<string, number | null> = {
ArrowRight: Math.min(100, cur + step),
ArrowUp: Math.min(100, cur + step),
ArrowLeft: Math.max(0, cur - step),
ArrowDown: Math.max(0, cur - step),
Home: 0,
End: 100,
Delete: null,
Backspace: null,
};
if (!(e.key in map)) return;
e.preventDefault();
const next = map[e.key]!;
setStepping({ v: next });
if (timer.current) clearTimeout(timer.current);
timer.current = setTimeout(() => set(next), 500);
};
const valueAt = (e: PointerEvent<HTMLSpanElement>, i: number) => {
const r = e.currentTarget.getBoundingClientRect();
const half = scale === "5half" && e.clientX - r.left < r.width / 2;
return (i + (half ? 0.5 : 1)) * 20;
};
const shown = hover ?? current ?? 0;
return (
<span className="rating-wrap" style={{ display: "inline-flex", alignItems: "center" }}>
<span
className="rating"
role="slider"
tabIndex={0}
aria-label={label}
aria-valuemin={0}
aria-valuemax={100}
aria-valuenow={current ?? undefined}
aria-valuetext={describeRating(current, scale)}
onKeyDown={onKey}
onBlur={flush}
onPointerLeave={() => setHover(null)}
>
{[0, 1, 2, 3, 4].map((i) => {
const fill = Math.max(0, Math.min(1, shown / 20 - i));
const id = `${clipBase}-${i}`;
return (
<span
key={i}
onPointerMove={(e) => setHover(valueAt(e, i))}
onClick={(e) => {
const v = valueAt(e as unknown as PointerEvent<HTMLSpanElement>, i);
set(v === value ? null : v);
}}
>
<svg viewBox="0 0 24 24" aria-hidden="true">
<defs>
<clipPath id={id}>
<rect x="0" y="0" width={24 * fill} height="24" />
</clipPath>
</defs>
<path className="empty" d={STAR} />
{fill > 0 && <path className="full" d={STAR} clipPath={`url(#${id})`} />}
</svg>
</span>
);
})}
</span>
{current !== null && (
<button type="button" className="rating-clear" onClick={() => set(null)}>
Clear
</button>
)}
</span>
);
}
+42
View File
@@ -0,0 +1,42 @@
/**
* Next-value builders for entry edits. Callers build from the latest cached entry (not a render's
* props), so quick successive edits (two tag adds) compose instead of overwriting each other.
*/
import type { PartialDate } from "./format.ts";
export function addTag(tags: string[], tag: string): string[] {
const t = tag.trim();
if (!t || tags.some((x) => x.toLowerCase() === t.toLowerCase())) return tags;
return [...tags, t];
}
export function removeTag(tags: string[], tag: string): string[] {
return tags.filter((x) => x !== tag);
}
/** Set a custom field value by definition id; null, "" and [] remove it. */
export function setCustomValue(fields: Record<string, unknown>, id: string, value: unknown): Record<string, unknown> {
const next = { ...fields };
if (value === null || value === undefined || value === "" || (Array.isArray(value) && value.length === 0)) delete next[id];
else next[id] = value;
return next;
}
/** The optimistic view of an entry after a PATCH. */
export function applyEntryPatch<T extends object>(entry: T, patch: Partial<T>): T {
return { ...entry, ...patch };
}
/**
* Reads/plays oldest first: by `finishedAt ?? startedAt` (partial dates compare as strings), then by
* when they were logged. Undated ones count as oldest.
*/
export function orderEngagements<G extends { createdAt: string; startedAt: PartialDate | null; finishedAt: PartialDate | null }>(list: G[]): G[] {
const key = (g: G) => (g.finishedAt?.value || g.startedAt?.value || "");
return [...list].sort((a, b) => {
const ka = key(a);
const kb = key(b);
if (ka !== kb) return ka < kb ? -1 : 1;
return a.createdAt < b.createdAt ? -1 : a.createdAt > b.createdAt ? 1 : 0;
});
}
+93
View File
@@ -0,0 +1,93 @@
Copyright 2022 The Gelasio Project Authors (https://github.com/SorkinType/Gelasio)
This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
https://openfontlicense.org
-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.
+93
View File
@@ -0,0 +1,93 @@
Copyright 2016 The Inter Project Authors (https://github.com/rsms/inter) Inter-Italic[opsz,wght].ttf: Copyright 2016 The Inter Project Authors (https://github.com/rsms/inter)
This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
http://scripts.sil.org/OFL
-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+314
View File
@@ -0,0 +1,314 @@
/** Display helpers. Pure functions: no DOM, no network. Partial dates never go through `Date`. */
export type Precision = "day" | "month" | "year" | "unknown";
export interface PartialDate {
value: string;
precision: Precision;
}
/** How ratings are shown and entered. Stored ratings are always integers 0–100. */
export type RatingScale = "5half" | "5" | "10" | "100";
export const MONTHS = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"];
/** Size of one step of the scale, in 0–100 units. */
export function ratingStep(scale: RatingScale): number {
return scale === "5half" ? 10 : scale === "5" ? 20 : scale === "10" ? 5 : 1;
}
/** Snap a 0–100 value to the scale's nearest step. */
export function snapRating(value: number, scale: RatingScale): number {
const step = ratingStep(scale);
return Math.min(100, Math.max(0, Math.round(value / step) * step));
}
/** `formatRating(90, "5half")` is `"★★★★½"`; null is `""`. */
export function formatRating(value: number | null | undefined, scale: RatingScale): string {
if (value === null || value === undefined) return "";
const v = snapRating(value, scale);
if (scale === "10") return `${v / 10}/10`;
if (scale === "100") return `${v}/100`;
const stars = v / 20;
const full = Math.floor(stars);
const half = stars - full >= 0.5;
return "★".repeat(full) + (half ? "½" : "") + "☆".repeat(5 - full - (half ? 1 : 0));
}
/** Screen-reader text for a rating: "4½ stars", "9 out of 10", "Not rated". */
export function describeRating(value: number | null | undefined, scale: RatingScale): string {
if (value === null || value === undefined) return "Not rated";
const v = snapRating(value, scale);
if (scale === "10") return `${v / 10} out of 10`;
if (scale === "100") return `${v} out of 100`;
const stars = v / 20;
const whole = Math.floor(stars);
const label = stars % 1 ? `${whole === 0 ? "" : whole}½` : String(whole);
return `${label} ${stars === 1 ? "star" : "stars"}`;
}
/** `{value:"2025-03",precision:"month"}` is `"Mar 2025"`. Unknown or missing dates are `""`. */
export function formatPartialDate(pd: PartialDate | null | undefined): string {
if (!pd || pd.precision === "unknown" || !pd.value) return "";
const y = pd.value.slice(0, 4);
if (pd.precision === "year") return y;
const month = MONTHS[Number(pd.value.slice(5, 7)) - 1] ?? "";
if (pd.precision === "month") return `${month} ${y}`;
return `${Number(pd.value.slice(8, 10))} ${month} ${y}`;
}
const isExact = (pd: PartialDate | null | undefined) => !pd || pd.precision === "day";
/**
* A read/play's dates: "2 Sep – 14 Sep 2026", "Mar 2025 (no exact date)", "Started 2 Sep 2026".
* Anything coarser than a day is flagged so a month never passes for a precise date.
*/
export function formatDateRange(start: PartialDate | null | undefined, end: PartialDate | null | undefined): string {
const s = formatPartialDate(start);
const e = formatPartialDate(end);
const suffix = isExact(start) && isExact(end) ? "" : " (no exact date)";
if (!s && !e) return "No dates";
if (s && !e) return `Started ${s}${suffix}`;
if (!s) return `${e}${suffix}`;
if (s === e) return `${e}${suffix}`;
// Drop the repeated year: "2 Sep – 14 Sep 2026".
const sameYear = start!.value.slice(0, 4) === end!.value.slice(0, 4) && start!.precision !== "year";
const left = sameYear ? s.slice(0, -5) : s;
return `${left} – ${e}${suffix}`;
}
/** Accepts `YYYY`, `YYYY-MM` and `YYYY-MM-DD`. Anything else (including `03/04/2019`) is null. */
export function parsePartialDate(input: string): PartialDate | null {
const s = input.trim();
const m = /^(\d{4})(?:-(\d{2})(?:-(\d{2}))?)?$/.exec(s);
if (!m) return null;
if (m[2] === undefined) return { value: s, precision: "year" };
const month = Number(m[2]);
if (month < 1 || month > 12) return null;
if (m[3] === undefined) return { value: s, precision: "month" };
const day = Number(m[3]);
const y = Number(m[1]);
const leap = (y % 4 === 0 && y % 100 !== 0) || y % 400 === 0;
const dim = [31, leap ? 29 : 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31][month - 1]!;
return day >= 1 && day <= dim ? { value: s, precision: "day" } : null;
}
const TYPE_LABELS: Record<string, string> = {
book: "Book",
comic: "Comic",
manga: "Manga",
game: "Game",
movie: "Movie",
show: "Show",
season: "Season",
episode: "Episode",
podcast: "Podcast",
podcast_episode: "Podcast episode",
audio_drama: "Audio drama",
artist: "Artist",
album: "Album",
track: "Track",
video: "Video",
other: "Other",
};
export function mediaTypeLabel(type: string): string {
return TYPE_LABELS[type] ?? type.charAt(0).toUpperCase() + type.slice(1).replace(/_/g, " ");
}
/** The cover colour family for a media type (DESIGN.md "Media type colours"). */
export function mediaColour(type: string): string {
if (["book", "comic", "manga"].includes(type)) return "#6b3fa0";
if (["podcast", "podcast_episode", "audio_drama"].includes(type)) return "#2f6b58";
if (type === "game") return "#c1652f";
if (["movie", "show", "season", "episode", "video"].includes(type)) return "#344b7a";
if (["artist", "album", "track"].includes(type)) return "#a33b62";
return "#4b4485";
}
const CHILD_NOUN: Record<string, [string, string]> = {
show: ["season", "seasons"],
season: ["episode", "episodes"],
podcast: ["episode", "episodes"],
audio_drama: ["episode", "episodes"],
manga: ["volume", "volumes"],
comic: ["issue", "issues"],
book: ["volume", "volumes"],
album: ["track", "tracks"],
artist: ["album", "albums"],
};
/** Children roll up as "season 3 of 5", "5 seasons" (none finished) or "all 200 episodes". */
export function rollupLabel(type: string, summary: { total: number; completed: number } | null): string | null {
if (!summary || summary.total === 0) return null;
const [one, many] = CHILD_NOUN[type] ?? ["part", "parts"];
const { total, completed } = summary;
if (completed === 0) return `${total} ${total === 1 ? one : many}`;
if (completed >= total) return total === 1 ? `1 ${one}` : `all ${total} ${many}`;
return `${one} ${completed} of ${total}`;
}
export type Status = "planned" | "in_progress" | "paused" | "completed" | "dropped" | "none";
export const STATUSES: Status[] = ["planned", "in_progress", "paused", "completed", "dropped", "none"];
/** Status in the words of the medium: "Reading", "Playing", "Watching", "Listening". */
export function statusLabel(status: Status, type = "other"): string {
if (status === "in_progress") {
const verb = engagementNouns(type).verb;
return verb;
}
return { planned: "Planned", paused: "Paused", completed: "Finished", dropped: "Dropped", none: "No status" }[status];
}
/** What one time through a work is called: reads, plays, watches, listens. */
export function engagementNouns(type: string): { box: string; one: string; verb: string; past: string; log: string } {
if (["book", "comic", "manga"].includes(type)) return { box: "Reads", one: "read", verb: "Reading", past: "Read", log: "Log a read" };
if (type === "game") return { box: "Plays", one: "play", verb: "Playing", past: "Played", log: "Log a play" };
if (["movie", "show", "season", "episode", "video"].includes(type))
return { box: "Watches", one: "watch", verb: "Watching", past: "Watched", log: "Log a watch" };
if (["podcast", "podcast_episode", "audio_drama", "artist", "album", "track"].includes(type))
return { box: "Listens", one: "listen", verb: "Listening", past: "Listened", log: "Log a listen" };
return { box: "Times through", one: "time", verb: "In progress", past: "Finished", log: "Log a time" };
}
/** "12 Oct" for a real timestamp, in local time. */
export function shortDate(iso: string): string {
const d = new Date(iso);
return `${d.getDate()} ${MONTHS[d.getMonth()]}`;
}
/** 1 → "1st", 2 → "2nd", 11 → "11th". */
export function ordinal(n: number): string {
const tens = n % 100;
if (tens >= 11 && tens <= 13) return `${n}th`;
return `${n}${["th", "st", "nd", "rd"][n % 10] ?? "th"}`;
}
/** "4 min ago", "3 h ago", "2 days ago", else "14 Sep 2026". For real timestamps only. */
export function timeAgo(iso: string, now: Date = new Date()): string {
const t = new Date(iso);
const s = Math.round((now.getTime() - t.getTime()) / 1000);
if (Number.isNaN(s)) return "";
if (s < 60) return "just now";
if (s < 3600) return `${Math.floor(s / 60)} min ago`;
if (s < 86400) return `${Math.floor(s / 3600)} h ago`;
if (s < 86400 * 7) return `${Math.floor(s / 86400)} ${s < 86400 * 2 ? "day" : "days"} ago`;
return `${t.getDate()} ${MONTHS[t.getMonth()]} ${t.getFullYear()}`;
}
/** The connection id in a plugin actor `plugin:<pluginId>:<connectionId>`, else null. */
export function actorConnectionId(actor: string): string | null {
const m = /^plugin:[^:]+:(.+)$/.exec(actor);
return m ? m[1]! : null;
}
// ---------------------------------------------------------------------------------------------
// Errors in plain words
// ---------------------------------------------------------------------------------------------
const FIELD_LABELS: Record<string, string> = {
startedAt: "Started",
finishedAt: "Finished",
acquiredAt: "When",
acquiredFrom: "Bought at",
priceCents: "Price",
privateNotes: "Private notes",
inviteCode: "Invite code",
coverUrl: "Cover",
workId: "Catalogue entry",
};
/** "customFields.Website" → "Website"; "displayName" → "Display name". */
function fieldLabel(field: string): string {
if (field.startsWith("customFields.")) return field.slice("customFields.".length);
const last = field.split(".").pop() ?? field;
if (FIELD_LABELS[last]) return FIELD_LABELS[last]!;
const words = last.replace(/([a-z])([A-Z])/g, "$1 $2").replace(/_/g, " ").toLowerCase();
return words.charAt(0).toUpperCase() + words.slice(1);
}
const TOO_LARGE = "That file is too large. The limit is 25 MB.";
/**
* An API error in the product's voice: what happened and how to fix it. Validation messages arrive as
* `field: reason` (core) or `field: Zod message`, several joined by "; "; the first one is described.
* Pass the user's rating scale so out-of-range ratings are described in the scale they see.
*/
export function friendlyError(err: unknown, opts: { ratingScale?: RatingScale } = {}): string {
if (!err || typeof err !== "object" || !("code" in err) || !("message" in err)) return "Something went wrong. Try again.";
const { status, code, message } = err as { status?: number; code: string; message: string };
if (status === 413) return TOO_LARGE;
if (code !== "validation") return message;
const first = message.split("; ")[0] ?? "";
const i = first.indexOf(": ");
const field = i > 0 ? first.slice(0, i) : "";
const reason = i > 0 ? first.slice(i + 2) : first;
const label = fieldLabel(field || "value");
if (field === "file" && /larger|large/.test(reason)) return TOO_LARGE;
if (field === "mime") return "That file type isn't supported. Use an image, a PDF or a text file.";
if (field === "password") return "Passwords need at least 8 characters.";
if (field === "username") return "Usernames are 2–32 characters: lowercase letters, digits and underscores.";
if (field === "currency" || reason === "not_cents") return "Price needs a number like 14 or 14.99, and a three-letter currency like GBP.";
if (reason === "not_url") return `${label} needs a full web address, starting with https://.`;
if (reason === "not_date" || reason === "bad_precision") return `${label} needs a date like 2025, 2025-03 or 2025-03-14.`;
if (reason === "out_of_range") {
const scale = opts.ratingScale ?? "5half";
const range = scale === "10" ? "0 and 10" : scale === "100" ? "0 and 100" : "0 and 5 stars";
return label === "Rating" ? `Pick a rating between ${range}.` : `Pick a rating for ${label} between ${range}.`;
}
if (reason === "required" || /^Too small: .*>=1 (character|item)/.test(reason) || /received undefined$/.test(reason)) return `${label} can't be empty.`;
if (reason === "too_long" || reason.startsWith("Too big")) return `${label} is too long. Shorten it and try again.`;
if (reason.startsWith("Too small")) return `${label} is too short.`;
if (reason === "not_number" || reason === "not_integer") return `${label} needs a number.`;
if (reason === "unknown_option" || reason === "duplicate_option") return `${label} has an option that no longer exists. Pick one from the list.`;
if (reason === "duplicate") return `${label} is already in use. Pick another.`;
return `${label} wasn't accepted. Check it and try again.`;
}
// ---------------------------------------------------------------------------------------------
// Review draft sync
// ---------------------------------------------------------------------------------------------
/**
* A text field edited locally and saved on blur. `synced` is the server value the text last agreed
* with; text that differs from it is unsaved and is never replaced. `pending` counts saves in flight
* (one entry's saves run serially, so they settle in order). While any is pending, incoming server
* values are only remembered in `incoming`. Values that are our own saves' echoes (optimistic updates
* show a save's `stored`, rollbacks show its `was`) are listed in `echoes` and ignored; any other
* value (an outside change, a Restore) is applied once the saves settle. So the outcome does not
* depend on the order in which rollbacks, refetches and settle callbacks land.
*/
export interface Draft {
text: string;
synced: string;
pending: number;
incoming: string;
echoes: string[];
}
export function newDraft(value: string): Draft {
return { text: value, synced: value, pending: 0, incoming: value, echoes: [] };
}
/** A server value arrived. Untouched text follows it; edited text is kept. Deferred while saving. */
export function draftIncoming(d: Draft, incoming: string): Draft {
if (d.pending > 0 || incoming === d.synced) return { ...d, incoming };
return { ...d, text: d.text === d.synced ? incoming : d.text, synced: incoming, incoming };
}
/** A save of `stored` started (`was` is the synced value before it): it counts as synced unless it fails. */
export function draftStartSave(d: Draft, save: { was: string; stored: string }): Draft {
return { ...d, synced: save.stored, pending: d.pending + 1, echoes: [...d.echoes, save.was, save.stored] };
}
/**
* The oldest pending save settled. Success: `stored` is on the server. Failure: back to `was`, so the
* local text still counts as unsaved. When the last pending save settles, a remembered incoming value
* that is not one of our own echoes is applied.
*/
export function draftSettled(d: Draft, save: { was: string; stored: string }, ok: boolean): Draft {
const pending = Math.max(0, d.pending - 1);
const next: Draft = { ...d, synced: ok ? save.stored : save.was, pending };
if (pending > 0) return next;
const settled: Draft = { ...next, echoes: [] };
return d.echoes.includes(d.incoming) ? settled : draftIncoming(settled, d.incoming);
}
+81
View File
@@ -0,0 +1,81 @@
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { useEffect, useRef, useState } from "react";
import { api, type ChangeEvent, type EntryPatch, type EntryView } from "./api.ts";
import { applyEntryPatch } from "./edits.ts";
/** Pending Changes-inbox events (first 200; the badge shows "200+" beyond that). */
export function usePendingChanges() {
return useQuery({ queryKey: ["changes", "pending"], queryFn: api.changes, staleTime: 30_000 });
}
export function useMe() {
return useQuery({ queryKey: ["me"], queryFn: api.me, retry: false, staleTime: Infinity });
}
export function useSources() {
return useQuery({ queryKey: ["sources"], queryFn: api.sources, staleTime: 60_000 });
}
/** Pending events that touch an entry or one of its reads/plays. */
export function changesFor(items: ChangeEvent[] | undefined, entryId: string, engagementIds: Set<string>): ChangeEvent[] {
return (items ?? []).filter(
(c) => c.targetId !== null && ((c.targetEntity === "entry" && c.targetId === entryId) || (c.targetEntity === "engagement" && engagementIds.has(c.targetId))),
);
}
export function useDebounced<T>(value: T, ms = 220): T {
const [v, setV] = useState(value);
useEffect(() => {
const t = setTimeout(() => setV(value), ms);
return () => clearTimeout(t);
}, [value, ms]);
return v;
}
/**
* Shared save path for every detail edit. The cached entry is updated optimistically (rolled back on
* error) and PATCHes for one entry run one at a time (mutation scope), in the order they were made.
* Use `edit(cur => patch)` when the next value depends on the current one (tags, custom fields): it
* builds from the latest cache, so quick successive edits compose.
*
* `onSettled` fires once for EVERY mutation of this hook (it is a hook-level option). Per-call
* `mutate(patch, { onSuccess })` callbacks only fire for the observer's latest call, so never use
* them for bookkeeping.
*/
export function useEntryPatch(id: string, opts: { onSettled?: (patch: EntryPatch, error: Error | null) => void } = {}) {
const qc = useQueryClient();
const onSettledRef = useRef(opts.onSettled);
onSettledRef.current = opts.onSettled;
const key = ["entry", id];
const mutationKey = ["entry-patch", id];
const m = useMutation({
mutationKey,
scope: { id: `entry-patch:${id}` },
mutationFn: (patch: EntryPatch) => api.patchEntry(id, patch),
onMutate: async (patch) => {
await qc.cancelQueries({ queryKey: key });
const prev = qc.getQueryData<EntryView>(key);
if (prev) qc.setQueryData<EntryView>(key, applyEntryPatch<EntryView>(prev, patch));
return { prev };
},
onError: (_e, _patch, ctx) => {
if (ctx?.prev) qc.setQueryData(key, ctx.prev);
void qc.invalidateQueries({ queryKey: key });
},
onSuccess: (entry) => {
// Only the last queued edit writes the server's answer, so earlier answers don't flash old values.
if (qc.isMutating({ mutationKey }) <= 1) qc.setQueryData(key, entry);
},
onSettled: (_entry, error, patch) => {
void qc.invalidateQueries({ queryKey: ["entries"] });
void qc.invalidateQueries({ queryKey: ["history", id] });
onSettledRef.current?.(patch, error);
},
});
const edit = (build: (cur: EntryView) => EntryPatch | null) => {
const cur = qc.getQueryData<EntryView>(key);
const patch = cur ? build(cur) : null;
if (patch) m.mutate(patch);
};
return { ...m, edit };
}
+74
View File
@@ -0,0 +1,74 @@
import { QueryCache, QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { StrictMode, type ReactNode } from "react";
import { createRoot } from "react-dom/client";
import { Redirect, Route, Switch } from "wouter";
import { ApiError } from "./api.ts";
import { friendlyError } from "./format.ts";
import { Shell } from "./components/Shell.tsx";
import { useMe } from "./hooks.ts";
import { Library } from "./routes/library.tsx";
import { Login } from "./routes/login.tsx";
import { NotYet } from "./routes/not-yet.tsx";
import { Register } from "./routes/register.tsx";
const queryClient: QueryClient = new QueryClient({
queryCache: new QueryCache({
// A 401 anywhere means the session ended: re-check who we are, which sends the user to sign in.
onError: (e, query) => {
if (e instanceof ApiError && e.status === 401 && query.queryKey[0] !== "me") void queryClient.invalidateQueries({ queryKey: ["me"] });
},
}),
defaultOptions: {
queries: {
staleTime: 15_000,
retry: (n, e) => !(e instanceof ApiError && e.status > 0 && e.status < 500) && n < 2,
},
},
});
function RequireAuth({ children }: { children: ReactNode }) {
const me = useMe();
if (me.isPending) return <div className="boot"><span className="logo">STTracker</span></div>;
if (me.error instanceof ApiError && me.error.status === 401) return <Redirect to="/login" replace />;
if (me.error) {
return (
<div className="auth">
<div className="auth-card" role="alert">
<span className="logo">STTracker</span>
<h1>Can't load your library</h1>
<p className="lede">{friendlyError(me.error)}</p>
<button className="btn primary" onClick={() => void me.refetch()}>Try again</button>
</div>
</div>
);
}
return <>{children}</>;
}
function App() {
return (
<Switch>
<Route path="/login" component={Login} />
<Route path="/register" component={Register} />
<Route>
<RequireAuth>
<Shell>
<Switch>
<Route path="/">{() => <Library selectedId={null} />}</Route>
<Route path="/entries/:id">{(p) => <Library selectedId={p.id} />}</Route>
<Route path="/:page/*?">{(p) => <NotYet page={p.page} />}</Route>
</Switch>
</Shell>
</RequireAuth>
</Route>
</Switch>
);
}
createRoot(document.getElementById("root")!).render(
<StrictMode>
<QueryClientProvider client={queryClient}>
<App />
</QueryClientProvider>
</StrictMode>,
);
+238
View File
@@ -0,0 +1,238 @@
import { useInfiniteQuery, useMutation, useQueryClient } from "@tanstack/react-query";
import { useRef, useState, type FormEvent } from "react";
import { Link, useLocation } from "wouter";
import { api, type EntryView } from "../api.ts";
import { Cover } from "../components/Cover.tsx";
import { EntryDetail } from "../components/EntryDetail.tsx";
import { Icon } from "../components/Icon.tsx";
import { describeRating, formatRating, friendlyError, mediaTypeLabel, rollupLabel, statusLabel, type Status } from "../format.ts";
import { useDebounced, usePendingChanges } from "../hooks.ts";
import { useRatingScale } from "../settings.ts";
const TYPE_FILTERS = ["book", "manga", "comic", "game", "movie", "show", "podcast", "audio_drama", "album", "artist", "video", "other"];
const STATUS_FILTERS: Status[] = ["in_progress", "planned", "completed", "paused", "dropped"];
/** The library: list pane (search, quick add, filters, rows) and the detail pane. */
export function Library({ selectedId }: { selectedId: string | null }) {
return (
<div className={`library${selectedId ? " has-selection" : ""}`}>
<ListPane selectedId={selectedId} />
<section className="detail-pane" aria-label="Entry">
{selectedId ? (
<EntryDetail key={selectedId} id={selectedId} />
) : (
<div className="detail-empty">
<p>
<strong>Nothing selected</strong>
Pick something from your library, or quick add a title and sort out the details later.
</p>
</div>
)}
</section>
</div>
);
}
// Filters survive navigating between entries (the list pane remounts on route change).
type Filters = { q: string; mediaType?: string; status?: Status; draft: boolean };
let savedFilters: Filters = { q: "", draft: false };
/** Forget per-session library state (on sign-out). */
export function resetLibraryState(): void {
savedFilters = { q: "", draft: false };
}
function ListPane({ selectedId }: { selectedId: string | null }) {
const [, navigate] = useLocation();
const qc = useQueryClient();
const scale = useRatingScale();
const [filters, setFiltersState] = useState(savedFilters);
const setFilters = (f: typeof filters) => {
savedFilters = f;
setFiltersState(f);
};
const q = useDebounced(filters.q.trim());
const query = { q: q || undefined, mediaType: filters.mediaType, status: filters.status, draft: filters.draft || undefined };
const entries = useInfiniteQuery({
queryKey: ["entries", query],
queryFn: ({ pageParam }) => api.entries({ ...query, cursor: pageParam, limit: 50 }),
initialPageParam: undefined as string | undefined,
getNextPageParam: (last) => last.nextCursor ?? undefined,
});
const changes = usePendingChanges();
const flagged = new Set((changes.data?.items ?? []).filter((c) => c.targetEntity === "entry").map((c) => c.targetId));
const [quickError, setQuickError] = useState<string | null>(null);
const quickInput = useRef<HTMLInputElement>(null);
const quick = useMutation({
mutationFn: api.quickAdd,
onSuccess: (entry, _t) => {
qc.setQueryData(["entry", entry.id], entry);
void qc.invalidateQueries({ queryKey: ["entries"] });
navigate(`/entries/${entry.id}`);
if (quickInput.current) quickInput.current.value = "";
quickInput.current?.focus();
},
onError: (e) => setQuickError(friendlyError(e)),
});
function onQuick(e: FormEvent<HTMLFormElement>) {
e.preventDefault();
const title = (quickInput.current?.value ?? "").trim();
if (!title || quick.isPending) return;
setQuickError(null);
quick.mutate(title);
}
const items = entries.data?.pages.flatMap((p) => p.items) ?? [];
const filtering = Boolean(q || filters.mediaType || filters.status || filters.draft);
return (
<section className="list-pane" aria-label="Library">
<div className="list-tools">
<form role="search" className="search" onSubmit={(e) => e.preventDefault()}>
<Icon name="search" />
<label className="sr-only" htmlFor="lib-search">Search your library</label>
<input
id="lib-search"
className="input"
type="search"
placeholder="Search your library…"
value={filters.q}
onChange={(e) => setFilters({ ...filters, q: e.target.value })}
/>
</form>
<form className="quick" onSubmit={onQuick}>
<span className="plus"><Icon name="plus" /></span>
<label className="sr-only" htmlFor="quick-add">Quick add</label>
<input
id="quick-add"
ref={quickInput}
name="title"
className="input"
placeholder="Quick add: type a title, sort it out later"
autoComplete="off"
maxLength={1000}
readOnly={quick.isPending}
aria-busy={quick.isPending}
aria-describedby={quickError ? "quick-error" : undefined}
/>
</form>
{quickError && <p className="error-text" id="quick-error" role="alert">{quickError}</p>}
<div className="filters">
<div className="chips" role="group" aria-label="Filter by status">
<button
type="button"
className="chip draft-chip"
aria-pressed={filters.draft}
onClick={() => setFilters({ ...filters, draft: !filters.draft })}
>
Draft
</button>
{STATUS_FILTERS.map((s) => (
<button
key={s}
type="button"
className="chip"
aria-pressed={filters.status === s}
onClick={() => setFilters({ ...filters, status: filters.status === s ? undefined : s })}
>
{statusLabel(s)}
</button>
))}
</div>
<div className="chips" role="group" aria-label="Filter by media type">
{TYPE_FILTERS.map((t) => (
<button
key={t}
type="button"
className="chip"
aria-pressed={filters.mediaType === t}
onClick={() => setFilters({ ...filters, mediaType: filters.mediaType === t ? undefined : t })}
>
{mediaTypeLabel(t)}
</button>
))}
</div>
</div>
</div>
{entries.isPending ? (
<ul className="list" aria-busy="true" aria-label="Loading your library">
{[0, 1, 2, 3].map((i) => (
<li key={i} className="row skeleton" aria-hidden="true">
<div className="cover" />
<div><div className="bar" /><div className="bar" /></div>
</li>
))}
</ul>
) : entries.error ? (
<div className="list-note" role="alert">
<p>{friendlyError(entries.error)}</p>
<button className="btn small" onClick={() => void entries.refetch()}>Try again</button>
</div>
) : items.length === 0 ? (
<div className="list-note">
{filtering ? (
<>
<p>Nothing matches these filters.</p>
<button className="btn small" onClick={() => setFilters({ q: "", draft: false })}>Clear filters</button>
</>
) : (
<p>
<strong>Your library is empty.</strong> Type a title into Quick add and press Enter. A title alone is
enough; you can link it to a catalogue entry later.
</p>
)}
</div>
) : (
<ul className="list">
{items.map((e) => (
<li key={e.id}>
<Row entry={e} selected={e.id === selectedId} flagged={flagged.has(e.id)} scale={scale} />
</li>
))}
{entries.hasNextPage && (
<li className="more">
<button className="btn small" disabled={entries.isFetchingNextPage} onClick={() => void entries.fetchNextPage()}>
{entries.isFetchingNextPage ? "Loading…" : "Show more"}
</button>
</li>
)}
</ul>
)}
</section>
);
}
/** `media type · meta`: children roll-up, else status, plus the year. */
export function rowMeta(e: EntryView): string {
const type = e.mediaTypeOverride ?? e.work.mediaType;
const parts = [mediaTypeLabel(type)];
const rollup = rollupLabel(type, e.childSummary);
if (rollup) parts.push(rollup);
else if (e.status !== "none") parts.push(statusLabel(e.status, type).toLowerCase());
if (e.work.year) parts.push(String(e.work.year));
return parts.join(" · ");
}
function Row({ entry: e, selected, flagged, scale }: { entry: EntryView; selected: boolean; flagged: boolean; scale: ReturnType<typeof useRatingScale> }) {
return (
<Link href={`/entries/${e.id}`} className="row" aria-current={selected ? "page" : undefined}>
<Cover url={e.work.coverUrl} mediaType={e.mediaTypeOverride ?? e.work.mediaType} draft={e.draft} />
<div style={{ minWidth: 0 }}>
<span className="title">{e.work.title}</span>
<div className="meta">{rowMeta(e)}</div>
</div>
<span className="end">
{flagged && <span className="flag" role="img" aria-label="A source changed this" title="A source changed this" />}
{e.draft ? (
<span className="draft">Draft</span>
) : e.rating !== null ? (
<span className="stars" role="img" aria-label={describeRating(e.rating, scale)}>{formatRating(e.rating, scale)}</span>
) : e.status === "in_progress" ? (
<span className="flag active" role="img" aria-label="In progress" title="In progress" />
) : null}
</span>
</Link>
);
}
+56
View File
@@ -0,0 +1,56 @@
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { useState, type FormEvent } from "react";
import { Link, Redirect, useLocation } from "wouter";
import { api } from "../api.ts";
import { friendlyError } from "../format.ts";
export function Login() {
const qc = useQueryClient();
const [, navigate] = useLocation();
const [error, setError] = useState<string | null>(null);
const [busy, setBusy] = useState(false);
const status = useQuery({ queryKey: ["auth-status"], queryFn: api.authStatus, staleTime: 0 });
async function submit(e: FormEvent<HTMLFormElement>) {
e.preventDefault();
const f = new FormData(e.currentTarget);
setBusy(true);
setError(null);
try {
const user = await api.login(String(f.get("username")), String(f.get("password")));
qc.setQueryData(["me"], user);
navigate("/", { replace: true });
} catch (err) {
setError(friendlyError(err));
setBusy(false);
}
}
// A fresh instance has nobody to sign in as: go straight to creating the admin account.
if (status.data?.hasUsers === false) return <Redirect to="/register" replace />;
return (
<main className="auth">
<div className="auth-card">
<span className="logo">STTracker</span>
<h1>Sign in</h1>
<p className="lede">Everything you read, watch, play and listen to, in one place that's yours.</p>
<form onSubmit={submit}>
{error && <p className="form-error" role="alert">{error}</p>}
<label className="field">
<span>Username</span>
<input className="input" name="username" autoComplete="username" autoCapitalize="none" spellCheck={false} required autoFocus />
</label>
<label className="field">
<span>Password</span>
<input className="input" name="password" type="password" autoComplete="current-password" required />
</label>
<button className="btn primary" disabled={busy}>{busy ? "Signing in…" : "Sign in"}</button>
</form>
<p className="alt">
Got an invite? <Link className="text-link" href="/register">Create your account</Link>
</p>
</div>
</main>
);
}
+24
View File
@@ -0,0 +1,24 @@
const NAMES: Record<string, string> = {
log: "Log",
lists: "Lists",
owned: "Owned",
changes: "Changes",
sources: "Sources",
settings: "Settings",
about: "About",
};
/** Sections that arrive in the next update. Unknown paths say so plainly. */
export function NotYet({ page }: { page: string }) {
const name = NAMES[page];
return (
<section className="page">
<h1>{name ?? "Nothing here"}</h1>
<p>
{name
? `${name} isn't built yet. Your data is safe; this screen arrives in the next update.`
: "This address doesn't match any page. Head back to your library."}
</p>
</section>
);
}
+97
View File
@@ -0,0 +1,97 @@
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { useState, type FormEvent } from "react";
import { Link, useLocation, useSearch } from "wouter";
import { ApiError, api } from "../api.ts";
import { friendlyError } from "../format.ts";
/**
* First-run screen. The first account becomes the admin; later accounts need an invite code.
* `GET auth/status` says which; `?invite=` prefills the code from an invite link.
*/
export function Register() {
const qc = useQueryClient();
const [, navigate] = useLocation();
const invite = new URLSearchParams(useSearch()).get("invite") ?? "";
const status = useQuery({ queryKey: ["auth-status"], queryFn: api.authStatus, staleTime: 0 });
const [inviteRequired, setInviteRequired] = useState(false);
const needInvite = invite !== "" || inviteRequired || status.data?.hasUsers === true;
const [error, setError] = useState<string | null>(null);
const [busy, setBusy] = useState(false);
async function submit(e: FormEvent<HTMLFormElement>) {
e.preventDefault();
const f = new FormData(e.currentTarget);
const username = String(f.get("username")).trim().toLowerCase();
const password = String(f.get("password"));
const displayName = String(f.get("displayName") ?? "").trim();
const inviteCode = String(f.get("inviteCode") ?? "").trim();
setBusy(true);
setError(null);
try {
await api.register({ username, password, displayName: displayName || undefined, inviteCode: inviteCode || undefined });
void qc.invalidateQueries({ queryKey: ["auth-status"] });
qc.setQueryData(["me"], await api.login(username, password));
navigate("/", { replace: true });
} catch (err) {
if (err instanceof ApiError && err.code === "invite_required") {
setInviteRequired(true);
setError(inviteCode ? "That invite code didn't work. Check it, or ask the admin for a new one." : "This STTracker already has an admin. Ask them for an invite code and enter it below.");
} else setError(friendlyError(err));
setBusy(false);
}
}
if (status.isPending && !invite) return <div className="boot"><span className="logo">STTracker</span></div>;
return (
<main className="auth">
<div className="auth-card">
<span className="logo">STTracker</span>
<h1>{needInvite ? "Create your account" : "Create the admin account"}</h1>
<p className="lede">
{needInvite
? "You've been invited to this STTracker. Pick a username and password."
: "This is the first account on this STTracker. It can invite friends later."}
</p>
<form onSubmit={submit}>
{error && <p className="form-error" role="alert">{error}</p>}
<label className="field">
<span>Your name (optional)</span>
<input className="input" name="displayName" autoComplete="name" maxLength={100} />
</label>
<label className="field">
<span>Username</span>
<input
className="input"
name="username"
autoComplete="username"
autoCapitalize="none"
spellCheck={false}
required
minLength={2}
maxLength={32}
pattern="[a-zA-Z0-9_]+"
aria-describedby="username-hint"
/>
<small className="hint" id="username-hint">2–32 lowercase letters, digits and underscores.</small>
</label>
<label className="field">
<span>Password</span>
<input className="input" name="password" type="password" autoComplete="new-password" required minLength={8} aria-describedby="password-hint" />
<small className="hint" id="password-hint">At least 8 characters.</small>
</label>
{needInvite && (
<label className="field">
<span>Invite code</span>
<input className="input" name="inviteCode" defaultValue={invite} required autoComplete="off" spellCheck={false} />
</label>
)}
<button className="btn primary" disabled={busy}>{busy ? "Creating…" : "Create account"}</button>
</form>
<p className="alt">
Already have an account? <Link className="text-link" href="/login">Sign in</Link>
</p>
</div>
</main>
);
}
+13
View File
@@ -0,0 +1,13 @@
import { useQuery } from "@tanstack/react-query";
import { api } from "./api.ts";
import type { RatingScale } from "./format.ts";
/** The signed-in user's preferences (GET me/preferences). Settings updates them with `api.updatePreferences`. */
export function usePreferences() {
return useQuery({ queryKey: ["preferences"], queryFn: api.preferences, staleTime: 5 * 60_000 });
}
/** The rating scale preference; five stars with halves until preferences load. */
export function useRatingScale(): RatingScale {
return usePreferences().data?.ratingScale ?? "5half";
}
+394
View File
@@ -0,0 +1,394 @@
/* STTracker app theme. Tokens are DESIGN.md's, verbatim; derived values sit below them. */
@font-face {
font-family: "Inter";
src: url("./fonts/inter-latin-wght-normal.woff2") format("woff2");
font-weight: 100 900;
font-style: normal;
font-display: swap;
}
/* Gelasio (OFL), official variable fonts from github.com/google/fonts ofl/gelasio, unmodified. */
@font-face {
font-family: "Gelasio";
src: url("./fonts/gelasio-wght.ttf") format("truetype");
font-weight: 400 700;
font-style: normal;
font-display: swap;
}
@font-face {
font-family: "Gelasio";
src: url("./fonts/gelasio-italic-wght.ttf") format("truetype");
font-weight: 400 700;
font-style: italic;
font-display: swap;
}
:root {
/* DESIGN.md */
--surface-0: #17152b;
--surface-1: #1e1b36;
--surface-2: #262247;
--surface-3: #2d2852;
--text: #ece9ff;
--text-muted: #a9a3d6;
--text-faint: #8d87c2;
--accent: #6d5dfc;
--rating: #fbbf24;
--warn: #f97316;
--ok: #34d399;
--media-books: #6b3fa0;
--media-audio: #2f6b58;
--media-games: #c1652f;
--media-film: #344b7a;
--media-music: #a33b62;
/* Derived (from the approved mockup) */
--nav-text: #b8b3e0;
--pill-text: #c9c4f0;
--dash: #4b4485;
--focus: #9d92ff;
--warn-bg: #3a2a1a;
--warn-line: #f9731666;
--warn-text: #fde2c8;
--on-warn: #1c0f04; /* white on --warn is 2.8:1; this is 6.4:1 */
--review-text: #e3e0fb;
--draft-stripes: repeating-linear-gradient(45deg, #3a356b 0 6px, #2d2852 6px 12px);
--r-control: 8px;
--r-box: 10px;
--r-pill: 999px;
--r-cover: 4px;
--font-ui: "Inter", ui-sans-serif, system-ui, -apple-system, "Segoe UI", sans-serif;
--font-review: "Gelasio", Georgia, "Times New Roman", serif;
--ease-out: cubic-bezier(0.16, 1, 0.3, 1);
color-scheme: dark;
}
* { box-sizing: border-box; margin: 0; padding: 0; }
html, body, #root { height: 100%; }
body {
background: var(--surface-1);
color: var(--text);
font-family: var(--font-ui);
font-size: 0.9rem;
line-height: 1.45;
font-feature-settings: "cv11", "ss01";
-webkit-font-smoothing: antialiased;
caret-color: var(--text);
}
::selection { background: #6d5dfc80; color: #fff; }
* { scrollbar-width: thin; scrollbar-color: var(--surface-3) transparent; }
button, input, select, textarea { font: inherit; color: inherit; }
button { cursor: pointer; background: none; border: 0; }
button:disabled { cursor: default; opacity: 0.55; }
a { color: inherit; text-decoration: none; }
a.text-link, .text-link { color: var(--focus); text-decoration: underline; text-underline-offset: 3px; text-decoration-thickness: 1px; }
:focus-visible { outline: 2px solid var(--focus); outline-offset: 2px; border-radius: 4px; }
.sr-only {
position: absolute; width: 1px; height: 1px; padding: 0; margin: -1px;
overflow: hidden; clip: rect(0 0 0 0); white-space: nowrap; border: 0;
}
.tabular { font-variant-numeric: tabular-nums; }
svg.icon { width: 16px; height: 16px; flex: none; stroke: currentColor; fill: none; stroke-width: 1.75; stroke-linecap: round; stroke-linejoin: round; }
/* ---------- Controls ---------- */
.input, .select, .textarea {
width: 100%;
background: var(--surface-3);
border: 1px solid transparent;
border-radius: var(--r-control);
padding: 8px 11px;
color: var(--text);
transition: border-color 0.15s, background 0.15s;
}
.input::placeholder, .textarea::placeholder { color: var(--text-muted); }
.input:hover, .select:hover, .textarea:hover { border-color: var(--dash); }
.input:focus-visible, .select:focus-visible, .textarea:focus-visible { outline: none; border-color: var(--focus); }
.select { appearance: none; padding-right: 30px; background-image: linear-gradient(45deg, transparent 50%, var(--text-muted) 50%), linear-gradient(135deg, var(--text-muted) 50%, transparent 50%); background-position: calc(100% - 15px) 52%, calc(100% - 10px) 52%; background-size: 5px 5px; background-repeat: no-repeat; }
.btn {
display: inline-flex; align-items: center; gap: 6px;
border-radius: var(--r-control); padding: 7px 13px; font-size: 0.82rem; font-weight: 500;
background: var(--surface-3); color: var(--text);
transition: background 0.15s;
}
.btn:hover:not(:disabled) { background: #3a346a; }
.btn.primary { background: var(--accent); color: #fff; }
.btn.primary:hover:not(:disabled) { background: #7c6dff; }
.btn.ghost { background: transparent; color: var(--text-muted); padding: 5px 8px; }
.btn.ghost:hover:not(:disabled) { background: var(--surface-3); color: var(--text); }
.btn.small { padding: 4px 9px; font-size: 0.76rem; }
.field { display: grid; gap: 5px; }
.field > span { font-size: 0.78rem; color: var(--text-muted); font-weight: 500; }
.hint { font-size: 0.76rem; color: var(--text-faint); }
.error-text { font-size: 0.8rem; color: #ffb4a8; }
.box .hint, .box .error-text { margin-top: 6px; }
/* ---------- Auth screens ---------- */
.auth {
min-height: 100%;
display: grid; place-items: center;
padding: 40px 20px;
background:
radial-gradient(60rem 40rem at 85% -10%, #6d5dfc22, transparent 60%),
radial-gradient(40rem 30rem at -10% 110%, #fbbf2412, transparent 60%),
var(--surface-0);
}
.auth-card { width: min(400px, 100%); }
.auth .logo { font-size: 1.15rem; margin: 0 0 28px; }
.auth h1 { font-size: 1.6rem; line-height: 1.2; font-weight: 650; letter-spacing: -0.015em; text-wrap: balance; }
.auth .lede { color: var(--text-muted); margin: 8px 0 26px; max-width: 42ch; }
.auth form { display: grid; gap: 14px; }
.auth form .btn.primary { justify-content: center; padding: 10px 14px; font-size: 0.9rem; margin-top: 6px; }
.auth .alt { margin-top: 22px; color: var(--text-muted); font-size: 0.85rem; }
.form-error { background: #3a1d24; border: 1px solid #f8717155; color: #ffd4d4; border-radius: var(--r-control); padding: 9px 12px; font-size: 0.84rem; }
/* ---------- Shell ---------- */
.logo { font-weight: 700; font-size: 1.05rem; color: var(--rating); letter-spacing: -0.01em; display: inline-block; }
.shell { display: grid; grid-template-columns: 190px minmax(0, 1fr); height: 100%; }
.side {
background: var(--surface-0); padding: 20px 14px 14px;
display: flex; flex-direction: column; overflow-y: auto;
}
.side .logo { margin: 0 8px 20px; }
.side nav { display: flex; flex-direction: column; gap: 1px; }
.nav {
padding: 7px 10px; border-radius: var(--r-control); color: var(--nav-text);
display: flex; justify-content: space-between; align-items: center; gap: 8px;
transition: background 0.15s, color 0.15s;
white-space: nowrap; overflow: hidden; text-overflow: ellipsis;
}
.nav:hover { background: #221f40; color: var(--text); }
.nav[aria-current="page"] { background: var(--surface-3); color: #fff; }
.nav .count { background: var(--warn); color: var(--on-warn); font-weight: 650; border-radius: var(--r-pill); font-size: 0.72rem; padding: 1px 7px; line-height: 1.4; }
.nav .dot { width: 7px; height: 7px; border-radius: 50%; background: var(--ok); flex: none; }
.nav .dot.paused { background: var(--dash); }
.nav .name { overflow: hidden; text-overflow: ellipsis; }
.grp { color: var(--text-faint); font-size: 0.78rem; margin: 18px 10px 6px; font-weight: 500; }
.side-foot { margin-top: auto; padding: 14px 10px 0; display: grid; gap: 2px; font-size: 0.8rem; }
.side-foot .who { color: var(--text-muted); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.side-foot button { justify-self: start; color: var(--text-faint); padding: 2px 0; }
.side-foot button:hover { color: var(--text); }
.main { min-width: 0; height: 100%; overflow: hidden; }
/* ---------- Library ---------- */
.library { display: grid; grid-template-columns: 360px minmax(0, 1fr); height: 100%; }
.list-pane { border-right: 1px solid var(--surface-3); display: flex; flex-direction: column; min-height: 0; }
.list-tools { padding: 18px 16px 10px; display: grid; gap: 12px; }
.search { position: relative; }
.search .icon { position: absolute; left: 11px; top: 50%; translate: 0 -50%; color: var(--text-muted); pointer-events: none; }
.search .input { padding-left: 33px; }
.quick { position: relative; }
.quick .input {
background: transparent; border: 1px dashed var(--dash); color: var(--pill-text); padding-left: 32px;
}
.quick .input::placeholder { color: var(--pill-text); }
.quick .input:hover { border-color: #6a62b0; }
.quick .input:focus-visible { border-style: solid; border-color: var(--rating); background: #221f40; }
.quick .plus { position: absolute; left: 11px; top: 50%; translate: 0 -50%; color: var(--rating); pointer-events: none; }
.quick .plus .icon { stroke-width: 2.25; }
.chips { display: flex; gap: 5px; overflow-x: auto; scrollbar-width: none; padding: 2px 24px 2px 2px; margin: -2px -16px -2px -2px; mask-image: linear-gradient(90deg, #000 calc(100% - 40px), transparent); }
.chips::-webkit-scrollbar { display: none; }
.chip {
flex: none; border-radius: var(--r-pill); padding: 3px 10px; font-size: 0.76rem;
background: transparent; color: var(--text-muted); border: 1px solid var(--surface-3);
transition: background 0.15s, color 0.15s, border-color 0.15s;
}
.chip:hover { color: var(--text); border-color: var(--dash); }
.chip[aria-pressed="true"] { background: var(--surface-3); color: #fff; border-color: var(--surface-3); }
.chip.draft-chip[aria-pressed="true"] { color: var(--rating); border-color: #fbbf2466; background: #fbbf2414; }
.filters { display: grid; gap: 6px; }
.list { list-style: none; overflow-y: auto; flex: 1; padding-bottom: 12px; }
.row {
display: grid; grid-template-columns: 40px minmax(0, 1fr) auto; gap: 12px; align-items: center;
padding: 9px 16px; transition: background 0.12s;
}
.row:hover { background: #24204a; }
.row[aria-current="page"] { background: var(--surface-3); }
.row:focus-visible { outline-offset: -2px; border-radius: 0; }
.row .title { font-size: 0.88rem; font-weight: 600; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; display: block; }
.row .meta { color: var(--text-muted); font-size: 0.76rem; margin-top: 2px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.row .end { display: flex; align-items: center; gap: 8px; }
.row .stars { font-size: 0.75rem; }
.flag { width: 8px; height: 8px; border-radius: 50%; background: var(--warn); flex: none; }
.flag.active { background: var(--accent); }
.draft { font-size: 0.7rem; color: var(--rating); border: 1px solid #fbbf2466; border-radius: 4px; padding: 1px 5px; line-height: 1.35; }
.stars { color: var(--rating); letter-spacing: 2px; white-space: nowrap; }
.list-note { padding: 18px 16px; color: var(--text-muted); font-size: 0.85rem; display: grid; gap: 10px; justify-items: start; }
.list-note strong { color: var(--text); font-weight: 600; }
.more { margin: 10px 16px; }
.skeleton .cover, .skeleton .bar { background: var(--surface-2); }
.skeleton .bar { height: 10px; border-radius: 4px; width: 70%; }
.skeleton .bar + .bar { width: 40%; margin-top: 7px; height: 8px; }
@media (prefers-reduced-motion: no-preference) {
.skeleton { animation: pulse 1.4s ease-in-out infinite; }
@keyframes pulse { 50% { opacity: 0.55; } }
}
/* ---------- Covers ---------- */
.cover { width: 40px; height: 56px; border-radius: 3px; overflow: hidden; flex: none; background: var(--surface-2); }
.cover img { width: 100%; height: 100%; object-fit: cover; display: block; }
.cover.is-draft { background: var(--draft-stripes); }
.cover.large { width: 110px; height: 160px; border-radius: 5px; box-shadow: 0 6px 20px #0006; }
.cover.small { width: 28px; height: 40px; border-radius: 3px; }
/* ---------- Detail ---------- */
.detail-pane { overflow-y: auto; min-width: 0; position: relative; }
.detail { padding: 22px 26px 40px; max-width: 980px; }
.detail-empty { height: 100%; display: grid; place-items: center; padding: 40px; text-align: center; color: var(--text-muted); }
.detail-empty p { max-width: 34ch; }
.detail-empty strong { display: block; color: var(--text); font-weight: 600; font-size: 1rem; margin-bottom: 6px; }
.back { display: none; }
.det-head { display: grid; grid-template-columns: 110px minmax(0, 1fr); gap: 20px; }
.det-head h1 { font-size: 1.45rem; line-height: 1.2; font-weight: 650; letter-spacing: -0.012em; text-wrap: balance; }
.det-head .m { color: var(--text-muted); margin: 4px 0 12px; }
.det-head .draft { margin-left: 8px; vertical-align: 3px; }
.det-controls { display: flex; align-items: center; gap: 14px; flex-wrap: wrap; }
.det-controls .select { width: auto; padding-top: 5px; padding-bottom: 5px; font-size: 0.82rem; }
.det-tools { margin-left: auto; display: flex; gap: 4px; }
.pills { display: flex; gap: 6px; flex-wrap: wrap; margin-top: 12px; align-items: center; }
.pill { background: var(--surface-3); padding: 3px 4px 3px 10px; border-radius: var(--r-pill); font-size: 0.76rem; color: var(--pill-text); display: inline-flex; align-items: center; gap: 2px; }
.pill button { width: 18px; height: 18px; border-radius: 50%; display: inline-grid; place-items: center; color: var(--text-muted); }
.pill button:hover { background: #ffffff14; color: var(--text); }
.pill button .icon { width: 11px; height: 11px; stroke-width: 2.25; }
.tag-input { background: transparent; border: 1px dashed var(--dash); border-radius: var(--r-pill); padding: 2px 10px; font-size: 0.76rem; width: 96px; color: var(--pill-text); }
.tag-input::placeholder { color: var(--text-muted); }
.tag-input:focus-visible { outline: none; border-style: solid; border-color: var(--focus); width: 150px; }
/* Rating control */
.rating { display: inline-flex; align-items: center; gap: 2px; border-radius: 6px; padding: 2px; cursor: pointer; }
.rating svg { width: 20px; height: 20px; display: block; }
.rating .empty { fill: none; stroke: #6a62b0; stroke-width: 1.4; }
.rating .full { fill: var(--rating); stroke: var(--rating); stroke-width: 1.4; }
.rating-clear { margin-left: 6px; font-size: 0.74rem; color: var(--text-faint); padding: 2px 4px; border-radius: 4px; }
.rating-clear:hover { color: var(--text); }
.rating-number { display: inline-flex; align-items: center; gap: 6px; color: var(--text-muted); font-size: 0.82rem; }
.rating-number .input { width: 72px; padding: 5px 8px; }
/* Change alert */
.alert {
margin: 18px 0 14px; background: var(--warn-bg); border: 1px solid var(--warn-line);
border-radius: var(--r-box); padding: 11px 14px; display: flex; gap: 12px; align-items: center;
}
.alert p { flex: 1; color: var(--warn-text); font-size: 0.85rem; line-height: 1.4; }
.alert p b { font-weight: 650; }
.alert .btn { background: var(--warn); color: var(--on-warn); font-weight: 600; border-radius: 6px; padding: 6px 12px; font-size: 0.8rem; white-space: nowrap; }
.alert .btn:hover { background: #fb8a3c; }
/* Boxes */
.det-grid { display: grid; grid-template-columns: 1fr 1fr; gap: 14px; margin-top: 18px; }
.alert + .det-grid, .alert + .link-box { margin-top: 0; }
.box { background: var(--surface-2); border-radius: var(--r-box); padding: 12px 14px; min-width: 0; }
.box.wide { grid-column: 1 / -1; }
.box-head { display: flex; align-items: center; justify-content: space-between; gap: 8px; margin-bottom: 8px; min-height: 22px; }
.box h2 { font-size: 0.78rem; color: var(--text-faint); font-weight: 600; }
.box .r { display: flex; justify-content: space-between; align-items: center; gap: 12px; padding: 4px 0; font-size: 0.84rem; }
.box .r > :first-child { color: var(--text-muted); }
.box .r > :last-child { text-align: right; }
.box .r .x { opacity: 0; transition: opacity 0.15s; }
.box .r:hover .x, .box .r .x:focus-visible { opacity: 1; }
@media (hover: none) { .box .r .x { opacity: 1; } }
.box .empty-line { color: var(--text-muted); font-size: 0.84rem; }
.box form { display: grid; gap: 10px; margin-top: 8px; }
.box form .two { display: grid; grid-template-columns: 1fr 1fr; gap: 8px; }
.box form .actions { display: flex; gap: 6px; }
.box .input, .box .select { padding: 6px 9px; font-size: 0.84rem; }
.box .search .input { padding-left: 33px; }
.copy + .copy { border-top: 1px solid var(--surface-3); margin-top: 6px; padding-top: 6px; }
.thumbs { display: flex; gap: 6px; flex-wrap: wrap; }
.thumb { width: 58px; height: 44px; border-radius: 4px; background-color: var(--surface-3); background-size: cover; background-position: center; display: grid; place-items: center; font-size: 0.66rem; color: var(--text-muted); overflow: hidden; text-align: center; padding: 2px; }
.thumb.upload { border: 1px dashed var(--dash); background: transparent; color: var(--text-muted); cursor: pointer; }
.thumb.upload:hover { color: var(--text); border-color: #6a62b0; }
.thumb.upload:focus-within { outline: 2px solid var(--focus); outline-offset: 2px; }
.src { display: flex; align-items: center; gap: 8px; padding: 4px 0; font-size: 0.84rem; }
.src i { width: 8px; height: 8px; border-radius: 50%; background: var(--ok); flex: none; }
.src i.paused { background: var(--dash); }
.children { list-style: none; }
.children a { display: grid; grid-template-columns: minmax(0, 1fr) auto; gap: 12px; padding: 5px 6px; margin: 0 -6px; border-radius: 6px; font-size: 0.84rem; }
.children a:hover { background: var(--surface-3); }
.children .t { overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.children .s { color: var(--text-muted); font-size: 0.76rem; margin-left: 8px; }
.children .stars { font-size: 0.72rem; }
.review textarea {
width: 100%; min-height: 7.5em; resize: vertical; background: transparent; border: 1px solid transparent;
border-radius: var(--r-control); padding: 6px 8px; margin: -6px -8px -4px; width: calc(100% + 16px);
font-family: var(--font-review); font-size: 0.95rem; line-height: 1.55; color: var(--review-text);
}
.review textarea::placeholder { color: var(--text-faint); font-style: italic; }
.review textarea:hover { border-color: var(--surface-3); }
.review textarea:focus-visible { outline: none; border-color: var(--focus); background: #211d40; }
.saved { font-size: 0.74rem; color: var(--text-faint); }
.custom { display: grid; gap: 10px; }
.custom .checks { display: flex; flex-wrap: wrap; gap: 4px 12px; font-size: 0.84rem; }
.custom label.inline { display: inline-flex; gap: 6px; align-items: center; font-size: 0.84rem; }
input[type="checkbox"] { accent-color: var(--accent); width: 15px; height: 15px; }
/* Link to catalogue */
.link-box { margin: 18px 0 14px; border: 1px solid #fbbf2440; background: #fbbf240a; }
.link-box p { color: var(--text-muted); font-size: 0.84rem; max-width: 70ch; }
.link-box .results { list-style: none; margin-top: 10px; display: grid; gap: 4px; }
.link-box .results li { display: grid; grid-template-columns: 28px minmax(0, 1fr) auto; gap: 10px; align-items: center; padding: 5px 6px; border-radius: 6px; font-size: 0.84rem; }
.link-box .results li:hover { background: var(--surface-3); }
.link-box .results .m { color: var(--text-muted); font-size: 0.76rem; }
.link-box .results .acts { display: flex; gap: 4px; }
.link-box .search { margin-top: 12px; max-width: 420px; }
.sub { font-size: 0.74rem; color: var(--text-faint); font-weight: 600; margin-top: 12px; }
/* History drawer */
.drawer {
position: fixed; top: 0; right: 0; bottom: 0; width: min(400px, 100vw); z-index: 20;
background: var(--surface-0); border-left: 1px solid var(--surface-3);
box-shadow: -18px 0 40px #0008; display: flex; flex-direction: column;
}
@media (prefers-reduced-motion: no-preference) {
.drawer { animation: drawer-in 0.32s var(--ease-out); }
@keyframes drawer-in { from { translate: 24px 0; opacity: 0; } }
}
.drawer header { display: flex; justify-content: space-between; align-items: center; padding: 18px 20px 12px; }
.drawer h2 { font-size: 1rem; font-weight: 650; }
.drawer ol { list-style: none; overflow-y: auto; padding: 0 20px 24px; }
.drawer li { border-top: 1px solid var(--surface-3); padding: 12px 0; display: grid; gap: 6px; }
.drawer .when { display: flex; justify-content: space-between; gap: 10px; font-size: 0.78rem; color: var(--text-muted); }
.drawer .who { color: var(--text); font-weight: 550; }
.drawer dl { display: grid; grid-template-columns: auto minmax(0, 1fr); gap: 3px 10px; font-size: 0.8rem; }
.drawer dt { color: var(--text-faint); }
.drawer dd { overflow-wrap: anywhere; }
.drawer dd s { color: var(--text-faint); text-decoration-color: #8d87c299; }
.drawer .arrow { color: var(--text-faint); margin: 0 4px; }
/* Placeholder pages */
.page { padding: 28px 32px; max-width: 60ch; }
.page h1 { font-size: 1.3rem; font-weight: 650; margin-bottom: 8px; }
.page p { color: var(--text-muted); }
.boot { height: 100%; display: grid; place-items: center; background: var(--surface-0); }
/* ---------- Responsive ---------- */
@media (max-width: 1180px) {
.library { grid-template-columns: 320px minmax(0, 1fr); }
.det-grid { grid-template-columns: 1fr; }
}
@media (max-width: 860px) {
.shell { grid-template-columns: 1fr; grid-template-rows: auto minmax(0, 1fr); }
.side { flex-direction: row; align-items: center; gap: 4px; padding: 10px 12px; overflow-x: auto; overflow-y: hidden; border-bottom: 1px solid var(--surface-3); scrollbar-width: none; }
.side::-webkit-scrollbar { display: none; }
.side .logo { margin: 0 10px 0 4px; }
.side nav { flex-direction: row; }
.side .grp, .side .src-nav, .side-foot .who { display: none; }
.side-foot { margin: 0 0 0 auto; padding: 0 4px; }
.side-foot button { white-space: nowrap; }
.library { grid-template-columns: 1fr; }
.library.has-selection .list-pane { display: none; }
.library:not(.has-selection) .detail-pane { display: none; }
.list-pane { border-right: 0; }
.back { display: inline-flex; margin: 0 0 14px -8px; }
.detail { padding: 16px 16px 40px; }
.det-head { grid-template-columns: 84px minmax(0, 1fr); gap: 14px; }
.cover.large { width: 84px; height: 122px; }
.det-head h1 { font-size: 1.2rem; }
.det-tools { margin-left: 0; }
.alert { flex-direction: column; align-items: stretch; }
.box form .two { grid-template-columns: 1fr; }
}
+92
View File
@@ -0,0 +1,92 @@
import { describe, expect, test } from "bun:test";
import { ApiError, request } from "../src/api.ts";
function fakeFetch(res: Response | (() => never)) {
const calls: { url: string; init: RequestInit }[] = [];
const fn = (async (url: string, init: RequestInit) => {
calls.push({ url, init });
if (typeof res === "function") return res();
return res.clone();
}) as unknown as typeof fetch;
return { fn, calls };
}
describe("request", () => {
test("throws an ApiError carrying code on a non-2xx JSON error", async () => {
const { fn } = fakeFetch(
new Response(JSON.stringify({ error: { code: "validation", message: "title: required" } }), {
status: 400,
headers: { "content-type": "application/json" },
}),
);
const err = (await request("POST", "entries/quick", { body: { title: "" }, fetchImpl: fn }).catch((e: unknown) => e)) as ApiError;
expect(err).toBeInstanceOf(ApiError);
expect(err.code).toBe("validation");
expect(err.status).toBe(400);
expect(err.message).toBe("title: required");
});
test("non-GET requests send the CSRF header, JSON and same-origin cookies", async () => {
const { fn, calls } = fakeFetch(new Response(JSON.stringify({ id: "e1" }), { status: 201 }));
const out = await request<{ id: string }>("POST", "entries/quick", { body: { title: "Dune" }, fetchImpl: fn });
expect(out).toEqual({ id: "e1" });
const { url, init } = calls[0]!;
expect(url).toBe("/api/v1/entries/quick");
const h = new Headers(init.headers);
expect(h.get("x-requested-with")).toBe("sttracker");
expect(h.get("content-type")).toBe("application/json");
expect(init.credentials).toBe("same-origin");
expect(init.body).toBe(JSON.stringify({ title: "Dune" }));
});
test("login and register also send the CSRF header and JSON content type", async () => {
const { fn, calls } = fakeFetch(new Response(JSON.stringify({ id: "u1" })));
await request("POST", "auth/login", { body: { username: "a", password: "b" }, fetchImpl: fn });
await request("POST", "history/h1/restore", { fetchImpl: fn });
for (const c of calls) {
const h = new Headers(c.init.headers);
expect(h.get("x-requested-with")).toBe("sttracker");
expect(h.get("content-type")).toBe("application/json");
}
});
test("multipart uploads send the CSRF header but let fetch set the content type", async () => {
const { fn, calls } = fakeFetch(new Response(JSON.stringify({ id: "a1" }), { status: 201 }));
const form = new FormData();
form.set("entity", "entry");
await request("POST", "attachments", { form, fetchImpl: fn });
const h = new Headers(calls[0]!.init.headers);
expect(h.get("x-requested-with")).toBe("sttracker");
expect(h.get("content-type")).toBeNull();
expect(calls[0]!.init.body).toBe(form);
});
test("GET builds the query string and skips empty values", async () => {
const { fn, calls } = fakeFetch(new Response(JSON.stringify({ items: [] })));
await request("GET", "entries", { query: { q: "dune", status: undefined, draft: true, limit: 50 }, fetchImpl: fn });
expect(calls[0]!.url).toBe("/api/v1/entries?q=dune&draft=true&limit=50");
});
test("204 resolves to undefined", async () => {
const { fn } = fakeFetch(new Response(null, { status: 204 }));
expect(await request("DELETE", "engagements/x", { fetchImpl: fn })).toBeUndefined();
});
test("a non-JSON error still becomes an ApiError", async () => {
const { fn } = fakeFetch(new Response("<html>Bad gateway</html>", { status: 502 }));
const err = (await request("GET", "me", { fetchImpl: fn }).catch((e: unknown) => e)) as ApiError;
expect(err).toBeInstanceOf(ApiError);
expect(err.code).toBe("http_502");
expect(err.status).toBe(502);
});
test("a network failure becomes an ApiError with code network", async () => {
const { fn } = fakeFetch(() => {
throw new TypeError("fetch failed");
});
const err = (await request("GET", "me", { fetchImpl: fn }).catch((e: unknown) => e)) as ApiError;
expect(err).toBeInstanceOf(ApiError);
expect(err.code).toBe("network");
expect(err.status).toBe(0);
});
});
+44
View File
@@ -0,0 +1,44 @@
import { describe, expect, test } from "bun:test";
import { addTag, applyEntryPatch, orderEngagements, removeTag, setCustomValue } from "../src/edits.ts";
describe("entry edit builders", () => {
test("two quick tag adds built from the latest cache keep both", () => {
// Each edit reads the cache as left by the previous (optimistic) edit, not the render's props.
let cache = { tags: [] as string[], rating: null as number | null };
const edit = (build: (cur: typeof cache) => Partial<typeof cache>) => (cache = applyEntryPatch(cache, build(cache)));
edit((cur) => ({ tags: addTag(cur.tags, "cosmic horror") }));
edit((cur) => ({ tags: addTag(cur.tags, "re-read") }));
expect(cache.tags).toEqual(["cosmic horror", "re-read"]);
});
test("addTag trims and ignores case duplicates; removeTag removes one", () => {
expect(addTag(["Horror"], " horror ")).toEqual(["Horror"]);
expect(addTag(["a"], " ")).toEqual(["a"]);
expect(removeTag(["a", "b"], "a")).toEqual(["b"]);
});
test("setCustomValue sets, and removes empty values", () => {
expect(setCustomValue({ a: 1 }, "b", "x")).toEqual({ a: 1, b: "x" });
expect(setCustomValue({ a: 1, b: "x" }, "b", "")).toEqual({ a: 1 });
expect(setCustomValue({ a: [1] }, "a", [])).toEqual({});
expect(setCustomValue({ a: 1 }, "a", null)).toEqual({});
});
});
describe("orderEngagements", () => {
const g = (id: string, createdAt: string, finished?: string, started?: string) => ({
id,
createdAt,
finishedAt: finished ? { value: finished, precision: "day" as const } : null,
startedAt: started ? { value: started, precision: "day" as const } : null,
});
test("sorts by finishedAt ?? startedAt, then createdAt; undated first", () => {
const list = [
g("late", "2026-01-01T00:00:00Z", "2026-09-14"),
g("early-logged-later", "2026-10-01T00:00:00Z", "2025-03"),
g("started-only", "2026-02-01T00:00:00Z", undefined, "2025-06-01"),
g("undated", "2026-03-01T00:00:00Z"),
];
expect(orderEngagements(list).map((x) => x.id)).toEqual(["undated", "early-logged-later", "started-only", "late"]);
});
});
+148
View File
@@ -0,0 +1,148 @@
import { describe, expect, test } from "bun:test";
import { draftIncoming, draftSettled, draftStartSave, newDraft, type Draft, formatDateRange, formatPartialDate, formatRating, friendlyError, parsePartialDate, rollupLabel, shortDate } from "../src/format.ts";
describe("formatRating", () => {
test("5 stars with halves", () => {
expect(formatRating(90, "5half")).toBe("★★★★½");
expect(formatRating(100, "5half")).toBe("★★★★★");
expect(formatRating(80, "5half")).toBe("★★★★☆");
expect(formatRating(0, "5half")).toBe("☆☆☆☆☆");
});
test("null is empty on every scale", () => {
expect(formatRating(null, "5half")).toBe("");
expect(formatRating(null, "10")).toBe("");
expect(formatRating(undefined, "100")).toBe("");
});
test("whole stars, 10 and 100 point scales", () => {
expect(formatRating(90, "5")).toBe("★★★★★");
expect(formatRating(70, "5")).toBe("★★★★☆");
expect(formatRating(90, "10")).toBe("9/10");
expect(formatRating(85, "10")).toBe("8.5/10");
expect(formatRating(73, "100")).toBe("73/100");
});
});
describe("formatPartialDate", () => {
test("renders by precision", () => {
expect(formatPartialDate({ value: "2025-03", precision: "month" })).toBe("Mar 2025");
expect(formatPartialDate({ value: "2019", precision: "year" })).toBe("2019");
expect(formatPartialDate({ value: "2026-09-14", precision: "day" })).toBe("14 Sep 2026");
expect(formatPartialDate({ value: "2026-09-04T23:30:00-05:00", precision: "day" })).toBe("4 Sep 2026");
expect(formatPartialDate({ value: "", precision: "unknown" })).toBe("");
expect(formatPartialDate(null)).toBe("");
});
});
describe("formatDateRange", () => {
test("flags dates that are not exact", () => {
expect(formatDateRange(null, { value: "2025-03", precision: "month" })).toBe("Mar 2025 (no exact date)");
expect(formatDateRange({ value: "2026-09-02", precision: "day" }, { value: "2026-09-14", precision: "day" })).toBe(
"2 Sep – 14 Sep 2026",
);
expect(formatDateRange({ value: "2025-12-30", precision: "day" }, { value: "2026-01-02", precision: "day" })).toBe(
"30 Dec 2025 – 2 Jan 2026",
);
expect(formatDateRange({ value: "2026-09-02", precision: "day" }, null)).toBe("Started 2 Sep 2026");
expect(formatDateRange(null, null)).toBe("No dates");
});
});
describe("parsePartialDate", () => {
test("accepts year, month and day, rejects the rest", () => {
expect(parsePartialDate("2019")).toEqual({ value: "2019", precision: "year" });
expect(parsePartialDate(" 2025-03 ")).toEqual({ value: "2025-03", precision: "month" });
expect(parsePartialDate("2024-02-29")).toEqual({ value: "2024-02-29", precision: "day" });
expect(parsePartialDate("2023-02-29")).toBeNull();
expect(parsePartialDate("03/04/2019")).toBeNull();
expect(parsePartialDate("")).toBeNull();
});
});
describe("rollupLabel", () => {
test("children roll up as 'season 3 of 5'", () => {
expect(rollupLabel("show", { total: 5, completed: 3 })).toBe("season 3 of 5");
expect(rollupLabel("show", { total: 5, completed: 0 })).toBe("5 seasons");
expect(rollupLabel("podcast", { total: 200, completed: 200 })).toBe("all 200 episodes");
expect(rollupLabel("book", null)).toBeNull();
});
});
describe("friendlyError", () => {
const v = (message: string, status = 400) => ({ status, code: "validation", message });
test("maps validation reasons to plain copy", () => {
expect(friendlyError(v("customFields.Website: not_url"))).toBe("Website needs a full web address, starting with https://.");
expect(friendlyError(v("finishedAt: not_date"))).toBe("Finished needs a date like 2025, 2025-03 or 2025-03-14.");
expect(friendlyError(v("rating: out_of_range"))).toBe("Pick a rating between 0 and 5 stars.");
expect(friendlyError(v("rating: out_of_range"), { ratingScale: "10" })).toBe("Pick a rating between 0 and 10.");
expect(friendlyError(v("customFields.Mood: out_of_range"), { ratingScale: "5" })).toBe("Pick a rating for Mood between 0 and 5 stars.");
expect(friendlyError(v("title: required"))).toBe("Title can't be empty.");
expect(friendlyError(v("review: too_long"))).toBe("Review is too long. Shorten it and try again.");
expect(friendlyError(v("password: Too small: expected string to have >=8 characters"))).toBe("Passwords need at least 8 characters.");
expect(friendlyError(v("title: Too big: expected string to have <=1000 characters"))).toBe("Title is too long. Shorten it and try again.");
expect(friendlyError(v("mime: unsupported file type"))).toBe("That file type isn't supported. Use an image, a PDF or a text file.");
expect(friendlyError(v("customFields.Mood: unknown_option"))).toBe("Mood has an option that no longer exists. Pick one from the list.");
expect(friendlyError(v("displayName: weird_reason"))).toBe("Display name wasn't accepted. Check it and try again.");
});
test("413 means the file is too large", () => {
expect(friendlyError({ status: 413, code: "validation", message: "Request body is too large (max 50 MB)." })).toBe("That file is too large. The limit is 25 MB.");
expect(friendlyError(v("file: larger than 25 MB"))).toBe("That file is too large. The limit is 25 MB.");
});
test("other API errors keep the server's message; unknown errors get a generic line", () => {
expect(friendlyError({ status: 401, code: "unauthenticated", message: "Wrong username or password." })).toBe("Wrong username or password.");
expect(friendlyError(new Error("boom"))).toBe("Something went wrong. Try again.");
});
});
test("shortDate", () => {
expect(shortDate("2026-10-12T10:00:00")).toBe("12 Oct");
});
describe("review draft sync", () => {
const draft = (text: string, synced: string): Draft => ({ ...newDraft(synced), text });
const pick = (d: Draft) => [d.text, d.synced, d.pending];
test("a rollback after a failed save keeps the local text", () => {
const save = { was: "First thoughts.", stored: "Second thoughts." };
let d = draftStartSave(draft("Second thoughts.", save.was), save);
d = draftIncoming(d, save.stored); // optimistic update
d = draftIncoming(d, save.was); // rollback
d = draftSettled(d, save, false);
expect(pick(d)).toEqual(["Second thoughts.", "First thoughts.", 0]);
});
// Two overlapping saves: "a" (was "old"), then "b" (was "a"). Incoming values mirror what the cache does.
const A = { was: "old", stored: "a" };
const B = { was: "a", stored: "b" };
const twoSaves = () => draftIncoming(draftIncoming(draftStartSave({ ...draftStartSave(draft("a", "old"), A), text: "b" }, B), "a"), "b");
test("two overlapping saves both settle and the flag clears (success, success)", () => {
let d = twoSaves();
expect(d.pending).toBe(2);
d = draftSettled(draftSettled(d, A, true), B, true);
expect(pick(d)).toEqual(["b", "b", 0]);
expect(d.echoes).toEqual([]);
});
test("two overlapping saves: fail then success ends synced; success then fail keeps text unsaved", () => {
let d = draftIncoming(twoSaves(), "old"); // A's rollback
d = draftSettled(draftSettled(d, A, false), B, true);
expect(pick(d)).toEqual(["b", "b", 0]);
let e = draftIncoming(twoSaves(), "a"); // B's rollback
e = draftSettled(draftSettled(e, A, true), B, false);
expect(pick(e)).toEqual(["b", "a", 0]);
});
test("an incoming value that arrives while saving is applied once saving ends", () => {
const save = { was: "x", stored: "y" };
let d = draftStartSave(draft("y", "x"), save);
d = draftIncoming(d, "Restored.");
expect(d.text).toBe("y");
d = draftSettled(d, save, true);
expect(pick(d)).toEqual(["Restored.", "Restored.", 0]);
});
test("a change from elsewhere replaces untouched text but not edited text", () => {
expect(draftIncoming(draft("Second.", "Second."), "First.").text).toBe("First.");
expect(draftIncoming(draft("Typed.", "Second."), "Other.").text).toBe("Typed.");
});
});
+132
View File
@@ -9,18 +9,150 @@
"typescript": "^5",
},
},
"apps/server": {
"name": "@sttracker/server",
"dependencies": {
"@sttracker/core": "workspace:*",
"@sttracker/db": "workspace:*",
"@sttracker/plugin-sdk": "workspace:*",
"kysely": "^0.29.7",
"zod": "^4.6.7",
},
},
"apps/web": {
"name": "@sttracker/web",
"dependencies": {
"@tanstack/react-query": "^5.105.0",
"react": "^19.3.1",
"react-dom": "^19.3.1",
"wouter": "^3.13.0",
},
"devDependencies": {
"@types/react": "^19.3.0",
"@types/react-dom": "^19.3.0",
},
},
"packages/core": {
"name": "@sttracker/core",
"dependencies": {
"@sttracker/db": "workspace:*",
"@sttracker/plugin-sdk": "workspace:*",
"fflate": "^0.8.3",
"kysely": "^0.29.7",
"zod": "^4.6.7",
},
},
"packages/db": {
"name": "@sttracker/db",
"dependencies": {
"kysely": "^0.29.7",
},
},
"packages/plugin-sdk": {
"name": "@sttracker/plugin-sdk",
"dependencies": {
"fflate": "^0.8.3",
},
},
"plugins/csv": {
"name": "@sttracker/plugin-csv",
"dependencies": {
"@sttracker/plugin-sdk": "workspace:*",
},
},
"plugins/hardcover": {
"name": "@sttracker/plugin-hardcover",
"dependencies": {
"@sttracker/plugin-sdk": "workspace:*",
},
},
"plugins/lastfm": {
"name": "@sttracker/plugin-lastfm",
"dependencies": {
"@sttracker/plugin-sdk": "workspace:*",
},
},
"plugins/listenbrainz": {
"name": "@sttracker/plugin-listenbrainz",
"dependencies": {
"@sttracker/plugin-sdk": "workspace:*",
"fflate": "^0.8.3",
},
},
"plugins/mangabaka": {
"name": "@sttracker/plugin-mangabaka",
"dependencies": {
"@sttracker/plugin-sdk": "workspace:*",
},
},
"plugins/trakt": {
"name": "@sttracker/plugin-trakt",
"dependencies": {
"@sttracker/plugin-sdk": "workspace:*",
},
},
},
"packages": {
"@sttracker/core": ["@sttracker/core@workspace:packages/core"],
"@sttracker/db": ["@sttracker/db@workspace:packages/db"],
"@sttracker/plugin-csv": ["@sttracker/plugin-csv@workspace:plugins/csv"],
"@sttracker/plugin-hardcover": ["@sttracker/plugin-hardcover@workspace:plugins/hardcover"],
"@sttracker/plugin-lastfm": ["@sttracker/plugin-lastfm@workspace:plugins/lastfm"],
"@sttracker/plugin-listenbrainz": ["@sttracker/plugin-listenbrainz@workspace:plugins/listenbrainz"],
"@sttracker/plugin-mangabaka": ["@sttracker/plugin-mangabaka@workspace:plugins/mangabaka"],
"@sttracker/plugin-sdk": ["@sttracker/plugin-sdk@workspace:packages/plugin-sdk"],
"@sttracker/plugin-trakt": ["@sttracker/plugin-trakt@workspace:plugins/trakt"],
"@sttracker/server": ["@sttracker/server@workspace:apps/server"],
"@sttracker/web": ["@sttracker/web@workspace:apps/web"],
"@tanstack/query-core": ["@tanstack/[email protected]", "", {}, "sha512-7p56vRmT/z4UJ6sctGIdl5WRUQyUv53IUpUwEKi2g97zMbO7zh8ZsCLffGAvCPBR4xD72e3VXsDyLwXHhoNTMQ=="],
"@tanstack/react-query": ["@tanstack/[email protected]", "", { "dependencies": { "@tanstack/query-core": "5.105.0" }, "peerDependencies": { "@types/react": "^18 || ^19", "react": "^18 || ^19" }, "optionalPeers": ["@types/react"] }, "sha512-y+hMuDERvgg6ay1da0V1a6ZC0ZONt7dELxSFkqRZuwm7p1TTZs0MoXrPlBRflIpRoYjvzQVYxycnLImpFLhWrw=="],
"@types/bun": ["@types/[email protected]", "", { "dependencies": { "bun-types": "1.4.4" } }, "sha512-9wkRDXUXK124pEnAutE6n/l18j+i9GC5IAHK8TmX/2nlySETmky0OU6yL5U8pW1AoKh++AN9auDCr647BfurjQ=="],
"@types/node": ["@types/[email protected]", "", { "dependencies": { "undici-types": "~8.9.0" } }, "sha512-MzQj+8Kx3XI40OmUAE6Qoh58ZMNstNNMwn2nfd3qWsfe7nDNSqJNzes4njhz6AHwrSHOffSLwXU6yb474Q4aog=="],
"@types/react": ["@types/[email protected]", "", { "dependencies": { "csstype": "^3.2.2" } }, "sha512-N0rFCuH9YoxG9/m61l9MfpJKfmLOVU0em7ipIz6TRgSSkvReLB9vL85GB+yr8Bs5leqpvg96JSwF4ZS1s4viQg=="],
"@types/react-dom": ["@types/[email protected]", "", { "peerDependencies": { "@types/react": "^19.3.0" } }, "sha512-ZI7bU42mZXXKHn/qNLEw2IrbiINU7X5+vfgdixBHkCNpYWXjKgfQ/P+uyGb5CjOLB9UcnTeg3rylQtV2hym44Q=="],
"bun-types": ["[email protected]", "", { "dependencies": { "@types/node": "*", "undici-types": "*" } }, "sha512-yjrJh5oGX7mMPKGd19YXyIjmEuj4G4zNC30bs7PM9Az7NQ3ndF3j6hbeRdvJTRQEhJ/FYQwW0a2moweEi2hMxQ=="],
"csstype": ["[email protected]", "", {}, "sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ=="],
"fflate": ["[email protected]", "", {}, "sha512-tbZNuJrLwGUp3zshBtdy4W+ORxZuIh8a5ilyIEQDC5rY1f3U20JMry0Ll3WBzU58EZKsEuJFXhb5gwv8CsPvgA=="],
"kysely": ["[email protected]", "", {}, "sha512-fmIq+7kv4WbFm2tcfznRzikqB4cuNpgZ40PSU2fAJ2/OO5XhojqnTPrmgrCjYSRJVwBln03XcRXJGnw9H099IQ=="],
"react": ["[email protected]", "", {}, "sha512-QNhWdt7nrgVWWQG58QuGsAR3jv3Hbe4mb4vz7w2csnHavciZJgda1CPNaldF6eVxv90ud3jUXpKlHL69gO38yw=="],
"react-dom": ["[email protected]", "", { "dependencies": { "scheduler": "^0.28.0" }, "peerDependencies": { "react": "^19.3.1" } }, "sha512-Y0bARz53lhEC1NMCASs0/j+faB/zqt3R8hpvg1e8+61ZVJ4UL0UdC4cBbXfZMcIuhrfgeILTLe+ZeBCavQPtKQ=="],
"regexparam": ["[email protected]", "", {}, "sha512-RSYAtP31mvYLkAHrOlh25pCNQ5hWnT106VukGaaFfuJrZFkGRX5GhUAdPqpSDXxOhA2c4akmRuplv1mRqnBn6Q=="],
"scheduler": ["[email protected]", "", {}, "sha512-juorfCmIkIw8tT+p5BXSm6PJjQF/ycEYmKyzURCIt/RaZIhL+PulbQ9Yu2z1HdOJDdqDTlxA1+xKBmHXJsczAw=="],
"typescript": ["[email protected]", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw=="],
"undici-types": ["[email protected]", "", {}, "sha512-EXE4ydL3S76dFb27UFjNVZHTyv/TLdATj/KUe29ptdcg6qXL48J2tZ3126E5kICj5jn781OR+XlVH1N6Mz8jbA=="],
"use-sync-external-store": ["[email protected]", "", { "peerDependencies": { "react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0" } }, "sha512-6L+EeigHMQhdaIPNIFUKwfWJSwWFQ8gJbJ2DLOs5sDIegTwR9fRxvnM3uciHKjIZhFz+KAv2emhWMRvDmMcY8A=="],
"wouter": ["[email protected]", "", { "dependencies": { "regexparam": "^3.0.0", "use-sync-external-store": "^1.0.0" }, "peerDependencies": { "react": ">=16.8.0" } }, "sha512-p1RoCuVyxwb3X5jLjDdkwDzlXj+/UGZqZb+806uaJOVEHvNfGkjET0KiQ/CGiDKC90rLsALMgMj4onNVKxgNdQ=="],
"zod": ["[email protected]", "", {}, "sha512-g8W2D2kU6a8untc053fX1uVBKzM7yqAxW/3zwk83+6RyF2+SKOuV+c0wBDhUwzV1PaGapmZH+rmcUzPf+3CGwg=="],
"@types/node/undici-types": ["[email protected]", "", {}, "sha512-KTDyRTYX8sWmKXAikPHHSyc63CRPETMctyjKFupcC6OBLXT3xsN0e9aF7m+mIXutFWpUXuedtowG7iLOzp0kQg=="],
}
}
+36 -11
View File
@@ -14,22 +14,47 @@ user layer or files a Changes-inbox event.
## Map
| Path | Responsibility | State |
|---|---|---|
| `apps/server` | `Bun.serve`: `/api/v1`, `/ingest/<plugin>`, job worker, static web | planned |
| `apps/web` | React app (HTML imports) | planned |
| `packages/core` | domain services: entries, matching, change engine, history, export | planned |
| `packages/db` | Kysely, SQLite/Postgres dialects, migrations | planned |
| `apps/server` | `main.ts` boots (config → `createDb` → `migrateToLatest` → `loadPlugins` → worker + scheduler + daily backup → `Bun.serve`). `app.ts` `createApp(deps)` = `/api/v1` route table + `/ingest/*` mount (404 in Phase 1), testable via `app.fetch`. `http.ts` = `route()`, auth (session cookie / Bearer app token), CSRF header, body caps, rate limit, error mapping. `routes/<area>.ts` one file per area; `openapi.ts` builds `/api/v1/openapi.json` from the table | exists |
| `apps/web` | React 19 app served by the server as an HTML import (`index.html` → `src/main.tsx`, `src/theme.css`). wouter routes in `main.tsx` (`/login`, `/register`, `/`, `/entries/:id`; other sections are placeholders until Task 18); `api.ts` = fetch client (`request`, `ApiError`, cookie auth, CSRF header, mirrored wire types); `format.ts` = pure display helpers and `friendlyError`; `edits.ts` = next-value builders for entry edits; `hooks.ts` = TanStack Query hooks incl. optimistic `useEntryPatch`; `settings.ts` = preferences; `components/` (shell, detail boxes, history drawer, rating control) and `routes/`. Never imports `packages/*` at runtime | exists |
| `packages/core` | domain services, one module per area (import by path, no barrel): `auth`, `crypto`, `works`, `matching`, `entries` (entries, library listing, quick add, per-user moves, match suggestions), `engagements`, `history` (`user_history`, restore), `lists`, `copies`, `custom-fields`, `attachments`, `preferences` (per-user rating scale and default visibility), `export` (per-user bundle, CSV, restore), `backup` (scheduled bundles + SQLite `VACUUM INTO`), `sync/` (the change engine: `snapshot` hashing and versions, `apply` per-record diff and apply, `run` = `runSync`, `changes` = Changes inbox `listChanges`/`resolveChange`, `listens` = `ingestListens`), `jobs` (queue, worker, scheduler), `registry` (`loadPlugins`), `connections` (CRUD, plugin context, `syncHandler`) | exists (partial) |
| `packages/db` | Kysely over one Bun.SQL dialect (SQLite + Postgres), ids, JSON helpers, migrations, `forEachDialect` test harness | exists (schema pending) |
| `packages/plugin-sdk` | plugin contract, `SourceRecord`, http/rate-limit helpers, fixture test harness | planned |
| `plugins/<id>` | one source integration each (hardcover, trakt, mangabaka, listenbrainz, lastfm, csv) | planned |
| `docs/design/mockups` | approved UI mockups (reference only) | exists |
## Data flow
Planned sync lifecycle:
1. The scheduler enqueues a job.
2. The worker loads the plugin and calls `pull` or `pullChanges` with an SDK context.
3. The plugin returns `SourceRecord[]`.
4. The core stores a new `source_record_versions` row for each changed hash and diffs the records into `change_events`.
5. Each event is either auto-applied (new record, or a field never edited by the user) or sent to the inbox.
6. Every user-layer write appends to `user_history`.
HTTP request (`apps/server`):
1. `Bun.serve` routes `/api/*` and `/ingest/*` to `app.fetch`; everything else gets the web app (HTML import).
2. `apiHandler` matches method + path in the route table, applies the per-IP rate limit (login/register), authenticates
(`session` cookie `stt_session`, Bearer app token with a scope, or either), requires `X-Requested-With: sttracker` on
every non-Bearer write (login included), reads the body (JSON ≤ 1 MB with `Content-Type: application/json`,
multipart ≤ 50 MB) and validates it with zod.
3. The handler calls one core function with the caller's `userId` and actor (`token:<name>` for Bearer, else `"user"`;
`field_sources` stores both as `"user"`). Core enforces ownership
(`NotFoundError` → 404). Errors become `{ error: { code, message } }`; unknown errors are logged and return 500 `internal`.
4. `POST sources/:id/run|import` only enqueue a `sync` job (lockKey `sync:<id>`, 409 when held); the worker runs it.
Sync lifecycle (`packages/core/src/sync/`; steps 1–2 are `jobs.ts` and `connections.ts`):
1. The scheduler enqueues a `sync` job (lockKey `sync:<connectionId>`) for each non-paused connection that is due.
2. The worker claims the job; `syncHandler` loads the connection and plugin and calls `runSync(deps, plugin, connectionId, mode)`, injecting
`makeContext` (http client, credentials, cursor). `mode` picks `pull`, `pullChanges` or `importFile`.
3. `runSync` opens a `source_runs` row and iterates the plugin's `SourceRecord`s one at a time.
- `listen` records are batched into `ingestListens` (insert-or-ignore into `listens`; no snapshot, no events).
- Every other record goes through `applyRecord`: hash its normalised form; same hash as
`source_records.current_hash` → only `last_seen` moves. Otherwise a new `source_record_versions` row
(gzipped `{normalized, raw}`) is written, and:
- **new record**: `matchWork` (own transaction, result committed on the record row so retries reuse it),
then in one apply transaction create/patch the entry, engagement (de-duplicating plays across sources)
or list, and write an `added` `auto_applied` event;
- **changed record**: fields that differ from the previous version are decided against `field_sources`
→ applied (`changed` `auto_applied`) or filed as `pending` `changed`/`conflict` events. Mapped records
are updated through `source_records.target_*`, never re-matched;
- `deleted: true` → `pending` `removed`; a new `mergedIntoKey` → `pending` `merged`.
4. After a `full` pull that completed and applied every record, records not seen are marked removed
(`source_records.deleted_at` + `pending` `removed`).
5. The run's counts, log and error land in `source_runs`; the connection gets `last_run_at` and `status`.
6. The user resolves `pending` events with `resolveChange` (accept / ignore / keep_mine).
7. Every user-layer write appends to `user_history` with the plugin actor `plugin:<pluginId>:<connectionId>`.
## External dependencies
- Database: SQLite file under `DATA_DIR` (default) or Postgres via `DATABASE_URL`. If unavailable, nothing works.
+306
View File
@@ -4,6 +4,289 @@ Append-only. Never edit or delete an entry. Supersede it with a new one that lin
One entry per decision a future maintainer would otherwise have to reverse-engineer.
No index: `grep -n '^## ' docs/DECISIONS.md` lists every entry (date, title, line number).
## 2026-10-30: Web app (Task 17): preferences, first run, optimistic edits, AA on orange, self-hosted fonts
**Status:** accepted
**Context:** Task 17 and its fix round 1 (controller rulings).
**Decision:**
- **Three small API routes for the web app.** `GET auth/status` (no auth) → `{ hasUsers }`: `/login` redirects a fresh
instance to `/register`, which shows "Create the admin account", or invite mode when users exist (also on
`invite_required`, and with `?invite=` prefilled). `GET attachments?entity=&entityId=` lists an owned record's
attachments with their links (404 when the record is not the caller's). `GET/PATCH me/preferences` →
`{ ratingScale: "5half"|"5"|"10"|"100", defaultVisibility }`, stored in `users.preferences` under the same keys
`defaultVisibility()` already reads (`packages/core/src/preferences.ts`); unknown stored keys are kept.
- **Rating scale** comes from preferences (default five stars with halves). Stored ratings stay integers 0–100; the web
formats them with `formatRating(value, scale)`.
- **Entry edits are optimistic and serial.** `useEntryPatch` updates the cached entry in `onMutate` (rollback on error),
runs one entry's PATCHes in order through a TanStack mutation `scope`, and `edit(cur => patch)` builds from the latest
cache, so two quick tag adds keep both. Only the last queued edit writes the server's answer back.
- **Keyboard rating saves once:** arrow steps update locally and save 500 ms after the last key, on blur, or on unmount,
so stepping does not write a history row per keypress.
- **Errors in the product's voice:** every error display goes through `friendlyError()` (`apps/web/src/format.ts`), which maps
`validation` field/reason pairs and 413 to plain copy; other API errors keep the server's message.
- **AA contrast on the approved palette:** dark text on `--warn`, `--text-faint` group labels, `--text-muted` placeholders on
`--surface-3` (recorded in DESIGN.md).
- **Fonts are vendored** (Inter woff2 from fontsource; Gelasio as the official variable TTFs from github.com/google/fonts
`ofl/gelasio`, unmodified, because no woff2 tooling was available offline) with their OFL texts.
- **The web app mirrors wire types** in `apps/web/src/api.ts` instead of importing `@sttracker/core` (apps/web has no
workspace dependency on it, and the browser bundle must not pull server code).
## 2026-10-30: Token actor in history, connection deleted_at, CSRF on every browser write
**Status:** accepted. Supersedes the **Actor**, **CSRF** and **Deleted connections** bullets of
"HTTP API: auth modes, CSRF header, error mapping, actor" below.
**Context:** Task 16 fix round 1 (controller rulings).
**Decision:**
- **Actor vs. owner.** API writes pass actor `token:<name>` for Bearer requests and `"user"` for sessions, so history
shows which script changed what (spec §6.9). `field_sources` stores `fieldOwner(actor)`: every actor that does not start
with `plugin:` is stored as `"user"`. A token edit is the user's edit, and syncs file a pending change instead of
overwriting it. List ownership, which is read from history, applies the same mapping.
- **CSRF.** `X-Requested-With: sttracker` is required on every non-GET request that has no Bearer token, including the
`auth: "none"` routes login and register (login CSRF). JSON-body routes also require `Content-Type: application/json`
(415 `unsupported_media_type` otherwise), so a cross-site simple form post cannot pass as JSON.
- **Deleted connections** carry `source_connections.deleted_at` (migration 0003). `deleteConnection` sets it and still pauses
the connection and wipes its credentials. Listing, lookups, updates, the scheduler and the sync handler ignore deleted rows.
A user-paused connection without credentials is a normal, visible, resumable connection.
- **Dev mode** (`Bun.serve({ development })`) is on only with `NODE_ENV=development` or `bun --hot` (detected through
`process.execArgv`; `Bun.argv` does not carry runtime flags). Everything else runs in production mode.
## 2026-10-30: HTTP API: auth modes, CSRF header, error mapping, actor
**Status:** accepted
**Context:** Task 16 (`apps/server`).
**Decision:**
- **Auth modes per route:** `none`, `session` (cookie only: tokens, sources create/edit/delete, device auth, bundle import,
admin), `token:<scope>`, `any:<scope>` (cookie or Bearer). A Bearer token valid for another scope gets 403, an unknown or
revoked one 401. App tokens cannot mint tokens or handle source credentials.
- **CSRF:** cookie-authenticated non-GET requests must send `X-Requested-With: sttracker` (403 otherwise). Browsers cannot
add it cross-site without a CORS preflight, which the server never grants. Bearer requests are exempt. Cookie is
`stt_session`, HttpOnly, SameSite=Lax, Secure when `PUBLIC_URL` is https, 30 days.
- **Error mapping:** zod and `ValidationError` → 400 `validation` (`field: reason`); `AuthError` →
401 (`invalid_credentials`, `unauthenticated`), 403 (`forbidden`), 400 with its own code (`invite_required`,
`username_taken`, `invalid_username`); `NotFoundError` → 404 `not_found`; `ImportError` → 400 with its code (409 for
`account_not_empty`, `conflict`); unique violations → 409 `conflict`; body over the cap → 413 `validation`;
rate limit → 429 `rate_limited`; anything else → 500 `internal` with a fixed message, detail logged only.
- **Limits:** JSON bodies 1 MB, multipart 50 MB (counted while streaming, not trusted from Content-Length).
Login and register: 10 per minute per client IP, in process (resets on restart; not shared across instances).
- **Actor:** every API write uses actor `"user"`, including token-authenticated ones, because the change engine only
protects fields whose `field_sources` is `"user"`. A `token:<name>` actor would let the next sync overwrite a script's edit.
- **Daily backup:** queued for the next 03:00 local with lockKey `"backup"` at boot, 5 s after each run (the finishing job
still holds the lock while it runs), and hourly as a safety net. All three are the same idempotent `enqueue`.
- **Deleted connections** (`paused` with no credentials) are hidden from `GET sources` and 404 on every `sources/:id` route.
- **File imports** travel in the job payload as base64 (`{ connectionId, file: { name, base64 } }`); `syncHandler` decodes
them into `runSync`'s `{ file }` mode, recorded as run mode `file`.
- **OpenAPI** 3.1 is generated from the route table (`z.toJSONSchema`, input side, unrepresentable → any): every route
with method, path, summary, security and request schema. Responses are not typed yet.
## 2026-10-30: Job failure semantics and plugin reload limits
**Status:** accepted
**Context:** Task 9 fix round 1.
**Decision:**
- **Failures before `runSync` never retry.** A missing/paused connection drops the job (`DropJobError`: `done`, message in
`last_error`). A missing plugin or undecryptable credentials (`CryptoError`) insert a failed `source_runs` row, set the
connection to `error`, advance `last_run_at` and throw `NonRetryableError` (job `failed`, no backoff). The scheduler still
picks `error` connections, one interval later, so fixing the cause heals them.
- **Stale locks.** Every worker tick re-queues `running` jobs whose `locked_at` is over an hour old (attempts unchanged), so a
crash cannot hold a lockKey forever. Job bookkeeping errors are logged (`[jobs]`), never unhandled; `stop()` never rejects
and aborts the `signal` passed to handlers (`handler(payload, { signal })`).
- **Reload** evicts the entry file from `require.cache` and imports with `?v=<Date.now()>` (Bun ignores the query alone for
cached `.ts`). Only entry files reload: changes in a plugin's other modules need a server restart.
- `credentials.save` and `updateConnection` read-merge-write in one transaction (row lock on Postgres). `ctx.log` is buffered and
appended to the run's `source_runs.log` when the run ends.
## 2026-10-30: Jobs, plugin registry, connections: rate limit in the manifest, secret split, per-connection lock
**Status:** accepted
**Context:** Task 9 (`packages/core/src/{jobs,registry,connections}.ts`).
**Decision:**
- **Rate limit lives in the manifest** (`PluginManifest.rateLimit?: { perMinute, burst? }`), not in plugin code. The host builds
`ctx.http` from it (default 60/min, burst 5, UA `STTracker/0.1 (+self-hosted)`), so every plugin is throttled the same way
and the per-plugin `RATE_PER_MINUTE` constants are gone.
- **Secrets are split off config.** Config fields typed `"secret"` in the manifest are stored only in the encrypted
`credentials_enc` blob (with OAuth tokens a plugin saves), never in `config`. `makePluginContext` merges them back into
`ctx.config`; `listConnections` returns plain config and `hasCredentials` only. `updateConnection` keeps secrets that are
omitted and replaces plain config wholesale. `credentials.save` merges and persists immediately, so a token refreshed
mid-pull survives a crash.
- **Constructor deviation.** `createConnection`/`updateConnection` take the plugin `manifest` as a trailing argument (needed
for the split), and `makePluginContext` is async (it decrypts) and takes the manifest before an optional `http`.
- **One sync per connection.** Sync jobs carry `lockKey = "sync:<connectionId>"`; `enqueue` returns null while a
`queued|running` job holds the key. On Postgres the check and insert run under `pg_advisory_xact_lock(hashtext(key))`.
Claiming is `update … where status = 'queued'` checked on affected rows. Retries back off `2^attempts` minutes, 5 attempts, then `failed`.
- **Mode choice.** The sync handler runs `full` when asked, when the plugin lacks `pullChanges`, or when no `ok` full run started
within 7 days; else `changes`. A failed run is rethrown so the worker retries it.
**Consequences:** Reload re-imports plugin modules with `?v=<Date.now()>`; old module instances are not freed by Bun.
## 2026-10-30: Sync engine: hash composition, anti-flip-flop, no revive, root ancestor, dedupe
**Status:** accepted
**Context:** Task 8 built the change engine (`packages/core/src/sync/`). Spec §8.3 fixes the outcomes; these
are the mechanics a maintainer would otherwise reverse-engineer.
**Decision:**
- **Hash composition.** sha256 of the stable-key-ordered JSON of `{kind, work, entry|engagement|listen|list,
deleted, mergedIntoKey}`. `key` and `raw` are excluded; `deleted` is always a boolean and `mergedIntoKey`
always present (null), so an omitted flag and an explicit `false` hash alike. A version row stores
base64(`Bun.gzipSync`) of `{normalized, raw}`: the normalised record is what the next change is diffed
against (field-level diff = previous version vs. incoming), the raw payload is kept for the record.
- **Per-field decision.** Only fields that differ from the previous version are considered, and of those
only values that differ from the current STTracker value. `field_sources[f]` unset or this connection's
actor → applied; `"user"` → `pending changed {field, from: current, to: incoming}`; another source →
`pending conflict {field, current, currentSource, incoming, incomingSource}`. A new record landing on an
entry that already exists goes through the same per-field decision, so a first sync never overwrites
what the user or another source wrote. Unchanged values are never sent, so a plugin never takes
ownership of a field it did not change.
- **Anti-flip-flop and superseding.** A pending `changed`/`conflict` for the same (record, field, incoming
value) is never filed twice. A different newer value for the same (record, field) resolves the older
pending event as `ignored` with `payload.supersededBy = <new event id>`, so at most one is pending per
field. If the source comes round to the value STTracker already holds, its pending events for that field
are resolved `ignored` with `payload.reason = "source_agrees"`. `keep_mine` re-stamps
`field_sources[f] = "user"`, so later values from any source become a user-decided `changed`, never
another source-vs-source conflict.
- **No revive.** If the mapped entry/engagement is soft-deleted, the record is still snapshotted but nothing
is applied and no event is filed; a new record on a work whose entry the user deleted adds nothing.
Only the user revives.
- **Root ancestor.** Creating an entry on a child work also creates a `status "none"` entry (plugin actor,
empty `field_sources`) on the root ancestor work, unless the user has one, live or deleted.
- **Mapped records are never re-matched.** Only new records go through `matchWork`. Its result is committed
at once as the record's row (`current_hash ""`, `target_entity "work"`), so a failed apply is retried
against the same work and an ID-less hint never leaks a second draft. A changed work hint refreshes the
mapped work via `updateWorkFromHint` (shared works only). `moveEntryToWork` re-points `source_records`
from a merged-away entry to the survivor in the same transaction, so a user's merge never orphans a source.
- **Engagement dedupe.** Among live engagements on the entry not already sourced by this connection:
finish dates match when both are datetimes ≤ 24 h apart as instants, or when both are day precision and
their dates (first 10 chars, computed with `Date.UTC`, no timezone shift) are ≤ 1 day apart; or start
datetimes are the same instant. A match attaches an `engagement_sources` row and fills fields that are
empty and that no source has set. Every other differing field goes through the per-field decision
(so a disagreeing rating is a `conflict`), except the date that identified the play, whose difference is
within the tolerance by definition.
- **Removal.** Absence counts only after a `full` pull whose iteration finished and whose records all
applied without error; any record error fails the run and skips removal. A removed record that comes
back clears `deleted_at` and retires its still-pending `removed` event as `ignored`. A merged-away record
(`merged_into_key` set, or a pending `merged` event) is expected to vanish: no removal, ever. Accepting
`removed` on an engagement other sources still report only deletes this connection's
`engagement_sources` row; the engagement is soft-deleted when no source remains. Run counts include only
records whose transaction committed.
- **Listens skip snapshots.** Listen records are not stored as `source_records` (they are high-volume and
append-only). `listens` insert-or-ignore is the idempotence; they are never removed. Per batch of 500:
one lookup of existing keys, each distinct recording resolved once, one multi-row insert.
- **Lists.** A new `list` record creates a user list (kind `list`, description `Imported from <plugin id>`),
items matched outside the transaction (an ID-less item first reuses the user's own private work with the
same media type and title). Lists have no `field_sources`; ownership is derived from `user_history`:
`name`/`description` belong to whoever last changed them, `items` to the user once any user edit exists.
A user-owned field becomes a pending `changed` (for items, the whole incoming item set); otherwise the
fields are applied and items replaced (`replaceListItems`, one history row).
- **Merge accept.** The target work is the merge target record's mapped entry's work, else `matchWork` on
its latest hint. Admin-merge events (`toWorkId`, no connection) resolve through the same `resolveChange`.
- **Gone targets.** Accept/keep_mine on an event whose target is deleted (or merged away with no live
survivor, found through the user's work alias) resolves it `ignored` with `payload.reason = "target_gone"`.
**Alternatives:**
- Diffing against the live entry instead of the previous version would re-file decided conflicts each run.
- Hashing `raw` would turn every cosmetic API change into a new version.
- Keeping one run "ok" despite failing records risks marking those records removed.
**Consequences:** Concurrent runs of one connection are not locked here (Task 9's worker; `docs/TODO.md`).
## 2026-10-30: Export format `sttracker-export/v1`: per-user zip, camelCase JSONL, deleted rows included, no secrets
**Status:** accepted
**Context:** Task 15 (`packages/core/src/{export,backup}.ts`) fixed the on-disk bundle format that backups and restore share.
**Decision:**
- **Files.** A zip (`fflate`) holding `manifest.json` (`{ format, exportedAt, user: { username, displayName } }`),
`works.jsonl`, `entries.jsonl`, `engagements.jsonl`, `listens.jsonl`, `lists.jsonl` (items nested),
`copies.jsonl`, `custom_fields.jsonl` (field definitions), `history.jsonl`, `attachments.jsonl` (links nested)
and `attachments/<sha256>` bytes. One JSON object per line, `\n` separated.
- **Keys.** camelCase; JSON columns are parsed into real JSON; `favorite` is a boolean. `user_id` columns are omitted
(the importing user owns everything). `works.jsonl` adds `private` and `externalIds`.
- **Deleted rows are included** with their `deletedAt`, and `fieldSources`/timestamps are kept, so a restore is lossless.
- **No secrets.** No users row beyond username/display name, no sessions, tokens, connections, credentials or source snapshots.
- **Merges and aliases.** `works.jsonl` rows carry `mergedIntoId` (the survivor and ancestors are exported too) and
`work_aliases.jsonl` holds the user's `{ fromWorkId, toWorkId }` rows. Both are additive to v1. Restore re-creates
tombstones (for works it creates) and aliases, so a later write to the old work still resolves to the moved entry.
- **Export is one snapshot:** all reads run in a single transaction (repeatable read on Postgres).
- **Restore** (`importUserBundle`) needs an account with no entries (`account_not_empty`) and `format === "sttracker-export/v1"`
(`unsupported_format`). Works with external IDs are re-matched (else created shared); works without become private.
Ids are remapped (UUIDs inside history `before`/`after` and custom-field value keys too). Rows are inserted directly, so no
new history is written. The import validates first (attachment mime/size/sha256, rating 0-100, status, visibility, 2 GB
uncompressed cap) and writes all rows, attachments included, in one transaction; errors are
`invalid_attachment`, `corrupt_bundle`, `bundle_too_large`, `conflict`. Works merged away at export time are exported as the tombstone row without `mergedIntoId`.
- **Backups** write `<dataDir>/backups/<ISO with ':'->'-'>/<username>.zip` per user plus `sttracker.db` (`VACUUM INTO`, SQLite only),
are built in `<stamp>.tmp` then renamed (modes 0700/0600), and prune to the newest `keep` (default 14).
**Consequences:** Any change to these files after release needs `v2` and a migration path for old bundles.
## 2026-10-30: User layer: per-user moves merge entries, keep aliases, and roll the library up by hierarchy
**Status:** accepted
**Context:** Task 6 built the user layer (`packages/core/src/{entries,engagements,history,lists,copies}.ts`).
Several behaviours are not obvious from the spec and would otherwise have to be reverse-engineered.
**Decision:**
- **Merge on move.** `moveEntryToWork` onto a work where the user already has an entry merges the two:
- Live target: the target's values win. Empty target fields are filled from the moved entry, carrying
the moved entry's `field_sources`. Tags are unioned and custom fields and overrides merged; their
`field_sources` is `"user"` if either side's was, otherwise the target's.
- Differing non-empty `review`/`private_notes` are concatenated (target, `\n\n---\n\n`, moved).
- A conflicting rating keeps the target's; the moved rating is recorded in the history row's `before`.
- Engagements are re-parented, each with an `entryId` history row (restorable), and the moved entry is
soft-deleted.
- Soft-deleted target: revived with the moved entry's values and `field_sources` as-is.
- **Alias bookkeeping.** A move writes `work_aliases (user, old → target)`, re-points aliases that led to
the old work, drops any alias leaving the target (no cycles), tombstones the user's own private draft,
and re-points this user's copies (with history) and list items. Writes to an aliased or merged work
(`upsertEntry`, copies, list items) resolve to the target. **Undoing a move** via `restoreVersion` deletes
the forward alias and writes no reverse alias, so later syncs onto the target land on the target.
- **Library cursor.** `listEntries` cursors are base64 JSON `{ sort, sortValue, id }`. The sort is carried
so a cursor cannot be replayed under another sort. For the title sort, `sortValue` is the SQL
`lower(w.title)` of the last row, never a JS lowercase, so SQL and cursor agree on non-ASCII titles.
- **Custom fields patches replace the whole map** (`customFields` in an entry, engagement or copy patch).
This makes restore exact. `field_sources` tracks the key `customFields` as a whole. Restore does not
re-check values against today's definitions, because it is the user's own past data.
- **List items are hard-deleted.** `list_items` has no `deleted_at`. The list's `user_history` snapshot
holds the ordered items, so `restoreVersion` brings removed items back.
- **Hierarchy visibility.** `childSummary` counts the user's live entries up to three levels below the
entry's work (child, grandchild, great-grandchild). The default library view (`parentId: null`) lists
entries with no ancestor, up to three levels, on which the user has a live entry. An episode whose show
is untracked therefore stays visible, and is rolled up once the show gets an entry. Drilling in
(`parentId: X`) applies the same rule relative to X: descendants up to three levels with no live-entry
work strictly between them and X. `childSummary` still counts every descendant entry.
- **Concurrent creates.** `upsertEntry` inserts with `ON CONFLICT (user_id, work_id) DO NOTHING`, then
patches through the update path, so a concurrent creator never aborts the caller's transaction.
**Alternatives:**
- Refusing a move onto an existing entry pushes the merge onto the user.
- Recursive CTEs for hierarchy are portable but heavier; three levels cover show → season → episode.
- A JS-lowercased cursor breaks on SQLite, whose `lower()` is ASCII-only.
**Consequences:** Undoing a move does not move copies or list items back, and hierarchy deeper than three
levels is not rolled up (both in `docs/TODO.md`).
## 2026-10-30: A single Kysely dialect over Bun.SQL for SQLite and Postgres
**Status:** accepted
**Context:** Kysely ships no driver for Bun. `Bun.SQL` speaks both `sqlite://` and `postgres://` behind one API.
**Decision:**
- `packages/db/src/dialect.ts` is one Kysely `Dialect` whose driver wraps `new SQL(url)`, choosing Kysely's
SQLite or Postgres adapter, compiler and introspector by URL scheme.
- Postgres connections come from `sql.reserve()`. Bun's SQLite adapter does not support `reserve()`, so SQLite
uses the single `SQL` instance as its only connection, serialised by a mutex so transactions never interleave.
- Transactions are plain `begin`/`commit`/`rollback` on the held connection.
**Alternatives:**
- Separate dialects over `bun:sqlite` and `Bun.sql`: two drivers to maintain for the same job.
- Community Bun dialect packages: an extra dependency, and none covers both databases.
**Consequences:** SQLite is effectively single-connection (fine for a self-hosted app). Savepoints and real
streaming are not implemented; `streamQuery` returns the whole result as one chunk.
## 2026-10-30: JSON columns are stored as text on both dialects
**Status:** accepted
**Context:** Spec §11 calls for `jsonb`. SQLite has no JSON type, so `jsonb` on Postgres would need
per-dialect read/write code in every service that touches a JSON column.
**Decision:** JSON columns are `text` everywhere, written with `toJson()` and read with `fromJson()` from
`@sttracker/db/json`. `fromJson` returns a caller-supplied fallback for null or invalid text.
**Alternatives:** `jsonb` on Postgres with per-dialect read/write code.
**Consequences:** No JSON operators or indexes in SQL; filtering on JSON content happens in app code. Can be
superseded later by a migration to `jsonb` if a query needs it.
## 2026-10-30: UUIDv7 instead of ULID for IDs
**Status:** accepted
**Context:** Spec and `AGENTS.md` say IDs are ULIDs generated in app code. Bun has `Bun.randomUUIDv7()` built in.
**Decision:** `newId()` in `@sttracker/db/ids` returns `Bun.randomUUIDv7()`. IDs are stored as `text`.
**Alternatives:** ULID, which needs a dependency or hand-written encoder.
**Consequences:** IDs are equally time-sortable as strings with no extra dependency. They are 36 characters
instead of 26. Where `AGENTS.md` or the spec say "ULID", read "UUIDv7".
## 2026-10-30: Product design decisions recorded in spec 001
**Status:** accepted
**Context:** Brainstorming with the owner produced the v1 design.
@@ -46,3 +329,26 @@ No index: `grep -n '^## ' docs/DECISIONS.md` lists every entry (date, title, lin
**Consequences:** Rule changes happen in one file, and the pointer files must stay thin. Plans are working
artefacts: kept after merge, they go stale and contradict the code, so the spec is the record.
## 2026-10-30: Catalogue matching is race-safe through unique keys plus whole-transaction retry
**Status:** accepted
**Context:** Two syncs matching the same new work at once both miss the lookup and both insert. On
Postgres the loser hit the `external_ids (namespace, value)` unique key and crashed the sync; seasons and
episodes had no key at all and were duplicated.
**Decision:**
- Migration `0002_works_position_unique` adds a unique index on `works (parent_id, position)`. NULLs are
distinct on both dialects, so top-level works and position-less children are unaffected.
- `matchWork` re-runs the whole resolve in a fresh transaction on a unique violation (SQLite
"UNIQUE constraint failed", Postgres SQLSTATE 23505), up to 3 attempts. Postgres aborts the transaction
on the error, so a statement-level retry is not possible.
- Adding missing external IDs to an existing work uses `ON CONFLICT DO NOTHING`.
- An ID match only counts if the work is shared or owned by the matching user, and private works can
never carry external IDs (`createWork` refuses; global merges accept only shared works).
- A global merge repoints children of `from` to `to` unless `to` already has a child at that position.
**Alternatives:**
- Advisory or table locks: dialect-specific, and SQLite already serialises writes.
- Upserts throughout `createWork`: would hide genuine conflicts and need per-dialect `RETURNING` handling.
**Consequences:** Parents resolved without IDs or position (title/year reuse) still have no unique key
and can be duplicated under concurrency; tracked in `docs/TODO.md`.
+31
View File
@@ -7,8 +7,20 @@ Small, specific, independently actionable. Remove only when actually done.
- [ ] Choose a linter/formatter (e.g. Biome), then add `lint` to `package.json`, `AGENTS.md` and `docs/PERMISSIONS.md`.
- [ ] Add CI on Gitea Actions: install, typecheck, and test on SQLite and Postgres. Add a guard that fails if `AGENTS.md` is over 150 lines.
- [ ] Agent permissions for Task 16 commands: add `bun run typecheck`, `bun apps/server/src/main.ts` and `bun --hot apps/server/src/main.ts` (all verified 2026-10-30) to `docs/PERMISSIONS.md`, `.claude/settings.json`, `.gemini/settings.json`, then fill Typecheck/Dev/Start in `AGENTS.md` Commands. Needs a human: agents may not edit their own permission files.
## Next
- [ ] API: response schemas in the OpenAPI document (requests only today).
- [ ] Deployment docs (Task 19): the server runs `Bun.serve` in production mode unless `NODE_ENV=development` or `bun --hot`, so Docker needs no `NODE_ENV`. Document `NODE_ENV=development` for local debugging.
- [ ] API: `restoreVersion`, `acceptSuggestion` and `resolveChange` take no actor, so history records `"user"` for them even when an app token calls them. Thread the actor through if script provenance matters there.
- [ ] API: login/register rate limit is per process and keyed on the socket IP; behind a reverse proxy every client shares one key. Add a trusted-proxy setting for `X-Forwarded-For`.
- [ ] API: file imports are base64 in the `jobs.payload` row (up to ~67 MB of text). Move uploads to `DATA_DIR/uploads/` and pass a path.
- [ ] API: `PATCH list-items/:id` only moves items; editing an item's note needs a core function.
- [ ] Phase 2 and Phase 3 plans (spec §12).
- [ ] Generate image thumbnails for attachments (Phase 1 sizes images with CSS).
- [ ] Web: convert the vendored Gelasio variable TTFs (~170 KB each) to latin-subset woff2 once fonttools/woff2 is available; Bun inlines fonts into the CSS bundle.
- [ ] Web/API: "Linked sources" is inferred from `plugin:` actors in `fieldSources`, so a source that matched an entry but never wrote a field is not shown. Add a per-entry source-links route.
- [ ] Web: the detail pane only lists attachments of the entry itself, not those linked to its reads or copies.
## Someday
@@ -23,4 +35,23 @@ Small, specific, independently actionable. Remove only when actually done.
file, and compare steps and token use.
## Known issues
- [ ] Web: in production the bundled HTML references `/../../chunk-*.css|js` (Bun bundles `../../web/index.html` from `apps/server`). Browsers normalise it to `/chunk-*` and it loads; check it behind a reverse proxy.
- [ ] Plugins: reload re-imports entry files only; edits to a plugin's other modules need a server restart.
- [ ] Plugins: rate limiting is per run (each run builds its own client), not per plugin (spec §11); share a limiter per plugin in Phase 2.
- [ ] Plugins: `ctx.log` reaches `source_runs.log` only after the run ends, so a crash mid-run loses it.
- [ ] Sync: two concurrent runs of one connection outside the job queue (e.g. a manual run) can still race on `source_records`.
Queued syncs are serialised by `lockKey`.
- [ ] Restore: a new child work whose parent+position collides with an existing child of a re-matched parent fails
with `ImportError("conflict")` instead of being merged or repositioned.
- [ ] Restore: an unlink of identical attachment bytes between the pre-write and the commit could remove the file;
mitigated by re-writing after commit, not locked.
- [ ] Export bundles hold attachment bytes in memory; very large libraries need a streaming zip.
- [ ] Admin merge does not merge colliding children recursively; colliding children stay under the tombstoned parent.
- [ ] Matching: a shared parent resolved without external IDs or position (title/year reuse in
`packages/core/src/matching.ts`) has no unique key, so concurrent matches can create it twice.
- [ ] Library search (`q`) and the title sort use the work's title, not the user's `overrides.title`
(overrides are a JSON text column). The `tag` filter matches the JSON text with LIKE.
- [ ] Library hierarchy rollup and orphan detection look three levels deep; deeper trees are not rolled up.
- [ ] Undoing a per-user move does not move copies or list items back, nor restore aliases that the move
re-pointed.
- [ ] `bun run typecheck` fails with TS18003 until source files exist under `apps/`, `packages/` or `plugins/`.
+642
View File
@@ -0,0 +1,642 @@
---
status: in-progress
---
# Phase 1, "Own your data": implementation plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task by task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** A self-hosted Bun app where users connect Hardcover, Trakt, MangaBaka, ListenBrainz, Last.fm and CSV/JSON sources. STTracker pulls and snapshots their data, detects platform-side changes into a Changes inbox, and never overwrites what the user wrote. Users can log anything without IDs and export or back up everything. SQLite or Postgres, shipped in Docker.
**Architecture:** Bun workspaces.
- `packages/db`: Kysely over a single custom dialect wrapping `Bun.SQL`, which speaks both `sqlite://` and `postgres://`.
- `packages/plugin-sdk`: the plugin contract.
- `packages/core`: domain services and the sync/change engine.
- `plugins/*`: source integrations. They return `SourceRecord`s and never touch the DB.
- `apps/server`: `Bun.serve` API, job worker, and serving the web app.
- `apps/web`: React app via HTML imports.
**Tech stack:** Bun 1.3.14, TypeScript 5.9, Kysely 0.28+, zod 4, fflate, React 19, wouter, @tanstack/react-query.
**Spec:** `docs/specs/001-sttracker-v1-design.md` (read §6–§11 before any task). Design: `DESIGN.md`, `docs/design/mockups/round2-app-scrapbook-river.html`.
## Global constraints
- Follow `AGENTS.md` rules. In particular: Bun-native APIs only; plugins import only `@sttracker/plugin-sdk`; syncs never overwrite user-edited fields.
- **Runtime deps are approved for this plan only:**
- `kysely`, `zod`, `fflate` (root);
- `react`, `react-dom`, `wouter`, `@tanstack/react-query` (apps/web);
- `@types/react`, `@types/react-dom` (dev).
- Ask before adding anything else.
- **IDs:** `newId()` from `@sttracker/db/ids`. It returns `Bun.randomUUIDv7()`. (UUIDv7 replaces the spec's ULID: it is equally time-sortable and built in. This is recorded in DECISIONS by Task 1.)
- **Timestamps:** ISO-8601 UTC strings (`new Date().toISOString()`) stored as `text` on both dialects.
- **JSON columns:** `text` on both dialects. Use `toJson()`/`fromJson()` from `@sttracker/db/json`. (A simplification of spec §11's jsonb; recorded in DECISIONS by Task 1.)
- **Ratings:** integer 0–100 or null.
- **Partial dates:** `{ value: string; precision: "day" | "month" | "year" | "unknown" }`. `value` is `YYYY-MM-DD`, `YYYY-MM`, `YYYY` or `""`, or a full ISO datetime when the source gives a time. Partial dates are never converted through `Date`, which would shift them by timezone.
- **Status enum:** `"planned" | "in_progress" | "paused" | "completed" | "dropped" | "none"`.
- **MediaType enum:** `book comic manga game movie show season episode podcast podcast_episode audio_drama artist album track video other`.
- **Package exports:** each package's `package.json` has `"exports": { "./*": "./src/*.ts" }`. Import by path (e.g. `@sttracker/core/entries`). There are no barrel `index.ts` files, which avoids merge conflicts between parallel tasks.
- **DB tests** use `forEachDialect()` from `@sttracker/db/testing` and must pass on SQLite always, and on Postgres when `TEST_POSTGRES_URL` is set. Local value: `TEST_POSTGRES_URL=postgres://postgres:[email protected]:55432/postgres` (container `sttracker-pg-test`).
- **Tests never hit the network.** Plugins are tested with fixtures in `plugins/<id>/test/fixtures/*.json`, written by hand from the documented API shapes, and played through `createFixtureHttp()`.
- **Attribution strings** are shown on the About page (exact text):
- "STTracker uses Trakt but is not endorsed or certified by Trakt."
- "Manga data from MangaBaka (CC BY-NC-SA 4.0)."
- "Music data from ListenBrainz and Last.fm."
- "Book data from Hardcover."
- **Commits:** Conventional Commits, ending with the line `Co-Authored-By: Claude Opus 5.5 <[email protected]>`.
## Review focus
1. **A full pull that fails part-way** (network error, 429s exhausted, thrown parser error) must mark *nothing* as removed. Only a pull that finished emits `removed` events. Test lives in Task 8.
2. **Running the same sync twice with unchanged source data** creates zero new `source_record_versions`, `change_events`, `engagements` or `listens`. Test lives in Task 8.
3. **A user edits a field, then the source changes it:** the user's value stays and a pending change appears. The same goes for two sources disagreeing: a pending `conflict`, with no flip-flopping on the next run. Test lives in Task 8 on both dialects.
4. **Messy CSV input** must round-trip without data loss: UTF-8 BOM, `;` delimiter, quoted newlines and quotes, empty rating, "2019" as a date, emoji titles. Test lives in Task 13.
5. **Rotating single-use refresh tokens (Trakt):** refreshed credentials are saved before any further request, and two syncs of the same connection never run at once. Tests live in Task 10 (save-before-continue) and Task 9 (connection lock).
---
## Dependency graph and waves
```
Wave A (parallel): T1 db-core T3 plugin-sdk
Wave B: T2 schema (needs T1)
Wave C (parallel): T4 auth T5 catalogue+matching T6 user-layer T7 attachments+crypto
T10 trakt T11 hardcover T12 mangabaka T13 csv/json T14 listenbrainz+lastfm (plugins need only T3)
Wave D (parallel): T8 sync engine (T3,T5,T6) T9 jobs+registry (T2,T3,T7) T15 export/backup (T5,T6,T7)
Wave E: T16 server API (all core)
Wave F (parallel): T17 web shell+library+detail T18 web changes/sources/lists/owned/settings/export T19 docker+CI+docs
```
Parallel tasks run in separate git worktrees branched from `feat/phase1` and are merged back into it after review.
---
### Task 1: `packages/db`: dialect, ids, json, migrations runner, test harness
**Files:**
- Create: `packages/db/package.json` (`@sttracker/db`, deps `kysely`), `packages/db/src/dialect.ts`, `packages/db/src/db.ts`, `packages/db/src/ids.ts`, `packages/db/src/json.ts`, `packages/db/src/migrate.ts`, `packages/db/src/testing.ts`
- Create: workspace `package.json` stubs with `exports` for `packages/core` (`@sttracker/core`), `packages/plugin-sdk` (`@sttracker/plugin-sdk`), `apps/server` (`@sttracker/server`), `apps/web` (`@sttracker/web`). Add all approved deps to the stubs now (`bun add` in each), so later parallel tasks never touch `bun.lock`.
- Modify: `docs/DECISIONS.md` (UUIDv7, JSON as text, single Bun.SQL dialect)
- Test: `packages/db/test/db.test.ts`
**Interfaces (produces):**
- `type Dialect = "sqlite" | "postgres"`
- `createDb(url: string): Promise<Db>`, where `interface Db { k: Kysely<Database>; dialect: Dialect; close(): Promise<void> }`. `url` is `sqlite://<path>`, `sqlite://:memory:` or `postgres://…`. `Database` is imported from `./schema` (Task 2). Until Task 2 lands, use a placeholder `export interface Database {}` in `src/schema.ts`.
- `newId(): string`, `nowIso(): string`
- `toJson(v: unknown): string`, `fromJson<T>(s: string | null | undefined, fallback: T): T`
- `migrateToLatest(db: Db): Promise<void>` uses Kysely `Migrator` with migrations imported statically from `src/migrations/index.ts` (`export const migrations: Record<string, Migration>`).
- `forEachDialect(name: string, fn: (getDb: () => Db) => void): void` registers a `describe` per dialect.
- Before each test it builds a fresh migrated DB: SQLite `:memory:`, or on Postgres a fresh `create database st_<random>` derived from `TEST_POSTGRES_URL`.
- It closes the DB after each test, and drops the Postgres DB in `afterAll`.
- [ ] **Step 1: Write failing tests** in `db.test.ts` using `forEachDialect("db basics", …)`:
- `"round-trips a row"`: create a temp table via `sql`, insert `{id: newId(), j: toJson({a: 1})}`, select, `expect(fromJson(row.j, {})).toEqual({a: 1})`.
- `"transactions roll back"`: `await expect(db.k.transaction().execute(async tx => { insert…; throw new Error("x") })).rejects.toThrow("x")`, then count is 0.
- `"newId is time-sortable"`: 1000 ids are already sorted.
- [ ] **Step 2:** Run `bun test packages/db` → FAIL (module not found).
- [ ] **Step 3: Implement.**
- `dialect.ts`: a Kysely `Dialect` whose `Driver` wraps `new SQL(url)`. `acquireConnection` returns a connection from `sql.reserve()`. `executeQuery` calls `conn.unsafe(compiled.sql, compiled.parameters)` and maps it to `{ rows, numAffectedRows }`. Transactions use `BEGIN`/`COMMIT`/`ROLLBACK` on the reserved connection.
- Adapter/compiler/introspector: `SqliteAdapter`/`SqliteQueryCompiler` for sqlite, and `PostgresAdapter`/`PostgresQueryCompiler` for postgres.
- For sqlite, run `PRAGMA foreign_keys = ON; PRAGMA journal_mode = WAL;` on create (skip WAL for `:memory:`).
- [ ] **Step 4:** `bun test packages/db` → PASS. Also run `TEST_POSTGRES_URL=… bun test packages/db` → PASS on both.
- [ ] **Step 5:** Commit `feat(db): Bun.SQL Kysely dialect for sqlite and postgres`.
### Task 2: Schema migration and `Database` types
**Files:** Create `packages/db/src/schema.ts` (Kysely table interfaces) and `packages/db/src/migrations/0001_initial.ts`; modify `packages/db/src/migrations/index.ts`. Test: `packages/db/test/schema.test.ts`.
**Interfaces (produces):** the `Database` interface with these tables. Column names are snake_case. Every table has `id text pk` unless noted. `ts` means ISO text. JSON is text.
- `users`(username unique, display_name, password_hash, role `admin|member`, preferences json, created_at, updated_at)
- `sessions`(user_id fk, token_hash unique, expires_at, created_at)
- `invites`(code_hash unique, created_by fk, used_by nullable, expires_at, created_at)
- `app_tokens`(user_id, name, token_hash unique, scopes json `("read"|"write"|"ingest")[]`, last_used_at, revoked_at, created_at)
- `works`(media_type, title, original_title, year int null, release_date null, creators json, description, cover_url, cover_attachment_id null, metadata json, parent_id null fk works, position json null, owner_user_id null fk users, merged_into_id null, created_at, updated_at)
- `external_ids`(work_id fk, namespace, value). Unique (namespace, value).
- `work_aliases`(user_id, from_work_id, to_work_id). Unique (user_id, from_work_id).
- `work_history`(work_id, before json, after json, actor, created_at)
- `entries`(user_id, work_id, status, rating int null, review null, private_notes null, favorite int 0/1, tags json, custom_fields json, visibility `private|friends|public`, overrides json, media_type_override null, field_sources json, created_at, updated_at, deleted_at null). Unique (user_id, work_id).
- `engagements`(entry_id, started_at null, started_precision, finished_at null, finished_precision, progress json, platform null, rating null, notes null, custom_fields json, field_sources json, created_at, updated_at, deleted_at)
- `engagement_sources`(engagement_id, connection_id, record_key). Unique (connection_id, record_key).
- `listens`(user_id, listened_at, artist, track, album null, duration_ms null, mbids json, work_id null, source, source_key, created_at). Unique (user_id, source, source_key). Index (user_id, listened_at).
- `lists`(user_id, name, description, kind `list|grouping|thread`, visibility, rating null, review null, created_at, updated_at, deleted_at)
- `list_items`(list_id, work_id, position int, note null)
- `custom_field_definitions`(user_id, name, type, options json, applies_to json `MediaType[] | "all"`, level `entry|engagement|copy`, created_at)
- `copies`(user_id, work_id null, title_hint null, format null, acquired_from null, acquired_at null, acquired_precision, price_cents int null, currency null, notes null, custom_fields json, source_ref null, created_at, updated_at, deleted_at)
- `attachments`(user_id, sha256, mime, size int, filename, caption null, taken_at null, visibility null, created_at)
- `attachment_links`(attachment_id, entity `entry|engagement|copy|list|work`, entity_id)
- `user_history`(user_id, entity, entity_id, before json, after json, actor, created_at)
- `match_suggestions`(user_id, work_id, candidate_work_id, score real, status `pending|accepted|rejected`, created_at)
- `source_connections`(user_id, plugin_id, name, config json, credentials_enc null, schedule_minutes int null, cursor null, status `active|paused|error`, last_run_at null, created_at)
- `source_runs`(connection_id, mode `full|changes|file`, started_at, finished_at null, status `running|ok|failed`, counts json, log text, error null)
- `source_records`(connection_id, key, kind, current_hash, first_seen, last_seen, deleted_at null, merged_into_key null, target_entity null, target_id null). Unique (connection_id, key).
- `source_record_versions`(record_id, hash, raw_gz text (base64 of gzip), seen_at)
- `change_events`(user_id, connection_id null, run_id null, record_id null, type `added|changed|removed|merged|conflict`, target_entity, target_id null, payload json, status `pending|auto_applied|accepted|ignored|kept_mine`, created_at, resolved_at null)
- `jobs`(type, payload json, run_at, status `queued|running|done|failed`, attempts int, last_error null, locked_at null, lock_key null, created_at). Index (status, run_at). Unique partial behaviour for lock_key is enforced in code (Task 9).
- [ ] **Step 1: Failing test** `forEachDialect("schema")`, `"all tables exist and unique constraints hold"`:
- insert a user, a work, and two `external_ids` rows with the same (namespace, value);
- expect the second insert to reject;
- insert duplicate `entries` (user, work) and expect a reject.
- [ ] **Step 2:** `bun test packages/db/test/schema.test.ts` → FAIL.
- [ ] **Step 3:** Write the migration using only the Kysely schema builder with portable types (`text`, `integer`, `real`), plus indexes for every fk column.
- [ ] **Step 4:** PASS on both dialects.
- [ ] **Step 5:** Commit `feat(db): initial schema`.
### Task 3: `packages/plugin-sdk`: contract, http client, fixture harness
**Files:** Create `packages/plugin-sdk/src/{types.ts,http.ts,testing.ts,partial-date.ts}`. Tests: `packages/plugin-sdk/test/{http.test.ts,partial-date.test.ts}`.
**Interfaces (produces):** `types.ts`, exactly these names:
```ts
export type MediaType = /* enum from Global constraints */;
export type Status = "planned" | "in_progress" | "paused" | "completed" | "dropped" | "none";
export type Precision = "day" | "month" | "year" | "unknown";
export interface PartialDate { value: string; precision: Precision }
export type Capability = "pull" | "pullChanges" | "push" | "ownedLibrary" | "ingest" | "lookup" | "importFile" | "exportFile";
export interface ConfigField { key: string; label: string; type: "string" | "secret" | "number" | "boolean" | "select"; required?: boolean; options?: string[]; help?: string }
export interface PluginManifest { id: string; name: string; version: string; sdkVersion: 1; mediaTypes: MediaType[]; capabilities: Capability[]; config: ConfigField[]; attribution?: string }
export interface WorkHint { title: string; mediaType: MediaType; year?: number; creators?: { name: string; role?: string }[]; externalIds: Record<string, string>; coverUrl?: string; description?: string; parent?: WorkHint; position?: { season?: number; episode?: number; index?: number } }
export interface EntryFields { status?: Status; rating?: number | null; review?: string | null; notes?: string | null; favorite?: boolean; tags?: string[] }
export interface EngagementFields { startedAt?: PartialDate | null; finishedAt?: PartialDate | null; progress?: Record<string, number | string>; platform?: string | null; rating?: number | null; notes?: string | null }
export interface ListenFields { listenedAt: string; artist: string; track: string; album?: string; durationMs?: number; mbids?: { recording?: string; release?: string; artist?: string } }
interface Base { key: string; raw: unknown; deleted?: boolean; mergedIntoKey?: string }
export type SourceRecord =
| (Base & { kind: "entry"; work: WorkHint; entry: EntryFields })
| (Base & { kind: "engagement"; work: WorkHint; engagement: EngagementFields })
| (Base & { kind: "listen"; listen: ListenFields })
| (Base & { kind: "list"; list: { name: string; description?: string; items: { work: WorkHint; note?: string }[] } });
export interface HttpClient { json<T>(url: string, init?: RequestInit & { query?: Record<string, string | number | undefined> }): Promise<T>; raw(url: string, init?: RequestInit): Promise<Response> }
export interface PluginContext {
http: HttpClient; log: (msg: string) => void; config: Record<string, unknown>;
credentials: { get(): Promise<Record<string, string> | null>; save(c: Record<string, string>): Promise<void> };
cursor: { get(): Promise<string | null>; set(c: string): Promise<void> };
now(): Date; signal: AbortSignal;
}
export interface DeviceAuth { start(ctx: PluginContext): Promise<{ userCode: string; verificationUrl: string; deviceCode: string; intervalSec: number; expiresInSec: number }>; poll(ctx: PluginContext, deviceCode: string): Promise<"pending" | Record<string, string>> }
export interface SourcePlugin {
manifest: PluginManifest;
deviceAuth?: DeviceAuth;
pull?(ctx: PluginContext): AsyncIterable<SourceRecord>;
pullChanges?(ctx: PluginContext): AsyncIterable<SourceRecord>;
lookup?(ctx: PluginContext, q: { title: string; mediaType?: MediaType; year?: number }): Promise<WorkHint[]>;
importFile?(ctx: PluginContext, file: { name: string; bytes: Uint8Array }): AsyncIterable<SourceRecord>;
}
export function definePlugin(p: SourcePlugin): SourcePlugin;
```
- `http.ts`: `createHttpClient(opts: { ratePerMinute?: number; burst?: number; maxRetries?: number; userAgent: string; fetchImpl?: typeof fetch; sleep?: (ms: number) => Promise<void> }): HttpClient`.
- Token-bucket rate limiting.
- Retries on 429/5xx with exponential backoff, honouring `Retry-After` (seconds or HTTP date) and `X-RateLimit-Reset-In`.
- Throws `HttpError { status, body }` after the retries run out.
- `json` sends `Accept: application/json` and appends `query`.
- `testing.ts`: `createFixtureHttp(routes: Record<string, unknown | ((req: Request) => unknown)>): { fetch: typeof fetch; calls: Request[] }`. The key is `"METHOD url-without-query"` or `"METHOD full-url"`. An unknown route throws. `createTestContext(partial?: Partial<PluginContext> & { fetch?: typeof fetch }): PluginContext & { saved: { credentials: Record<string,string>[]; cursor: string[] } }` provides in-memory credentials and cursor. `collect<T>(it: AsyncIterable<T>): Promise<T[]>`.
- `partial-date.ts`: `parsePartialDate(input: string | null | undefined): PartialDate | null` accepts `2019`, `2019-03`, `2019-03-04`, full ISO datetime, `03/04/2019`→null (ambiguous), and `""`→null. `comparePartialDates(a, b): number`.
- [ ] **Step 1: Failing tests.**
- `http.test.ts`:
- `"retries 429 honouring Retry-After"`: fetch returns 429 `Retry-After: 2` then 200; `sleep` was called with 2000; the result parsed.
- `"gives up after maxRetries with HttpError"`.
- `"rate limits to ratePerMinute"`: with a fake clock, 3 calls at `ratePerMinute: 60, burst: 1` sleep ~1000ms between calls.
- `partial-date.test.ts`: a table of the inputs above to the expected `{value, precision}`.
- [ ] **Step 2:** FAIL.
- [ ] **Step 3:** Implement.
- [ ] **Step 4:** `bun test packages/plugin-sdk` → PASS.
- [ ] **Step 5:** Commit `feat(sdk): plugin contract, http client, fixture harness`.
### Task 4: Auth: users, sessions, invites, app tokens
**Files:** Create `packages/core/src/auth.ts`. Test: `packages/core/test/auth.test.ts`.
**Interfaces (produces):**
- `registerUser(db: Db, input: { username: string; password: string; displayName?: string; inviteCode?: string }): Promise<User>`. The first user becomes `admin` and needs no invite. Later users need a valid, unused, unexpired invite or it throws `AuthError("invite_required")`. Usernames match `/^[a-z0-9_]{2,32}$/`, lowercased, unique.
- `login(db, username, password): Promise<{ user: User; sessionToken: string }>`. Sessions last 30 days. The token is 32 random bytes in base64url; only its SHA-256 is stored.
- `getSessionUser(db, token): Promise<User | null>`, `logout(db, token)`.
- `createInvite(db, adminId, ttlDays = 7): Promise<string>` (admins only, else `AuthError("forbidden")`).
- `createAppToken(db, userId, name, scopes): Promise<{ id: string; token: string }>`. The token is formatted `stt_<base64url>` and only shown once. `verifyAppToken(db, token, scope): Promise<User | null>` updates `last_used_at`. `revokeAppToken(db, userId, id)`.
- `type User = { id: string; username: string; displayName: string; role: "admin" | "member" }`. Passwords use `Bun.password.hash` (argon2id).
- [ ] **Step 1: Failing tests** (`forEachDialect`):
- first user is admin;
- second registration without invite → `invite_required`;
- with invite → member, and the invite cannot be reused;
- wrong password → `AuthError("invalid_credentials")`;
- session round-trip and logout;
- app token: `verifyAppToken` with a missing scope → null; revoked → null.
- [ ] **Steps 2–4:** FAIL, then implement, then PASS on both dialects.
- [ ] **Step 5:** Commit `feat(core): users, sessions, invites, app tokens`.
### Task 5: Catalogue: works, external IDs, matching
**Files:** Create `packages/core/src/works.ts` and `packages/core/src/matching.ts`. Tests: `packages/core/test/{works,matching}.test.ts`.
**Interfaces (produces):**
- `createWork(db, input: { hint: WorkHint; ownerUserId?: string | null; parentId?: string | null }, actor: string): Promise<string>` inserts the work and its external_ids.
- `getWork(db, id): Promise<Work | null>` follows `merged_into_id` chains.
- `updateWorkFromHint(db, workId, hint, actor)` fills or updates shared-work display fields, writes `work_history`, adds missing external IDs, and never touches private works.
- `findWorkByExternalIds(db, ids: Record<string,string>): Promise<string | null>`.
- `searchWorks(db, userId, q: string, opts?: { mediaType?: MediaType; limit?: number }): Promise<Work[]>` covers shared works plus the user's private ones, with a case-insensitive `LIKE` on title.
- `matchWork(db, userId, hint: WorkHint, actor: string): Promise<{ workId: string; created: boolean; draft: boolean }>` follows spec §8.4:
1. By external ID: link, and update display fields.
2. If `hint.parent` + `hint.position`: resolve the parent recursively, then find the child by (parent_id, position). If not found, create a shared child.
3. Hint has external IDs: create a shared work.
4. Otherwise: create a private draft (`owner_user_id = userId`) and store up to 5 `match_suggestions`, where `fuzzyScore ≥ 0.6`.
- Shows, seasons and episodes create the whole chain.
- `fuzzyScore(a: {title, year?, creators?}, b: same): number` returns 0..1. Normalise case, punctuation, leading "the/a/an" and diacritics. Score is title token-set Jaccard × 0.7, + year match 0.2 (±1 year counts as 0.1), + any creator match 0.1.
- `resolveAlias(db, userId, workId): Promise<string>`.
- `acceptSuggestion(db, userId, suggestionId)` moves the user's entry from the draft work to the candidate (via `moveEntryToWork` from T6, imported lazily to avoid a cycle). `rejectSuggestion(db, userId, suggestionId)`.
- `adminMergeWorks(db, adminUserId, fromId, toId)` (admin only, else `AuthError("forbidden")`) sets `merged_into_id`, moves the external_ids, writes `work_history`, and repoints every user's entries unless the user already has an entry on `toId`. In that case the user's `from` entry is kept and they get a `pending` `merged` change event.
- `type Work` mirrors the row with JSON fields parsed.
- [ ] **Step 1: Failing tests:**
- `"match by isbn13 links existing"`;
- `"episode hint creates show→season→episode chain once"` (calling it twice → same ids, 3 works total);
- `"hint without ids creates private draft with suggestions"`;
- `"private works invisible to other users in search"`;
- `fuzzyScore("The Expanse: Leviathan Wakes", "Leviathan Wakes")` > 0.6;
- `fuzzyScore("Dune","Dune Messiah")` < 0.6;
- `"Pokémon"` vs `"Pokemon"` = 1 when year and creators are equal or absent.
- [ ] **Steps 2–4:** FAIL, then implement, then PASS on both dialects.
- [ ] **Step 5:** Commit `feat(core): works catalogue and matching`.
### Task 6: User layer: entries, engagements, history, lists, copies, custom fields
**Files:** Create `packages/core/src/{history.ts,entries.ts,engagements.ts,lists.ts,copies.ts,custom-fields.ts}`. Tests: one test file per source file in `packages/core/test/`.
**Interfaces (produces):**
- `recordHistory(tx, { userId, entity, entityId, before, after, actor })` stores only the changed keys.
- `getHistory(db, userId, entity, entityId)` and `restoreVersion(db, userId, historyId)` (re-applies `before` as actor `user`).
- `type Actor = "user" | \`plugin:${string}:${string}\` | \`token:${string}\``. Plugin actors are `plugin:<pluginId>:<connectionId>`.
- `upsertEntry(db, userId, workId, patch: Partial<EntryPatch>, actor: Actor): Promise<Entry>`. `EntryPatch` keys: `status rating review privateNotes favorite tags customFields visibility overrides mediaTypeOverride`.
- It sets `field_sources[key] = actor` for each patched key.
- It writes history.
- It validates rating 0–100 and custom fields against definitions.
- Defaults: `status "none"`, `visibility` from user prefs (`"private"`).
- `getEntry(db, userId, entryId)`.
- `listEntries(db, userId, filter: { mediaType?; status?; tag?; draft?: boolean; q?; parentId?: string | null; sort?: "updated" | "title" | "rating"; limit?; cursor? }): Promise<{ items: EntryView[]; nextCursor: string | null }>`.
- `EntryView` = entry + work (with overrides applied) + `draft` (the work has no external IDs) + `childSummary: { total: number; completed: number } | null`.
- The default `parentId: null` lists top-level only.
- `softDeleteEntry`, `moveEntryToWork(db, userId, entryId, targetWorkId, actor)` (writes a `work_aliases` row).
- `quickAdd(db, userId, { title, mediaType? = "other" }): Promise<EntryView>` creates a private draft work, then the entry.
- `addEngagement` / `updateEngagement` / `softDeleteEngagement` (the same field_sources and history rules), plus `listEngagements(db, userId, entryId)`.
- Lists:
- `createList`, `updateList`, `addListItem`, `moveListItem`, `removeListItem`, `getList`, `listLists`;
- `kind` is `list|grouping|thread`, and a grouping may carry `rating`/`review`.
- Copies: `createCopy`, `updateCopy`, `softDeleteCopy`, `listCopies(db, userId, { workId? })`. Price is stored in cents.
- Custom fields: `defineCustomField`, `listCustomFields(db, userId, mediaType?)`, `validateCustomValues(defs, values)`, which throws `ValidationError(field, reason)`.
- [ ] **Step 1: Failing tests:**
- `"upsertEntry records field_sources and history"`;
- `"restoreVersion brings back previous review"`;
- `"rating 101 rejected"`;
- `"quickAdd creates draft entry"` (`draft === true`);
- `"listEntries rolls up children"`: a show with 3 episode entries, 2 completed → `childSummary {total:3, completed:2}`, and episodes are not listed at the top level;
- `"engagement with year precision keeps '2019'"`;
- `"moveEntryToWork leaves alias"`;
- `"custom select field rejects unknown option"`;
- `"grouping list can be rated"`.
- [ ] **Steps 2–4:** FAIL, then implement, then PASS on both dialects.
- [ ] **Step 5:** Commit `feat(core): entries, engagements, history, lists, copies, custom fields`.
### Task 7: Attachments and secret encryption
**Files:** Create `packages/core/src/attachments.ts` and `packages/core/src/crypto.ts`. Tests: `packages/core/test/{attachments,crypto}.test.ts`.
**Interfaces (produces):**
- `encryptJson(secret: string, value: unknown): Promise<string>` and `decryptJson<T>(secret: string, blob: string): Promise<T>`. Use AES-256-GCM via WebCrypto, with the key = HKDF-SHA256 of `secret` and info `"sttracker-credentials"`. The blob is `v1.<b64 iv>.<b64 ciphertext>`. A wrong secret throws `CryptoError`.
- `storeAttachment(db, dataDir, userId, file: { bytes: Uint8Array; mime: string; filename: string; caption?: string; takenAt?: string }, link: { entity; entityId }): Promise<Attachment>`. Content is addressed by sha256 at `${dataDir}/attachments/ab/cd/<sha256>` and deduplicated. Allowed mimes are `image/*`, `application/pdf` and `text/plain`, with a max of 25 MB (else `ValidationError`).
- `readAttachment(db, dataDir, userId, id): Promise<{ file: BunFile; mime }>` checks the owner.
- `listAttachments(db, userId, entity, entityId)`, `unlinkAttachment(...)`.
- Thumbnails are out of Phase 1. The UI sizes images with CSS (recorded in TODO).
- [ ] **Step 1: Failing tests:** encrypt/decrypt round-trip; wrong secret throws; the same bytes stored twice produce one file and two links; `.exe` mime rejected; another user's read → null.
- [ ] **Steps 2–4:** FAIL, then implement, then PASS.
- [ ] **Step 5:** Commit `feat(core): attachments and credential encryption`.
### Task 8: Sync engine: snapshots, diff, apply, Changes inbox
**Files:** Create `packages/core/src/sync/{snapshot.ts,apply.ts,run.ts,changes.ts,listens.ts}`. Tests: `packages/core/test/sync/*.test.ts`.
**Interfaces:**
- **Consumes:** `SourcePlugin`, `SourceRecord`, `PluginContext` (T3); `matchWork`, `updateWorkFromHint` (T5); `upsertEntry`, `addEngagement`, `moveEntryToWork`, `softDeleteEntry`, `softDeleteEngagement` (T6).
- **Produces:**
- `runSync(deps: { db: Db; makeContext: (conn: Connection, runId: string) => PluginContext }, plugin: SourcePlugin, connectionId: string, mode: "full" | "changes" | { file: { name: string; bytes: Uint8Array } }): Promise<RunSummary>`.
- `RunSummary = { runId; status: "ok" | "failed"; counts: { seen; added; changed; removed; merged; conflicts; autoApplied; pending; listens }; error?: string }`.
- `listChanges(db, userId, { status?: "pending" | "all"; connectionId?; limit?; cursor? })`.
- `resolveChange(db, userId, changeId, action: "accept" | "ignore" | "keep_mine"): Promise<void>`.
- `ingestListens(db, userId, source: string, records: SourceRecord[]): Promise<{ inserted: number }>`.
**Rules (spec §8.3; implement exactly):**
1. **Hash** = sha256 of a stable-key-ordered JSON of `{kind, work, entry|engagement|listen|list, deleted, mergedIntoKey}`. The raw payload is excluded from the hash, but stored in a version gzipped with `Bun.gzipSync` and base64-encoded. A new `source_record_versions` row is written only when the hash differs from `current_hash`.
2. **New record (`added`):** match the work. For `entry`, `upsertEntry` with only the fields present, under the plugin actor. For `engagement`, ensure an entry exists, then de-duplicate: if any non-deleted engagement on that entry has `finished_at` within 24 h of the incoming one (day-precision values compare as dates) or an equal `started_at` datetime, attach an `engagement_sources` row instead of inserting. Write a `change_events` row with status `auto_applied`.
3. **Changed record:** for each field that differs from the previous normalized version:
- `field_sources[f]` is unset or equals this connection's actor → apply, `auto_applied`;
- it is `"user"` → `pending` `changed` event with `{field, from, to}`, and the current value is kept;
- it is another connection's actor and the value differs → `pending` `conflict` event with `{field, current, currentSource, incoming, incomingSource}`.
- A pending event for the same (record, field, incoming value) is never duplicated. That is the anti-flip-flop rule.
4. **Removed:** only when the mode is `"full"`, the plugin iteration completed without throwing, and the record was not seen in this run, or when `deleted: true` arrives in any mode. Set `source_records.deleted_at` and write a `pending` `removed` event. Accepting it soft-deletes the target entry or engagement.
5. **Merged:** when `mergedIntoKey` is set, write a `pending` `merged` event. Accepting it calls `moveEntryToWork` to the work matched for the target record's hint.
6. **Listens:** go to `ingestListens`. Insert-or-ignore on (user, source, source_key), and match a work only via `mbids.recording` (→ an existing work, or a shared track work created with a `musicbrainz_recording` id). There is no change event per listen; counts go in the run summary.
7. **`resolveChange`:** `accept` applies the incoming value with actor `plugin:…` and sets `field_sources` to that actor. `keep_mine` sets `field_sources[f] = "user"` for conflicts. `ignore` does nothing to data. All three set `status` and `resolved_at`.
8. **Run bookkeeping:** set `source_runs.status`, `counts`, `log` and `error`; `source_connections.last_run_at`; and set `status = "error"` on failure.
- [ ] **Step 1: Failing tests** using a fake in-test plugin whose `pull` yields records from an array:
- `"first full pull creates entries and auto_applied events"`;
- `"second identical pull creates nothing new"` (Review focus 2: counts versions, events, engagements and listens before/after);
- `"failed pull marks nothing removed"` (Review focus 1: the plugin yields 1 of 2 records, then throws);
- `"record missing from completed full pull → pending removed; accept soft-deletes entry"`;
- `"user-edited rating not overwritten; pending changed event"` (Review focus 3);
- `"two connections disagree → one pending conflict, not duplicated on rerun; keep_mine stops further conflicts for that value"`;
- `"same play from two sources dedupes into one engagement with two engagement_sources"`;
- `"merged record → pending merged; accept moves entry"`;
- `"listens insert-or-ignore and mbid matching"`.
- All of these run on both dialects.
- [ ] **Steps 2–4:** FAIL, then implement, then PASS on both dialects.
- [ ] **Step 5:** Commit `feat(core): sync engine and changes inbox`.
### Task 9: Jobs, scheduler, plugin registry, connections
**Files:** Create `packages/core/src/{jobs.ts,registry.ts,connections.ts}`. Tests: `packages/core/test/{jobs,registry,connections}.test.ts`, plus fixture plugin `packages/core/test/fixtures/plugins/echo/index.ts`.
**Interfaces (produces):**
- `enqueue(db, type: string, payload: unknown, opts?: { runAt?: string; lockKey?: string }): Promise<string | null>` returns null when a `queued|running` job with the same `lockKey` exists. Sync jobs use `lockKey = "sync:<connectionId>"` (Review focus 5).
- `startWorker(db, handlers: Record<string, (payload: any) => Promise<void>>, opts?: { pollMs?: number; concurrency?: number }): { stop(): Promise<void> }`.
- A job is claimed atomically (`update … where id = ? and status = 'queued'` and check affected rows).
- Retry with backoff `2^attempts` minutes up to 5 attempts, then `failed`.
- `startScheduler(db, opts?: { tickMs?: number }): { stop() }` enqueues `sync` for each active connection whose `last_run_at + schedule_minutes` has passed.
- `loadPlugins(dirs: string[]): Promise<PluginRegistry>`, where `PluginRegistry = { get(id): SourcePlugin | undefined; list(): PluginManifest[]; reload(): Promise<void> }`.
- Each subdirectory with `index.ts` whose default export passes `definePlugin` shape checks is registered (`sdkVersion === 1`, unique id).
- `reload` re-imports with `?v=<Date.now()>`.
- A bad plugin is logged and skipped, never fatal.
- Connections:
- `createConnection(db, secret, userId, { pluginId, name, config, credentials?, scheduleMinutes? })`, `updateConnection`, `deleteConnection` (soft: `status "paused"` + credentials cleared), `listConnections(db, userId)`;
- `makePluginContext(db, secret, conn, runId, http: HttpClient): PluginContext`. Its credentials `save` encrypts and persists *immediately* (Review focus 5), and its cursor is persisted on `set`.
- `syncHandler(deps)` is the `"sync"` job handler: it loads the connection and plugin, calls `runSync` with `mode` from the payload (default `pullChanges` if the plugin supports it, otherwise `full`), and runs a weekly `full` regardless.
- [ ] **Step 1: Failing tests:**
- `"enqueue with same lockKey returns null"`;
- `"worker retries then fails"`;
- `"scheduler enqueues due connections only"`;
- `"registry loads echo plugin and skips broken one"`;
- `"credentials.save persists encrypted before pull continues"` (the echo plugin saves creds, then throws; the DB holds the new encrypted creds).
- [ ] **Steps 2–4:** FAIL, then implement, then PASS on both dialects.
- [ ] **Step 5:** Commit `feat(core): jobs, scheduler, plugin registry, connections`.
### Task 10: Trakt plugin
**Files:** `plugins/trakt/{package.json,index.ts,src/*.ts,test/trakt.test.ts,test/fixtures/*.json}`.
**Interfaces:** `definePlugin` with:
- manifest `id "trakt"`, capabilities `["pull","pullChanges","lookup"]`, mediaTypes `movie show season episode`;
- config: `clientId` (string, required), `clientSecret` (secret, required);
- `attribution` = the exact Trakt line from Global constraints;
- `deviceAuth` via `POST /oauth/device/code` and `/oauth/device/token`.
Requests use base `https://api.trakt.tv` with headers `trakt-api-version: 2` and `trakt-api-key: clientId`, at `ratePerMinute: 100`. Before each run, refresh the access token if it expires within 24 h (`POST /oauth/token`, `grant_type=refresh_token`), then **`ctx.credentials.save` before any other request**.
**pull** yields:
- `GET /sync/history?page=&limit=250` (paged by `X-Pagination-Page-Count`): each history item → an `engagement` record with key `history:<id>`, and `finishedAt` = `watched_at` at day precision but keeping the full datetime in `value`. Movies: hint `movie` with ids `{trakt, tmdb_movie, imdb}`. Episodes: hint `episode` with `parent` season → `parent` show (`tmdb_show`, `tvdb`, `imdb`, `trakt`) and `position {season, episode}`.
- `GET /sync/ratings`: `entry` records with key `rating:<type>:<traktId>`, `rating = rating × 10`.
- `GET /sync/watchlist`: `entry` with status `planned`, key `watchlist:<type>:<traktId>`.
- `GET /users/me/lists` + `/users/me/lists/:id/items`: `list` records.
**pullChanges:** `GET /sync/last_activities`, compared with the cursor (JSON of the last activity timestamps). For the categories that changed, re-fetch:
- history with `start_at = previous cursor time`;
- ratings and watchlist in full.
- Set the new cursor at the end.
**lookup:** `GET /search/movie,show?query=`.
- [ ] **Step 1: Failing tests with fixtures:** history paging over 2 pages; episode hint chain and position; rating 8 → 80; watchlist → planned; a token near expiry is refreshed and saved before the first `/sync` call (assert the order of `fixture.calls` and `ctx.saved.credentials`); `pullChanges` with an unchanged `last_activities` yields nothing.
- [ ] **Steps 2–4:** FAIL, then implement, then PASS (`bun test plugins/trakt`).
- [ ] **Step 5:** Commit `feat(plugins): trakt`.
### Task 11: Hardcover plugin
**Files:** `plugins/hardcover/...` (same layout as Task 10).
**Interfaces:** manifest `id "hardcover"`, capabilities `["pull","pullChanges","lookup"]`, mediaTypes `book`, config `token` (secret, required: personal access token, sent as `Authorization: Bearer`), and attribution "Book data from Hardcover." Requests are `POST https://api.hardcover.app/v1/graphql` at `ratePerMinute: 50`. GraphQL errors throw `HttpError`.
**pull** pages `me { user_books(limit: 100, offset) { id status_id rating review_raw private_notes starred updated_at merged_at original_book_id book { id title release_year contributions { author { name } } book_series { position series { id name } } } edition { id isbn_13 isbn_10 asin } user_book_reads { id started_at finished_at progress_pages } } }` and yields:
- `entry` with key `ub:<id>`:
- status map: 1→planned, 2→in_progress, 3→completed, 4→paused, 5→dropped, 6→none;
- `rating` ×20 (Hardcover is 0–5 with halves);
- `review_raw` → review, `private_notes` → notes, starred → favorite;
- hint ids `{hardcover_book, isbn13, isbn10, asin, hardcover_edition}`; if there is a series, `parent` = series hint (`hardcover_series`) with `position {index}`.
- `engagement` for each read with key `read:<id>`, day-precision dates.
- When `merged_at` is set and `original_book_id ≠ book.id`, also set `mergedIntoKey` on a synthetic record with key `ub:<id>@book:<original_book_id>`, so the engine sees the old identity merged into the new one.
- Lists via `me { lists { id name description list_books { book {…} } } }`.
**pullChanges:** the same query filtered by `updated_at > cursor`. Deletions are not reported, so the weekly full pull detects them.
- [ ] **Step 1: Failing tests:** status and rating mapping (4.5 → 90); series parent hint; re-reads → 2 engagements; merge produces `mergedIntoKey`; GraphQL `errors` → throws; offset paging stops on a short page.
- [ ] **Steps 2–4:** FAIL, then implement, then PASS.
- [ ] **Step 5:** Commit `feat(plugins): hardcover`.
### Task 12: MangaBaka plugin
**Files:** `plugins/mangabaka/...`.
**Interfaces:** manifest `id "mangabaka"`, capabilities `["pull","pullChanges","lookup"]`, mediaTypes `manga comic`, config `token` (secret, required, header `x-api-key`), and attribution "Manga data from MangaBaka (CC BY-NC-SA 4.0)." Base `https://api.mangabaka.org`, `ratePerMinute: 150`.
**pull** pages `GET /v1/my/library?page=` and yields `entry` records keyed `series:<series_id>`:
- state map: considering|plan_to_read→planned, reading|rereading→in_progress, paused→paused, dropped→dropped, completed→completed;
- rating is already 0–100;
- `note` → notes;
- progress → an `engagement` record keyed `progress:<series_id>` with `progress {chapter, volume}` and start/finish dates;
- hint ids `{mangabaka, anilist, mal, mangaupdates, kitsu}` from `source.*`.
- If the series `state === "merged"`, set `mergedIntoKey = "series:<merged_with>"`.
**pullChanges:** `GET /v2/my/library/changes?cursor=`. Upserts yield records, deletes yield `deleted: true`, and the new cursor is saved.
- [ ] **Step 1: Failing tests:** state mapping table; a merged series sets `mergedIntoKey`; the changes feed yields a deletion; cursor saved.
- [ ] **Steps 2–4:** FAIL, then implement, then PASS.
- [ ] **Step 5:** Commit `feat(plugins): mangabaka`.
### Task 13: CSV/JSON import plugin and CSV codec
**Files:** `packages/plugin-sdk/src/csv.ts` (the codec lives in the SDK so core can reuse it), `plugins/csv/{index.ts,src/mapping.ts,test/*.test.ts,test/fixtures/*}`.
**Interfaces (produces):**
- `parseCsv(text: string, opts?: { delimiter?: "," | ";" | "\t" }): string[][]` (RFC 4180). It strips a BOM and auto-detects the delimiter from the header line.
- `toCsv(rows: string[][]): string` quotes as needed and uses `\r\n`. This is reused by Task 15.
- The manifest is `id "csv"`, capabilities `["importFile"]`, and config `mapping` (string: JSON `{ column: field }`, where the fields are `title mediaType status rating ratingScale(5|10|100) review finishedAt startedAt tags isbn13 imdb tmdb_movie igdb notes`).
- `importFile`:
- **`.csv`:** maps rows into `entry` records (plus an `engagement` when dates are present), keyed `row:<sha1 of title+mediaType+finishedAt>`.
- **`.json`:** accepts an array of objects with the same field names.
- **Without a mapping**, it guesses from header names, case-insensitively, including the Goodreads export columns (`Title`, `Author`, `My Rating`, `Date Read`, `ISBN13`, `Exclusive Shelf`, `My Review`).
- [ ] **Step 1: Failing tests (Review focus 4):** a fixture `messy.csv` with BOM, `;`, quoted newline and `""` quotes, an empty rating, `Date Read` = `2019`, and an emoji title; `parseCsv(toCsv(x))` deep-equals `x`; Goodreads fixture headers are auto-mapped; `Exclusive Shelf` `to-read` → planned and `read` → completed; `My Rating` 4 at scale 5 → 80.
- [ ] **Steps 2–4:** FAIL, then implement, then PASS.
- [ ] **Step 5:** Commit `feat(plugins): csv and json import`.
### Task 14: ListenBrainz and Last.fm plugins
**Files:** `plugins/listenbrainz/...` and `plugins/lastfm/...`.
**ListenBrainz:** manifest `id "listenbrainz"`, capabilities `["pull","pullChanges"]`, mediaTypes `track album artist`, config `username` (string) + `token` (secret, `Authorization: Token`). Base `https://api.listenbrainz.org`. Respect `X-RateLimit-Reset-In` (already handled in T3).
- **pull (full backfill):**
1. `POST /1/export/` (if the response is 400 "already pending", fall through to listing).
2. Poll `GET /1/export/list` every 30 s via `ctx.now`/an injected sleep until the newest export's status is `complete`.
3. Download `GET /1/export/{id}/download`.
4. Unzip with `fflate.unzipSync`.
5. Stream `listens/**/*.jsonl` → `listen` records keyed `<listened_at>:<recording_msid>`, with `mbids` from `track_metadata.mbid_mapping`.
6. `feedback.jsonl` with score 1 → an `entry` with `favorite: true`, track hint ids `{musicbrainz_recording}`.
- **pullChanges:** `GET /1/user/{u}/listens?min_ts=<cursor>&count=1000` loops until empty, and the cursor becomes the max `listened_at`.
**Last.fm:** manifest `id "lastfm"`, capabilities `["pull","pullChanges"]`, config `username` + `apiKey` (secret). Base `https://ws.audioscrobbler.com/2.0/`, `format=json`, `ratePerMinute: 240`.
- **pull:** pages `user.getRecentTracks&limit=200&page=`, skipping the `@attr.nowplaying` track, yielding `listen` records keyed `<uts>:<artist>:<track>` with MBIDs when non-empty. Also `user.getLovedTracks` → favorite `entry` records.
- **pullChanges:** `from=<cursor>`.
- Error code 29 → throw `HttpError` with status 429 so the client retries.
- [ ] **Step 1: Failing tests:**
- LB: export flow over fixtures (a pending then complete list, plus a zip built in-test with `fflate.zipSync`); a feedback love → favorite entry; the incremental cursor advances.
- Last.fm: the nowplaying track is skipped; empty MBIDs are omitted; error 29 is retried (via the fixture fetch returning 29 once).
- [ ] **Steps 2–4:** FAIL, then implement, then PASS.
- [ ] **Step 5:** Commit `feat(plugins): listenbrainz and lastfm`.
### Task 15: Export bundle, restore, scheduled backups
**Files:** Create `packages/core/src/export.ts` and `packages/core/src/backup.ts`. Tests: `packages/core/test/{export,backup}.test.ts`.
**Interfaces (produces):**
- `exportUserBundle(db, dataDir, userId): Promise<Uint8Array>` returns a zip (fflate) containing:
- `manifest.json` `{ format: "sttracker-export/v1", exportedAt, user: { username, displayName } }`;
- `works.jsonl` (every referenced work, plus ancestors, with `externalIds`);
- `entries.jsonl`, `engagements.jsonl`, `listens.jsonl`, `lists.jsonl` (with items), `copies.jsonl`, `custom_fields.jsonl`, `history.jsonl`, `attachments.jsonl`;
- `attachments/<sha256>` file bytes.
- It excludes credentials, sessions and tokens.
- In all files, camelCase keys mirror the `Entry`/`Work` view types.
- `exportUserCsv(db, userId): Promise<Record<string, string>>` returns `{ "entries.csv": …, "engagements.csv": …, "listens.csv": … }` using `toCsv` from `@sttracker/plugin-sdk/csv` (Task 13).
- `importUserBundle(db, dataDir, userId, zip: Uint8Array): Promise<{ works: number; entries: number }>` refuses unless the user has no entries (`ImportError("account_not_empty")`). It re-matches works by external IDs; unmatched works become private works; old IDs are remapped to new ones.
- `runBackup(db, dataDir, opts: { keep: number }): Promise<string>` writes `${dataDir}/backups/<ISO>/` containing one bundle per user, plus on SQLite `VACUUM INTO 'sttracker.db'`. It prunes to the newest `keep` (default 14).
- `"backup"` job handler: `backupHandler(deps)`.
- [ ] **Step 1: Failing tests:**
- `"export then import into fresh user reproduces entries, ratings, engagements with precision, lists and attachments"`;
- `"bundle has no credential fields"` (grep the zip text for `password`, `token`, `credentials`);
- `"import into non-empty account refused"`;
- `"backup prunes to keep"`.
- All on both dialects (VACUUM is SQLite only).
- [ ] **Steps 2–4:** FAIL, then implement, then PASS.
- [ ] **Step 5:** Commit `feat(core): export bundle, restore, backups`.
### Task 16: `apps/server`: API, auth middleware, ingest mount, OpenAPI, boot
**Files:** Create `apps/server/src/{main.ts,config.ts,http.ts,routes/*.ts,openapi.ts}` and `apps/server/test/api.test.ts`.
**Interfaces:**
- **Config** (`config.ts`) reads env:
- `DATABASE_URL` (default `sqlite://${DATA_DIR}/sttracker.db`);
- `DATA_DIR` (default `./data`);
- `STTRACKER_SECRET` (required; refuse to boot without it, with the message "Set STTRACKER_SECRET to a long random string. It encrypts source credentials.");
- `PUBLIC_URL`, `PORT` (default 3000);
- `PLUGIN_DIRS` (default `plugins,${DATA_DIR}/plugins`).
- `createApp(deps: { db; registry; secret; dataDir }): { fetch(req: Request): Promise<Response>; routes }` is testable without listening. `main.ts` migrates, loads plugins, starts the worker and scheduler, schedules a daily `backup` job, and calls `Bun.serve({ routes: { "/*": index.html }, fetch: app.fetch })`.
- `http.ts`: `route({ method, path, auth: "none" | "session" | "token:read" | "token:write" | "any:read" | "any:write", body?: ZodType, query?: ZodType, handler })`.
- JSON errors have the shape `{ error: { code, message } }`, with codes `400 validation`, `401 unauthenticated`, `403 forbidden`, `404 not_found`, `409 conflict`.
- Session cookie: `stt_session`, HttpOnly, SameSite=Lax, Secure when `PUBLIC_URL` is https.
- `any:*` accepts a session or a Bearer app token.
- **Routes under `/api/v1`** (bodies validated with zod; each calls the core function named in Tasks 4–15):
- auth: `POST auth/register`, `POST auth/login`, `POST auth/logout`, `GET me`, `GET/POST/DELETE me/tokens`, `POST admin/invites`;
- entries: `GET entries` (the `listEntries` filter as query), `POST entries/quick`, `GET/PATCH/DELETE entries/:id`, `GET entries/:id/history`, `POST history/:id/restore`, `POST entries/:id/move`;
- engagements: `GET/POST entries/:id/engagements`, `PATCH/DELETE engagements/:id`;
- works: `GET works/search`, `GET works/:id`, `GET works/:id/suggestions`, `POST suggestions/:id/accept|reject`;
- lists: `GET/POST lists`, `GET/PATCH/DELETE lists/:id`, `POST lists/:id/items`, `PATCH/DELETE list-items/:id`;
- copies: `GET/POST copies`, `PATCH/DELETE copies/:id`;
- custom fields: `GET/POST custom-fields`;
- attachments: `POST attachments` (multipart: `file`, `entity`, `entityId`, `caption`), `GET attachments/:id`, `DELETE attachments/:id/links/:linkId`;
- plugins and sources: `GET plugins` (manifests), `GET/POST sources`, `PATCH/DELETE sources/:id`, `POST sources/:id/run` (enqueue, 409 if locked), `POST sources/:id/import` (multipart file → enqueue a file-mode run), `POST sources/:id/device/start`, `POST sources/:id/device/poll`, `GET sources/:id/runs`;
- changes: `GET changes`, `POST changes/:id/resolve` `{action}`;
- export: `GET export/bundle` (zip download), `GET export/csv?file=entries.csv`, `POST import/bundle`;
- `GET openapi.json` (generated from the route table with `z.toJSONSchema`);
- `GET about` (attributions from plugin manifests).
- `/ingest/<pluginId>/*` → 404 in Phase 1 (the mount point exists, and no plugin has `ingest`).
- `POST admin/plugins/reload` (admin) → `registry.reload()`.
- [ ] **Step 1: Failing tests** (`forEachDialect`, calling `app.fetch` directly):
- register → login cookie → `GET me`;
- unauthenticated `GET entries` → 401 JSON;
- quick add, then list shows a draft;
- PATCH rating 150 → 400 `validation`;
- an app token with `read` cannot PATCH (403);
- another user's entry → 404;
- `POST sources` with the echo plugin, then `POST sources/:id/run` twice → the second returns 409;
- export bundle content-type is `application/zip`;
- `openapi.json` lists `/api/v1/entries`;
- booting without `STTRACKER_SECRET` throws the exact message.
- [ ] **Steps 2–4:** FAIL, then implement, then PASS.
- [ ] Update `AGENTS.md` Commands with Dev (`bun --hot apps/server/src/main.ts`), verified by booting and hitting `/api/v1/openapi.json`. Update `docs/PERMISSIONS.md` and the tool files accordingly.
- [ ] **Step 5:** Commit `feat(server): REST API, auth, sources, changes, export`.
### Task 17: Web app: shell, auth, library, quick add, entry detail
**Files:** Create `apps/web/{index.html,src/main.tsx,src/api.ts,src/theme.css,src/routes/*.tsx,src/components/*.tsx}` and `apps/web/test/*.test.tsx` (logic only: `api.ts` and formatters).
Follow `DESIGN.md` tokens exactly and use impeccable `craft`. Routes (wouter):
- `/login`, `/register` (the first-run screen when no users exist: "Create the admin account");
- `/` (library);
- `/entries/:id`.
Library:
- The three-pane layout from the round 2 mockup: sidebar nav (Library, Log, Lists, Owned, Changes with a pending-count badge in `--warn`, a Sources list, Settings), the list pane, and the detail pane.
- Search, filter chips by media type and status, and a "Draft" filter.
- The Quick add box, with the placeholder text "Quick add: type a title, sort it out later". Enter creates the entry and selects it.
- Rows show cover (or the striped draft cover), title, `media type · meta`, and stars or the amber `Draft` badge. Children roll up as "season 3 of 5"-style meta from `childSummary`.
Detail:
- Cover, title, editable rating (stars in the user's scale; the setting defaults to 5 with halves), status select, tags, and custom fields.
- The orange change alert when pending changes target this entry, with "See what changed" → `/changes?entry=:id`.
- Boxes for Reads/Plays (engagements with partial dates rendered by precision: "Mar 2025 (no exact date)"), Your copy, Attachments (upload, thumbnails via CSS), Linked sources, and Review (Gelasio, Markdown textarea).
- Children with their own ratings, a History drawer with restore, and "Link to catalogue" for drafts (suggestions plus search).
Formatters: `formatRating(value0to100, scale)` and `formatPartialDate(pd)`.
- [ ] **Step 1: Failing tests:** `formatRating(90, "5half")` = `"★★★★½"`, `formatRating(null, …)` = `""`, `formatPartialDate({value:"2025-03",precision:"month"})` = `"Mar 2025"`, `{value:"2019",precision:"year"}` = `"2019"`; `api.ts` throws an `ApiError` carrying `code` on a non-2xx JSON error.
- [ ] **Steps 2–4:** FAIL, then implement, then PASS.
- [ ] **Step 5:** Manual check: run the server, register, quick add, and open the detail pane. Take a screenshot and compare it with the mockup. Fix glaring deviations.
- [ ] **Step 6:** Commit `feat(web): app shell, library, entry detail`.
### Task 18: Web app: Changes, Sources, Log, Lists, Owned, Settings, Export, About
**Files:** `apps/web/src/routes/{changes,sources,log,lists,owned,settings,about}.tsx` plus components. This task depends on Task 17's `api.ts`, `theme.css` and layout components; create new components only.
- **Changes:**
- Grouped by run.
- Each card shows `type`, the entry title, a field-level before/after (or both sources' values for conflicts), and buttons "Use the new value" (accept), "Keep mine" (keep_mine) and "Ignore".
- Removed events say "<Source> no longer has this. Your copy here is safe." with buttons "Delete here too" (accept) and "Keep it" (ignore).
- Merged events say "<Source> merged this into <target title>." with "Follow the merge".
- **Sources:**
- Plugin cards from `GET plugins`, and a connect form rendered from the `config` fields (secrets as password inputs).
- The device-code flow modal shows the `userCode` and the verification link, and polls.
- Per connection: status, last run, "Sync now", schedule select (off/hourly/6h/daily), runs log, and file upload for `importFile` plugins.
- **Log:** a chronological feed of engagements and listens (paged).
- **Lists:** create lists of kind list, grouping or thread; add works via search; drag-to-reorder (up/down buttons are acceptable); a grouping can be rated.
- **Owned:** a copies table with add/edit (bought where, when, price, format, notes).
- **Settings:** profile, rating scale, default visibility, custom fields editor, app tokens (create shows the token once with "Copy"; revoke), invites (admin), "Reload plugins" (admin), and Export (download bundle and CSVs, import a bundle into an empty account).
- **About:** the attributions.
- [ ] **Step 1: Failing tests** for pure helpers: `describeChange(event)` returns the exact strings above for the removed/merged/conflict fixtures; `scheduleLabel(minutes)`.
- [ ] **Steps 2–4:** FAIL, then implement, then PASS.
- [ ] **Step 5:** Manual check of the Changes flow against a fake connection (echo plugin) in the running app.
- [ ] **Step 6:** Commit `feat(web): changes inbox, sources, log, lists, owned, settings`.
### Task 19: Docker, compose, CI, docs
**Files:** Create `Dockerfile`, `docker-compose.yml`, `.dockerignore`, `.env.example` and `.gitea/workflows/ci.yml`. Modify `AGENTS.md` (Commands), `docs/ARCHITECTURE.md` (replace "planned" with reality), `docs/PERMISSIONS.md` and the tool files, `docs/TODO.md` and `README.md` (Quick start).
- **Dockerfile:** `oven/bun:1.3` base; `bun install --frozen-lockfile --production`; copy the source; `VOLUME /data`; `ENV DATA_DIR=/data`; `EXPOSE 3000`; `CMD ["bun","apps/server/src/main.ts"]`; run as a non-root `bun` user.
- **Compose:** the `sttracker` service with `./data:/data`, env from `.env`; plus a `postgres:17-alpine` service under `profiles: ["postgres"]`, with a comment showing the `DATABASE_URL` for it.
- **CI:** on push/PR, a postgres:17-alpine service, then `bun install --frozen-lockfile`, `bun run typecheck`, `bun test` with `TEST_POSTGRES_URL` set, and `test $(wc -l < AGENTS.md) -le 150`.
- [ ] **Step 1:** `docker build -t sttracker:dev .` → succeeds.
- [ ] **Step 2:** `docker run --rm -e STTRACKER_SECRET=test -p 3999:3000 sttracker:dev` → `curl -s localhost:3999/api/v1/openapi.json` returns JSON.
- [ ] **Step 3:** Repeat with the compose postgres profile and `DATABASE_URL` → boots and migrates.
- [ ] **Step 4:** Update the docs. Every command in `AGENTS.md` must be verified in this session (typecheck now passes; `bun test` passes on both DBs).
- [ ] **Step 5:** Commit `chore: docker, compose, CI, docs`.
---
## Completion
- [ ] `bun run typecheck` clean. `bun test` passes with and without `TEST_POSTGRES_URL`.
- [ ] Whole-branch review (superpowers:requesting-code-review). Fix the findings.
- [ ] `git rm docs/plans/001-phase1-own-your-data.md`. Set the spec front matter to `status: implemented`. Merge `feat/phase1` → `main` via superpowers:finishing-a-development-branch.
+15
View File
@@ -0,0 +1,15 @@
{
"name": "@sttracker/core",
"private": true,
"type": "module",
"exports": {
"./*": "./src/*.ts"
},
"dependencies": {
"@sttracker/db": "workspace:*",
"@sttracker/plugin-sdk": "workspace:*",
"fflate": "^0.8.3",
"kysely": "^0.29.7",
"zod": "^4.6.7"
}
}
+263
View File
@@ -0,0 +1,263 @@
import { mkdir, rename, rm } from "node:fs/promises";
import { dirname, join } from "node:path";
import type { Db } from "@sttracker/db/db";
import { newId, nowIso } from "@sttracker/db/ids";
import type { AttachmentEntity, AttachmentsTable } from "@sttracker/db/schema";
import { NotFoundError, ValidationError } from "./errors.ts";
export type Attachment = AttachmentsTable;
export const MAX_ATTACHMENT_BYTES = 25 * 1024 * 1024;
/** Lowercase and strip parameters (`; charset=...`). */
export function normalizeMime(mime: string): string {
return (mime.split(";")[0] ?? "").trim().toLowerCase();
}
export function mimeAllowed(mime: string): boolean {
if (mime === "image/svg+xml") return false; // scriptable; never serve user SVG
return mime.startsWith("image/") || mime === "application/pdf" || mime === "text/plain";
}
const locks = new Map<string, Promise<void>>();
/** Serialise file mutations per sha within this process. */
async function withLock<T>(key: string, fn: () => Promise<T>): Promise<T> {
const prev = locks.get(key) ?? Promise.resolve();
const run = prev.then(fn, fn);
const tail = run.then(
() => undefined,
() => undefined,
);
locks.set(key, tail);
try {
return await run;
} finally {
if (locks.get(key) === tail) locks.delete(key);
}
}
function pathFor(dataDir: string, sha: string): string {
return join(dataDir, "attachments", sha.slice(0, 2), sha.slice(2, 4), sha);
}
export function sha256Hex(bytes: Uint8Array): string {
return new Bun.CryptoHasher("sha256").update(bytes).digest("hex");
}
/**
* Write bytes to the content-addressed store (idempotent, deduplicated) and return their sha256.
* Callers that insert rows afterwards should call it again after committing, since an unlink of
* identical bytes in between may have removed the file.
*/
export async function writeAttachmentFile(dataDir: string, bytes: Uint8Array): Promise<string> {
const sha256 = sha256Hex(bytes);
const path = pathFor(dataDir, sha256);
await withLock(sha256, async () => {
if (await Bun.file(path).exists()) return;
await mkdir(dirname(path), { recursive: true });
const tmp = `${path}.tmp-${newId()}`;
try {
await Bun.write(tmp, bytes);
await rename(tmp, path);
} catch (e) {
await rm(tmp, { force: true });
throw e;
}
});
return sha256;
}
/**
* Store a file content-addressed by sha256 (deduplicated on disk) and link it to an entity.
* Each call creates a new attachment row and link; identical bytes share one file.
*/
export async function storeAttachment(
db: Db,
dataDir: string,
userId: string,
file: { bytes: Uint8Array; mime: string; filename: string; caption?: string; takenAt?: string },
link: { entity: AttachmentEntity; entityId: string },
): Promise<Attachment> {
const mime = normalizeMime(file.mime);
if (!mimeAllowed(mime)) throw new ValidationError("mime", "unsupported file type");
if (file.bytes.byteLength > MAX_ATTACHMENT_BYTES) throw new ValidationError("file", "larger than 25 MB");
const sha256 = new Bun.CryptoHasher("sha256").update(file.bytes).digest("hex");
const path = pathFor(dataDir, sha256);
const row: Attachment = {
id: newId(),
user_id: userId,
sha256,
mime,
size: file.bytes.byteLength,
filename: file.filename,
caption: file.caption ?? null,
taken_at: file.takenAt ?? null,
visibility: null,
created_at: nowIso(),
};
await withLock(sha256, async () => {
if (!(await Bun.file(path).exists())) {
await mkdir(dirname(path), { recursive: true });
const tmp = `${path}.tmp-${newId()}`;
try {
await Bun.write(tmp, file.bytes);
await rename(tmp, path);
} catch (e) {
await rm(tmp, { force: true });
throw e;
}
}
await db.k.transaction().execute(async (tx) => {
await tx.insertInto("attachments").values(row).execute();
await tx
.insertInto("attachment_links")
.values({ id: newId(), attachment_id: row.id, entity: link.entity, entity_id: link.entityId })
.execute();
});
});
return row;
}
/** Open an attachment's file. Returns null when it does not exist or belongs to another user. */
export async function readAttachment(
db: Db,
dataDir: string,
userId: string,
id: string,
): Promise<{ file: ReturnType<typeof Bun.file>; mime: string; filename: string } | null> {
const row = await db.k
.selectFrom("attachments")
.select(["sha256", "mime", "filename"])
.where("id", "=", id)
.where("user_id", "=", userId)
.executeTakeFirst();
if (!row) return null;
return { file: Bun.file(pathFor(dataDir, row.sha256)), mime: row.mime, filename: row.filename };
}
/** Attachments of `userId` linked to the given entity, oldest first. */
export async function listAttachments(
db: Db,
userId: string,
entity: AttachmentEntity,
entityId: string,
): Promise<Attachment[]> {
return db.k
.selectFrom("attachments")
.innerJoin("attachment_links", "attachment_links.attachment_id", "attachments.id")
.selectAll("attachments")
.where("attachments.user_id", "=", userId)
.where("attachment_links.entity", "=", entity)
.where("attachment_links.entity_id", "=", entityId)
.orderBy("attachments.created_at")
.orderBy("attachments.id")
.execute();
}
/**
* Remove an attachment's link to an entity. When no links remain the attachment row is
* deleted, and the file too if no other row references its content. No-op for other users.
*/
export async function unlinkAttachment(
db: Db,
dataDir: string,
userId: string,
attachmentId: string,
entity: AttachmentEntity,
entityId: string,
): Promise<void> {
const att = await db.k
.selectFrom("attachments")
.select("sha256")
.where("id", "=", attachmentId)
.where("user_id", "=", userId)
.executeTakeFirst();
if (!att) return;
await withLock(att.sha256, async () => {
const orphan = await db.k.transaction().execute(async (tx) => {
await tx
.deleteFrom("attachment_links")
.where("attachment_id", "=", attachmentId)
.where("entity", "=", entity)
.where("entity_id", "=", entityId)
.execute();
const left = await tx
.selectFrom("attachment_links")
.select("id")
.where("attachment_id", "=", attachmentId)
.executeTakeFirst();
if (left) return false;
await tx.deleteFrom("attachments").where("id", "=", attachmentId).execute();
const shared = await tx
.selectFrom("attachments")
.select("id")
.where("sha256", "=", att.sha256)
.executeTakeFirst();
return !shared;
});
if (orphan) await rm(pathFor(dataDir, att.sha256), { force: true });
});
}
const LINK_ENTITIES: readonly AttachmentEntity[] = ["entry", "engagement", "copy", "list", "work"];
/**
* Throw `NotFoundError` unless `entityId` is a live `entity` the user may attach to: their own entry,
* engagement, copy or list, or a work that is shared or theirs. Call before `storeAttachment`.
*/
export async function requireLinkTarget(db: Db, userId: string, entity: AttachmentEntity, entityId: string): Promise<void> {
if (!LINK_ENTITIES.includes(entity)) throw new ValidationError("entity", "invalid");
const k = db.k;
let found: unknown;
switch (entity) {
case "entry":
found = await k.selectFrom("entries").select("id").where("id", "=", entityId).where("user_id", "=", userId).where("deleted_at", "is", null).executeTakeFirst();
break;
case "engagement":
found = await k
.selectFrom("engagements as g")
.innerJoin("entries as e", "e.id", "g.entry_id")
.select("g.id")
.where("g.id", "=", entityId)
.where("e.user_id", "=", userId)
.where("g.deleted_at", "is", null)
.where("e.deleted_at", "is", null)
.executeTakeFirst();
break;
case "copy":
found = await k.selectFrom("copies").select("id").where("id", "=", entityId).where("user_id", "=", userId).where("deleted_at", "is", null).executeTakeFirst();
break;
case "list":
found = await k.selectFrom("lists").select("id").where("id", "=", entityId).where("user_id", "=", userId).where("deleted_at", "is", null).executeTakeFirst();
break;
case "work":
found = await k
.selectFrom("works")
.select("id")
.where("id", "=", entityId)
.where((eb) => eb.or([eb("owner_user_id", "is", null), eb("owner_user_id", "=", userId)]))
.executeTakeFirst();
break;
}
if (!found) throw new NotFoundError(entity);
}
/** The links of one of the user's attachments (empty for another user's or a missing attachment). */
export async function listAttachmentLinks(
db: Db,
userId: string,
attachmentId: string,
): Promise<{ id: string; entity: AttachmentEntity; entityId: string }[]> {
const rows = await db.k
.selectFrom("attachment_links as l")
.innerJoin("attachments as a", "a.id", "l.attachment_id")
.select(["l.id", "l.entity", "l.entity_id"])
.where("a.id", "=", attachmentId)
.where("a.user_id", "=", userId)
.orderBy("l.id")
.execute();
return rows.map((r) => ({ id: r.id, entity: r.entity, entityId: r.entity_id }));
}
+241
View File
@@ -0,0 +1,241 @@
import type { TokenScope } from "@sttracker/db/schema";
import type { Db } from "@sttracker/db/db";
import { newId, nowIso } from "@sttracker/db/ids";
import { fromJson, toJson } from "@sttracker/db/json";
import { sql } from "kysely";
import { AuthError } from "./errors.ts";
export type User = { id: string; username: string; displayName: string; role: "admin" | "member" };
const USERNAME_RE = /^[a-z0-9_]{2,32}$/;
const DAY_MS = 86_400_000;
const SESSION_DAYS = 30;
function sha256Hex(s: string): string {
return new Bun.CryptoHasher("sha256").update(s).digest("hex");
}
let dummy: Promise<string> | undefined;
/** Fixed argon2id hash, computed once, so unknown-user logins cost the same as real ones. */
function dummyHash(): Promise<string> {
return (dummy ??= Bun.password.hash("sttracker-dummy-password", { algorithm: "argon2id" }));
}
function randomToken(bytes = 32): string {
return crypto.getRandomValues(new Uint8Array(bytes)).toBase64({ alphabet: "base64url", omitPadding: true });
}
function toUser(r: { id: string; username: string; display_name: string; role: "admin" | "member" }): User {
return { id: r.id, username: r.username, displayName: r.display_name, role: r.role };
}
export async function registerUser(
db: Db,
input: { username: string; password: string; displayName?: string; inviteCode?: string },
): Promise<User> {
const username = input.username.toLowerCase();
if (!USERNAME_RE.test(username)) throw new AuthError("invalid_username");
const passwordHash = await Bun.password.hash(input.password, { algorithm: "argon2id" });
return db.k.transaction().execute(async (tx) => {
// Serialise registrations on Postgres so two first-time sign-ups cannot both become admin.
// SQLite is already serialised by its single connection.
if (db.dialect === "postgres") await sql`select pg_advisory_xact_lock(727401)`.execute(tx);
const anyUser = await tx.selectFrom("users").select("id").limit(1).executeTakeFirst();
if (anyUser && !input.inviteCode) throw new AuthError("invite_required");
const existing = await tx.selectFrom("users").select("id").where("username", "=", username).executeTakeFirst();
if (existing) throw new AuthError("username_taken");
const id = newId();
const now = nowIso();
const role = anyUser ? "member" : "admin";
const display_name = input.displayName ?? username;
await tx
.insertInto("users")
.values({
id,
username,
display_name,
password_hash: passwordHash,
role,
preferences: toJson({}),
created_at: now,
updated_at: now,
})
.execute();
// used_by references users.id, so claim after the insert; the tx rolls back on failure.
if (anyUser) {
const claimed = await tx
.updateTable("invites")
.set({ used_by: id })
.where("code_hash", "=", sha256Hex(input.inviteCode!))
.where("used_by", "is", null)
.where("expires_at", ">", now)
.executeTakeFirst();
if (Number(claimed.numUpdatedRows) !== 1) throw new AuthError("invite_required");
}
return { id, username, displayName: display_name, role };
});
}
export async function login(
db: Db,
username: string,
password: string,
): Promise<{ user: User; sessionToken: string }> {
const row = await db.k
.selectFrom("users")
.selectAll()
.where("username", "=", username.toLowerCase())
.executeTakeFirst();
const hash = row?.password_hash ?? (await dummyHash());
const ok = await Bun.password.verify(password, hash);
if (!row || !ok) {
throw new AuthError("invalid_credentials");
}
const sessionToken = randomToken();
await db.k
.insertInto("sessions")
.values({
id: newId(),
user_id: row.id,
token_hash: sha256Hex(sessionToken),
expires_at: new Date(Date.now() + SESSION_DAYS * DAY_MS).toISOString(),
created_at: nowIso(),
})
.execute();
return { user: toUser(row), sessionToken };
}
/** True once any account exists (the first one becomes admin). */
export async function hasUsers(db: Db): Promise<boolean> {
return (await db.k.selectFrom("users").select("id").limit(1).executeTakeFirst()) !== undefined;
}
export async function getSessionUser(db: Db, token: string): Promise<User | null> {
const row = await db.k
.selectFrom("sessions")
.innerJoin("users", "users.id", "sessions.user_id")
.select(["users.id", "users.username", "users.display_name", "users.role"])
.where("sessions.token_hash", "=", sha256Hex(token))
.where("sessions.expires_at", ">", nowIso())
.executeTakeFirst();
return row ? toUser(row) : null;
}
export async function logout(db: Db, token: string): Promise<void> {
await db.k.deleteFrom("sessions").where("token_hash", "=", sha256Hex(token)).execute();
}
/** Admin-only. Returns the invite code (shown once; only its hash is stored). */
export async function createInvite(db: Db, adminId: string, ttlDays = 7): Promise<string> {
const admin = await db.k.selectFrom("users").select("role").where("id", "=", adminId).executeTakeFirst();
if (admin?.role !== "admin") throw new AuthError("forbidden");
const code = randomToken(16);
await db.k
.insertInto("invites")
.values({
id: newId(),
code_hash: sha256Hex(code),
created_by: adminId,
used_by: null,
expires_at: new Date(Date.now() + ttlDays * DAY_MS).toISOString(),
created_at: nowIso(),
})
.execute();
return code;
}
export async function createAppToken(
db: Db,
userId: string,
name: string,
scopes: TokenScope[],
): Promise<{ id: string; token: string }> {
const id = newId();
const token = `stt_${randomToken()}`;
await db.k
.insertInto("app_tokens")
.values({
id,
user_id: userId,
name,
token_hash: sha256Hex(token),
scopes: toJson(scopes),
last_used_at: null,
revoked_at: null,
created_at: nowIso(),
})
.execute();
return { id, token };
}
export async function verifyAppToken(db: Db, token: string, scope: TokenScope): Promise<User | null> {
return (await authenticateAppToken(db, token, scope))?.user ?? null;
}
/** Like `verifyAppToken`, also returning the token's name (API writes record actor `token:<name>`). */
export async function authenticateAppToken(db: Db, token: string, scope: TokenScope): Promise<{ user: User; tokenName: string } | null> {
const row = await db.k
.selectFrom("app_tokens")
.innerJoin("users", "users.id", "app_tokens.user_id")
.select([
"app_tokens.id as token_id",
"app_tokens.name as token_name",
"app_tokens.scopes",
"users.id",
"users.username",
"users.display_name",
"users.role",
])
.where("app_tokens.token_hash", "=", sha256Hex(token))
.where("app_tokens.revoked_at", "is", null)
.executeTakeFirst();
if (!row || !fromJson<string[]>(row.scopes, []).includes(scope)) return null;
await db.k.updateTable("app_tokens").set({ last_used_at: nowIso() }).where("id", "=", row.token_id).execute();
return { user: toUser(row), tokenName: row.token_name };
}
/** Revoke one of the user's live tokens. Returns false when no such live token exists (unknown, foreign or revoked). */
export async function revokeAppToken(db: Db, userId: string, id: string): Promise<boolean> {
const r = await db.k
.updateTable("app_tokens")
.set({ revoked_at: nowIso() })
.where("id", "=", id)
.where("user_id", "=", userId)
.where("revoked_at", "is", null)
.executeTakeFirst();
return Number(r.numUpdatedRows) === 1;
}
export interface AppTokenInfo {
id: string;
name: string;
scopes: TokenScope[];
lastUsedAt: string | null;
revokedAt: string | null;
createdAt: string;
}
/** The user's app tokens, newest first. Metadata only: the token and its hash are never returned. */
export async function listAppTokens(db: Db, userId: string): Promise<AppTokenInfo[]> {
const rows = await db.k
.selectFrom("app_tokens")
.select(["id", "name", "scopes", "last_used_at", "revoked_at", "created_at"])
.where("user_id", "=", userId)
.orderBy("created_at", "desc")
.orderBy("id", "desc")
.execute();
return rows.map((r) => ({
id: r.id,
name: r.name,
scopes: fromJson<TokenScope[]>(r.scopes, []),
lastUsedAt: r.last_used_at,
revokedAt: r.revoked_at,
createdAt: r.created_at,
}));
}
+59
View File
@@ -0,0 +1,59 @@
import { chmod, mkdir, readdir, rename, rm } from "node:fs/promises";
import { join } from "node:path";
import { sql } from "kysely";
import type { Db } from "@sttracker/db/db";
import { exportUserBundle } from "./export.ts";
const STAMP_RE = /^\d{4}-\d{2}-\d{2}T\d{2}-\d{2}-\d{2}\.\d{3}Z$/;
/**
* Write `${dataDir}/backups/<ISO>/` with one bundle per user (plus `sttracker.db` on SQLite),
* then prune to the newest `keep` (default 14). Returns the new directory.
*/
export async function runBackup(db: Db, dataDir: string, opts: { keep?: number } = {}): Promise<string> {
const keep = Math.max(1, opts.keep ?? 14);
const root = join(dataDir, "backups");
await mkdir(root, { recursive: true });
await chmod(root, 0o700);
// Two backups in the same millisecond must not share a directory.
const stamp = (ms: number) => new Date(ms).toISOString().replaceAll(":", "-");
const taken = new Set(await readdir(root));
let t = Date.now();
while (taken.has(stamp(t))) t++;
const dir = join(root, stamp(t));
// Build in `<stamp>.tmp` and rename at the end, so a failed run never leaves a counted partial backup.
const tmp = `${dir}.tmp`;
await rm(tmp, { recursive: true, force: true });
await mkdir(tmp);
await chmod(tmp, 0o700);
try {
const users = await db.k.selectFrom("users").select(["id", "username"]).orderBy("id").execute();
for (const u of users) {
const name = u.username.replace(/[^A-Za-z0-9._-]/g, "_");
const file = join(tmp, `${name}.zip`);
await Bun.write(file, await exportUserBundle(db, dataDir, u.id));
await chmod(file, 0o600);
}
if (db.dialect === "sqlite") {
const file = join(tmp, "sttracker.db");
await sql.raw(`VACUUM INTO '${file.replaceAll("'", "''")}'`).execute(db.k);
await chmod(file, 0o600);
}
await rename(tmp, dir);
} catch (e) {
await rm(tmp, { recursive: true, force: true });
throw e;
}
const all = (await readdir(root)).filter((n) => STAMP_RE.test(n)).sort();
for (const old of all.slice(0, Math.max(0, all.length - keep))) await rm(join(root, old), { recursive: true, force: true });
return dir;
}
/** Job handler for the `"backup"` job type. */
export function backupHandler(deps: { db: Db; dataDir: string; keep?: number }): () => Promise<void> {
return async () => {
await runBackup(deps.db, deps.dataDir, { keep: deps.keep });
};
}
+414
View File
@@ -0,0 +1,414 @@
import type { Db } from "@sttracker/db/db";
import { newId, nowIso } from "@sttracker/db/ids";
import { fromJson, toJson } from "@sttracker/db/json";
import type { ConnectionStatus, RunMode, RunStatus } from "@sttracker/db/schema";
import { createHttpClient, type HttpClientOptions } from "@sttracker/plugin-sdk/http";
import type { HttpClient, PluginContext, PluginManifest } from "@sttracker/plugin-sdk/types";
import { decryptJson, encryptJson } from "./crypto.ts";
import { CryptoError, NotFoundError, ValidationError } from "./errors.ts";
import { DropJobError, NonRetryableError, type JobContext } from "./jobs.ts";
import type { PluginRegistry } from "./registry.ts";
import { runSync, type Connection, type RunSummary, type SyncMode } from "./sync/run.ts";
const WEEK_MS = 7 * 24 * 60 * 60 * 1000;
export interface ConnectionView {
id: string;
pluginId: string;
name: string;
config: Record<string, unknown>;
scheduleMinutes: number | null;
status: ConnectionStatus;
lastRunAt: string | null;
createdAt: string;
hasCredentials: boolean;
}
export interface ConnectionInput {
pluginId: string;
name: string;
config: Record<string, unknown>;
credentials?: Record<string, string>;
scheduleMinutes?: number | null;
}
type Creds = Record<string, string>;
/** Split incoming config: fields typed `"secret"` in the manifest go to the credentials blob, the rest stay config. */
function splitConfig(manifest: PluginManifest, config: Record<string, unknown>): { plain: Record<string, unknown>; secrets: Creds } {
const secretKeys = new Set(manifest.config.filter((f) => f.type === "secret").map((f) => f.key));
const plain: Record<string, unknown> = {};
const secrets: Creds = {};
for (const [k, v] of Object.entries(config)) {
if (!secretKeys.has(k)) plain[k] = v;
else if (typeof v === "string" && v !== "") secrets[k] = v;
else if (v !== undefined && v !== null && v !== "") throw new ValidationError(k, "secret_must_be_string");
}
return { plain, secrets };
}
async function readCreds(secret: string, enc: string | null): Promise<Creds | null> {
return enc === null ? null : decryptJson<Creds>(secret, enc);
}
const encOrNull = async (secret: string, c: Creds): Promise<string | null> => (Object.keys(c).length === 0 ? null : encryptJson(secret, c));
function checkSchedule(m: number | null | undefined): void {
if (m !== undefined && m !== null && (!Number.isInteger(m) || m < 1)) throw new ValidationError("scheduleMinutes", "must_be_positive_integer");
}
export async function createConnection(db: Db, secret: string, userId: string, input: ConnectionInput, manifest: PluginManifest): Promise<string> {
if (manifest.id !== input.pluginId) throw new ValidationError("pluginId", "manifest_mismatch");
if (input.name.trim() === "") throw new ValidationError("name", "required");
checkSchedule(input.scheduleMinutes);
const { plain, secrets } = splitConfig(manifest, input.config);
const id = newId();
await db.k
.insertInto("source_connections")
.values({
id,
user_id: userId,
plugin_id: input.pluginId,
name: input.name.trim(),
config: toJson(plain),
credentials_enc: await encOrNull(secret, { ...secrets, ...input.credentials }),
schedule_minutes: input.scheduleMinutes ?? null,
cursor: null,
status: "active",
last_run_at: null,
created_at: nowIso(),
deleted_at: null,
})
.execute();
return id;
}
export interface ConnectionPatch {
name?: string;
config?: Record<string, unknown>;
credentials?: Record<string, string>;
scheduleMinutes?: number | null;
status?: ConnectionStatus;
}
/** Secrets omitted from `config` are kept; plain config is replaced wholesale. Read-modify-write runs in one transaction. */
export async function updateConnection(db: Db, secret: string, userId: string, id: string, patch: ConnectionPatch, manifest: PluginManifest): Promise<void> {
checkSchedule(patch.scheduleMinutes);
await db.k.transaction().execute(async (tx) => {
let q = tx.selectFrom("source_connections").selectAll().where("id", "=", id).where("user_id", "=", userId).where("deleted_at", "is", null);
if (db.dialect === "postgres") q = q.forUpdate();
const row = await q.executeTakeFirst();
if (!row) throw new NotFoundError("connection");
if (row.plugin_id !== manifest.id) throw new ValidationError("pluginId", "manifest_mismatch");
const set: Record<string, unknown> = {};
if (patch.name !== undefined) {
if (patch.name.trim() === "") throw new ValidationError("name", "required");
set.name = patch.name.trim();
}
if (patch.scheduleMinutes !== undefined) set.schedule_minutes = patch.scheduleMinutes;
if (patch.status !== undefined) set.status = patch.status;
if (patch.config !== undefined || patch.credentials !== undefined) {
const split = patch.config ? splitConfig(manifest, patch.config) : { plain: undefined, secrets: {} };
if (split.plain) set.config = toJson(split.plain);
const merged = { ...((await readCreds(secret, row.credentials_enc)) ?? {}), ...split.secrets, ...patch.credentials };
set.credentials_enc = await encOrNull(secret, merged);
}
if (Object.keys(set).length > 0) await tx.updateTable("source_connections").set(set).where("id", "=", id).execute();
});
}
/** Soft delete: `deleted_at` is set, the connection is paused and its credentials are wiped. Records and runs stay. */
export async function deleteConnection(db: Db, userId: string, id: string): Promise<void> {
const r = await db.k
.updateTable("source_connections")
.set({ status: "paused", credentials_enc: null, deleted_at: nowIso() })
.where("id", "=", id)
.where("user_id", "=", userId)
.where("deleted_at", "is", null)
.executeTakeFirst();
if (r.numUpdatedRows !== 1n) throw new NotFoundError("connection");
}
/** The user's live (not deleted) connections. Never returns secrets: plain config plus `hasCredentials`. */
export async function listConnections(db: Db, userId: string): Promise<ConnectionView[]> {
const rows = await db.k
.selectFrom("source_connections")
.selectAll()
.where("user_id", "=", userId)
.where("deleted_at", "is", null)
.orderBy("created_at")
.execute();
return rows.map((r) => ({
id: r.id,
pluginId: r.plugin_id,
name: r.name,
config: fromJson<Record<string, unknown>>(r.config, {}),
scheduleMinutes: r.schedule_minutes,
status: r.status,
lastRunAt: r.last_run_at,
createdAt: r.created_at,
hasCredentials: r.credentials_enc !== null,
}));
}
async function ownedRow(db: Db, userId: string, id: string) {
const r = await db.k
.selectFrom("source_connections")
.selectAll()
.where("id", "=", id)
.where("user_id", "=", userId)
.where("deleted_at", "is", null)
.executeTakeFirst();
if (!r) throw new NotFoundError("connection");
return r;
}
/** One of the user's live connections, without secrets. Deleted or foreign connections throw `NotFoundError`. */
export async function getConnection(db: Db, userId: string, id: string): Promise<ConnectionView> {
const r = await ownedRow(db, userId, id);
return {
id: r.id,
pluginId: r.plugin_id,
name: r.name,
config: fromJson<Record<string, unknown>>(r.config, {}),
scheduleMinutes: r.schedule_minutes,
status: r.status,
lastRunAt: r.last_run_at,
createdAt: r.created_at,
hasCredentials: r.credentials_enc !== null,
};
}
export interface RunView {
id: string;
mode: RunMode;
status: RunStatus;
startedAt: string;
finishedAt: string | null;
counts: Record<string, number>;
log: string;
error: string | null;
}
/** Runs of one of the user's connections, newest first (at most `limit`, default 50). */
export async function listRuns(db: Db, userId: string, connectionId: string, limit = 50): Promise<RunView[]> {
await ownedRow(db, userId, connectionId);
const rows = await db.k
.selectFrom("source_runs")
.selectAll()
.where("connection_id", "=", connectionId)
.orderBy("started_at", "desc")
.orderBy("id", "desc")
.limit(Math.min(Math.max(limit, 1), 200))
.execute();
return rows.map((r) => ({
id: r.id,
mode: r.mode,
status: r.status,
startedAt: r.started_at,
finishedAt: r.finished_at,
counts: fromJson<Record<string, number>>(r.counts, {}),
log: r.log,
error: r.error,
}));
}
async function deviceContext(db: Db, secret: string, registry: PluginRegistry, userId: string, id: string) {
const r = await ownedRow(db, userId, id);
const plugin = registry.get(r.plugin_id);
if (!plugin) throw new ValidationError("pluginId", "not_installed");
if (!plugin.deviceAuth) throw new ValidationError("pluginId", "no_device_auth");
const conn: Connection = {
id: r.id,
userId: r.user_id,
pluginId: r.plugin_id,
name: r.name,
config: fromJson<Record<string, unknown>>(r.config, {}),
credentialsEnc: r.credentials_enc,
scheduleMinutes: r.schedule_minutes,
cursor: r.cursor,
status: r.status,
lastRunAt: r.last_run_at,
createdAt: r.created_at,
};
const ctx = await makePluginContext(db, secret, conn, "device", plugin.manifest);
return { deviceAuth: plugin.deviceAuth, ctx };
}
/** Start a device-code sign-in for one of the user's connections. */
export async function deviceStart(db: Db, secret: string, registry: PluginRegistry, userId: string, id: string) {
const { deviceAuth, ctx } = await deviceContext(db, secret, registry, userId, id);
return deviceAuth.start(ctx);
}
/** Poll a device-code sign-in. On success the returned credentials are merged into the encrypted blob. */
export async function devicePoll(db: Db, secret: string, registry: PluginRegistry, userId: string, id: string, deviceCode: string): Promise<"pending" | "connected"> {
const { deviceAuth, ctx } = await deviceContext(db, secret, registry, userId, id);
const res = await deviceAuth.poll(ctx, deviceCode);
if (res === "pending") return "pending";
await ctx.credentials.save(res);
return "connected";
}
export const USER_AGENT = "STTracker/0.1 (+self-hosted)";
export interface ContextOpts {
/** Replaces the rate-limited client entirely. */
http?: HttpClient;
/** Overrides for the built client (fetch, sleep, clock); used by tests. */
httpOptions?: Pick<HttpClientOptions, "fetchImpl" | "sleep" | "now">;
signal?: AbortSignal;
/** Receives `ctx.log` messages. */
onLog?: (msg: string) => void;
}
/** Synchronous context builder over already-decrypted credentials. */
function buildContext(db: Db, secret: string, conn: Connection, manifest: PluginManifest, initial: Creds | null, opts: ContextOpts): PluginContext {
let creds = initial;
let cursor = conn.cursor;
return {
http:
opts.http ??
createHttpClient({
ratePerMinute: manifest.rateLimit?.perMinute ?? 60,
burst: manifest.rateLimit?.burst ?? 5,
userAgent: USER_AGENT,
...opts.httpOptions,
}),
log: (msg) => opts.onLog?.(msg),
config: { ...conn.config, ...(initial ?? {}) },
credentials: {
get: async () => creds,
async save(c) {
// Read the current blob, merge, and write in one transaction so concurrent saves cannot lose keys.
creds = await db.k.transaction().execute(async (tx) => {
let q = tx.selectFrom("source_connections").select("credentials_enc").where("id", "=", conn.id);
if (db.dialect === "postgres") q = q.forUpdate();
const row = await q.executeTakeFirstOrThrow();
const merged = { ...((await readCreds(secret, row.credentials_enc)) ?? {}), ...c };
await tx.updateTable("source_connections").set({ credentials_enc: await encryptJson(secret, merged) }).where("id", "=", conn.id).execute();
return merged;
});
},
},
cursor: {
get: async () => cursor,
async set(c) {
await db.k.updateTable("source_connections").set({ cursor: c }).where("id", "=", conn.id).execute();
cursor = c;
},
},
now: () => new Date(),
signal: opts.signal ?? new AbortController().signal,
};
}
/**
* Build the plugin context for one run. `ctx.http` is rate limited from `manifest.rateLimit` (default 60/min, burst 5)
* unless `opts.http` is given. `credentials.save` merges into the current blob and persists it encrypted immediately;
* `cursor.set` persists immediately.
*/
export async function makePluginContext(db: Db, secret: string, conn: Connection, _runId: string, manifest: PluginManifest, opts: ContextOpts = {}): Promise<PluginContext> {
return buildContext(db, secret, conn, manifest, await readCreds(secret, conn.credentialsEnc), opts);
}
export interface SyncHandlerDeps {
db: Db;
secret: string;
registry: PluginRegistry;
http?: HttpClient;
}
const MSG_CRYPTO = "Cannot decrypt this source's credentials. Was STTRACKER_SECRET changed? Reconnect the source to fix it.";
/** Record a run that failed before `runSync` could start, flag the connection, and stop the job without retry. */
async function failBeforeRun(db: Db, connectionId: string, mode: RunMode, message: string): Promise<never> {
const now = nowIso();
await db.k
.insertInto("source_runs")
.values({ id: newId(), connection_id: connectionId, mode, started_at: now, finished_at: now, status: "failed", counts: "{}", log: message, error: message })
.execute();
await db.k.updateTable("source_connections").set({ status: "error", last_run_at: now }).where("id", "=", connectionId).execute();
throw new NonRetryableError(message);
}
/**
* The `"sync"` job handler. Payload `{ connectionId, mode? }`. Runs `full` when asked, when the plugin cannot
* `pullChanges`, or when the last successful full run is over a week old; otherwise `changes`.
* A missing or paused connection drops the job. A missing plugin or undecryptable credentials record a failed
* run, mark the connection `error` and fail the job without retry. A failed run is rethrown so the worker retries.
* `ctx.log` output is appended to the run's `source_runs.log`.
*/
export interface SyncPayload {
connectionId: string;
mode?: "full" | "changes";
/** File mode: run the plugin's `importFile` over these bytes instead of pulling. */
file?: { name: string; base64: string };
}
export function syncHandler(deps: SyncHandlerDeps): (payload: SyncPayload, jc?: JobContext) => Promise<RunSummary | void> {
return async (payload, jc) => {
const { db } = deps;
const full = await db.k.selectFrom("source_connections").selectAll().where("id", "=", payload.connectionId).executeTakeFirst();
if (!full) throw new DropJobError("connection not found");
if (full.deleted_at !== null) throw new DropJobError("connection is deleted");
if (full.status === "paused") throw new DropJobError("connection is paused");
const plugin = deps.registry.get(full.plugin_id);
if (!plugin) return failBeforeRun(db, full.id, payload.file ? "file" : (payload.mode ?? "full"), `Plugin '${full.plugin_id}' is not installed or failed to load.`);
let mode: SyncMode = "full";
if (payload.file) {
mode = { file: { name: payload.file.name, bytes: new Uint8Array(Buffer.from(payload.file.base64, "base64")) } };
} else if (plugin.pullChanges && payload.mode !== "full") {
const cutoff = new Date(Date.now() - WEEK_MS).toISOString();
const recentFull = await db.k
.selectFrom("source_runs")
.select("id")
.where("connection_id", "=", full.id)
.where("mode", "=", "full")
.where("status", "=", "ok")
.where("started_at", ">=", cutoff)
.executeTakeFirst();
if (recentFull) mode = "changes";
}
const conn: Connection = {
id: full.id,
userId: full.user_id,
pluginId: full.plugin_id,
name: full.name,
config: fromJson<Record<string, unknown>>(full.config, {}),
credentialsEnc: full.credentials_enc,
scheduleMinutes: full.schedule_minutes,
cursor: full.cursor,
status: full.status,
lastRunAt: full.last_run_at,
createdAt: full.created_at,
};
let creds: Creds | null;
try {
creds = await readCreds(deps.secret, conn.credentialsEnc);
} catch (e) {
if (e instanceof CryptoError) return failBeforeRun(db, full.id, typeof mode === "string" ? mode : "file", MSG_CRYPTO);
throw e;
}
const logs: string[] = [];
let runId: string | null = null;
const makeContext = (c: Connection, id: string) => {
runId = id;
return buildContext(db, deps.secret, c, plugin.manifest, creds, { http: deps.http, signal: jc?.signal, onLog: (m) => void logs.push(m) });
};
let summary: RunSummary;
try {
summary = await runSync({ db, makeContext }, plugin, full.id, mode);
} finally {
if (runId !== null && logs.length > 0) {
const cur = await db.k.selectFrom("source_runs").select("log").where("id", "=", runId).executeTakeFirst();
if (cur) {
const log = cur.log === "" ? logs.join("\n") : `${logs.join("\n")}\n${cur.log}`;
await db.k.updateTable("source_runs").set({ log }).where("id", "=", runId).execute();
}
}
}
if (summary.status === "failed") throw new Error(summary.error ?? "sync failed");
return summary;
};
}
+247
View File
@@ -0,0 +1,247 @@
import type { Db } from "@sttracker/db/db";
import { newId, nowIso } from "@sttracker/db/ids";
import { fromJson, toJson } from "@sttracker/db/json";
import type { Database } from "@sttracker/db/schema";
import type { PartialDate } from "@sttracker/plugin-sdk/types";
import { customFieldsFor, validateCustomValues } from "./custom-fields.ts";
import { checkKeys, checkText, fromStoredDate, resolveVisibleWork, toStoredDate } from "./entries.ts";
import { NotFoundError, ValidationError } from "./errors.ts";
import { type Actor, type Executor, inTx, recordHistory } from "./history.ts";
/** An owned copy (spec §6.7). Always private. `workId` is null until matched; then `titleHint` names it. */
export interface CopyPatch {
workId: string | null;
titleHint: string | null;
format: string | null;
acquiredFrom: string | null;
acquiredAt: PartialDate | null;
/** Integer minor units (cents). */
priceCents: number | null;
/** ISO 4217 code, e.g. "GBP". */
currency: string | null;
notes: string | null;
customFields: Record<string, unknown>;
sourceRef: string | null;
}
const COPY_KEYS = [
"workId",
"titleHint",
"format",
"acquiredFrom",
"acquiredAt",
"priceCents",
"currency",
"notes",
"customFields",
"sourceRef",
] as const satisfies readonly (keyof CopyPatch)[];
export interface Copy extends CopyPatch {
id: string;
userId: string;
createdAt: string;
updatedAt: string;
deletedAt: string | null;
}
type Row = Database["copies"];
function toCopy(r: Row): Copy {
return {
id: r.id,
userId: r.user_id,
workId: r.work_id,
titleHint: r.title_hint,
format: r.format,
acquiredFrom: r.acquired_from,
acquiredAt: fromStoredDate(r.acquired_at, r.acquired_precision),
priceCents: r.price_cents,
currency: r.currency,
notes: r.notes,
customFields: fromJson<Record<string, unknown>>(r.custom_fields, {}),
sourceRef: r.source_ref,
createdAt: r.created_at,
updatedAt: r.updated_at,
deletedAt: r.deleted_at,
};
}
function snapshot(c: Copy): Record<string, unknown> {
const s: Record<string, unknown> = { deletedAt: c.deletedAt };
for (const k of COPY_KEYS) s[k] = c[k];
return s;
}
/** Validate a patch into columns. `workId` must be visible to the user; custom fields use copy-level definitions. */
async function toColumns(
k: Executor,
userId: string,
patch: Partial<CopyPatch>,
currentWorkId: string | null,
): Promise<Partial<Row>> {
checkKeys("patch", patch, COPY_KEYS);
const c: Partial<Row> = {};
let workId = currentWorkId;
if ("workId" in patch) {
workId = patch.workId === null ? null : (await resolveVisibleWork(k, userId, String(patch.workId))).id;
c.work_id = workId;
}
if ("titleHint" in patch) c.title_hint = checkText("titleHint", patch.titleHint);
if ("format" in patch) c.format = checkText("format", patch.format);
if ("acquiredFrom" in patch) c.acquired_from = checkText("acquiredFrom", patch.acquiredFrom);
if ("acquiredAt" in patch) {
const d = toStoredDate("acquiredAt", patch.acquiredAt);
c.acquired_at = d.value;
c.acquired_precision = d.precision;
}
if ("priceCents" in patch) {
const p = patch.priceCents;
if (p !== null && (typeof p !== "number" || !Number.isInteger(p) || p < 0)) {
throw new ValidationError("priceCents", "not_cents");
}
c.price_cents = p;
}
if ("currency" in patch) {
const cur = checkText("currency", patch.currency);
if (cur !== null && !/^[A-Z]{3}$/.test(cur)) throw new ValidationError("currency", "invalid");
c.currency = cur;
}
if ("notes" in patch) c.notes = checkText("notes", patch.notes);
if ("sourceRef" in patch) c.source_ref = checkText("sourceRef", patch.sourceRef);
if ("customFields" in patch) {
let mediaType: string | null = null;
if (workId) {
const w = await k.selectFrom("works").select("media_type").where("id", "=", workId).executeTakeFirst();
mediaType = w?.media_type ?? null;
}
const defs = await customFieldsFor(k, userId, { mediaType, level: "copy" });
validateCustomValues(defs, patch.customFields as Record<string, unknown>);
c.custom_fields = toJson(patch.customFields);
}
return c;
}
async function load(k: Executor, userId: string, copyId: string, includeDeleted = false): Promise<Copy> {
let q = k.selectFrom("copies").selectAll().where("id", "=", copyId).where("user_id", "=", userId);
if (!includeDeleted) q = q.where("deleted_at", "is", null);
const r = await q.executeTakeFirst();
if (!r) throw new NotFoundError("copy");
return toCopy(r);
}
async function write(
k: Executor,
current: Copy,
patch: Partial<CopyPatch>,
actor: Actor,
extra: Partial<Pick<Row, "deleted_at">> = {},
): Promise<Copy> {
const cols = await toColumns(k, current.userId, patch, current.workId);
const next = { ...current, ...patchFromCols(current, cols) };
if (next.workId === null && !next.titleHint?.trim()) throw new ValidationError("titleHint", "required");
await k
.updateTable("copies")
.set({ ...cols, ...extra, updated_at: nowIso() })
.where("id", "=", current.id)
.execute();
const after = await load(k, current.userId, current.id, true);
await recordHistory(k, {
userId: current.userId,
entity: "copy",
entityId: current.id,
before: snapshot(current),
after: snapshot(after),
actor,
});
return after;
}
function patchFromCols(current: Copy, cols: Partial<Row>): Pick<Copy, "workId" | "titleHint"> {
return {
workId: "work_id" in cols ? (cols.work_id ?? null) : current.workId,
titleHint: "title_hint" in cols ? (cols.title_hint ?? null) : current.titleHint,
};
}
/** Create an owned copy. Needs a `workId` or a `titleHint`. Writes history. */
export async function createCopy(
db: Db,
userId: string,
input: Partial<CopyPatch>,
actor: Actor = "user",
tx?: Executor,
): Promise<Copy> {
return inTx(db, tx, async (k) => {
const cols = await toColumns(k, userId, input, null);
if (!cols.work_id && !cols.title_hint?.trim()) throw new ValidationError("titleHint", "required");
const now = nowIso();
const row: Row = {
id: newId(),
user_id: userId,
work_id: null,
title_hint: null,
format: null,
acquired_from: null,
acquired_at: null,
acquired_precision: "unknown",
price_cents: null,
currency: null,
notes: null,
custom_fields: toJson({}),
source_ref: null,
...cols,
created_at: now,
updated_at: now,
deleted_at: null,
};
await k.insertInto("copies").values(row).execute();
const copy = toCopy(row);
await recordHistory(k, { userId, entity: "copy", entityId: copy.id, before: null, after: snapshot(copy), actor });
return copy;
});
}
/** Patch a live copy; only the keys present are touched. Writes history. */
export async function updateCopy(
db: Db,
userId: string,
copyId: string,
patch: Partial<CopyPatch>,
actor: Actor = "user",
tx?: Executor,
): Promise<Copy> {
return inTx(db, tx, async (k) => write(k, await load(k, userId, copyId), patch, actor));
}
/** Soft-delete a copy. Writes history. */
export async function softDeleteCopy(
db: Db,
userId: string,
copyId: string,
actor: Actor = "user",
tx?: Executor,
): Promise<void> {
await inTx(db, tx, async (k) => write(k, await load(k, userId, copyId), {}, actor, { deleted_at: nowIso() }));
}
/** The user's live copies, optionally of one work, newest first. */
export async function listCopies(db: Db, userId: string, filter: { workId?: string } = {}): Promise<Copy[]> {
let q = db.k.selectFrom("copies").selectAll().where("user_id", "=", userId).where("deleted_at", "is", null);
if (filter.workId) q = q.where("work_id", "=", filter.workId);
const rows = await q.orderBy("created_at", "desc").orderBy("id", "desc").execute();
return rows.map(toCopy);
}
/** Called by `restoreVersion`: re-apply a history row's `before` values as actor "user". */
export async function restoreCopyFromHistory(
k: Executor,
userId: string,
copyId: string,
before: Record<string, unknown>,
): Promise<void> {
const current = await load(k, userId, copyId, true);
const patch: Record<string, unknown> = {};
for (const key of COPY_KEYS) if (key in before) patch[key] = before[key];
const extra = "deletedAt" in before ? { deleted_at: (before.deletedAt as string | null) ?? null } : {};
await write(k, current, patch as Partial<CopyPatch>, "user", extra);
}
+45
View File
@@ -0,0 +1,45 @@
import { CryptoError } from "./errors.ts";
const SALT = new TextEncoder().encode("sttracker-v1");
const INFO = new TextEncoder().encode("sttracker-credentials");
function b64(bytes: Uint8Array): string {
return Buffer.from(bytes).toString("base64");
}
async function deriveKey(secret: string): Promise<CryptoKey> {
const material = await crypto.subtle.importKey("raw", new TextEncoder().encode(secret), "HKDF", false, [
"deriveKey",
]);
return crypto.subtle.deriveKey(
{ name: "HKDF", hash: "SHA-256", salt: SALT, info: INFO },
material,
{ name: "AES-GCM", length: 256 },
false,
["encrypt", "decrypt"],
);
}
/** Encrypt a JSON value with AES-256-GCM. Returns `v1.<b64 iv>.<b64 ciphertext>`. */
export async function encryptJson(secret: string, value: unknown): Promise<string> {
const key = await deriveKey(secret);
const iv = crypto.getRandomValues(new Uint8Array(12));
const plain = new TextEncoder().encode(JSON.stringify(value));
const ct = new Uint8Array(await crypto.subtle.encrypt({ name: "AES-GCM", iv }, key, plain));
return `v1.${b64(iv)}.${b64(ct)}`;
}
/** Decrypt a blob from {@link encryptJson}. Throws `CryptoError` on a wrong secret or bad blob. */
export async function decryptJson<T>(secret: string, blob: string): Promise<T> {
const parts = blob.split(".");
if (parts.length !== 3 || parts[0] !== "v1") throw new CryptoError("malformed credential blob");
try {
const iv = new Uint8Array(Buffer.from(parts[1]!, "base64"));
const ct = new Uint8Array(Buffer.from(parts[2]!, "base64"));
const key = await deriveKey(secret);
const plain = await crypto.subtle.decrypt({ name: "AES-GCM", iv }, key, ct);
return JSON.parse(new TextDecoder().decode(plain)) as T;
} catch (cause) {
throw new CryptoError("decryption failed", { cause });
}
}
+176
View File
@@ -0,0 +1,176 @@
import type { Kysely } from "kysely";
import type { Db } from "@sttracker/db/db";
import { newId, nowIso } from "@sttracker/db/ids";
import { fromJson, toJson } from "@sttracker/db/json";
import type { CustomFieldLevel, Database } from "@sttracker/db/schema";
import { parsePartialDate } from "@sttracker/plugin-sdk/partial-date";
import type { MediaType } from "@sttracker/plugin-sdk/types";
import { ValidationError } from "./errors.ts";
export const CUSTOM_FIELD_TYPES = [
"text",
"long_text",
"number",
"date",
"boolean",
"single_select",
"multi_select",
"url",
"rating",
] as const;
export type CustomFieldType = (typeof CUSTOM_FIELD_TYPES)[number];
const LEVELS: readonly CustomFieldLevel[] = ["entry", "engagement", "copy"];
/**
* A user-defined field. Values live in the `custom_fields` JSON of the matching row
* (entry, engagement or copy), keyed by definition **id** so renaming a field keeps its values.
*/
export interface CustomFieldDef {
id: string;
userId: string;
name: string;
type: CustomFieldType;
options: string[];
appliesTo: string[] | "all";
level: CustomFieldLevel;
createdAt: string;
}
export interface CustomFieldInput {
name: string;
type: CustomFieldType;
options?: string[];
appliesTo?: MediaType[] | string[] | "all";
level?: CustomFieldLevel;
}
type Row = Database["custom_field_definitions"];
function toDef(r: { [K in keyof Row]: Row[K] }): CustomFieldDef {
return {
id: r.id,
userId: r.user_id,
name: r.name,
type: r.type as CustomFieldType,
options: fromJson<string[]>(r.options, []),
appliesTo: fromJson<string[] | "all">(r.applies_to, "all"),
level: r.level,
createdAt: r.created_at,
};
}
/** Create a custom field definition. Names are unique per user and level (case-insensitive). */
export async function defineCustomField(db: Db, userId: string, input: CustomFieldInput): Promise<CustomFieldDef> {
const name = typeof input.name === "string" ? input.name.trim() : "";
if (!name) throw new ValidationError("name", "required");
if (!CUSTOM_FIELD_TYPES.includes(input.type)) throw new ValidationError("type", "invalid");
const level = input.level ?? "entry";
if (!LEVELS.includes(level)) throw new ValidationError("level", "invalid");
const isSelect = input.type === "single_select" || input.type === "multi_select";
const options = [...new Set((input.options ?? []).map((o) => String(o).trim()).filter(Boolean))];
if (isSelect && options.length === 0) throw new ValidationError("options", "required");
const appliesTo = input.appliesTo ?? "all";
if (appliesTo !== "all" && (!Array.isArray(appliesTo) || appliesTo.length === 0)) {
throw new ValidationError("appliesTo", "invalid");
}
const existing = await db.k
.selectFrom("custom_field_definitions")
.select("name")
.where("user_id", "=", userId)
.where("level", "=", level)
.execute();
if (existing.some((r) => r.name.toLowerCase() === name.toLowerCase())) {
throw new ValidationError("name", "duplicate");
}
const row: Row = {
id: newId(),
user_id: userId,
name,
type: input.type,
options: toJson(isSelect ? options : []),
applies_to: toJson(appliesTo),
level,
created_at: nowIso(),
};
await db.k.insertInto("custom_field_definitions").values(row).execute();
return toDef(row);
}
/** A user's definitions, optionally only those applying to `mediaType` (and/or one level). */
export async function listCustomFields(
db: Db,
userId: string,
mediaType?: string,
level?: CustomFieldLevel,
): Promise<CustomFieldDef[]> {
return customFieldsFor(db.k, userId, { mediaType, level });
}
/** Same as `listCustomFields`, usable with a transaction handle. */
export async function customFieldsFor(
k: Kysely<Database>,
userId: string,
opts: { mediaType?: string | null; level?: CustomFieldLevel },
): Promise<CustomFieldDef[]> {
let q = k.selectFrom("custom_field_definitions").selectAll().where("user_id", "=", userId);
if (opts.level) q = q.where("level", "=", opts.level);
const rows = await q.orderBy("created_at").orderBy("id").execute();
return rows
.map(toDef)
.filter((d) => opts.mediaType == null || d.appliesTo === "all" || d.appliesTo.includes(opts.mediaType));
}
/**
* Check `values` (keyed by definition id) against `defs`. `null` clears a value and is always allowed.
* Throws `ValidationError("customFields.<name or key>", reason)`.
*/
export function validateCustomValues(defs: CustomFieldDef[], values: Record<string, unknown>): void {
if (values === null || typeof values !== "object" || Array.isArray(values)) {
throw new ValidationError("customFields", "not_object");
}
const byId = new Map(defs.map((d) => [d.id, d]));
for (const [key, v] of Object.entries(values)) {
const def = byId.get(key);
if (!def) throw new ValidationError(`customFields.${key}`, "unknown_field");
if (v === null) continue;
const reason = checkValue(def, v);
if (reason) throw new ValidationError(`customFields.${def.name}`, reason);
}
}
function checkValue(def: CustomFieldDef, v: unknown): string | null {
switch (def.type) {
case "text":
case "long_text":
return typeof v === "string" ? null : "not_text";
case "number":
return typeof v === "number" && Number.isFinite(v) ? null : "not_number";
case "date":
return typeof v === "string" && parsePartialDate(v) ? null : "not_date";
case "boolean":
return typeof v === "boolean" ? null : "not_boolean";
case "single_select":
if (typeof v !== "string") return "not_text";
return def.options.includes(v) ? null : "unknown_option";
case "multi_select":
if (!Array.isArray(v)) return "not_list";
if (!v.every((x) => typeof x === "string" && def.options.includes(x))) return "unknown_option";
return new Set(v).size === v.length ? null : "duplicate_option";
case "url":
return typeof v === "string" && isHttpUrl(v) ? null : "not_url";
case "rating":
return typeof v === "number" && Number.isInteger(v) && v >= 0 && v <= 100 ? null : "out_of_range";
}
}
function isHttpUrl(s: string): boolean {
try {
const u = new URL(s);
return u.protocol === "http:" || u.protocol === "https:";
} catch {
return false;
}
}
+276
View File
@@ -0,0 +1,276 @@
import { sql } from "kysely";
import type { Db } from "@sttracker/db/db";
import { newId, nowIso } from "@sttracker/db/ids";
import { fromJson, toJson } from "@sttracker/db/json";
import type { Database } from "@sttracker/db/schema";
import type { PartialDate } from "@sttracker/plugin-sdk/types";
import { customFieldsFor, validateCustomValues } from "./custom-fields.ts";
import {
checkKeys,
checkRating,
checkText,
type FieldSources,
fromStoredDate,
requireEntry,
toStoredDate,
} from "./entries.ts";
import { NotFoundError, ValidationError } from "./errors.ts";
import { type Actor, type Executor, fieldOwner, inTx, recordHistory } from "./history.ts";
/** One time through a work (spec §6.3). Dates are partial; `null` means unknown. */
export interface EngagementPatch {
startedAt: PartialDate | null;
finishedAt: PartialDate | null;
progress: Record<string, number | string>;
platform: string | null;
rating: number | null;
notes: string | null;
customFields: Record<string, unknown>;
}
export const ENGAGEMENT_PATCH_KEYS = [
"startedAt",
"finishedAt",
"progress",
"platform",
"rating",
"notes",
"customFields",
] as const satisfies readonly (keyof EngagementPatch)[];
export interface Engagement extends EngagementPatch {
id: string;
entryId: string;
fieldSources: FieldSources;
createdAt: string;
updatedAt: string;
deletedAt: string | null;
}
type Row = Database["engagements"];
function toEngagement(r: Row): Engagement {
return {
id: r.id,
entryId: r.entry_id,
startedAt: fromStoredDate(r.started_at, r.started_precision),
finishedAt: fromStoredDate(r.finished_at, r.finished_precision),
progress: fromJson<Record<string, number | string>>(r.progress, {}),
platform: r.platform,
rating: r.rating,
notes: r.notes,
customFields: fromJson<Record<string, unknown>>(r.custom_fields, {}),
fieldSources: fromJson<FieldSources>(r.field_sources, {}),
createdAt: r.created_at,
updatedAt: r.updated_at,
deletedAt: r.deleted_at,
};
}
function snapshot(e: Engagement): Record<string, unknown> {
const s: Record<string, unknown> = { entryId: e.entryId, deletedAt: e.deletedAt };
for (const k of ENGAGEMENT_PATCH_KEYS) s[k] = e[k];
return s;
}
/** Validate a patch and turn it into columns. */
async function toColumns(
k: Executor,
userId: string,
mediaType: string,
patch: Partial<EngagementPatch>,
): Promise<Partial<Row>> {
checkKeys("patch", patch, ENGAGEMENT_PATCH_KEYS);
const c: Partial<Row> = {};
if ("startedAt" in patch) {
const d = toStoredDate("startedAt", patch.startedAt);
c.started_at = d.value;
c.started_precision = d.precision;
}
if ("finishedAt" in patch) {
const d = toStoredDate("finishedAt", patch.finishedAt);
c.finished_at = d.value;
c.finished_precision = d.precision;
}
if ("progress" in patch) {
const p = patch.progress;
if (p === null || typeof p !== "object" || Array.isArray(p)) throw new ValidationError("progress", "not_object");
for (const [key, v] of Object.entries(p)) {
if (typeof v !== "string" && (typeof v !== "number" || !Number.isFinite(v))) {
throw new ValidationError(`progress.${key}`, "invalid");
}
}
c.progress = toJson(p);
}
if ("platform" in patch) c.platform = checkText("platform", patch.platform);
if ("rating" in patch) c.rating = checkRating("rating", patch.rating);
if ("notes" in patch) c.notes = checkText("notes", patch.notes);
if ("customFields" in patch) {
const defs = await customFieldsFor(k, userId, { mediaType, level: "engagement" });
validateCustomValues(defs, patch.customFields as Record<string, unknown>);
c.custom_fields = toJson(patch.customFields);
}
return c;
}
/** Load an engagement whose entry belongs to the user. */
async function load(
k: Executor,
userId: string,
engagementId: string,
opts: { includeDeleted?: boolean } = {},
): Promise<{ eg: Engagement; mediaType: string }> {
let q = k
.selectFrom("engagements as g")
.innerJoin("entries as e", "e.id", "g.entry_id")
.innerJoin("works as w", "w.id", "e.work_id")
.selectAll("g")
.select(["e.media_type_override", "w.media_type"])
.where("g.id", "=", engagementId)
.where("e.user_id", "=", userId);
if (!opts.includeDeleted) q = q.where("g.deleted_at", "is", null).where("e.deleted_at", "is", null);
const r = await q.executeTakeFirst();
if (!r) throw new NotFoundError("engagement");
return { eg: toEngagement(r), mediaType: r.media_type_override ?? r.media_type };
}
async function write(
k: Executor,
userId: string,
current: Engagement,
mediaType: string,
patch: Partial<EngagementPatch>,
actor: Actor,
extra: Partial<Pick<Row, "deleted_at" | "entry_id">> = {},
): Promise<Engagement> {
const cols = await toColumns(k, userId, mediaType, patch);
const fieldSources = { ...current.fieldSources };
for (const key of Object.keys(patch)) fieldSources[key] = fieldOwner(actor) as Actor;
await k
.updateTable("engagements")
.set({ ...cols, ...extra, field_sources: toJson(fieldSources), updated_at: nowIso() })
.where("id", "=", current.id)
.execute();
const { eg } = await load(k, userId, current.id, { includeDeleted: true });
await recordHistory(k, {
userId,
entity: "engagement",
entityId: current.id,
before: snapshot(current),
after: snapshot(eg),
actor,
});
return eg;
}
/** Add an engagement to the user's live entry. Each patched key gets `field_sources[key] = actor`. */
export async function addEngagement(
db: Db,
userId: string,
entryId: string,
patch: Partial<EngagementPatch>,
actor: Actor,
tx?: Executor,
): Promise<Engagement> {
return inTx(db, tx, async (k) => {
const { mediaType } = await requireEntry(k, userId, entryId);
const cols = await toColumns(k, userId, mediaType, patch);
const fieldSources: FieldSources = {};
for (const key of Object.keys(patch)) fieldSources[key] = fieldOwner(actor) as Actor;
const now = nowIso();
const row: Row = {
id: newId(),
entry_id: entryId,
started_at: null,
started_precision: "unknown",
finished_at: null,
finished_precision: "unknown",
progress: toJson({}),
platform: null,
rating: null,
notes: null,
custom_fields: toJson({}),
...cols,
field_sources: toJson(fieldSources),
created_at: now,
updated_at: now,
deleted_at: null,
};
await k.insertInto("engagements").values(row).execute();
const eg = toEngagement(row);
await recordHistory(k, { userId, entity: "engagement", entityId: eg.id, before: null, after: snapshot(eg), actor });
return eg;
});
}
/** Patch a live engagement; only the keys present are touched. Writes history. */
export async function updateEngagement(
db: Db,
userId: string,
engagementId: string,
patch: Partial<EngagementPatch>,
actor: Actor,
tx?: Executor,
): Promise<Engagement> {
return inTx(db, tx, async (k) => {
const { eg, mediaType } = await load(k, userId, engagementId);
if (Object.keys(patch).length === 0) return eg;
return write(k, userId, eg, mediaType, patch, actor);
});
}
/** Soft-delete an engagement. Writes history. */
export async function softDeleteEngagement(
db: Db,
userId: string,
engagementId: string,
actor: Actor,
tx?: Executor,
): Promise<void> {
await inTx(db, tx, async (k) => {
const { eg, mediaType } = await load(k, userId, engagementId);
await write(k, userId, eg, mediaType, {}, actor, { deleted_at: nowIso() });
});
}
/** The live engagements of the user's live entry, by start date (undated last), then creation. */
export async function listEngagements(db: Db, userId: string, entryId: string): Promise<Engagement[]> {
await requireEntry(db.k, userId, entryId);
const rows = await db.k
.selectFrom("engagements")
.selectAll()
.where("entry_id", "=", entryId)
.where("deleted_at", "is", null)
.orderBy(sql`case when started_at is null then 1 else 0 end`)
.orderBy("started_at")
.orderBy("created_at")
.orderBy("id")
.execute();
return rows.map(toEngagement);
}
/** Called by `restoreVersion`: re-apply a history row's `before` values as actor "user". */
export async function restoreEngagementFromHistory(
k: Executor,
userId: string,
engagementId: string,
before: Record<string, unknown>,
): Promise<void> {
const { eg, mediaType } = await load(k, userId, engagementId, { includeDeleted: true });
const patch: Record<string, unknown> = {};
for (const key of ENGAGEMENT_PATCH_KEYS) if (key in before) patch[key] = before[key];
const extra: Partial<Pick<Row, "deleted_at" | "entry_id">> =
"deletedAt" in before ? { deleted_at: (before.deletedAt as string | null) ?? null } : {};
if (typeof before.entryId === "string" && before.entryId !== eg.entryId) {
// Undo a merge's re-parenting; the old entry must be this user's.
const owner = await k
.selectFrom("entries")
.select(["id", "deleted_at"])
.where("id", "=", before.entryId)
.where("user_id", "=", userId)
.executeTakeFirst();
if (!owner) throw new NotFoundError("entry");
if (owner.deleted_at !== null) throw new ValidationError("entryId", "target entry is deleted");
extra.entry_id = before.entryId;
}
await write(k, userId, eg, mediaType, patch as Partial<EngagementPatch>, "user", extra);
}
File diff suppressed because it is too large Load Diff
+42
View File
@@ -0,0 +1,42 @@
/** Authentication or authorisation failure; `code` names the reason (e.g. "invalid_credentials"). */
export class AuthError extends Error {
override readonly name = "AuthError";
constructor(readonly code: string) {
super(code);
}
}
/** Input rejected by validation. */
export class ValidationError extends Error {
override readonly name = "ValidationError";
readonly code = "validation";
constructor(
readonly field: string,
readonly reason: string,
) {
super(`${field}: ${reason}`);
}
}
/** Encryption or decryption failure (e.g. wrong key, tampered ciphertext). */
export class CryptoError extends Error {
override readonly name = "CryptoError";
readonly code = "crypto";
}
/** Import or restore failure; `code` names the reason. */
export class ImportError extends Error {
override readonly name = "ImportError";
constructor(readonly code: string) {
super(code);
}
}
/** A requested record does not exist (or is not visible to the caller). */
export class NotFoundError extends Error {
override readonly name = "NotFoundError";
readonly code = "not_found";
constructor(readonly what: string) {
super(`${what} not found`);
}
}
+474
View File
@@ -0,0 +1,474 @@
import { strFromU8, strToU8, unzipSync, zipSync } from "fflate";
import type { MediaType } from "@sttracker/plugin-sdk/types";
import { toCsv } from "@sttracker/plugin-sdk/csv";
import type { Db } from "@sttracker/db/db";
import { newId } from "@sttracker/db/ids";
import { fromJson, toJson } from "@sttracker/db/json";
import type { AttachmentEntity, EntryStatus, Visibility } from "@sttracker/db/schema";
import { MAX_ATTACHMENT_BYTES, mimeAllowed, normalizeMime, sha256Hex, writeAttachmentFile } from "./attachments.ts";
import { isUniqueViolation } from "./matching.ts";
import { ImportError } from "./errors.ts";
import { createWorkIn, findWorkByExternalIdsIn, type K, type Position } from "./works.ts";
export const EXPORT_FORMAT = "sttracker-export/v1";
// Export rows are loosely typed records: the zip is untrusted input on import.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
type Rec = Record<string, any>;
const camel = (s: string) => s.replace(/_([a-z0-9])/g, (_, c: string) => c.toUpperCase());
const snake = (s: string) => s.replace(/[A-Z]/g, (c) => `_${c.toLowerCase()}`);
const JSON_COLS = {
works: ["creators", "metadata", "position"],
entries: ["tags", "custom_fields", "overrides", "field_sources"],
engagements: ["progress", "custom_fields", "field_sources"],
listens: ["mbids"],
copies: ["custom_fields"],
defs: ["options", "applies_to"],
history: ["before", "after"],
} as const;
/** DB row -> camelCase export record, JSON columns parsed, `drop` columns removed. */
function out(row: Rec, jsonCols: readonly string[], drop: readonly string[] = []): Rec {
const o: Rec = {};
for (const [k, v] of Object.entries(row)) {
if (drop.includes(k)) continue;
o[camel(k)] = jsonCols.includes(k) ? fromJson(v as string, null) : v;
}
return o;
}
/** camelCase export record -> snake_case row, JSON columns serialised. */
function inn(rec: Rec, jsonCols: readonly string[]): Rec {
const o: Rec = {};
for (const [k, v] of Object.entries(rec)) {
const col = snake(k);
o[col] = jsonCols.includes(col) ? toJson(v) : v;
}
return o;
}
const jsonl = (rows: Rec[]) => strToU8(rows.map((r) => JSON.stringify(r) + "\n").join(""));
function readJsonl(files: Record<string, Uint8Array>, name: string): Rec[] {
const f = files[name];
if (!f) return [];
return strFromU8(f)
.split("\n")
.filter((l) => l.trim() !== "")
.map((l) => JSON.parse(l) as Rec);
}
const CHUNK = 500;
const MAX_UNCOMPRESSED = 2 * 1024 * 1024 * 1024;
/** Run `fn` over `ids` in batches (keeps `IN (...)` lists under driver parameter limits). */
async function inChunks<T>(ids: readonly string[], fn: (chunk: string[]) => Promise<T[]>): Promise<T[]> {
const all: T[] = [];
for (let i = 0; i < ids.length; i += CHUNK) all.push(...(await fn(ids.slice(i, i + CHUNK))));
return all;
}
/**
* Per-user export bundle (zip). Includes soft-deleted rows so a restore is lossless; never credentials.
* All reads happen in one transaction (repeatable read on Postgres) so the bundle is a consistent snapshot.
*/
export function exportUserBundle(db: Db, dataDir: string, userId: string): Promise<Uint8Array> {
const tb = db.k.transaction();
return (db.dialect === "postgres" ? tb.setIsolationLevel("repeatable read") : tb).execute((tx) => buildBundle(tx, dataDir, userId));
}
async function buildBundle(k: K, dataDir: string, userId: string): Promise<Uint8Array> {
const user = await k.selectFrom("users").select(["username", "display_name"]).where("id", "=", userId).executeTakeFirstOrThrow();
const entries = await k.selectFrom("entries").selectAll().where("user_id", "=", userId).orderBy("id").execute();
const engagements = (
await k.selectFrom("engagements").innerJoin("entries", "entries.id", "engagements.entry_id").selectAll("engagements").where("entries.user_id", "=", userId).orderBy("engagements.id").execute()
);
const listens = await k.selectFrom("listens").selectAll().where("user_id", "=", userId).orderBy("id").execute();
const lists = await k.selectFrom("lists").selectAll().where("user_id", "=", userId).orderBy("id").execute();
const items = await inChunks(lists.map((l) => l.id), (c) =>
k.selectFrom("list_items").selectAll().where("list_id", "in", c).orderBy("position").orderBy("id").execute(),
);
const copies = await k.selectFrom("copies").selectAll().where("user_id", "=", userId).orderBy("id").execute();
const defs = await k.selectFrom("custom_field_definitions").selectAll().where("user_id", "=", userId).orderBy("id").execute();
const history = await k.selectFrom("user_history").selectAll().where("user_id", "=", userId).orderBy("created_at").orderBy("id").execute();
const atts = await k.selectFrom("attachments").selectAll().where("user_id", "=", userId).orderBy("id").execute();
const links = await inChunks(atts.map((a) => a.id), (c) => k.selectFrom("attachment_links").selectAll().where("attachment_id", "in", c).execute());
const aliases = await k.selectFrom("work_aliases").selectAll().where("user_id", "=", userId).orderBy("id").execute();
// Works: everything referenced, plus ancestors.
const workIds = new Set<string>();
for (const e of entries) workIds.add(e.work_id);
for (const i of items) workIds.add(i.work_id);
for (const c of copies) if (c.work_id) workIds.add(c.work_id);
for (const l of listens) if (l.work_id) workIds.add(l.work_id);
for (const a of aliases) {
workIds.add(a.from_work_id);
workIds.add(a.to_work_id);
}
const works = new Map<string, Rec>();
let frontier = [...workIds];
while (frontier.length) {
const rows = await inChunks(frontier, (c) => k.selectFrom("works").selectAll().where("id", "in", c).execute());
frontier = [];
for (const w of rows) {
works.set(w.id, w as Rec);
// Ancestors and merge survivors are part of the bundle too.
for (const ref of [w.parent_id, w.merged_into_id]) if (ref && !works.has(ref)) frontier.push(ref);
}
}
const extIds = await inChunks([...works.keys()], (c) => k.selectFrom("external_ids").selectAll().where("work_id", "in", c).orderBy("namespace").execute());
const extByWork = new Map<string, Record<string, string>>();
for (const x of extIds) {
const m = extByWork.get(x.work_id) ?? {};
m[x.namespace] = x.value;
extByWork.set(x.work_id, m);
}
const workRecs = [...works.values()]
.sort((a, b) => (a.id < b.id ? -1 : 1))
.map((w) => ({
...out(w, JSON_COLS.works, ["owner_user_id", "cover_attachment_id"]),
private: w.owner_user_id != null,
externalIds: extByWork.get(w.id) ?? {},
}));
const files: Record<string, Uint8Array> = {};
const attRecs: Rec[] = [];
for (const a of atts) {
const file = Bun.file(`${dataDir}/attachments/${a.sha256.slice(0, 2)}/${a.sha256.slice(2, 4)}/${a.sha256}`);
if (!(await file.exists())) continue;
files[`attachments/${a.sha256}`] = new Uint8Array(await file.arrayBuffer());
attRecs.push({
...out(a, [], ["user_id"]),
links: links.filter((l) => l.attachment_id === a.id).map((l) => ({ entity: l.entity, entityId: l.entity_id })),
});
}
const listRecs = lists.map((l) => ({
...out(l, [], ["user_id"]),
items: items.filter((i) => i.list_id === l.id).map((i) => out(i, [], ["list_id"])),
}));
const manifest = {
format: EXPORT_FORMAT,
exportedAt: new Date().toISOString(),
user: { username: user.username, displayName: user.display_name },
};
return zipSync({
"manifest.json": strToU8(JSON.stringify(manifest, null, 2)),
"works.jsonl": jsonl(workRecs),
"entries.jsonl": jsonl(entries.map((e) => ({ ...out(e, JSON_COLS.entries, ["user_id"]), favorite: e.favorite !== 0 }))),
"engagements.jsonl": jsonl(engagements.map((e) => out(e, JSON_COLS.engagements))),
"listens.jsonl": jsonl(listens.map((l) => out(l, JSON_COLS.listens, ["user_id"]))),
"lists.jsonl": jsonl(listRecs),
"copies.jsonl": jsonl(copies.map((c) => out(c, JSON_COLS.copies, ["user_id"]))),
"custom_fields.jsonl": jsonl(defs.map((d) => out(d, JSON_COLS.defs, ["user_id"]))),
"work_aliases.jsonl": jsonl(aliases.map((a) => ({ fromWorkId: a.from_work_id, toWorkId: a.to_work_id }))),
"history.jsonl": jsonl(history.map((h) => out(h, JSON_COLS.history, ["user_id"]))),
"attachments.jsonl": jsonl(attRecs),
...files,
});
}
/** CSV per entity type (live rows only): entries, engagements, listens. */
export async function exportUserCsv(db: Db, userId: string): Promise<Record<string, string>> {
const k = db.k;
const s = (v: unknown) => (v == null ? "" : String(v));
const entries = await k
.selectFrom("entries")
.innerJoin("works", "works.id", "entries.work_id")
.select(["entries.id", "entries.work_id", "works.title", "works.media_type", "works.year", "entries.status", "entries.rating", "entries.favorite", "entries.tags", "entries.review", "entries.created_at", "entries.updated_at"])
.where("entries.user_id", "=", userId)
.where("entries.deleted_at", "is", null)
.orderBy("entries.id")
.execute();
const engagements = await k
.selectFrom("engagements")
.innerJoin("entries", "entries.id", "engagements.entry_id")
.innerJoin("works", "works.id", "entries.work_id")
.select(["engagements.id", "engagements.entry_id", "works.title", "engagements.started_at", "engagements.started_precision", "engagements.finished_at", "engagements.finished_precision", "engagements.platform", "engagements.rating", "engagements.notes"])
.where("entries.user_id", "=", userId)
.where("entries.deleted_at", "is", null)
.where("engagements.deleted_at", "is", null)
.orderBy("engagements.id")
.execute();
const listens = await k.selectFrom("listens").selectAll().where("user_id", "=", userId).orderBy("listened_at").orderBy("id").execute();
return {
"entries.csv": toCsv([
["id", "workId", "title", "mediaType", "year", "status", "rating", "favorite", "tags", "review", "createdAt", "updatedAt"],
...entries.map((e) => [e.id, e.work_id, e.title, e.media_type, s(e.year), e.status, s(e.rating), e.favorite ? "true" : "false", fromJson<string[]>(e.tags, []).join("; "), s(e.review), e.created_at, e.updated_at]),
]),
"engagements.csv": toCsv([
["id", "entryId", "title", "startedAt", "startedPrecision", "finishedAt", "finishedPrecision", "platform", "rating", "notes"],
...engagements.map((e) => [e.id, e.entry_id, e.title, s(e.started_at), e.started_precision, s(e.finished_at), e.finished_precision, s(e.platform), s(e.rating), s(e.notes)]),
]),
"listens.csv": toCsv([
["id", "listenedAt", "artist", "track", "album", "durationMs", "source"],
...listens.map((l) => [l.id, l.listened_at, l.artist, l.track, s(l.album), s(l.duration_ms), l.source]),
]),
};
}
const UUID_RE = /[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/g;
const STATUSES: readonly EntryStatus[] = ["planned", "in_progress", "paused", "completed", "dropped", "none"];
const VISIBILITIES: readonly Visibility[] = ["private", "friends", "public"];
function corrupt(): never {
throw new ImportError("corrupt_bundle");
}
function checkRatingValue(v: unknown): void {
if (v != null && !(typeof v === "number" && Number.isInteger(v) && v >= 0 && v <= 100)) corrupt();
}
function checkRows(rows: Rec[], opts: { status?: boolean; visibility?: boolean; rating?: boolean }): void {
for (const r of rows) {
if (opts.rating) checkRatingValue(r.rating);
if (opts.status && !STATUSES.includes(r.status)) corrupt();
if (opts.visibility && !VISIBILITIES.includes(r.visibility)) corrupt();
}
}
/**
* Restore a bundle into an account with no entries. Works are re-matched by external id. Everything is
* validated before anything is written; rows (attachments included) are inserted in one transaction.
*/
export async function importUserBundle(
db: Db,
dataDir: string,
userId: string,
zip: Uint8Array,
): Promise<{ works: number; entries: number }> {
let files: Record<string, Uint8Array>;
let total = 0;
try {
files = unzipSync(zip, {
filter: (f) => {
total += f.originalSize;
if (total > MAX_UNCOMPRESSED) throw new ImportError("bundle_too_large");
return true;
},
});
} catch (e) {
if (e instanceof ImportError) throw e;
throw new ImportError("unsupported_format");
}
const mf = files["manifest.json"];
let manifest: Rec = {};
try {
manifest = mf ? (JSON.parse(strFromU8(mf)) as Rec) : {};
} catch {
/* handled below */
}
if (manifest.format !== EXPORT_FORMAT) throw new ImportError("unsupported_format");
const existing = await db.k.selectFrom("entries").select("id").where("user_id", "=", userId).limit(1).executeTakeFirst();
if (existing) throw new ImportError("account_not_empty");
let works: Rec[], entries: Rec[], engagements: Rec[], listens: Rec[], lists: Rec[], copies: Rec[];
let defs: Rec[], history: Rec[], atts: Rec[], aliases: Rec[];
try {
works = readJsonl(files, "works.jsonl");
entries = readJsonl(files, "entries.jsonl");
engagements = readJsonl(files, "engagements.jsonl");
listens = readJsonl(files, "listens.jsonl");
lists = readJsonl(files, "lists.jsonl");
copies = readJsonl(files, "copies.jsonl");
defs = readJsonl(files, "custom_fields.jsonl");
history = readJsonl(files, "history.jsonl");
atts = readJsonl(files, "attachments.jsonl");
aliases = readJsonl(files, "work_aliases.jsonl");
} catch {
return corrupt();
}
checkRows(entries, { rating: true, status: true, visibility: true });
checkRows(engagements, { rating: true });
checkRows(lists, { rating: true, visibility: true });
// Attachments: validate everything before writing anything.
for (const a of atts) {
const bytes = files[`attachments/${a.sha256}`];
if (
!bytes ||
!mimeAllowed(normalizeMime(String(a.mime ?? ""))) ||
bytes.byteLength > MAX_ATTACHMENT_BYTES ||
sha256Hex(bytes) !== a.sha256
) {
throw new ImportError("invalid_attachment");
}
}
for (const a of atts) await writeAttachmentFile(dataDir, files[`attachments/${a.sha256}`]!);
const ids = new Map<string, string>();
const remap = (old: unknown): string | null => (typeof old === "string" ? (ids.get(old) ?? null) : null);
const remapCustom = (v: unknown): Rec => {
const o: Rec = {};
for (const [key, val] of Object.entries((v ?? {}) as Rec)) o[ids.get(key) ?? key] = val;
return o;
};
// Parents before children.
const byId = new Map(works.map((w) => [w.id as string, w]));
const depth = (w: Rec): number => {
let d = 0;
const seen = new Set<string>();
for (let p = w.parentId ? byId.get(w.parentId) : undefined; p && !seen.has(p.id); p = p.parentId ? byId.get(p.parentId) : undefined) {
seen.add(p.id);
d++;
}
return d;
};
const ordered = works.map((w) => ({ w, d: depth(w) })).sort((a, b) => a.d - b.d).map((x) => x.w);
try {
await db.k.transaction().execute(async (tx) => {
const created = new Set<string>();
for (const w of ordered) {
const externalIds = (w.externalIds ?? {}) as Record<string, string>;
const hasExt = Object.keys(externalIds).length > 0;
const matched = hasExt ? await findWorkByExternalIdsIn(tx, externalIds, userId) : null;
if (matched) {
ids.set(w.id, matched);
continue;
}
const newWorkId = await createWorkIn(
tx,
{
hint: {
title: w.title,
mediaType: w.mediaType as MediaType,
year: w.year ?? undefined,
creators: w.creators ?? [],
externalIds,
coverUrl: w.coverUrl ?? "",
description: w.description ?? "",
position: (w.position ?? undefined) as Position | undefined,
},
ownerUserId: hasExt ? null : userId,
parentId: remap(w.parentId),
},
"import",
);
await tx
.updateTable("works")
.set({
original_title: w.originalTitle ?? "",
release_date: w.releaseDate ?? null,
metadata: toJson(w.metadata ?? {}),
created_at: w.createdAt,
updated_at: w.updatedAt,
})
.where("id", "=", newWorkId)
.execute();
ids.set(w.id, newWorkId);
created.add(w.id);
}
// Tombstones: only for works created here (a matched existing work keeps its own state).
for (const w of works) {
const target = remap(w.mergedIntoId);
if (!created.has(w.id) || !target) continue;
await tx.updateTable("works").set({ merged_into_id: target }).where("id", "=", ids.get(w.id)!).execute();
}
for (const a of aliases) {
const from = remap(a.fromWorkId);
const to = remap(a.toWorkId);
if (!from || !to || from === to) continue;
await tx.insertInto("work_aliases").values({ id: newId(), user_id: userId, from_work_id: from, to_work_id: to }).execute();
}
for (const d of defs) {
const id = newId();
ids.set(d.id, id);
await tx.insertInto("custom_field_definitions").values({ ...inn(d, JSON_COLS.defs), id, user_id: userId } as never).execute();
}
for (const e of entries) {
const id = newId();
ids.set(e.id, id);
const workId = remap(e.workId);
if (!workId) corrupt();
await tx
.insertInto("entries")
.values({ ...inn(e, JSON_COLS.entries), id, user_id: userId, work_id: workId, favorite: e.favorite ? 1 : 0, custom_fields: toJson(remapCustom(e.customFields)) } as never)
.execute();
}
for (const g of engagements) {
const id = newId();
ids.set(g.id, id);
const entryId = remap(g.entryId);
if (!entryId) corrupt();
await tx
.insertInto("engagements")
.values({ ...inn(g, JSON_COLS.engagements), id, entry_id: entryId, custom_fields: toJson(remapCustom(g.customFields)) } as never)
.execute();
}
for (const l of listens) {
await tx
.insertInto("listens")
.values({ ...inn(l, JSON_COLS.listens), id: newId(), user_id: userId, work_id: remap(l.workId) } as never)
.onConflict((oc) => oc.doNothing())
.execute();
}
for (const l of lists) {
const id = newId();
ids.set(l.id, id);
const { items, ...rest } = l;
await tx.insertInto("lists").values({ ...inn(rest, []), id, user_id: userId } as never).execute();
for (const it of (items ?? []) as Rec[]) {
const workId = remap(it.workId);
if (!workId) continue;
await tx.insertInto("list_items").values({ ...inn(it, []), id: newId(), list_id: id, work_id: workId } as never).execute();
}
}
for (const c of copies) {
const id = newId();
ids.set(c.id, id);
await tx
.insertInto("copies")
.values({ ...inn(c, JSON_COLS.copies), id, user_id: userId, work_id: remap(c.workId), custom_fields: toJson(remapCustom(c.customFields)) } as never)
.execute();
}
for (const a of atts) {
const links = ((a.links ?? []) as Rec[])
.map((l) => ({ entity: l.entity as AttachmentEntity, entityId: remap(l.entityId) }))
.filter((l): l is { entity: AttachmentEntity; entityId: string } => l.entityId != null);
if (links.length === 0) continue;
const id = newId();
const { links: _links, ...attRow } = a;
await tx
.insertInto("attachments")
.values({ ...inn(attRow, []), id, user_id: userId, mime: normalizeMime(a.mime), size: files[`attachments/${a.sha256}`]!.byteLength } as never)
.execute();
for (const l of links) {
await tx.insertInto("attachment_links").values({ id: newId(), attachment_id: id, entity: l.entity, entity_id: l.entityId }).execute();
}
}
// History last, so every referenced id has been remapped.
for (const h of history) {
const sub = (v: unknown) => toJson(JSON.parse(JSON.stringify(v).replace(UUID_RE, (m) => ids.get(m) ?? m)));
await tx
.insertInto("user_history")
.values({ ...inn(h, []), id: newId(), user_id: userId, entity_id: remap(h.entityId) ?? h.entityId, before: sub(h.before), after: sub(h.after) } as never)
.execute();
}
});
} catch (e) {
if (e instanceof ImportError) throw e;
if (isUniqueViolation(e)) throw new ImportError("conflict");
throw e;
}
// An unlink of identical bytes between the pre-write and the commit may have removed a file.
for (const a of atts) await writeAttachmentFile(dataDir, files[`attachments/${a.sha256}`]!);
return { works: works.length, entries: entries.length };
}
+145
View File
@@ -0,0 +1,145 @@
import type { Kysely } from "kysely";
import type { Db } from "@sttracker/db/db";
import { newId, nowIso } from "@sttracker/db/ids";
import { fromJson, toJson } from "@sttracker/db/json";
import type { Database } from "@sttracker/db/schema";
import { restoreCopyFromHistory } from "./copies.ts";
import { restoreEngagementFromHistory } from "./engagements.ts";
import { restoreEntryFromHistory } from "./entries.ts";
import { NotFoundError } from "./errors.ts";
import { restoreListFromHistory } from "./lists.ts";
/** Who made a change. Plugin actors are `plugin:<pluginId>:<connectionId>`; API tokens are `token:<name>`. */
export type Actor = "user" | `plugin:${string}:${string}` | `token:${string}`;
/**
* Who owns a field in `field_sources` after `actor` writes it. Only plugin actors keep their name; the user,
* through the UI or an API token, is always `"user"`, so syncs never overwrite a field a script edited.
* History rows keep the full actor.
*/
export function fieldOwner(actor: string): string {
return actor.startsWith("plugin:") ? actor : "user";
}
/** User-layer entities that write `user_history`. */
export type HistoryEntity = "entry" | "engagement" | "copy" | "list";
/** A Kysely handle: the root instance or a transaction. */
export type Executor = Kysely<Database>;
export interface HistoryRecord {
id: string;
entity: string;
entityId: string;
before: Record<string, unknown>;
after: Record<string, unknown>;
actor: string;
createdAt: string;
}
/**
* Run `fn` in the caller's transaction when given, otherwise in a new one.
* Inside `fn` only the passed handle may be used (SQLite deadlocks on `db.k` mid-transaction).
*/
export function inTx<T>(db: Db, tx: Executor | undefined, fn: (tx: Executor) => Promise<T>): Promise<T> {
return tx ? fn(tx) : db.k.transaction().execute((t) => fn(t));
}
/**
* Write a `user_history` row holding only the keys whose values differ between `before` and `after`
* (compared as JSON). `null` before/after means the entity did not exist / no longer exists.
* Returns the history id, or null when nothing changed (no row written).
*/
export async function recordHistory(
tx: Executor | Db,
input: {
userId: string;
entity: HistoryEntity;
entityId: string;
before: Record<string, unknown> | null;
after: Record<string, unknown> | null;
actor: Actor;
},
): Promise<string | null> {
const k = "k" in tx ? tx.k : tx;
const before = input.before ?? {};
const after = input.after ?? {};
const b: Record<string, unknown> = {};
const a: Record<string, unknown> = {};
for (const key of new Set([...Object.keys(before), ...Object.keys(after)])) {
if (toJson(before[key]) === toJson(after[key])) continue;
if (key in before) b[key] = before[key];
if (key in after) a[key] = after[key];
}
if (Object.keys(a).length === 0 && Object.keys(b).length === 0) return null;
const id = newId();
await k
.insertInto("user_history")
.values({
id,
user_id: input.userId,
entity: input.entity,
entity_id: input.entityId,
before: toJson(b),
after: toJson(a),
actor: input.actor,
created_at: nowIso(),
})
.execute();
return id;
}
/** The user's history for one entity, newest first. Another user's entity yields `[]`. */
export async function getHistory(db: Db, userId: string, entity: string, entityId: string): Promise<HistoryRecord[]> {
const rows = await db.k
.selectFrom("user_history")
.selectAll()
.where("user_id", "=", userId)
.where("entity", "=", entity)
.where("entity_id", "=", entityId)
.orderBy("created_at", "desc")
.orderBy("id", "desc")
.execute();
return rows.map(toRecord);
}
/**
* Re-apply a history row's `before` values as actor `user` (which writes a new history row and sets
* `field_sources` for the restored fields to `"user"`). Throws `NotFoundError` for another user's row.
*/
export async function restoreVersion(db: Db, userId: string, historyId: string): Promise<void> {
const row = await db.k
.selectFrom("user_history")
.selectAll()
.where("id", "=", historyId)
.where("user_id", "=", userId)
.executeTakeFirst();
if (!row) throw new NotFoundError("history");
const h = toRecord(row);
await db.k.transaction().execute(async (tx) => {
switch (h.entity as HistoryEntity) {
case "entry":
return restoreEntryFromHistory(tx, userId, h.entityId, h.before);
case "engagement":
return restoreEngagementFromHistory(tx, userId, h.entityId, h.before);
case "copy":
return restoreCopyFromHistory(tx, userId, h.entityId, h.before);
case "list":
return restoreListFromHistory(tx, userId, h.entityId, h.before);
default:
throw new NotFoundError("history");
}
});
}
function toRecord(r: Database["user_history"]): HistoryRecord {
return {
id: r.id,
entity: r.entity,
entityId: r.entity_id,
before: fromJson<Record<string, unknown>>(r.before, {}),
after: fromJson<Record<string, unknown>>(r.after, {}),
actor: r.actor,
createdAt: r.created_at,
};
}
+222
View File
@@ -0,0 +1,222 @@
import type { Db } from "@sttracker/db/db";
import { newId, nowIso } from "@sttracker/db/ids";
import { fromJson, toJson } from "@sttracker/db/json";
import { sql } from "kysely";
const MAX_ATTEMPTS = 5;
const STALE_LOCK_MS = 60 * 60 * 1000;
const HEARTBEAT_MS = 5 * 60 * 1000;
export interface JobContext {
/** Aborted when the worker is stopped. */
signal: AbortSignal;
}
export type JobHandler = (payload: any, ctx: JobContext) => Promise<unknown>;
/** Throw from a handler to fail the job at once, with no retry. */
export class NonRetryableError extends Error {
override readonly name = "NonRetryableError";
}
/** Throw from a handler to drop the job: it is marked `done` and the message is kept in `last_error`. */
export class DropJobError extends Error {
override readonly name = "DropJobError";
}
const logErr = (err: unknown) => console.error("[jobs]", err);
/**
* Queue a job. Returns its id, or null when a `queued|running` job already holds `lockKey`.
* On Postgres the check and insert run under an advisory transaction lock so two callers cannot both pass.
*/
export async function enqueue(
db: Db,
type: string,
payload: unknown,
opts: { runAt?: string; lockKey?: string } = {},
): Promise<string | null> {
const lockKey = opts.lockKey ?? null;
return db.k.transaction().execute(async (tx) => {
if (lockKey !== null) {
if (db.dialect === "postgres") await sql`select pg_advisory_xact_lock(hashtext(${lockKey}))`.execute(tx);
const held = await tx
.selectFrom("jobs")
.select("id")
.where("lock_key", "=", lockKey)
.where("status", "in", ["queued", "running"])
.executeTakeFirst();
if (held) return null;
}
const id = newId();
const now = nowIso();
await tx
.insertInto("jobs")
.values({
id,
type,
payload: toJson(payload ?? null),
run_at: opts.runAt ?? now,
status: "queued",
attempts: 0,
last_error: null,
locked_at: null,
lock_key: lockKey,
created_at: now,
})
.execute();
return id;
});
}
/** Claim one queued job atomically; true when this caller won it. */
async function claim(db: Db, id: string): Promise<boolean> {
const r = await db.k
.updateTable("jobs")
.set({ status: "running", locked_at: nowIso() })
.where("id", "=", id)
.where("status", "=", "queued")
.executeTakeFirst();
return r.numUpdatedRows === 1n;
}
async function runJob(db: Db, handlers: Record<string, JobHandler>, job: { id: string; type: string; payload: string; attempts: number }, signal: AbortSignal, heartbeatMs: number): Promise<void> {
// Keep `locked_at` fresh so another process's reaper does not steal a long-running job.
const beat = setInterval(() => {
db.k.updateTable("jobs").set({ locked_at: nowIso() }).where("id", "=", job.id).where("status", "=", "running").execute().catch(logErr);
}, heartbeatMs);
try {
try {
const h = handlers[job.type];
if (!h) throw new NonRetryableError(`no handler for job type "${job.type}"`);
await h(fromJson<unknown>(job.payload, null), { signal });
await db.k.updateTable("jobs").set({ status: "done", locked_at: null, last_error: null }).where("id", "=", job.id).execute();
} catch (e) {
const attempts = job.attempts + 1;
const err = e instanceof Error ? e.message : String(e);
if (signal.aborted && !(e instanceof DropJobError) && !(e instanceof NonRetryableError)) {
// Interrupted by shutdown: back to the queue untouched, no attempt burned, no backoff.
await db.k.updateTable("jobs").set({ status: "queued", locked_at: null }).where("id", "=", job.id).execute();
} else if (e instanceof DropJobError) {
await db.k.updateTable("jobs").set({ status: "done", locked_at: null, last_error: err }).where("id", "=", job.id).execute();
} else if (e instanceof NonRetryableError || attempts >= MAX_ATTEMPTS) {
await db.k.updateTable("jobs").set({ status: "failed", attempts, last_error: err, locked_at: null }).where("id", "=", job.id).execute();
} else {
const runAt = new Date(Date.now() + 2 ** attempts * 60_000).toISOString();
await db.k
.updateTable("jobs")
.set({ status: "queued", attempts, last_error: err, run_at: runAt, locked_at: null })
.where("id", "=", job.id)
.execute();
}
}
} catch (e) {
logErr(e);
} finally {
clearInterval(beat);
}
}
/** Re-queue jobs a crashed process left `running`, so their lockKey cannot be held forever. */
async function reapStale(db: Db, staleMs: number, mine: Set<string>): Promise<void> {
const cutoff = new Date(Date.now() - staleMs).toISOString();
let q = db.k.updateTable("jobs").set({ status: "queued", locked_at: null }).where("status", "=", "running").where("locked_at", "<", cutoff);
if (mine.size > 0) q = q.where("id", "not in", [...mine]);
await q.execute();
}
/** Poll for due jobs and run them with bounded concurrency. Failures retry with backoff `2^attempts` minutes, then `failed`. */
export function startWorker(
db: Db,
handlers: Record<string, JobHandler>,
opts: { pollMs?: number; concurrency?: number; staleMs?: number; heartbeatMs?: number } = {},
): { stop(): Promise<void> } {
const pollMs = opts.pollMs ?? 1000;
const concurrency = opts.concurrency ?? 2;
const staleMs = opts.staleMs ?? STALE_LOCK_MS;
const heartbeatMs = opts.heartbeatMs ?? HEARTBEAT_MS;
const inflight = new Set<Promise<void>>();
const mine = new Set<string>();
const abort = new AbortController();
let stopped = false;
let timer: ReturnType<typeof setTimeout> | undefined;
let tickP: Promise<void> = Promise.resolve();
async function tick(): Promise<void> {
await reapStale(db, staleMs, mine);
while (!stopped && inflight.size < concurrency) {
const due = await db.k
.selectFrom("jobs")
.select(["id", "type", "payload", "attempts"])
.where("status", "=", "queued")
.where("run_at", "<=", nowIso())
.orderBy("run_at")
.limit(concurrency - inflight.size)
.execute();
let started = 0;
for (const job of due) {
if (stopped || inflight.size >= concurrency) break;
if (!(await claim(db, job.id))) continue;
started++;
mine.add(job.id);
const p: Promise<void> = runJob(db, handlers, job, abort.signal, heartbeatMs)
.catch(logErr)
.finally(() => {
inflight.delete(p);
mine.delete(job.id);
});
inflight.add(p);
}
if (started === 0) break;
}
}
function schedule() {
if (stopped) return;
timer = setTimeout(() => {
tickP = tick().catch(logErr).finally(schedule);
}, pollMs);
}
tickP = tick().catch(logErr).finally(schedule);
return {
async stop() {
stopped = true;
abort.abort();
if (timer) clearTimeout(timer);
await tickP.catch(logErr);
await Promise.allSettled([...inflight]);
},
};
}
/** Every `tickMs`, enqueue a `sync` job for each non-paused (active or errored) connection whose schedule has come due. */
export function startScheduler(db: Db, opts: { tickMs?: number } = {}): { stop(): void; tickNow(): Promise<void> } {
const tickMs = opts.tickMs ?? 60_000;
let stopped = false;
async function tick() {
const rows = await db.k
.selectFrom("source_connections")
.select(["id", "schedule_minutes", "last_run_at"])
.where("status", "!=", "paused")
.where("deleted_at", "is", null)
.where("schedule_minutes", "is not", null)
.execute();
const now = Date.now();
for (const r of rows) {
if (stopped) return;
const due = r.last_run_at === null || Date.parse(r.last_run_at) + r.schedule_minutes! * 60_000 <= now;
if (due) await enqueue(db, "sync", { connectionId: r.id }, { lockKey: `sync:${r.id}` });
}
}
const run = () => tick().catch(logErr);
void run();
const timer = setInterval(() => void run(), tickMs);
return {
/** Run one scheduling pass now (tests, manual triggers). */
tickNow: run,
stop() {
stopped = true;
clearInterval(timer);
},
};
}
+406
View File
@@ -0,0 +1,406 @@
import type { Db } from "@sttracker/db/db";
import { newId, nowIso } from "@sttracker/db/ids";
import type { Database, ListKind, Visibility } from "@sttracker/db/schema";
import { checkKeys, checkRating, checkText, defaultVisibility, resolveVisibleWork, VISIBILITIES } from "./entries.ts";
import { NotFoundError, ValidationError } from "./errors.ts";
import { type Actor, type Executor, inTx, recordHistory } from "./history.ts";
const KINDS: readonly ListKind[] = ["list", "grouping", "thread"];
/** Spec §6.5. Only a `grouping` may carry `rating`/`review`. */
export interface ListPatch {
name: string;
description: string;
kind: ListKind;
visibility: Visibility;
rating: number | null;
review: string | null;
}
const LIST_KEYS = ["name", "description", "kind", "visibility", "rating", "review"] as const satisfies readonly (keyof ListPatch)[];
export interface List extends ListPatch {
id: string;
userId: string;
createdAt: string;
updatedAt: string;
deletedAt: string | null;
}
export interface ListItem {
id: string;
listId: string;
workId: string;
/** 0-based, dense. */
position: number;
note: string | null;
}
export interface ListWithItems extends List {
items: (ListItem & { work: { id: string; title: string; mediaType: string; year: number | null; coverUrl: string } })[];
}
type Row = Database["lists"];
type ItemRow = Database["list_items"];
function toList(r: Row): List {
return {
id: r.id,
userId: r.user_id,
name: r.name,
description: r.description,
kind: r.kind,
visibility: r.visibility,
rating: r.rating,
review: r.review,
createdAt: r.created_at,
updatedAt: r.updated_at,
deletedAt: r.deleted_at,
};
}
const toItem = (r: ItemRow): ListItem => ({
id: r.id,
listId: r.list_id,
workId: r.work_id,
position: r.position,
note: r.note,
});
/** The history snapshot: list fields, `deletedAt`, and the ordered items (so removals are restorable). */
async function snapshot(k: Executor, l: List): Promise<Record<string, unknown>> {
const s: Record<string, unknown> = { deletedAt: l.deletedAt };
for (const key of LIST_KEYS) s[key] = l[key];
s.items = (await items(k, l.id)).map((i) => ({ id: i.id, workId: i.workId, note: i.note }));
return s;
}
function normalise(patch: Partial<ListPatch>): Partial<ListPatch> {
checkKeys("patch", patch, LIST_KEYS);
const out: Partial<ListPatch> = {};
if ("name" in patch) {
if (typeof patch.name !== "string" || !patch.name.trim()) throw new ValidationError("name", "required");
out.name = patch.name.trim();
}
if ("description" in patch) out.description = checkText("description", patch.description) ?? "";
if ("kind" in patch) {
if (!KINDS.includes(patch.kind as ListKind)) throw new ValidationError("kind", "invalid");
out.kind = patch.kind;
}
if ("visibility" in patch) {
if (!VISIBILITIES.includes(patch.visibility as Visibility)) throw new ValidationError("visibility", "invalid");
out.visibility = patch.visibility;
}
if ("rating" in patch) out.rating = checkRating("rating", patch.rating);
if ("review" in patch) out.review = checkText("review", patch.review);
return out;
}
function checkGrouping(l: Pick<ListPatch, "kind" | "rating" | "review">): void {
if (l.kind !== "grouping" && (l.rating !== null || l.review !== null)) {
throw new ValidationError(l.rating !== null ? "rating" : "review", "grouping_only");
}
}
function toColumns(p: Partial<ListPatch>): Partial<Row> {
const c: Partial<Row> = {};
if (p.name !== undefined) c.name = p.name;
if (p.description !== undefined) c.description = p.description;
if (p.kind !== undefined) c.kind = p.kind;
if (p.visibility !== undefined) c.visibility = p.visibility;
if (p.rating !== undefined) c.rating = p.rating;
if (p.review !== undefined) c.review = p.review;
return c;
}
async function load(k: Executor, userId: string, listId: string, includeDeleted = false): Promise<List> {
let q = k.selectFrom("lists").selectAll().where("id", "=", listId).where("user_id", "=", userId);
if (!includeDeleted) q = q.where("deleted_at", "is", null);
const r = await q.executeTakeFirst();
if (!r) throw new NotFoundError("list");
return toList(r);
}
async function items(k: Executor, listId: string): Promise<ListItem[]> {
const rows = await k
.selectFrom("list_items")
.selectAll()
.where("list_id", "=", listId)
.orderBy("position")
.orderBy("id")
.execute();
return rows.map(toItem);
}
/** Rewrite positions to 0..n-1 in the given order. */
async function renumber(k: Executor, ordered: ListItem[]): Promise<void> {
for (const [i, it] of ordered.entries()) {
if (it.position !== i) await k.updateTable("list_items").set({ position: i }).where("id", "=", it.id).execute();
}
}
/** Run `change` on a list (live, user's), then bump `updated_at` and record history. */
async function mutate<T>(
k: Executor,
userId: string,
listId: string,
actor: Actor,
change: (l: List) => Promise<T>,
opts: { includeDeleted?: boolean } = {},
): Promise<T> {
const before = await load(k, userId, listId, opts.includeDeleted);
const beforeSnap = await snapshot(k, before);
const result = await change(before);
await k.updateTable("lists").set({ updated_at: nowIso() }).where("id", "=", listId).execute();
const after = await load(k, userId, listId, true);
await recordHistory(k, {
userId,
entity: "list",
entityId: listId,
before: beforeSnap,
after: await snapshot(k, after),
actor,
});
return result;
}
/** Create a list. Default kind `list`, visibility from user preferences. */
export async function createList(
db: Db,
userId: string,
input: Partial<ListPatch> & { name: string },
actor: Actor = "user",
tx?: Executor,
): Promise<List> {
return inTx(db, tx, async (k) => {
const p = normalise(input);
if (p.name === undefined) throw new ValidationError("name", "required");
const now = nowIso();
const row: Row = {
id: newId(),
user_id: userId,
name: p.name,
description: p.description ?? "",
kind: p.kind ?? "list",
visibility: p.visibility ?? (await defaultVisibility(k, userId)),
rating: p.rating ?? null,
review: p.review ?? null,
created_at: now,
updated_at: now,
deleted_at: null,
};
checkGrouping(row);
await k.insertInto("lists").values(row).execute();
const list = toList(row);
await recordHistory(k, {
userId,
entity: "list",
entityId: list.id,
before: null,
after: await snapshot(k, list),
actor,
});
return list;
});
}
/** Patch a list's own fields. */
export async function updateList(
db: Db,
userId: string,
listId: string,
patch: Partial<ListPatch>,
actor: Actor = "user",
tx?: Executor,
): Promise<List> {
return inTx(db, tx, async (k) => {
const p = normalise(patch);
await mutate(k, userId, listId, actor, async (l) => {
checkGrouping({ ...l, ...p });
const cols = toColumns(p);
if (Object.keys(cols).length) await k.updateTable("lists").set(cols).where("id", "=", listId).execute();
});
return load(k, userId, listId);
});
}
/** Soft-delete a list (its items stay, for restore). */
export async function softDeleteList(
db: Db,
userId: string,
listId: string,
actor: Actor = "user",
tx?: Executor,
): Promise<void> {
await inTx(db, tx, (k) =>
mutate(k, userId, listId, actor, async () => {
await k.updateTable("lists").set({ deleted_at: nowIso() }).where("id", "=", listId).execute();
}),
);
}
/** Add a work to a list, at `position` (0-based; default: the end). */
export async function addListItem(
db: Db,
userId: string,
listId: string,
input: { workId: string; note?: string | null; position?: number },
actor: Actor = "user",
tx?: Executor,
): Promise<ListItem> {
return inTx(db, tx, (k) =>
mutate(k, userId, listId, actor, async () => {
const work = await resolveVisibleWork(k, userId, input.workId);
const note = input.note === undefined ? null : checkText("note", input.note);
const current = await items(k, listId);
const at = clampIndex(input.position ?? current.length, current.length);
const item: ListItem = { id: newId(), listId, workId: work.id, position: current.length, note };
await k
.insertInto("list_items")
.values({ id: item.id, list_id: listId, work_id: item.workId, position: item.position, note })
.execute();
current.splice(at, 0, item);
await renumber(k, current);
return { ...item, position: at };
}),
);
}
/** Replace a list's items with `next`, in order (one history row). Used by the sync engine. */
export async function replaceListItems(
db: Db,
userId: string,
listId: string,
next: { workId: string; note?: string | null }[],
actor: Actor = "user",
tx?: Executor,
): Promise<void> {
await inTx(db, tx, (k) =>
mutate(k, userId, listId, actor, async () => {
const rows: ItemRow[] = [];
for (const [i, it] of next.entries()) {
const work = await resolveVisibleWork(k, userId, it.workId);
rows.push({ id: newId(), list_id: listId, work_id: work.id, position: i, note: it.note == null ? null : checkText("note", it.note) });
}
await k.deleteFrom("list_items").where("list_id", "=", listId).execute();
if (rows.length > 0) await k.insertInto("list_items").values(rows).execute();
}),
);
}
function clampIndex(i: number, max: number): number {
if (typeof i !== "number" || !Number.isInteger(i)) throw new ValidationError("position", "not_integer");
return Math.max(0, Math.min(i, max));
}
/** Load an item of one of the user's live lists. */
async function loadItem(k: Executor, userId: string, itemId: string): Promise<ListItem> {
const r = await k
.selectFrom("list_items as i")
.innerJoin("lists as l", "l.id", "i.list_id")
.selectAll("i")
.where("i.id", "=", itemId)
.where("l.user_id", "=", userId)
.where("l.deleted_at", "is", null)
.executeTakeFirst();
if (!r) throw new NotFoundError("list_item");
return toItem(r);
}
/** Move an item to `toIndex` (clamped to the list bounds). */
export async function moveListItem(
db: Db,
userId: string,
itemId: string,
toIndex: number,
actor: Actor = "user",
tx?: Executor,
): Promise<void> {
await inTx(db, tx, async (k) => {
const item = await loadItem(k, userId, itemId);
await mutate(k, userId, item.listId, actor, async () => {
const current = (await items(k, item.listId)).filter((i) => i.id !== itemId);
current.splice(clampIndex(toIndex, current.length), 0, item);
await renumber(k, current);
});
});
}
/** Remove an item. The list's history keeps it, so `restoreVersion` can bring it back. */
export async function removeListItem(
db: Db,
userId: string,
itemId: string,
actor: Actor = "user",
tx?: Executor,
): Promise<void> {
await inTx(db, tx, async (k) => {
const item = await loadItem(k, userId, itemId);
await mutate(k, userId, item.listId, actor, async () => {
await k.deleteFrom("list_items").where("id", "=", itemId).execute();
await renumber(k, await items(k, item.listId));
});
});
}
/** A live list of the user's with its items in order. */
export async function getList(db: Db, userId: string, listId: string): Promise<ListWithItems> {
const list = await load(db.k, userId, listId);
const rows = await db.k
.selectFrom("list_items as i")
.innerJoin("works as w", "w.id", "i.work_id")
.selectAll("i")
.select(["w.title", "w.media_type", "w.year", "w.cover_url"])
.where("i.list_id", "=", listId)
.orderBy("i.position")
.orderBy("i.id")
.execute();
return {
...list,
items: rows.map((r) => ({
...toItem(r),
work: { id: r.work_id, title: r.title, mediaType: r.media_type, year: r.year, coverUrl: r.cover_url },
})),
};
}
/** The user's live lists, most recently updated first. */
export async function listLists(db: Db, userId: string, filter: { kind?: ListKind } = {}): Promise<List[]> {
let q = db.k.selectFrom("lists").selectAll().where("user_id", "=", userId).where("deleted_at", "is", null);
if (filter.kind) q = q.where("kind", "=", filter.kind);
const rows = await q.orderBy("updated_at", "desc").orderBy("id", "desc").execute();
return rows.map(toList);
}
/** Called by `restoreVersion`: re-apply a history row's `before` (fields, deletion and items) as actor "user". */
export async function restoreListFromHistory(
k: Executor,
userId: string,
listId: string,
before: Record<string, unknown>,
): Promise<void> {
const patch: Record<string, unknown> = {};
for (const key of LIST_KEYS) if (key in before) patch[key] = before[key];
const p = normalise(patch as Partial<ListPatch>);
await mutate(
k,
userId,
listId,
"user",
async (l) => {
checkGrouping({ ...l, ...p });
const cols: Partial<Row> = toColumns(p);
if ("deletedAt" in before) cols.deleted_at = (before.deletedAt as string | null) ?? null;
if (Object.keys(cols).length) await k.updateTable("lists").set(cols).where("id", "=", listId).execute();
if (Array.isArray(before.items)) {
await k.deleteFrom("list_items").where("list_id", "=", listId).execute();
for (const [position, it] of (before.items as { id: string; workId: string; note: string | null }[]).entries()) {
await k
.insertInto("list_items")
.values({ id: it.id, list_id: listId, work_id: it.workId, position, note: it.note ?? null })
.execute();
}
}
},
{ includeDeleted: true },
);
}
+192
View File
@@ -0,0 +1,192 @@
import { sql } from "kysely";
import type { WorkHint } from "@sttracker/plugin-sdk/types";
import type { Db } from "@sttracker/db/db";
import { newId, nowIso } from "@sttracker/db/ids";
import {
createWorkIn,
finalWorkId,
findWorkByExternalIdsIn,
normalizePosition,
resolveAliasIn,
toWork,
updateWorkFromHintIn,
type K,
} from "./works.ts";
export interface FuzzyInput {
title: string;
year?: number | null;
creators?: { name: string }[];
}
function normalize(s: string): string {
return s
.normalize("NFD")
.replace(/\p{M}+/gu, "")
.toLowerCase()
.replace(/[^\p{L}\p{N}]+/gu, " ")
.trim()
.replace(/^(the|a|an)\s+/, "");
}
function tokens(s: string): string[] {
return normalize(s).split(" ").filter(Boolean);
}
/**
* 0..1 similarity. Title token-set Jaccard weighs 0.7; year 0.2 (±1 year counts half); any shared
* creator 0.1. A component missing on either side is left out and the weights renormalised, so a
* title-only comparison can still reach 1.
*/
export function fuzzyScore(a: FuzzyInput, b: FuzzyInput): number {
const ta = new Set(tokens(a.title));
const tb = new Set(tokens(b.title));
let inter = 0;
for (const t of ta) if (tb.has(t)) inter++;
const union = ta.size + tb.size - inter;
const jaccard = union === 0 ? 0 : inter / union;
let score = jaccard * 0.7;
let weight = 0.7;
if (a.year != null && b.year != null) {
const d = Math.abs(a.year - b.year);
score += d === 0 ? 0.2 : d === 1 ? 0.1 : 0;
weight += 0.2;
}
const ca = new Set((a.creators ?? []).map((c) => normalize(c.name)).filter(Boolean));
const cb = (b.creators ?? []).map((c) => normalize(c.name)).filter(Boolean);
if (ca.size > 0 && cb.length > 0) {
if (cb.some((n) => ca.has(n))) score += 0.1;
weight += 0.1;
}
return score / weight;
}
interface Resolved {
workId: string;
created: boolean;
draft: boolean;
}
async function resolve(k: K, userId: string | null, hint: WorkHint, actor: string): Promise<Resolved> {
// 1. Any external ID on a work this user may see (parents resolve with userId null: shared only).
const byId = await findWorkByExternalIdsIn(k, hint.externalIds, userId);
if (byId) {
await updateWorkFromHintIn(k, byId, hint, actor);
const workId = userId ? await resolveAliasIn(k, userId, byId) : byId;
const owner = await k.selectFrom("works").select("owner_user_id").where("id", "=", workId).executeTakeFirstOrThrow();
return { workId, created: false, draft: owner.owner_user_id !== null };
}
// 2. Parent + position.
const parentId = hint.parent ? (await resolve(k, null, hint.parent, actor)).workId : null;
const position = normalizePosition(hint.position);
if (parentId && position) {
const child = await k
.selectFrom("works")
.select("id")
.where("parent_id", "=", parentId)
.where("position", "=", position)
.where("owner_user_id", "is", null)
.executeTakeFirst();
if (child) {
// A merged tombstone still holds its (parent_id, position) slot: follow it to the surviving work.
const workId = await finalWorkId(k, child.id);
await updateWorkFromHintIn(k, workId, hint, actor);
return { workId, created: false, draft: false };
}
return { workId: await createWorkIn(k, { hint, parentId }, actor), created: true, draft: false };
}
// 3. External IDs but no match: new shared work.
if (Object.keys(hint.externalIds).length > 0) {
return { workId: await createWorkIn(k, { hint, parentId }, actor), created: true, draft: false };
}
if (userId === null) {
// Parent of a shared child: reuse an identical shared work, else create one.
let q = k
.selectFrom("works")
.select("id")
.where("owner_user_id", "is", null)
.where("merged_into_id", "is", null)
.where("media_type", "=", hint.mediaType)
.where(sql<boolean>`lower(title) = ${hint.title.toLowerCase()}`);
q = hint.year === undefined ? q : q.where("year", "=", hint.year);
q = parentId ? q.where("parent_id", "=", parentId) : q.where("parent_id", "is", null);
const same = await q.executeTakeFirst();
if (same) return { workId: same.id, created: false, draft: false };
return { workId: await createWorkIn(k, { hint, parentId }, actor), created: true, draft: false };
}
// 4. Private draft plus fuzzy suggestions.
const workId = await createWorkIn(k, { hint, ownerUserId: userId }, actor);
await suggest(k, userId, workId, hint);
return { workId, created: true, draft: true };
}
async function suggest(k: K, userId: string, draftId: string, hint: WorkHint): Promise<void> {
// Bound the candidate set with LIKE on the significant title tokens (OR), so "The Expanse: Leviathan
// Wakes" still finds "Leviathan Wakes". Tokens of 3+ chars, first five; falls back to the first token.
const all = tokens(hint.title);
const sig = all.filter((t) => t.length >= 3).slice(0, 5);
const likeTokens = sig.length > 0 ? sig : all.slice(0, 1);
if (likeTokens.length === 0) return;
const rows = await k
.selectFrom("works")
.selectAll()
.where("merged_into_id", "is", null)
.where("id", "!=", draftId)
.where((eb) =>
eb.or([
eb.and([eb("owner_user_id", "is", null), eb("media_type", "=", hint.mediaType)]),
eb("owner_user_id", "=", userId),
]),
)
.where((eb) =>
eb.or(likeTokens.map((t) => eb(sql<string>`lower(title)`, "like", `%${t}%`))),
)
.limit(200)
.execute();
const scored = rows
.map(toWork)
.map((w) => ({ id: w.id, score: fuzzyScore(hint, w) }))
.filter((c) => c.score >= 0.6)
.sort((a, b) => b.score - a.score)
.slice(0, 5);
const now = nowIso();
for (const c of scored) {
await k
.insertInto("match_suggestions")
.values({ id: newId(), user_id: userId, work_id: draftId, candidate_work_id: c.id, score: c.score, status: "pending", created_at: now })
.execute();
}
}
const MAX_ATTEMPTS = 3;
/** SQLite "UNIQUE constraint failed", or Postgres SQLSTATE 23505, anywhere in the cause chain. */
export function isUniqueViolation(err: unknown): boolean {
for (let e: unknown = err, depth = 0; e && typeof e === "object" && depth < 5; e = (e as { cause?: unknown }).cause, depth++) {
const { code, errno, message } = e as { code?: unknown; errno?: unknown; message?: unknown };
if (code === "23505" || errno === "23505") return true;
if (typeof message === "string" && (message.includes("UNIQUE constraint failed") || message.includes("duplicate key"))) {
return true;
}
}
return false;
}
/**
* Resolve a hint to a work per spec §8.4. Never auto-links on fuzzy matches; those become suggestions.
* A concurrent match may create the same work first (unique external id or parent+position); the losing
* transaction is discarded and the whole resolve re-run, which then finds the winner's rows.
*/
export async function matchWork(db: Db, userId: string, hint: WorkHint, actor: string): Promise<Resolved> {
for (let attempt = 1; ; attempt++) {
try {
return await db.k.transaction().execute((tx) => resolve(tx, userId, hint, actor));
} catch (err) {
if (attempt >= MAX_ATTEMPTS || !isUniqueViolation(err)) throw err;
}
}
}
+54
View File
@@ -0,0 +1,54 @@
import type { Db } from "@sttracker/db/db";
import type { Executor } from "./history.ts";
import { fromJson, toJson } from "@sttracker/db/json";
import { nowIso } from "@sttracker/db/ids";
import type { Visibility } from "@sttracker/db/schema";
import { VISIBILITIES } from "./entries.ts";
import { NotFoundError, ValidationError } from "./errors.ts";
export const RATING_SCALES = ["5half", "5", "10", "100"] as const;
export type RatingScale = (typeof RATING_SCALES)[number];
/** Per-user settings, stored as JSON in `users.preferences`. Keys match what `defaultVisibility()` reads. */
export interface Preferences {
ratingScale: RatingScale;
defaultVisibility: Visibility;
}
export const DEFAULT_PREFERENCES: Preferences = { ratingScale: "5half", defaultVisibility: "private" };
async function readRaw(k: Executor, userId: string, lock = false): Promise<Record<string, unknown>> {
let q = k.selectFrom("users").select("preferences").where("id", "=", userId);
if (lock) q = q.forUpdate();
const u = await q.executeTakeFirst();
if (!u) throw new NotFoundError("user");
return fromJson<Record<string, unknown>>(u.preferences, {});
}
/** The user's preferences; missing or invalid stored values fall back to the defaults. */
export async function getPreferences(db: Db, userId: string): Promise<Preferences> {
return toPreferences(await readRaw(db.k, userId));
}
function toPreferences(raw: Record<string, unknown>): Preferences {
return {
ratingScale: (RATING_SCALES as readonly unknown[]).includes(raw.ratingScale) ? (raw.ratingScale as RatingScale) : DEFAULT_PREFERENCES.ratingScale,
defaultVisibility: (VISIBILITIES as readonly unknown[]).includes(raw.defaultVisibility)
? (raw.defaultVisibility as Visibility)
: DEFAULT_PREFERENCES.defaultVisibility,
};
}
/** Merge a partial update into the stored preferences (unknown stored keys are kept). */
export async function updatePreferences(db: Db, userId: string, patch: Partial<Preferences>): Promise<Preferences> {
if (patch.ratingScale !== undefined && !RATING_SCALES.includes(patch.ratingScale)) throw new ValidationError("ratingScale", "invalid");
if (patch.defaultVisibility !== undefined && !VISIBILITIES.includes(patch.defaultVisibility)) throw new ValidationError("defaultVisibility", "invalid");
// Read-modify-write in one transaction, with the user row locked on Postgres (SQLite runs one
// connection), so concurrent updates of different keys don't drop each other.
return db.k.transaction().execute(async (tx) => {
const raw = await readRaw(tx, userId, db.dialect === "postgres");
const next = { ...raw, ...Object.fromEntries(Object.entries(patch).filter(([, v]) => v !== undefined)) };
await tx.updateTable("users").set({ preferences: toJson(next), updated_at: nowIso() }).where("id", "=", userId).execute();
return toPreferences(next);
});
}
+65
View File
@@ -0,0 +1,65 @@
import { readdir } from "node:fs/promises";
import { join } from "node:path";
import { pathToFileURL } from "node:url";
import type { PluginManifest, SourcePlugin } from "@sttracker/plugin-sdk/types";
export interface PluginRegistry {
get(id: string): SourcePlugin | undefined;
list(): PluginManifest[];
reload(): Promise<void>;
}
function validShape(p: unknown): p is SourcePlugin {
const m = (p as SourcePlugin | undefined)?.manifest;
return (
!!m &&
typeof m.id === "string" &&
m.id.trim() !== "" &&
m.sdkVersion === 1 &&
typeof m.name === "string" &&
Array.isArray(m.capabilities) &&
Array.isArray(m.config)
);
}
async function scan(dirs: string[]): Promise<Map<string, SourcePlugin>> {
const found = new Map<string, SourcePlugin>();
const stamp = Date.now();
for (const dir of dirs) {
let names: string[];
try {
names = (await readdir(dir, { withFileTypes: true })).filter((d) => d.isDirectory()).map((d) => d.name).sort();
} catch (e) {
console.warn(`plugins: cannot read ${dir}: ${e instanceof Error ? e.message : String(e)}`);
continue;
}
for (const name of names) {
const file = join(dir, name, "index.ts");
if (!(await Bun.file(file).exists())) continue;
try {
// Bun ignores the `?v=` query for cached .ts modules; evicting the cache entry is what makes reload work.
delete require.cache[file];
const mod = (await import(`${pathToFileURL(file).href}?v=${stamp}`)) as { default?: unknown };
if (!validShape(mod.default)) throw new Error("default export is not a valid plugin (manifest, sdkVersion 1)");
const id = mod.default.manifest.id;
if (found.has(id)) throw new Error(`duplicate plugin id "${id}"`);
found.set(id, mod.default);
} catch (e) {
console.warn(`plugins: skipping ${file}: ${e instanceof Error ? e.message : String(e)}`);
}
}
}
return found;
}
/** Load every `<dir>/<name>/index.ts` plugin. A bad plugin is logged and skipped, never fatal. */
export async function loadPlugins(dirs: string[]): Promise<PluginRegistry> {
let plugins = await scan(dirs);
return {
get: (id) => plugins.get(id),
list: () => [...plugins.values()].map((p) => p.manifest),
async reload() {
plugins = await scan(dirs);
},
};
}
+815
View File
@@ -0,0 +1,815 @@
import { sql } from "kysely";
import type { Db } from "@sttracker/db/db";
import { newId, nowIso } from "@sttracker/db/ids";
import { fromJson, toJson } from "@sttracker/db/json";
import type { ChangeStatus, ChangeType, Database } from "@sttracker/db/schema";
import type { EngagementFields, EntryFields, PartialDate, SourceRecord, WorkHint } from "@sttracker/plugin-sdk/types";
import { addEngagement, type EngagementPatch, updateEngagement } from "../engagements.ts";
import { type EntryPatch, fromStoredDate, resolveVisibleWork, upsertEntry } from "../entries.ts";
import { type Actor, type Executor, fieldOwner } from "../history.ts";
import { createList, replaceListItems, updateList } from "../lists.ts";
import { matchWork } from "../matching.ts";
import { resolveAliasIn, updateWorkFromHintIn } from "../works.ts";
import { decodeVersion, encodeVersion, hashRecord, type NormalizedRecord, normalizeRecord, sameValue } from "./snapshot.ts";
export interface RunCounts {
seen: number;
added: number;
changed: number;
removed: number;
merged: number;
conflicts: number;
autoApplied: number;
pending: number;
listens: number;
}
export function emptyCounts(): RunCounts {
return { seen: 0, added: 0, changed: 0, removed: 0, merged: 0, conflicts: 0, autoApplied: 0, pending: 0, listens: 0 };
}
/** Everything one run needs while applying records. */
export interface SyncState {
db: Db;
userId: string;
connectionId: string;
pluginId: string;
runId: string;
actor: Actor;
counts: RunCounts;
log: string[];
}
/**
* Run `fn` against scratch counts and log, and fold them into the run's only when `fn` resolves, so
* the run summary reflects committed work only.
*/
export async function committed(st: SyncState, fn: (local: SyncState) => Promise<void>): Promise<void> {
const local: SyncState = { ...st, counts: emptyCounts(), log: [] };
await fn(local);
for (const key of Object.keys(local.counts) as (keyof RunCounts)[]) st.counts[key] += local.counts[key];
st.log.push(...local.log);
}
type SourceRecordRow = Database["source_records"];
type Fields = Record<string, unknown>;
type ListItemValue = { workId: string; note: string | null };
/**
* `source_records.current_hash` of a record that has been matched (its work id stored as
* `target_entity "work"`) but whose apply has not committed yet. A retry reuses the stored work.
*/
const PENDING_HASH = "";
// ---------------------------------------------------------------------------------------------
// Field mapping between SourceRecord fields and the user layer's patch keys
// ---------------------------------------------------------------------------------------------
/** SDK `EntryFields` → `EntryPatch` keys (`notes` is the entry's private notes). Absent keys stay absent. */
export function entryPatchFrom(f: EntryFields | Record<string, unknown> | undefined): Fields {
const src = (f ?? {}) as EntryFields;
const out: Fields = {};
if (src.status !== undefined) out.status = src.status;
if (src.rating !== undefined) out.rating = src.rating;
if (src.review !== undefined) out.review = src.review;
if (src.notes !== undefined) out.privateNotes = src.notes;
if (src.favorite !== undefined) out.favorite = src.favorite;
if (src.tags !== undefined) out.tags = src.tags;
return out;
}
/** SDK `EngagementFields` → `EngagementPatch` keys. An "unknown"-precision date is `null`. */
export function engagementPatchFrom(f: EngagementFields | Record<string, unknown> | undefined): Fields {
const src = (f ?? {}) as EngagementFields;
const date = (d: PartialDate | null) => (d === null || d.precision === "unknown" ? null : d);
const out: Fields = {};
if (src.startedAt !== undefined) out.startedAt = date(src.startedAt);
if (src.finishedAt !== undefined) out.finishedAt = date(src.finishedAt);
if (src.progress !== undefined) out.progress = src.progress;
if (src.platform !== undefined) out.platform = src.platform;
if (src.rating !== undefined) out.rating = src.rating;
if (src.notes !== undefined) out.notes = src.notes;
return out;
}
/** A list record's own fields (items are matched separately). */
function listPatchFrom(l: Record<string, unknown> | undefined): Fields {
const out: Fields = {};
if (typeof l?.name === "string") out.name = l.name;
if (typeof l?.description === "string") out.description = l.description;
return out;
}
/** An entry row's current values under the sync-managed patch keys. */
export function entryValues(r: Database["entries"]): Fields {
return {
status: r.status,
rating: r.rating,
review: r.review,
privateNotes: r.private_notes,
favorite: Boolean(r.favorite),
tags: fromJson<string[]>(r.tags, []),
};
}
/** An engagement row's current values under the sync-managed patch keys. */
export function engagementValues(r: Database["engagements"]): Fields {
return {
startedAt: fromStoredDate(r.started_at, r.started_precision),
finishedAt: fromStoredDate(r.finished_at, r.finished_precision),
progress: fromJson<Record<string, unknown>>(r.progress, {}),
platform: r.platform,
rating: r.rating,
notes: r.notes,
};
}
/** A list's current values: name, description and ordered items. */
export async function listValues(k: Executor, r: Database["lists"]): Promise<Fields> {
const items = await k.selectFrom("list_items").select(["work_id", "note"]).where("list_id", "=", r.id).orderBy("position").orderBy("id").execute();
return { name: r.name, description: r.description, items: items.map((i): ListItemValue => ({ workId: i.work_id, note: i.note })) };
}
/**
* Lists carry no `field_sources`, so ownership is read from their history: `name`/`description` belong
* to whoever last changed them, and `items` to the user once any user edit exists, else to the source.
*/
export async function listFieldSources(k: Executor, listId: string, sourceActor: Actor): Promise<Record<string, string>> {
const rows = await k
.selectFrom("user_history")
.select(["after", "actor"])
.where("entity", "=", "list")
.where("entity_id", "=", listId)
.orderBy("created_at")
.orderBy("id")
.execute();
const fs: Record<string, string> = { items: sourceActor };
for (const r of rows) {
const after = fromJson<Fields>(r.after, {});
for (const f of ["name", "description"]) if (f in after) fs[f] = fieldOwner(r.actor);
if (fieldOwner(r.actor) === "user") fs.items = "user";
}
return fs;
}
function patchOf(n: NormalizedRecord | undefined): Fields {
if (!n) return {};
if (n.kind === "entry") return entryPatchFrom(n.entry);
if (n.kind === "engagement") return engagementPatchFrom(n.engagement);
if (n.kind === "list") return listPatchFrom(n.list);
return {};
}
// ---------------------------------------------------------------------------------------------
// Change events
// ---------------------------------------------------------------------------------------------
async function writeEvent(
k: Executor,
st: SyncState,
e: { id?: string; recordId: string; type: ChangeType; status: ChangeStatus; targetEntity: string; targetId: string | null; payload: Fields },
): Promise<void> {
await k
.insertInto("change_events")
.values({
id: e.id ?? newId(),
user_id: st.userId,
connection_id: st.connectionId,
run_id: st.runId,
record_id: e.recordId,
type: e.type,
target_entity: e.targetEntity,
target_id: e.targetId,
payload: toJson(e.payload),
status: e.status,
created_at: nowIso(),
resolved_at: null,
})
.execute();
if (e.status === "auto_applied") st.counts.autoApplied++;
if (e.status === "pending") st.counts.pending++;
if (e.status === "pending" && e.type === "conflict") st.counts.conflicts++;
}
const incomingOf = (p: Fields) => ("incoming" in p ? p.incoming : p.to);
/** Pending changed/conflict events of a record for one field. */
async function pendingFieldEvents(k: Executor, recordId: string, field: string) {
const rows = await k
.selectFrom("change_events")
.select(["id", "payload"])
.where("record_id", "=", recordId)
.where("status", "=", "pending")
.where("type", "in", ["changed", "conflict"])
.execute();
return rows.map((r) => ({ id: r.id, payload: fromJson<Fields>(r.payload, {}) })).filter((r) => r.payload.field === field);
}
/** Resolve stale pending events as `ignored`, recording why in their payload. */
async function retire(k: Executor, stale: { id: string; payload: Fields }[], extra: Fields): Promise<void> {
for (const s of stale) {
await k
.updateTable("change_events")
.set({ status: "ignored", resolved_at: nowIso(), payload: toJson({ ...s.payload, ...extra }) })
.where("id", "=", s.id)
.execute();
}
}
interface Target {
entity: "entry" | "engagement" | "list";
id: string;
current: Fields;
fieldSources: Record<string, string>;
}
/**
* Decide each incoming field against the target's field ownership (spec §8.3):
* unset or this connection → applied; `"user"` → pending `changed`; another source → pending `conflict`.
* Values equal to the current one are skipped, so the plugin never takes ownership of a field it did not change.
* Anti-flip-flop: a pending event for the same (record, field, incoming value) is never filed twice, and a
* newer incoming value supersedes an older pending one (`ignored`, `payload.supersededBy`).
* Returns the patch that may be applied, with each applied field's previous value.
*/
async function decideFields(
k: Executor,
st: SyncState,
recordId: string,
t: Target,
incoming: Fields,
): Promise<{ patch: Fields; from: Fields }> {
const patch: Fields = {};
const from: Fields = {};
for (const [field, value] of Object.entries(incoming)) {
const current = t.current[field];
const pending = await pendingFieldEvents(k, recordId, field);
if (sameValue(current, value)) {
// The source now agrees with what STTracker holds: its older proposals are moot.
await retire(k, pending, { reason: "source_agrees" });
continue;
}
const source = t.fieldSources[field];
if (source === undefined || source === st.actor) {
patch[field] = value;
from[field] = current ?? null;
continue;
}
if (pending.some((p) => sameValue(incomingOf(p.payload), value))) continue;
const id = newId();
await retire(k, pending, { supersededBy: id });
if (source === "user") {
await writeEvent(k, st, {
id,
recordId,
type: "changed",
status: "pending",
targetEntity: t.entity,
targetId: t.id,
payload: { field, from: current ?? null, to: value },
});
} else {
await writeEvent(k, st, {
id,
recordId,
type: "conflict",
status: "pending",
targetEntity: t.entity,
targetId: t.id,
payload: { field, current: current ?? null, currentSource: source, incoming: value, incomingSource: st.actor },
});
}
}
return { patch, from };
}
/** Write an allowed patch to the target as this connection's actor. */
export async function applyPatch(k: Executor, db: Db, userId: string, t: Pick<Target, "entity" | "id">, patch: Fields, actor: Actor): Promise<void> {
if (Object.keys(patch).length === 0) return;
if (t.entity === "entry") {
const row = await k.selectFrom("entries").select("work_id").where("id", "=", t.id).executeTakeFirstOrThrow();
await upsertEntry(db, userId, row.work_id, patch as Partial<EntryPatch>, actor, k);
} else if (t.entity === "engagement") {
await updateEngagement(db, userId, t.id, patch as Partial<EngagementPatch>, actor, k);
} else {
const { items, ...own } = patch;
if (Object.keys(own).length > 0) await updateList(db, userId, t.id, own, actor, k);
if (items !== undefined) await replaceListItems(db, userId, t.id, items as ListItemValue[], actor, k);
}
}
/** The live target of a mapped record, or null (unmapped, gone, or soft-deleted: never revived). */
async function loadTarget(k: Executor, st: SyncState, entity: string | null, id: string | null): Promise<Target | null> {
if (!id) return null;
if (entity === "entry") {
const r = await k
.selectFrom("entries")
.selectAll()
.where("id", "=", id)
.where("user_id", "=", st.userId)
.where("deleted_at", "is", null)
.executeTakeFirst();
return r ? { entity, id, current: entryValues(r), fieldSources: fromJson(r.field_sources, {}) } : null;
}
if (entity === "engagement") {
const r = await k
.selectFrom("engagements as g")
.innerJoin("entries as e", "e.id", "g.entry_id")
.selectAll("g")
.where("g.id", "=", id)
.where("e.user_id", "=", st.userId)
.where("g.deleted_at", "is", null)
.where("e.deleted_at", "is", null)
.executeTakeFirst();
return r ? { entity, id, current: engagementValues(r), fieldSources: fromJson(r.field_sources, {}) } : null;
}
if (entity === "list") {
const r = await k
.selectFrom("lists")
.selectAll()
.where("id", "=", id)
.where("user_id", "=", st.userId)
.where("deleted_at", "is", null)
.executeTakeFirst();
return r ? { entity, id, current: await listValues(k, r), fieldSources: await listFieldSources(k, id, st.actor) } : null;
}
return null;
}
// ---------------------------------------------------------------------------------------------
// Entry creation helpers
// ---------------------------------------------------------------------------------------------
async function findEntryRow(k: Executor, userId: string, workId: string) {
return k.selectFrom("entries").selectAll().where("user_id", "=", userId).where("work_id", "=", workId).executeTakeFirst();
}
/**
* Hierarchy ruling: when the engine creates an entry on a child work, the user also gets an entry on
* the root ancestor (status "none", plugin actor) unless one exists. An existing root entry, live or
* deleted, is never touched.
*/
async function ensureRootEntry(k: Executor, st: SyncState, workId: string): Promise<void> {
let root = workId;
for (let hop = 0; hop < 32; hop++) {
const w = await k.selectFrom("works").select("parent_id").where("id", "=", root).executeTakeFirst();
if (!w?.parent_id) break;
root = w.parent_id;
}
if (root === workId) return;
root = await resolveAliasIn(k, st.userId, root);
if (await findEntryRow(k, st.userId, root)) return;
await upsertEntry(st.db, st.userId, root, {}, st.actor, k);
}
/**
* The user's live entry for a work, created (with root ancestor) when missing. Null when the user
* deleted it: syncs never revive.
*/
async function liveEntryFor(
k: Executor,
st: SyncState,
workId: string,
patch: Fields,
): Promise<{ row: Database["entries"]; created: boolean } | null> {
const work = await resolveVisibleWork(k, st.userId, workId);
const row = await findEntryRow(k, st.userId, work.id);
if (row) return row.deleted_at === null ? { row, created: false } : null;
const e = await upsertEntry(st.db, st.userId, work.id, patch as Partial<EntryPatch>, st.actor, k);
await ensureRootEntry(k, st, work.id);
const created = await k.selectFrom("entries").selectAll().where("id", "=", e.id).executeTakeFirstOrThrow();
return { row: created, created: true };
}
// ---------------------------------------------------------------------------------------------
// Engagement de-duplication (spec §8.3: same play from two sources)
// ---------------------------------------------------------------------------------------------
const DAY_MS = 86_400_000;
const hasTime = (v: string) => v.length > 10 && v[10] === "T";
const dayNumber = (v: string) => Date.UTC(Number(v.slice(0, 4)), Number(v.slice(5, 7)) - 1, Number(v.slice(8, 10))) / DAY_MS;
/**
* Finish dates "within 24 h". Two datetimes compare as instants. Otherwise day-precision values compare
* as calendar dates (a datetime's date part is its first 10 characters) and match when at most one day
* apart. Month/year/unknown precision never matches.
*/
export function finishedWithin24h(a: PartialDate, b: PartialDate): boolean {
if (a.precision !== "day" || b.precision !== "day") return false;
if (hasTime(a.value) && hasTime(b.value)) {
const x = Date.parse(a.value);
const y = Date.parse(b.value);
if (!Number.isNaN(x) && !Number.isNaN(y)) return Math.abs(x - y) <= DAY_MS;
}
return Math.abs(dayNumber(a.value) - dayNumber(b.value)) <= 1;
}
/** Equal start datetimes (both carry a time). Compared as instants, falling back to string equality. */
export function sameStartInstant(a: PartialDate, b: PartialDate): boolean {
if (!hasTime(a.value) || !hasTime(b.value)) return false;
const x = Date.parse(a.value);
const y = Date.parse(b.value);
return Number.isNaN(x) || Number.isNaN(y) ? a.value === b.value : x === y;
}
/**
* A live engagement on the entry that is the same play, not already reported by this connection.
* Also returns the date field that matched (its values differ only within the dedupe tolerance).
*/
async function findDuplicateEngagement(k: Executor, st: SyncState, entryId: string, incoming: Fields) {
const fin = incoming.finishedAt as PartialDate | null | undefined;
const start = incoming.startedAt as PartialDate | null | undefined;
if (!fin && !start) return null;
const rows = await k
.selectFrom("engagements")
.selectAll()
.where("entry_id", "=", entryId)
.where("deleted_at", "is", null)
.where((eb) =>
eb.not(
eb.exists(
eb
.selectFrom("engagement_sources as s")
.select("s.id")
.whereRef("s.engagement_id", "=", "engagements.id")
.where("s.connection_id", "=", st.connectionId),
),
),
)
.orderBy("id")
.execute();
for (const r of rows) {
const v = engagementValues(r);
const f = v.finishedAt as PartialDate | null;
const s = v.startedAt as PartialDate | null;
if (fin && f && finishedWithin24h(fin, f)) return { row: r, matchedOn: "finishedAt" };
if (start && s && sameStartInstant(start, s)) return { row: r, matchedOn: "startedAt" };
}
return null;
}
const isEmptyValue = (v: unknown) =>
v === null || v === undefined || (typeof v === "object" && !Array.isArray(v) && Object.keys(v as object).length === 0);
// ---------------------------------------------------------------------------------------------
// List items
// ---------------------------------------------------------------------------------------------
/**
* Match list item hints to works, outside any transaction. An ID-less hint first reuses the user's own
* live private work with the same media type and title, so re-matching never piles up drafts.
*/
async function matchListItems(st: SyncState, items: { work: WorkHint; note?: string }[]): Promise<ListItemValue[]> {
const out: ListItemValue[] = [];
const seen = new Set<string>();
for (const it of items) {
let workId: string | undefined;
if (Object.keys(it.work.externalIds).length === 0) {
workId = (
await st.db.k
.selectFrom("works")
.select("id")
.where("owner_user_id", "=", st.userId)
.where("merged_into_id", "is", null)
.where("media_type", "=", it.work.mediaType)
.where(sql<boolean>`lower(title) = ${it.work.title.toLowerCase()}`)
.orderBy("id")
.executeTakeFirst()
)?.id;
}
workId ??= (await matchWork(st.db, st.userId, it.work, st.actor)).workId;
if (seen.has(workId)) continue;
seen.add(workId);
out.push({ workId, note: it.note ?? null });
}
return out;
}
// ---------------------------------------------------------------------------------------------
// Records
// ---------------------------------------------------------------------------------------------
/**
* Snapshot one non-listen record and apply it. Call outside any transaction (matchWork has its own).
* Counts land in `st` only once the record's work has committed.
*/
export function applyRecord(st: SyncState, rec: SourceRecord): Promise<void> {
return committed(st, async (local) => {
const hash = hashRecord(rec);
const existing = await local.db.k
.selectFrom("source_records")
.selectAll()
.where("connection_id", "=", local.connectionId)
.where("key", "=", rec.key)
.executeTakeFirst();
if (!existing || existing.current_hash === PENDING_HASH) return addRecord(local, rec, hash, existing ?? null);
if (existing.current_hash === hash) {
await local.db.k.transaction().execute(async (tx) => {
await tx.updateTable("source_records").set({ last_seen: nowIso() }).where("id", "=", existing.id).execute();
if (existing.deleted_at !== null && rec.deleted !== true) await reappeared(tx, existing);
});
return;
}
return changeRecord(local, existing, rec, hash);
});
}
async function addRecord(st: SyncState, rec: SourceRecord, hash: string, pendingRow: SourceRecordRow | null): Promise<void> {
const applicable = rec.deleted !== true && !rec.mergedIntoKey && rec.kind !== "listen";
let recordId = pendingRow?.id ?? null;
let workId: string | null = null;
let items: ListItemValue[] = [];
if (applicable && (rec.kind === "entry" || rec.kind === "engagement")) {
if (pendingRow?.target_entity === "work" && pendingRow.target_id) {
workId = pendingRow.target_id;
} else {
// Matching runs in its own transaction (it retries on unique violations). Its result is committed
// right away, so a failed apply is retried against the same work instead of a fresh draft.
workId = (await matchWork(st.db, st.userId, rec.work, st.actor)).workId;
recordId = await persistMatched(st, rec, workId, recordId);
}
} else if (applicable && rec.kind === "list") {
items = await matchListItems(st, rec.list.items);
}
await st.db.k.transaction().execute(async (tx) => {
const now = nowIso();
const row = {
current_hash: hash,
last_seen: now,
deleted_at: rec.deleted === true ? now : null,
merged_into_key: rec.mergedIntoKey ?? null,
target_entity: rec.kind === "listen" ? null : rec.kind,
target_id: null,
};
if (recordId) {
await tx.updateTable("source_records").set(row).where("id", "=", recordId).execute();
} else {
recordId = newId();
await tx
.insertInto("source_records")
.values({ id: recordId, connection_id: st.connectionId, key: rec.key, kind: rec.kind, first_seen: now, ...row })
.execute();
}
const id = recordId;
await tx.insertInto("source_record_versions").values({ id: newId(), record_id: id, hash, raw_gz: encodeVersion(rec), seen_at: now }).execute();
if (!applicable) return;
let targetId: string | null = null;
if (rec.kind === "entry" && workId) targetId = await addEntryRecord(tx, st, id, rec.key, workId, entryPatchFrom(rec.entry));
else if (rec.kind === "engagement" && workId) targetId = await addEngagementRecord(tx, st, id, rec.key, workId, engagementPatchFrom(rec.engagement));
else if (rec.kind === "list") targetId = await addListRecord(tx, st, id, rec.key, rec.list.name, items);
if (targetId) {
await tx.updateTable("source_records").set({ target_id: targetId }).where("id", "=", id).execute();
st.counts.added++;
}
});
}
/** Commit the record row with its matched work (`target_entity "work"`, pending hash) before applying. */
async function persistMatched(st: SyncState, rec: SourceRecord, workId: string, recordId: string | null): Promise<string> {
const now = nowIso();
if (recordId) {
await st.db.k.updateTable("source_records").set({ target_entity: "work", target_id: workId, last_seen: now }).where("id", "=", recordId).execute();
return recordId;
}
const id = newId();
await st.db.k
.insertInto("source_records")
.values({
id,
connection_id: st.connectionId,
key: rec.key,
kind: rec.kind,
current_hash: PENDING_HASH,
first_seen: now,
last_seen: now,
deleted_at: null,
merged_into_key: null,
target_entity: "work",
target_id: workId,
})
.execute();
return id;
}
async function addEntryRecord(k: Executor, st: SyncState, recordId: string, key: string, workId: string, incoming: Fields): Promise<string | null> {
const live = await liveEntryFor(k, st, workId, incoming);
if (!live) {
st.log.push(`record ${key}: the user deleted this entry; not revived`);
return null;
}
let applied = Object.keys(incoming);
if (!live.created) {
// The user (or another source) already has this entry: decide field by field.
const t: Target = { entity: "entry", id: live.row.id, current: entryValues(live.row), fieldSources: fromJson(live.row.field_sources, {}) };
const { patch } = await decideFields(k, st, recordId, t, incoming);
await applyPatch(k, st.db, st.userId, t, patch, st.actor);
applied = Object.keys(patch);
}
await writeEvent(k, st, {
recordId,
type: "added",
status: "auto_applied",
targetEntity: "entry",
targetId: live.row.id,
payload: { key, kind: "entry", fields: applied },
});
return live.row.id;
}
async function addEngagementRecord(k: Executor, st: SyncState, recordId: string, key: string, workId: string, incoming: Fields): Promise<string | null> {
const live = await liveEntryFor(k, st, workId, {});
if (!live) {
st.log.push(`record ${key}: the user deleted this entry; engagement not added`);
return null;
}
const dup = await findDuplicateEngagement(k, st, live.row.id, incoming);
let engagementId: string;
let payload: Fields;
if (dup) {
// Same play from another source: attach. Fields nobody has set yet are filled; any other differing
// field goes through the per-field rules (conflict / pending changed). The date that identified the
// play differs only within the dedupe tolerance and is left alone.
const t: Target = {
entity: "engagement",
id: dup.row.id,
current: engagementValues(dup.row),
fieldSources: fromJson(dup.row.field_sources, {}),
};
const fill: Fields = {};
const rest: Fields = {};
for (const [f, v] of Object.entries(incoming)) {
if (f === dup.matchedOn) continue;
if (t.fieldSources[f] === undefined && isEmptyValue(t.current[f]) && !isEmptyValue(v)) fill[f] = v;
else rest[f] = v;
}
const { patch } = await decideFields(k, st, recordId, t, rest);
await applyPatch(k, st.db, st.userId, t, { ...fill, ...patch }, st.actor);
engagementId = dup.row.id;
payload = { key, kind: "engagement", fields: Object.keys({ ...fill, ...patch }), dedupedInto: dup.row.id };
} else {
const eg = await addEngagement(st.db, st.userId, live.row.id, incoming as Partial<EngagementPatch>, st.actor, k);
engagementId = eg.id;
payload = { key, kind: "engagement", fields: Object.keys(incoming) };
}
await k
.insertInto("engagement_sources")
.values({ id: newId(), engagement_id: engagementId, connection_id: st.connectionId, record_key: key })
.onConflict((oc) => oc.columns(["connection_id", "record_key"]).doNothing())
.execute();
await writeEvent(k, st, { recordId, type: "added", status: "auto_applied", targetEntity: "engagement", targetId: engagementId, payload });
return engagementId;
}
async function addListRecord(k: Executor, st: SyncState, recordId: string, key: string, name: string, items: ListItemValue[]): Promise<string> {
const list = await createList(st.db, st.userId, { name, description: `Imported from ${st.pluginId}`, kind: "list" }, st.actor, k);
if (items.length > 0) await replaceListItems(st.db, st.userId, list.id, items, st.actor, k);
await writeEvent(k, st, {
recordId,
type: "added",
status: "auto_applied",
targetEntity: "list",
targetId: list.id,
payload: { key, kind: "list", fields: ["name", "description", "items"] },
});
return list.id;
}
async function changeRecord(st: SyncState, existing: SourceRecordRow, rec: SourceRecord, hash: string): Promise<void> {
const prevRow = await st.db.k
.selectFrom("source_record_versions")
.select("raw_gz")
.where("record_id", "=", existing.id)
.where("hash", "=", existing.current_hash)
.orderBy("seen_at", "desc")
.orderBy("id", "desc")
.executeTakeFirst();
const prev = prevRow ? decodeVersion(prevRow.raw_gz).normalized : undefined;
const next = normalizeRecord(rec);
// List items are matched before the transaction (matchWork has its own), and only when they changed.
const itemsChanged = rec.kind === "list" && !rec.deleted && !sameValue(prev?.list?.items, next.list?.items);
const items = itemsChanged && rec.kind === "list" ? await matchListItems(st, rec.list.items) : null;
await st.db.k.transaction().execute(async (tx) => {
const now = nowIso();
// A merged record that is later reported deleted stays "merged": its disappearance is expected.
const mergedInto = rec.mergedIntoKey ?? (rec.deleted === true ? existing.merged_into_key : null);
await tx.insertInto("source_record_versions").values({ id: newId(), record_id: existing.id, hash, raw_gz: encodeVersion(rec), seen_at: now }).execute();
await tx
.updateTable("source_records")
.set({ current_hash: hash, last_seen: now, merged_into_key: mergedInto })
.where("id", "=", existing.id)
.execute();
if (rec.deleted === true) {
if (existing.deleted_at === null) await markRemoved(tx, st, { ...existing, merged_into_key: mergedInto });
return;
}
st.counts.changed++;
if (existing.deleted_at !== null) await reappeared(tx, existing);
if (rec.mergedIntoKey && rec.mergedIntoKey !== (prev?.mergedIntoKey ?? null)) await fileMerged(tx, st, existing, rec.mergedIntoKey);
const target = await loadTarget(tx, st, existing.target_entity, existing.target_id);
if (!target) return; // unmapped, or the user deleted it: never revived
const before = patchOf(prev);
const after = patchOf(next);
const incoming: Fields = {};
for (const [f, v] of Object.entries(after)) if (!sameValue(before[f], v)) incoming[f] = v;
if (items) incoming.items = items;
const { patch, from } = await decideFields(tx, st, existing.id, target, incoming);
await applyPatch(tx, st.db, st.userId, target, patch, st.actor);
for (const [field, to] of Object.entries(patch)) {
await writeEvent(tx, st, {
recordId: existing.id,
type: "changed",
status: "auto_applied",
targetEntity: target.entity,
targetId: target.id,
payload: { field, from: from[field], to },
});
}
// The source's description of the work changed: refresh the (shared) work it maps to; never re-match.
if ((rec.kind === "entry" || rec.kind === "engagement") && prev?.work !== undefined && !sameValue(prev.work, next.work)) {
const workId = await targetWorkId(tx, target);
if (workId) await updateWorkFromHintIn(tx, workId, rec.work, st.actor);
}
});
}
async function targetWorkId(k: Executor, t: Target): Promise<string | null> {
if (t.entity === "list") return null;
const entryId =
t.entity === "entry"
? t.id
: (await k.selectFrom("engagements").select("entry_id").where("id", "=", t.id).executeTakeFirst())?.entry_id;
if (!entryId) return null;
return (await k.selectFrom("entries").select("work_id").where("id", "=", entryId).executeTakeFirst())?.work_id ?? null;
}
/** The record is back at the source: clear `deleted_at` and retire still-pending removal events for it. */
async function reappeared(k: Executor, rec: SourceRecordRow): Promise<void> {
await k.updateTable("source_records").set({ deleted_at: null }).where("id", "=", rec.id).execute();
await k
.updateTable("change_events")
.set({ status: "ignored", resolved_at: nowIso() })
.where("record_id", "=", rec.id)
.where("type", "=", "removed")
.where("status", "=", "pending")
.execute();
}
/**
* Set `source_records.deleted_at` and file a pending `removed` event when there is a live target.
* A record merged into another (its `merged_into_key` set, or a pending `merged` event) is expected
* to vanish: no removal is counted or filed for it, ever.
*/
export async function markRemoved(k: Executor, st: SyncState, rec: SourceRecordRow): Promise<void> {
await k.updateTable("source_records").set({ deleted_at: nowIso() }).where("id", "=", rec.id).execute();
if (rec.merged_into_key) return;
const pendingMerge = await k
.selectFrom("change_events")
.select("id")
.where("record_id", "=", rec.id)
.where("type", "=", "merged")
.where("status", "=", "pending")
.executeTakeFirst();
if (pendingMerge) return;
st.counts.removed++;
const target = await loadTarget(k, st, rec.target_entity, rec.target_id);
if (!target) return;
const already = await k
.selectFrom("change_events")
.select("id")
.where("record_id", "=", rec.id)
.where("type", "=", "removed")
.where("status", "=", "pending")
.executeTakeFirst();
if (already) return;
await writeEvent(k, st, {
recordId: rec.id,
type: "removed",
status: "pending",
targetEntity: target.entity,
targetId: target.id,
payload: { key: rec.key, kind: rec.kind },
});
}
/** A pending `merged` event on the entry this record maps to (an engagement record's entry). */
async function fileMerged(k: Executor, st: SyncState, rec: SourceRecordRow, intoKey: string): Promise<void> {
st.counts.merged++;
const target = await loadTarget(k, st, rec.target_entity, rec.target_id);
if (!target || target.entity === "list") return;
const entryId =
target.entity === "entry"
? target.id
: (await k.selectFrom("engagements").select("entry_id").where("id", "=", target.id).executeTakeFirstOrThrow()).entry_id;
await writeEvent(k, st, {
recordId: rec.id,
type: "merged",
status: "pending",
targetEntity: "entry",
targetId: entryId,
payload: { key: rec.key, intoKey },
});
}
+257
View File
@@ -0,0 +1,257 @@
import type { Db } from "@sttracker/db/db";
import { nowIso } from "@sttracker/db/ids";
import { fromJson, toJson } from "@sttracker/db/json";
import type { ChangeStatus, ChangeType, Database } from "@sttracker/db/schema";
import type { WorkHint } from "@sttracker/plugin-sdk/types";
import { type EngagementPatch, softDeleteEngagement, updateEngagement } from "../engagements.ts";
import { type EntryPatch, moveEntryToWork, softDeleteEntry, upsertEntry } from "../entries.ts";
import { NotFoundError, ValidationError } from "../errors.ts";
import type { Actor, Executor } from "../history.ts";
import { softDeleteList } from "../lists.ts";
import { matchWork } from "../matching.ts";
import { resolveAliasIn } from "../works.ts";
import { applyPatch, engagementValues, entryValues } from "./apply.ts";
import { decodeVersion } from "./snapshot.ts";
export interface ChangeEvent {
id: string;
type: ChangeType;
status: ChangeStatus;
connectionId: string | null;
runId: string | null;
recordId: string | null;
targetEntity: string;
targetId: string | null;
payload: Record<string, unknown>;
createdAt: string;
resolvedAt: string | null;
}
function toChange(r: Database["change_events"]): ChangeEvent {
return {
id: r.id,
type: r.type,
status: r.status,
connectionId: r.connection_id,
runId: r.run_id,
recordId: r.record_id,
targetEntity: r.target_entity,
targetId: r.target_id,
payload: fromJson<Record<string, unknown>>(r.payload, {}),
createdAt: r.created_at,
resolvedAt: r.resolved_at,
};
}
/**
* The user's change events, newest first. `status` defaults to "pending" (the inbox); "all" includes
* auto-applied and resolved ones. `cursor` is the `nextCursor` of the previous page.
*/
export async function listChanges(
db: Db,
userId: string,
opts: { status?: "pending" | "all"; connectionId?: string; limit?: number; cursor?: string } = {},
): Promise<{ items: ChangeEvent[]; nextCursor: string | null }> {
const limit = Math.min(Math.max(opts.limit ?? 50, 1), 200);
let q = db.k.selectFrom("change_events").selectAll().where("user_id", "=", userId);
if ((opts.status ?? "pending") === "pending") q = q.where("status", "=", "pending");
if (opts.connectionId) q = q.where("connection_id", "=", opts.connectionId);
if (opts.cursor) q = q.where("id", "<", opts.cursor);
const rows = await q.orderBy("id", "desc").limit(limit + 1).execute();
const items = rows.slice(0, limit).map(toChange);
return { items, nextCursor: rows.length > limit ? items[items.length - 1]!.id : null };
}
const STATUS_FOR = { accept: "accepted", ignore: "ignored", keep_mine: "kept_mine" } as const;
/**
* Resolve a pending event.
* - `accept`: changed/conflict applies the incoming value as the event's plugin actor (which then owns
* the field); removed soft-deletes the target (an engagement other sources still report only loses
* this connection's `engagement_sources` row); merged moves the entry to the merge target's work.
* - `keep_mine`: changed/conflict set `field_sources[field] = "user"`, so that source stops overriding it.
* - `ignore`: data untouched.
* All three set `status` and `resolved_at`. Accept/keep_mine on a target that is gone (deleted, or merged
* away with no live survivor) resolves the event as `ignored` with `payload.reason = "target_gone"`.
*/
export async function resolveChange(
db: Db,
userId: string,
changeId: string,
action: "accept" | "ignore" | "keep_mine",
): Promise<void> {
if (!Object.hasOwn(STATUS_FOR, action)) throw new ValidationError("action", "invalid");
const row = await db.k.selectFrom("change_events").selectAll().where("id", "=", changeId).where("user_id", "=", userId).executeTakeFirst();
if (!row) throw new NotFoundError("change");
if (row.status !== "pending") throw new ValidationError("status", "not_pending");
const ev = toChange(row);
const actor = await actorFor(db, ev.connectionId);
// matchWork runs its own transaction; resolve the merge target before opening ours.
const mergeTarget = action === "accept" && ev.type === "merged" ? await mergeTargetWork(db, userId, ev, actor) : null;
await db.k.transaction().execute(async (tx) => {
// Claim the event first so a concurrent resolve of the same event becomes a no-op failure.
const claimed = await tx
.updateTable("change_events")
.set({ status: STATUS_FOR[action], resolved_at: nowIso() })
.where("id", "=", ev.id)
.where("status", "=", "pending")
.executeTakeFirst();
if (Number(claimed.numUpdatedRows) === 0) throw new ValidationError("status", "not_pending");
if (action === "ignore") return;
const isField = ev.type === "changed" || ev.type === "conflict";
if (action === "keep_mine" && !isField) return;
const targetId = await liveTargetId(tx, userId, ev);
if (!targetId) {
await tx
.updateTable("change_events")
.set({ status: "ignored", payload: toJson({ ...ev.payload, reason: "target_gone" }) })
.where("id", "=", ev.id)
.execute();
return;
}
if (isField) {
const field = String(ev.payload.field);
const value = action === "keep_mine" ? undefined : ev.type === "conflict" ? ev.payload.incoming : ev.payload.to;
await setField(db, tx, userId, ev.targetEntity, targetId, field, value, action === "keep_mine" ? "user" : actor);
return;
}
if (ev.type === "removed") return removeTarget(db, tx, userId, ev, targetId, actor);
if (ev.type === "merged" && mergeTarget) return moveTo(db, tx, userId, targetId, mergeTarget, actor);
});
}
async function actorFor(db: Db, connectionId: string | null): Promise<Actor> {
if (!connectionId) return "user";
const c = await db.k.selectFrom("source_connections").select(["id", "plugin_id"]).where("id", "=", connectionId).executeTakeFirst();
return c ? `plugin:${c.plugin_id}:${c.id}` : "user";
}
/**
* The event's target if it is still live. A soft-deleted entry that was merged into another (the user's
* work alias leads to a work with a live entry) resolves to that survivor. Otherwise null.
*/
async function liveTargetId(k: Executor, userId: string, ev: ChangeEvent): Promise<string | null> {
if (!ev.targetId) return null;
if (ev.targetEntity === "entry") {
const r = await k.selectFrom("entries").select(["id", "work_id", "deleted_at"]).where("id", "=", ev.targetId).where("user_id", "=", userId).executeTakeFirst();
if (!r) return null;
if (r.deleted_at === null) return r.id;
const to = await resolveAliasIn(k, userId, r.work_id);
if (to === r.work_id) return null;
const survivor = await k
.selectFrom("entries")
.select("id")
.where("user_id", "=", userId)
.where("work_id", "=", to)
.where("deleted_at", "is", null)
.executeTakeFirst();
return survivor?.id ?? null;
}
if (ev.targetEntity === "engagement") {
const r = await k
.selectFrom("engagements as g")
.innerJoin("entries as e", "e.id", "g.entry_id")
.select("g.id")
.where("g.id", "=", ev.targetId)
.where("e.user_id", "=", userId)
.where("g.deleted_at", "is", null)
.where("e.deleted_at", "is", null)
.executeTakeFirst();
return r?.id ?? null;
}
if (ev.targetEntity === "list") {
const r = await k
.selectFrom("lists")
.select("id")
.where("id", "=", ev.targetId)
.where("user_id", "=", userId)
.where("deleted_at", "is", null)
.executeTakeFirst();
return r?.id ?? null;
}
return null;
}
/** Write `value` to a live target's field as `actor`; with `value` undefined, re-stamp the current value. */
async function setField(db: Db, k: Executor, userId: string, entity: string, id: string, field: string, value: unknown, actor: Actor): Promise<void> {
if (entity === "entry") {
const r = await k.selectFrom("entries").selectAll().where("id", "=", id).executeTakeFirstOrThrow();
const v = value === undefined ? entryValues(r)[field] : value;
await upsertEntry(db, userId, r.work_id, { [field]: v } as Partial<EntryPatch>, actor, k);
} else if (entity === "engagement") {
const r = await k.selectFrom("engagements").selectAll().where("id", "=", id).executeTakeFirstOrThrow();
const v = value === undefined ? engagementValues(r)[field] : value;
await updateEngagement(db, userId, id, { [field]: v } as Partial<EngagementPatch>, actor, k);
} else if (entity === "list") {
// Lists have no field_sources: ownership comes from history, so keep_mine needs no write.
if (value === undefined) return;
await applyPatch(k, db, userId, { entity: "list", id }, { [field]: value }, actor);
} else {
throw new ValidationError("targetEntity", "unsupported");
}
}
async function removeTarget(db: Db, k: Executor, userId: string, ev: ChangeEvent, targetId: string, actor: Actor): Promise<void> {
if (ev.targetEntity === "entry") return softDeleteEntry(db, userId, targetId, actor, k);
if (ev.targetEntity === "list") return softDeleteList(db, userId, targetId, actor, k);
if (ev.targetEntity === "engagement") {
// The same play may still be reported by other sources: detach this one, delete only when none remain.
if (ev.connectionId) {
await k.deleteFrom("engagement_sources").where("engagement_id", "=", targetId).where("connection_id", "=", ev.connectionId).execute();
}
const left = await k.selectFrom("engagement_sources").select("id").where("engagement_id", "=", targetId).executeTakeFirst();
if (left) return;
await softDeleteEngagement(db, userId, targetId, actor, k);
}
}
/**
* The work a merged event moves its entry to. An admin merge carries `toWorkId`. A source merge names
* the target record (`intoKey`): its mapped entry's work when it has one, else the work matched for
* the target record's latest hint.
*/
async function mergeTargetWork(db: Db, userId: string, ev: ChangeEvent, actor: Actor): Promise<string> {
if (typeof ev.payload.toWorkId === "string") return ev.payload.toWorkId;
const intoKey = ev.payload.intoKey;
if (!ev.connectionId || typeof intoKey !== "string") throw new ValidationError("payload", "no_merge_target");
const target = await db.k
.selectFrom("source_records")
.selectAll()
.where("connection_id", "=", ev.connectionId)
.where("key", "=", intoKey)
.executeTakeFirst();
if (!target) throw new ValidationError("intoKey", "unknown_record");
// A record matched but not yet applied stores its work directly.
if (target.target_entity === "work" && target.target_id) return target.target_id;
if (target.target_id && (target.target_entity === "entry" || target.target_entity === "engagement")) {
const entryId =
target.target_entity === "entry"
? target.target_id
: (await db.k.selectFrom("engagements").select("entry_id").where("id", "=", target.target_id).executeTakeFirst())?.entry_id;
const entry = entryId
? await db.k.selectFrom("entries").select("work_id").where("id", "=", entryId).where("user_id", "=", userId).executeTakeFirst()
: undefined;
if (entry) return entry.work_id;
}
const version = await db.k
.selectFrom("source_record_versions")
.select("raw_gz")
.where("record_id", "=", target.id)
.where("hash", "=", target.current_hash)
.orderBy("seen_at", "desc")
.orderBy("id", "desc")
.executeTakeFirst();
const hint = version ? (decodeVersion(version.raw_gz).normalized.work as WorkHint | undefined) : undefined;
if (!hint) throw new ValidationError("intoKey", "no_work_hint");
return (await matchWork(db, userId, hint, actor)).workId;
}
/** Move the live entry to the work; `moveEntryToWork` re-points source records if it merges. */
async function moveTo(db: Db, k: Executor, userId: string, entryId: string, workId: string, actor: Actor): Promise<void> {
const entry = await k.selectFrom("entries").select("work_id").where("id", "=", entryId).executeTakeFirstOrThrow();
if (entry.work_id === workId) return;
await moveEntryToWork(db, userId, entryId, workId, actor, k);
}
+95
View File
@@ -0,0 +1,95 @@
import type { Db } from "@sttracker/db/db";
import { newId, nowIso } from "@sttracker/db/ids";
import { toJson } from "@sttracker/db/json";
import type { ListenFields, SourceRecord } from "@sttracker/plugin-sdk/types";
import type { Actor } from "../history.ts";
import { isUniqueViolation } from "../matching.ts";
import { createWork, findWorkByExternalIds } from "../works.ts";
/** The one external-ID namespace the engine names itself (spec §6.4). */
export const RECORDING_NAMESPACE = "musicbrainz_recording";
/**
* The shared track work for a MusicBrainz recording: an existing work carrying the id, else a new
* shared `track` work. Never renames an existing work, and never creates drafts.
*/
async function trackWork(db: Db, recording: string, l: ListenFields, actor: Actor): Promise<string> {
const ids = { [RECORDING_NAMESPACE]: recording };
for (let attempt = 1; ; attempt++) {
const found = await findWorkByExternalIds(db, ids);
if (found) return found;
try {
return await createWork(
db,
{ hint: { title: l.track.trim() || "Untitled track", mediaType: "track", externalIds: ids, creators: l.artist ? [{ name: l.artist }] : [] } },
actor,
);
} catch (err) {
// A concurrent ingest created it first: look again.
if (attempt >= 3 || !isUniqueViolation(err)) throw err;
}
}
}
/** Listens per query/transaction. 500 rows × 12 columns stays well under both dialects' parameter limits. */
const BATCH = 500;
/**
* Append listens (spec §6.4): insert-or-ignore on (user, source, source_key), where `source_key` is the
* record key. A work is linked only through `mbids.recording`. Non-listen records are ignored.
* No change events are written. Per batch: one lookup of existing keys, each distinct recording resolved
* once (outside the transaction; `createWork` has its own), then one multi-row insert.
*/
export async function ingestListens(
db: Db,
userId: string,
source: string,
records: SourceRecord[],
actor: Actor = `plugin:${source}:ingest`,
): Promise<{ inserted: number }> {
const listens = records.filter((r): r is Extract<SourceRecord, { kind: "listen" }> => r.kind === "listen");
let inserted = 0;
for (let i = 0; i < listens.length; i += BATCH) {
const batch = new Map<string, ListenFields>();
for (const r of listens.slice(i, i + BATCH)) if (!batch.has(r.key)) batch.set(r.key, r.listen);
const existing = await db.k
.selectFrom("listens")
.select("source_key")
.where("user_id", "=", userId)
.where("source", "=", source)
.where("source_key", "in", [...batch.keys()])
.execute();
for (const e of existing) batch.delete(e.source_key);
if (batch.size === 0) continue;
const works = new Map<string, string>();
for (const l of batch.values()) {
const rec = l.mbids?.recording;
if (rec && !works.has(rec)) works.set(rec, await trackWork(db, rec, l, actor));
}
const now = nowIso();
const rows = [...batch].map(([key, l]) => ({
id: newId(),
user_id: userId,
listened_at: l.listenedAt,
artist: l.artist,
track: l.track,
album: l.album ?? null,
duration_ms: l.durationMs ?? null,
mbids: toJson(l.mbids ?? {}),
work_id: l.mbids?.recording ? works.get(l.mbids.recording)! : null,
source,
source_key: key,
created_at: now,
}));
const res = await db.k.transaction().execute((tx) =>
tx
.insertInto("listens")
.values(rows)
.onConflict((oc) => oc.columns(["user_id", "source", "source_key"]).doNothing())
.executeTakeFirst(),
);
inserted += Number(res.numInsertedOrUpdatedRows ?? 0n);
}
return { inserted };
}
+164
View File
@@ -0,0 +1,164 @@
import type { Db } from "@sttracker/db/db";
import { newId, nowIso } from "@sttracker/db/ids";
import { fromJson, toJson } from "@sttracker/db/json";
import type { ConnectionStatus, Database, RunMode } from "@sttracker/db/schema";
import type { PluginContext, SourcePlugin, SourceRecord } from "@sttracker/plugin-sdk/types";
import { NotFoundError, ValidationError } from "../errors.ts";
import type { Actor } from "../history.ts";
import { applyRecord, committed, emptyCounts, markRemoved, type RunCounts, type SyncState } from "./apply.ts";
import { ingestListens } from "./listens.ts";
/** A `source_connections` row with JSON parsed. Credentials stay encrypted; the caller's `makeContext` decrypts. */
export interface Connection {
id: string;
userId: string;
pluginId: string;
name: string;
config: Record<string, unknown>;
credentialsEnc: string | null;
scheduleMinutes: number | null;
cursor: string | null;
status: ConnectionStatus;
lastRunAt: string | null;
createdAt: string;
}
export interface SyncDeps {
db: Db;
/** Builds the plugin context (http client, credentials, cursor, logging). Injected by the caller. */
makeContext: (conn: Connection, runId: string) => PluginContext;
}
export type SyncMode = "full" | "changes" | { file: { name: string; bytes: Uint8Array } };
export interface RunSummary {
runId: string;
status: "ok" | "failed";
counts: RunCounts;
error?: string;
}
const LISTEN_BATCH = 500;
function toConnection(r: Database["source_connections"]): Connection {
return {
id: r.id,
userId: r.user_id,
pluginId: r.plugin_id,
name: r.name,
config: fromJson<Record<string, unknown>>(r.config, {}),
credentialsEnc: r.credentials_enc,
scheduleMinutes: r.schedule_minutes,
cursor: r.cursor,
status: r.status,
lastRunAt: r.last_run_at,
createdAt: r.created_at,
};
}
function iterate(plugin: SourcePlugin, ctx: PluginContext, mode: SyncMode): AsyncIterable<SourceRecord> {
const id = plugin.manifest.id;
if (mode === "full") {
if (!plugin.pull) throw new Error(`plugin "${id}" cannot pull`);
return plugin.pull(ctx);
}
if (mode === "changes") {
if (!plugin.pullChanges) throw new Error(`plugin "${id}" cannot pull changes`);
return plugin.pullChanges(ctx);
}
if (!plugin.importFile) throw new Error(`plugin "${id}" cannot import files`);
return plugin.importFile(ctx, mode.file);
}
const message = (e: unknown) => (e instanceof Error ? e.message : String(e));
/**
* One sync run (spec §8.3): pull records, snapshot and diff each against its previous version, apply
* or file change events, and append listens. Absent records are marked removed only after a `full`
* pull whose iteration finished and whose records all applied. Any failure marks the run `failed` and
* the connection `error`; the summary is returned, not thrown.
*/
export async function runSync(deps: SyncDeps, plugin: SourcePlugin, connectionId: string, mode: SyncMode): Promise<RunSummary> {
const { db } = deps;
const row = await db.k.selectFrom("source_connections").selectAll().where("id", "=", connectionId).executeTakeFirst();
if (!row) throw new NotFoundError("connection");
if (row.plugin_id !== plugin.manifest.id) throw new ValidationError("plugin", "connection_plugin_mismatch");
const conn = toConnection(row);
const runId = newId();
const runMode: RunMode = typeof mode === "string" ? mode : "file";
await db.k
.insertInto("source_runs")
.values({ id: runId, connection_id: conn.id, mode: runMode, started_at: nowIso(), finished_at: null, status: "running", counts: toJson(emptyCounts()), log: "", error: null })
.execute();
const actor: Actor = `plugin:${conn.pluginId}:${conn.id}`;
const st: SyncState = { db, userId: conn.userId, connectionId: conn.id, pluginId: conn.pluginId, runId, actor, counts: emptyCounts(), log: [] };
const seen = new Set<string>();
const errors: string[] = [];
let pullError: string | null = null;
let listens: SourceRecord[] = [];
const flushListens = async () => {
if (listens.length === 0) return;
const batch = listens;
listens = [];
try {
st.counts.listens += (await ingestListens(db, conn.userId, conn.pluginId, batch, actor)).inserted;
} catch (e) {
errors.push(`listens: ${message(e)}`);
}
};
try {
const ctx = deps.makeContext(conn, runId);
for await (const rec of iterate(plugin, ctx, mode)) {
st.counts.seen++;
if (rec.kind === "listen") {
listens.push(rec);
if (listens.length >= LISTEN_BATCH) await flushListens();
continue;
}
seen.add(rec.key);
try {
await applyRecord(st, rec);
} catch (e) {
errors.push(`record ${rec.key}: ${message(e)}`);
}
}
} catch (e) {
pullError = message(e);
}
await flushListens();
// Removal by absence: only a completed full pull with every record applied can prove a record is gone.
if (mode === "full" && pullError === null && errors.length === 0) {
try {
const live = await db.k
.selectFrom("source_records")
.selectAll()
.where("connection_id", "=", conn.id)
.where("deleted_at", "is", null)
.execute();
for (const r of live) {
if (seen.has(r.key)) continue;
await committed(st, (local) => db.k.transaction().execute((tx) => markRemoved(tx, local, r)));
}
} catch (e) {
errors.push(`removal: ${message(e)}`);
}
}
const failed = pullError !== null || errors.length > 0;
const error = pullError ?? (errors.length > 0 ? `${errors.length} error(s); first: ${errors[0]}` : undefined);
const log = [...st.log, ...errors];
if (pullError !== null) log.push(`pull failed: ${pullError}`);
const now = nowIso();
await db.k
.updateTable("source_runs")
.set({ finished_at: now, status: failed ? "failed" : "ok", counts: toJson(st.counts), log: log.join("\n"), error: error ?? null })
.where("id", "=", runId)
.execute();
const nextStatus: ConnectionStatus = failed ? "error" : conn.status === "error" ? "active" : conn.status;
await db.k.updateTable("source_connections").set({ last_run_at: now, status: nextStatus }).where("id", "=", conn.id).execute();
return { runId, status: failed ? "failed" : "ok", counts: st.counts, ...(error !== undefined ? { error } : {}) };
}
+75
View File
@@ -0,0 +1,75 @@
import type { SourceRecord } from "@sttracker/plugin-sdk/types";
/**
* The part of a `SourceRecord` that the change engine compares: everything except `key` and `raw`.
* `deleted` is always a boolean and `mergedIntoKey` always present (null when unset), so an absent
* flag and an explicit `false` hash the same.
*/
export interface NormalizedRecord {
kind: SourceRecord["kind"];
work?: unknown;
entry?: Record<string, unknown>;
engagement?: Record<string, unknown>;
listen?: Record<string, unknown>;
list?: Record<string, unknown>;
deleted: boolean;
mergedIntoKey: string | null;
}
/** JSON with object keys sorted at every depth; `undefined` members are dropped, as in `JSON.stringify`. */
export function stableStringify(v: unknown): string {
return JSON.stringify(sortKeys(v)) ?? "null";
}
function sortKeys(v: unknown): unknown {
if (Array.isArray(v)) return v.map((x) => (x === undefined ? null : sortKeys(x)));
if (v === null || typeof v !== "object") return v;
const out: Record<string, unknown> = {};
for (const k of Object.keys(v).sort()) {
const x = (v as Record<string, unknown>)[k];
if (x !== undefined) out[k] = sortKeys(x);
}
return out;
}
/** True when two values serialise identically (`undefined` counts as `null`). */
export function sameValue(a: unknown, b: unknown): boolean {
return stableStringify(a ?? null) === stableStringify(b ?? null);
}
export function normalizeRecord(r: SourceRecord): NormalizedRecord {
const base = { kind: r.kind, deleted: r.deleted === true, mergedIntoKey: r.mergedIntoKey ?? null };
const plain = (o: unknown) => JSON.parse(stableStringify(o)) as Record<string, unknown>;
switch (r.kind) {
case "entry":
return { ...base, work: plain(r.work), entry: plain(r.entry) };
case "engagement":
return { ...base, work: plain(r.work), engagement: plain(r.engagement) };
case "listen":
return { ...base, listen: plain(r.listen) };
case "list":
return { ...base, list: plain(r.list) };
}
}
/** sha256 (hex) of the stable JSON of the normalised record. The raw payload is not part of it. */
export function hashRecord(r: SourceRecord): string {
return new Bun.CryptoHasher("sha256").update(stableStringify(normalizeRecord(r))).digest("hex");
}
/** What a `source_record_versions.raw_gz` holds: the normalised record (for diffing) and the raw payload. */
export interface StoredVersion {
normalized: NormalizedRecord;
raw: unknown;
}
/** gzip (`Bun.gzipSync`) + base64 of `{ normalized, raw }`. */
export function encodeVersion(r: SourceRecord): string {
const json = JSON.stringify({ normalized: normalizeRecord(r), raw: r.raw ?? null });
return Buffer.from(Bun.gzipSync(new TextEncoder().encode(json))).toString("base64");
}
export function decodeVersion(s: string): StoredVersion {
const bytes = Bun.gunzipSync(Buffer.from(s, "base64"));
return JSON.parse(new TextDecoder().decode(bytes)) as StoredVersion;
}
+330
View File
@@ -0,0 +1,330 @@
import { sql, type Kysely } from "kysely";
import type { MediaType, WorkHint } from "@sttracker/plugin-sdk/types";
import type { Db } from "@sttracker/db/db";
import { newId, nowIso } from "@sttracker/db/ids";
import { fromJson, toJson } from "@sttracker/db/json";
import type { Database, WorksTable } from "@sttracker/db/schema";
import { AuthError, NotFoundError, ValidationError } from "./errors.ts";
export type Creator = { name: string; role?: string };
export type Position = { season?: number; episode?: number; index?: number };
/** A `works` row with JSON columns parsed. Other fields keep their column names. */
export interface Work extends Omit<WorksTable, "creators" | "metadata" | "position"> {
creators: Creator[];
metadata: Record<string, unknown>;
position: Position | null;
}
/** Any Kysely handle: the root one or a transaction. */
export type K = Kysely<Database>;
/** Normalised JSON for `works.position`: keys in a fixed order (season, episode, index), or null when empty. */
export function normalizePosition(p: Position | undefined | null): string | null {
if (!p) return null;
const o: Position = {};
if (p.season !== undefined) o.season = p.season;
if (p.episode !== undefined) o.episode = p.episode;
if (p.index !== undefined) o.index = p.index;
return Object.keys(o).length === 0 ? null : toJson(o);
}
export function toWork(r: WorksTable): Work {
return {
...r,
creators: fromJson<Creator[]>(r.creators, []),
metadata: fromJson<Record<string, unknown>>(r.metadata, {}),
position: fromJson<Position | null>(r.position, null),
};
}
export async function createWorkIn(
k: K,
input: { hint: WorkHint; ownerUserId?: string | null; parentId?: string | null },
actor: string,
): Promise<string> {
const { hint } = input;
if (hint.title.trim() === "") throw new ValidationError("title", "required");
if (input.ownerUserId && Object.keys(hint.externalIds).length > 0) {
throw new ValidationError("externalIds", "private works cannot carry external ids");
}
const id = newId();
const now = nowIso();
await k
.insertInto("works")
.values({
id,
media_type: hint.mediaType,
title: hint.title,
original_title: "",
year: hint.year ?? null,
release_date: null,
creators: toJson(hint.creators ?? []),
description: hint.description ?? "",
cover_url: hint.coverUrl ?? "",
cover_attachment_id: null,
metadata: "{}",
parent_id: input.parentId ?? null,
position: normalizePosition(hint.position),
owner_user_id: input.ownerUserId ?? null,
merged_into_id: null,
created_at: now,
updated_at: now,
})
.execute();
for (const [namespace, value] of Object.entries(hint.externalIds)) {
await k.insertInto("external_ids").values({ id: newId(), work_id: id, namespace, value }).execute();
}
await k
.insertInto("work_history")
.values({ id: newId(), work_id: id, before: "{}", after: toJson({ created: true, title: hint.title }), actor, created_at: now })
.execute();
return id;
}
/** Insert the work and its external IDs. */
export function createWork(
db: Db,
input: { hint: WorkHint; ownerUserId?: string | null; parentId?: string | null },
actor: string,
): Promise<string> {
return db.k.transaction().execute((tx) => createWorkIn(tx, input, actor));
}
export async function getWorkIn(k: K, id: string): Promise<Work | null> {
const seen = new Set<string>();
let cur: string | null = id;
while (cur && !seen.has(cur)) {
seen.add(cur);
const row: WorksTable | undefined = await k.selectFrom("works").selectAll().where("id", "=", cur).executeTakeFirst();
if (!row) return null;
if (!row.merged_into_id) return toWork(row);
cur = row.merged_into_id;
}
return null;
}
/** Fetch a work, following `merged_into_id` tombstones to the surviving one. */
export function getWork(db: Db, id: string): Promise<Work | null> {
return getWorkIn(db.k, id);
}
/** Follow `merged_into_id` to the surviving id (the input id if it is not merged). */
export async function finalWorkId(k: K, id: string): Promise<string> {
return (await getWorkIn(k, id))?.id ?? id;
}
/**
* First work matching any (namespace, value) pair, following merges. Only works visible to `userId`
* count (shared, or private to that user); with no user, only shared works. Others are treated as absent.
*/
export async function findWorkByExternalIdsIn(
k: K,
ids: Record<string, string>,
userId: string | null = null,
): Promise<string | null> {
for (const [namespace, value] of Object.entries(ids)) {
const row = await k
.selectFrom("external_ids")
.select("work_id")
.where("namespace", "=", namespace)
.where("value", "=", value)
.executeTakeFirst();
if (!row) continue;
const work = await getWorkIn(k, row.work_id);
if (work && (work.owner_user_id === null || work.owner_user_id === userId)) return work.id;
}
return null;
}
/** First work visible to `userId` (shared works only when omitted) matching any (namespace, value) pair. */
export function findWorkByExternalIds(db: Db, ids: Record<string, string>, userId: string | null = null): Promise<string | null> {
return findWorkByExternalIdsIn(db.k, ids, userId);
}
async function addMissingExternalIds(k: K, workId: string, ids: Record<string, string>): Promise<string[]> {
const added: string[] = [];
for (const [namespace, value] of Object.entries(ids)) {
const exists = await k
.selectFrom("external_ids")
.select("id")
.where("namespace", "=", namespace)
.where("value", "=", value)
.executeTakeFirst();
if (exists) continue;
// A concurrent sync may have added the same pair since the check: skip it rather than fail.
const res = await k
.insertInto("external_ids")
.values({ id: newId(), work_id: workId, namespace, value })
.onConflict((oc) => oc.columns(["namespace", "value"]).doNothing())
.executeTakeFirst();
if (Number(res.numInsertedOrUpdatedRows ?? 0n) > 0) added.push(`${namespace}:${value}`);
}
return added;
}
export async function updateWorkFromHintIn(k: K, workId: string, hint: WorkHint, actor: string): Promise<void> {
const row = await k.selectFrom("works").selectAll().where("id", "=", workId).executeTakeFirst();
if (!row || row.owner_user_id !== null) return; // private works are never touched by syncs
const before: Record<string, unknown> = {};
const after: Record<string, unknown> = {};
const patch: Partial<WorksTable> = {};
const set = <F extends "title" | "year" | "creators" | "description" | "cover_url">(field: F, next: WorksTable[F] | undefined, empty: boolean) => {
if (next === undefined || empty || next === row[field]) return;
before[field] = row[field];
after[field] = next;
patch[field] = next;
};
set("title", hint.title, hint.title.trim() === "");
set("year", hint.year, hint.year === undefined);
set("creators", hint.creators ? toJson(hint.creators) : undefined, !hint.creators || hint.creators.length === 0);
set("description", hint.description, !hint.description);
set("cover_url", hint.coverUrl, !hint.coverUrl);
const addedIds = await addMissingExternalIds(k, workId, hint.externalIds);
if (addedIds.length > 0) after.externalIds = addedIds;
if (Object.keys(after).length === 0) return;
const now = nowIso();
if (Object.keys(patch).length > 0) {
await k.updateTable("works").set({ ...patch, updated_at: now }).where("id", "=", workId).execute();
}
await k
.insertInto("work_history")
.values({ id: newId(), work_id: workId, before: toJson(before), after: toJson(after), actor, created_at: now })
.execute();
}
/** Fill or update a shared work's display fields from a hint, add missing external IDs, log to `work_history`. */
export function updateWorkFromHint(db: Db, workId: string, hint: WorkHint, actor: string): Promise<void> {
return db.k.transaction().execute((tx) => updateWorkFromHintIn(tx, workId, hint, actor));
}
function escapeLike(s: string): string {
return s.replace(/[!%_]/g, (c) => `!${c}`);
}
/** Shared works plus the user's own private ones, case-insensitive substring match on title. */
export async function searchWorks(
db: Db,
userId: string,
q: string,
opts: { mediaType?: MediaType; limit?: number } = {},
): Promise<Work[]> {
let qb = db.k
.selectFrom("works")
.selectAll()
.where("merged_into_id", "is", null)
.where((eb) => eb.or([eb("owner_user_id", "is", null), eb("owner_user_id", "=", userId)]))
.where(sql<boolean>`lower(title) like ${`%${escapeLike(q.toLowerCase())}%`} escape '!'`);
if (opts.mediaType) qb = qb.where("media_type", "=", opts.mediaType);
const rows = await qb.orderBy("title").limit(opts.limit ?? 50).execute();
return rows.map(toWork);
}
/** Follow the user's aliases and global merge tombstones to the work id they should see. */
export async function resolveAliasIn(k: K, userId: string, workId: string): Promise<string> {
const seen = new Set<string>();
let cur = workId;
while (!seen.has(cur)) {
seen.add(cur);
const alias = await k
.selectFrom("work_aliases")
.select("to_work_id")
.where("user_id", "=", userId)
.where("from_work_id", "=", cur)
.executeTakeFirst();
if (alias) {
cur = alias.to_work_id;
continue;
}
const w = await k.selectFrom("works").select("merged_into_id").where("id", "=", cur).executeTakeFirst();
if (!w?.merged_into_id) break;
cur = w.merged_into_id;
}
return cur;
}
export function resolveAlias(db: Db, userId: string, workId: string): Promise<string> {
return resolveAliasIn(db.k, userId, workId);
}
/**
* Admin-only global merge of shared work `fromId` into `toId`. Leaves a `merged_into_id` tombstone and
* moves external IDs and the children whose position `toId` does not already have. Both works must
* be shared. Users without an entry on `toId` get their entry repointed; users who already
* have one keep their `from` entry and get a pending `merged` change event.
*/
export async function adminMergeWorks(db: Db, adminUserId: string, fromId: string, toId: string): Promise<void> {
await db.k.transaction().execute(async (tx) => {
const admin = await tx.selectFrom("users").select("role").where("id", "=", adminUserId).executeTakeFirst();
if (admin?.role !== "admin") throw new AuthError("forbidden");
if (fromId === toId) throw new ValidationError("toId", "cannot merge a work into itself");
const from = await tx.selectFrom("works").selectAll().where("id", "=", fromId).executeTakeFirst();
const to = await tx.selectFrom("works").selectAll().where("id", "=", toId).executeTakeFirst();
if (!from) throw new NotFoundError("work");
if (!to) throw new NotFoundError("work");
if (from.merged_into_id) throw new ValidationError("fromId", "already merged");
if (to.merged_into_id) throw new ValidationError("toId", "already merged");
if (from.owner_user_id !== null) throw new ValidationError("fromId", "only shared works can be merged");
if (to.owner_user_id !== null) throw new ValidationError("toId", "only shared works can be merged");
const now = nowIso();
await tx.updateTable("works").set({ merged_into_id: toId, updated_at: now }).where("id", "=", fromId).execute();
const moved = await tx.selectFrom("external_ids").select(["namespace", "value"]).where("work_id", "=", fromId).execute();
await tx.updateTable("external_ids").set({ work_id: toId }).where("work_id", "=", fromId).execute();
// Repoint children so matching through `toId` finds them. A child whose position `toId` already
// has stays under `fromId` (unique parent_id+position); it remains reachable via the tombstone.
const toPositions = new Set(
(await tx.selectFrom("works").select("position").where("parent_id", "=", toId).where("position", "is not", null).execute()).map(
(r) => r.position,
),
);
const children = await tx.selectFrom("works").select(["id", "position"]).where("parent_id", "=", fromId).execute();
const movedChildren: string[] = [];
for (const c of children) {
if (c.position !== null && toPositions.has(c.position)) continue;
await tx.updateTable("works").set({ parent_id: toId, updated_at: now }).where("id", "=", c.id).execute();
movedChildren.push(c.id);
}
const entries = await tx.selectFrom("entries").select(["id", "user_id"]).where("work_id", "=", fromId).execute();
for (const e of entries) {
const existing = await tx
.selectFrom("entries")
.select("id")
.where("user_id", "=", e.user_id)
.where("work_id", "=", toId)
.executeTakeFirst();
if (existing) {
await tx
.insertInto("change_events")
.values({
id: newId(),
user_id: e.user_id,
connection_id: null,
run_id: null,
record_id: null,
type: "merged",
status: "pending",
target_entity: "entry",
target_id: e.id,
payload: toJson({ fromWorkId: fromId, toWorkId: toId }),
created_at: now,
resolved_at: null,
})
.execute();
} else {
await tx.updateTable("entries").set({ work_id: toId, updated_at: now }).where("id", "=", e.id).execute();
}
}
await tx
.insertInto("work_history")
.values({ id: newId(), work_id: fromId, before: toJson({ merged_into_id: null }), after: toJson({ merged_into_id: toId }), actor: adminUserId, created_at: now })
.execute();
await tx
.insertInto("work_history")
.values({ id: newId(), work_id: toId, before: "{}", after: toJson({ mergedFrom: fromId, externalIds: moved, children: movedChildren }), actor: adminUserId, created_at: now })
.execute();
});
}
+249
View File
@@ -0,0 +1,249 @@
import { expect, spyOn, test } from "bun:test";
import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { newId, nowIso } from "@sttracker/db/ids";
import { forEachDialect } from "@sttracker/db/testing";
import { definePlugin, type SourcePlugin } from "@sttracker/plugin-sdk/types";
import { listAttachmentLinks, readAttachment, requireLinkTarget, storeAttachment } from "../src/attachments.ts";
import { authenticateAppToken, createAppToken, listAppTokens, revokeAppToken } from "../src/auth.ts";
import { getHistory } from "../src/history.ts";
import { fieldOwner } from "../src/history.ts";
import { getEntry, upsertEntry } from "../src/entries.ts";
import { listEngagements } from "../src/engagements.ts";
import { createConnection, deleteConnection, devicePoll, deviceStart, getConnection, listConnections, listRuns, syncHandler, updateConnection } from "../src/connections.ts";
import { startScheduler } from "../src/jobs.ts";
import { DropJobError } from "../src/jobs.ts";
import { listSuggestions, quickAdd } from "../src/entries.ts";
import { addEngagement } from "../src/engagements.ts";
import { NotFoundError, ValidationError } from "../src/errors.ts";
import type { PluginRegistry } from "../src/registry.ts";
import { loadPlugins } from "../src/registry.ts";
import { makeUser, makeWork } from "./user-layer-fixtures.ts";
const SECRET = "test-secret";
const warn = spyOn(console, "warn").mockImplementation(() => {});
const reg = await loadPlugins([join(import.meta.dir, "fixtures/plugins")]);
warn.mockRestore();
const echo = reg.get("echo")!;
const fileish: SourcePlugin = definePlugin({
manifest: { id: "fileish", name: "Fileish", version: "1", sdkVersion: 1, mediaTypes: ["book"], capabilities: ["importFile"], config: [] },
async *importFile(_ctx, file) {
yield {
kind: "entry",
key: file.name,
raw: new TextDecoder().decode(file.bytes),
work: { title: new TextDecoder().decode(file.bytes), mediaType: "book", externalIds: { isbn13: "9780000000009" } },
entry: { status: "planned" },
};
},
deviceAuth: {
start: async () => ({ userCode: "ABCD", verificationUrl: "https://x.test/device", deviceCode: "dc", intervalSec: 5, expiresInSec: 600 }),
poll: async (_ctx, deviceCode) => (deviceCode === "dc" ? { access_token: "tok" } : "pending"),
},
});
let flipStatus: "planned" | "completed" = "planned";
const flip: SourcePlugin = definePlugin({
manifest: { id: "flip", name: "Flip", version: "1", sdkVersion: 1, mediaTypes: ["book"], capabilities: ["pull"], config: [] },
async *pull() {
yield {
kind: "entry",
key: "flip-1",
raw: flipStatus,
work: { title: "Flip", mediaType: "book", externalIds: { isbn13: "9780000000010" } },
entry: { status: flipStatus, rating: flipStatus === "planned" ? 40 : 90 },
};
},
});
const extra: Record<string, SourcePlugin> = { fileish, flip };
const reg2: PluginRegistry = { get: (id) => extra[id] ?? reg.get(id), list: () => [...reg.list(), fileish.manifest, flip.manifest], reload: async () => {} };
forEachDialect("api support", (getDb) => {
test("listAppTokens returns metadata only, revoked flagged, scoped to user", async () => {
const db = getDb();
const u = await makeUser(db);
const other = await makeUser(db, "bob");
const t = await createAppToken(db, u, "script", ["read"]);
await createAppToken(db, other, "x", ["write"]);
expect(await revokeAppToken(db, other, t.id)).toBe(false);
expect(await revokeAppToken(db, u, t.id)).toBe(true);
expect(await revokeAppToken(db, u, t.id)).toBe(false);
const list = await listAppTokens(db, u);
expect(list).toHaveLength(1);
expect(list[0]).toMatchObject({ id: t.id, name: "script", scopes: ["read"], lastUsedAt: null });
expect(list[0]!.revokedAt).not.toBeNull();
expect(JSON.stringify(list)).not.toContain("hash");
});
test("fieldOwner: only plugin actors keep their name in field_sources", () => {
expect(fieldOwner("user")).toBe("user");
expect(fieldOwner("token:script")).toBe("user");
expect(fieldOwner("plugin:echo:c1")).toBe("plugin:echo:c1");
});
test("authenticateAppToken returns the user and the token name", async () => {
const db = getDb();
const u = await makeUser(db);
const t = await createAppToken(db, u, "script", ["write"]);
expect(await authenticateAppToken(db, t.token, "write")).toMatchObject({ user: { id: u }, tokenName: "script" });
expect(await authenticateAppToken(db, t.token, "read")).toBeNull();
});
test("a token write keeps its actor in history, is owned by the user, and a later sync files a pending change", async () => {
const db = getDb();
const u = await makeUser(db);
flipStatus = "planned";
const id = await createConnection(db, SECRET, u, { pluginId: "flip", name: "F", config: {} }, flip.manifest);
const sync = syncHandler({ db, secret: SECRET, registry: reg2 });
await sync({ connectionId: id, mode: "full" });
const entry = await db.k.selectFrom("entries").selectAll().where("user_id", "=", u).executeTakeFirstOrThrow();
await upsertEntry(db, u, entry.work_id, { status: "dropped" }, "token:script");
const view = await getEntry(db, u, entry.id);
expect(view.fieldSources.status).toBe("user");
expect(view.fieldSources.rating).toBe(`plugin:flip:${id}`);
const hist = await getHistory(db, u, "entry", entry.id);
expect(hist[0]!.actor).toBe("token:script");
flipStatus = "completed";
await sync({ connectionId: id, mode: "full" });
const after = await getEntry(db, u, entry.id);
expect(after.status).toBe("dropped");
expect(after.rating).toBe(90);
const pending = await db.k.selectFrom("change_events").selectAll().where("user_id", "=", u).where("status", "=", "pending").execute();
expect(pending.map((p) => [p.type, JSON.parse(p.payload).field])).toEqual([["changed", "status"]]);
});
test("token-authored engagement fields are owned by the user", async () => {
const db = getDb();
const u = await makeUser(db);
const e = await quickAdd(db, u, { title: "Dune" });
await addEngagement(db, u, e.id, { platform: "kindle" }, "token:script");
const [g] = await listEngagements(db, u, e.id);
expect(g!.fieldSources).toEqual({ platform: "user" });
});
test("listSuggestions lists pending suggestions for a visible work", async () => {
const db = getDb();
const u = await makeUser(db);
const draft = await quickAdd(db, u, { title: "Dune??", mediaType: "book" });
const good = await makeWork(db, { title: "Dune", externalIds: { isbn13: "1" } });
await db.k
.insertInto("match_suggestions")
.values({ id: newId(), user_id: u, work_id: draft.workId, candidate_work_id: good, score: 0.75, status: "pending", created_at: nowIso() })
.execute();
const s = await listSuggestions(db, u, draft.workId);
expect(s).toHaveLength(1);
expect(s[0]).toMatchObject({ workId: draft.workId, candidateWorkId: good, score: 0.75, status: "pending" });
const other = await makeUser(db, "bob");
await expect(listSuggestions(db, other, draft.workId)).rejects.toBeInstanceOf(NotFoundError);
// Asking by an id the user aliased away lists the canonical work's suggestions.
const old = await makeWork(db, { title: "Old Dune", ownerUserId: u });
await db.k.insertInto("work_aliases").values({ id: newId(), user_id: u, from_work_id: old, to_work_id: draft.workId }).execute();
expect((await listSuggestions(db, u, old)).map((x) => x.workId)).toEqual([draft.workId]);
});
test("requireLinkTarget checks ownership per entity; links and filename are readable", async () => {
const db = getDb();
const dir = await mkdtemp(join(tmpdir(), "stt-att-"));
try {
const u = await makeUser(db);
const bob = await makeUser(db, "bob");
const e = await quickAdd(db, u, { title: "Dune" });
const g = await addEngagement(db, u, e.id, {}, "user");
await requireLinkTarget(db, u, "entry", e.id);
await requireLinkTarget(db, u, "engagement", g.id);
await requireLinkTarget(db, u, "work", e.workId);
await expect(requireLinkTarget(db, bob, "entry", e.id)).rejects.toBeInstanceOf(NotFoundError);
await expect(requireLinkTarget(db, bob, "engagement", g.id)).rejects.toBeInstanceOf(NotFoundError);
await expect(requireLinkTarget(db, bob, "work", e.workId)).rejects.toBeInstanceOf(NotFoundError);
await expect(requireLinkTarget(db, u, "nope" as never, e.id)).rejects.toBeInstanceOf(ValidationError);
const a = await storeAttachment(db, dir, u, { bytes: new Uint8Array([1]), mime: "image/png", filename: "a.png" }, { entity: "entry", entityId: e.id });
const links = await listAttachmentLinks(db, u, a.id);
expect(links).toEqual([{ id: expect.any(String), entity: "entry", entityId: e.id }]);
expect(await listAttachmentLinks(db, bob, a.id)).toEqual([]);
expect((await readAttachment(db, dir, u, a.id))?.filename).toBe("a.png");
} finally {
await rm(dir, { recursive: true, force: true });
}
});
test("getConnection hides other users and deleted connections; listRuns is newest first", async () => {
const db = getDb();
const u = await makeUser(db);
const bob = await makeUser(db, "bob");
const id = await createConnection(db, SECRET, u, { pluginId: "echo", name: "E", config: {} }, echo.manifest);
expect((await getConnection(db, u, id)).pluginId).toBe("echo");
await expect(getConnection(db, bob, id)).rejects.toBeInstanceOf(NotFoundError);
const sync = syncHandler({ db, secret: SECRET, registry: reg });
await sync({ connectionId: id, mode: "full" });
await sync({ connectionId: id, mode: "full" });
const runs = await listRuns(db, u, id);
expect(runs).toHaveLength(2);
expect(runs[0]!.startedAt >= runs[1]!.startedAt).toBe(true);
expect(runs[0]).toMatchObject({ mode: "full", status: "ok" });
await expect(listRuns(db, bob, id)).rejects.toBeInstanceOf(NotFoundError);
await deleteConnection(db, u, id);
await expect(getConnection(db, u, id)).rejects.toBeInstanceOf(NotFoundError);
});
test("a paused connection without credentials stays listed and can be unpaused; a deleted one is gone", async () => {
const db = getDb();
const u = await makeUser(db);
const id = await createConnection(db, SECRET, u, { pluginId: "echo", name: "E", config: {} }, echo.manifest);
await updateConnection(db, SECRET, u, id, { status: "paused" }, echo.manifest);
expect((await listConnections(db, u)).map((c) => c.id)).toEqual([id]);
expect((await getConnection(db, u, id)).status).toBe("paused");
await updateConnection(db, SECRET, u, id, { status: "active" }, echo.manifest);
expect((await getConnection(db, u, id)).status).toBe("active");
await deleteConnection(db, u, id);
const row = await db.k.selectFrom("source_connections").selectAll().where("id", "=", id).executeTakeFirstOrThrow();
expect(row).toMatchObject({ status: "paused", credentials_enc: null });
expect(row.deleted_at).not.toBeNull();
expect(await listConnections(db, u)).toEqual([]);
await expect(getConnection(db, u, id)).rejects.toBeInstanceOf(NotFoundError);
await expect(updateConnection(db, SECRET, u, id, { status: "active" }, echo.manifest)).rejects.toBeInstanceOf(NotFoundError);
await expect(deleteConnection(db, u, id)).rejects.toBeInstanceOf(NotFoundError);
});
test("deleted connections are never scheduled and their queued syncs are dropped", async () => {
const db = getDb();
const u = await makeUser(db);
const id = await createConnection(db, SECRET, u, { pluginId: "echo", name: "E", config: {}, scheduleMinutes: 1 }, echo.manifest);
// Simulate a delete that left status "error" (e.g. a row edited by hand): deleted_at alone must exclude it.
await deleteConnection(db, u, id);
await db.k.updateTable("source_connections").set({ status: "error" }).where("id", "=", id).execute();
const sched = startScheduler(db, { tickMs: 3_600_000 });
await sched.tickNow();
sched.stop();
expect(await db.k.selectFrom("jobs").select("id").execute()).toEqual([]);
const sync = syncHandler({ db, secret: SECRET, registry: reg });
await expect(sync({ connectionId: id })).rejects.toBeInstanceOf(DropJobError);
});
test("syncHandler runs file mode from a base64 payload", async () => {
const db = getDb();
const u = await makeUser(db);
const id = await createConnection(db, SECRET, u, { pluginId: "fileish", name: "F", config: {} }, fileish.manifest);
const sync = syncHandler({ db, secret: SECRET, registry: reg2 });
const s = await sync({ connectionId: id, file: { name: "x.txt", base64: Buffer.from("Dune").toString("base64") } });
expect(s).toMatchObject({ status: "ok" });
expect((await listRuns(db, u, id))[0]!.mode).toBe("file");
const entries = await db.k.selectFrom("entries").select("id").where("user_id", "=", u).execute();
expect(entries).toHaveLength(1);
});
test("deviceStart and devicePoll store credentials on success", async () => {
const db = getDb();
const u = await makeUser(db);
const id = await createConnection(db, SECRET, u, { pluginId: "fileish", name: "F", config: {} }, fileish.manifest);
expect(await deviceStart(db, SECRET, reg2, u, id)).toMatchObject({ userCode: "ABCD", deviceCode: "dc" });
expect(await devicePoll(db, SECRET, reg2, u, id, "other")).toBe("pending");
expect(await devicePoll(db, SECRET, reg2, u, id, "dc")).toBe("connected");
expect(await getConnection(db, u, id)).toMatchObject({ hasCredentials: true });
const e = await createConnection(db, SECRET, u, { pluginId: "echo", name: "E", config: {} }, echo.manifest);
await expect(deviceStart(db, SECRET, reg2, u, e)).rejects.toBeInstanceOf(ValidationError);
});
});
+173
View File
@@ -0,0 +1,173 @@
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
import { mkdtemp, readdir, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { newId, nowIso } from "@sttracker/db/ids";
import { forEachDialect } from "@sttracker/db/testing";
import type { Db } from "@sttracker/db/db";
import {
listAttachments,
readAttachment,
storeAttachment,
unlinkAttachment,
} from "../src/attachments.ts";
import { ValidationError } from "../src/errors.ts";
async function addUser(db: Db, username: string): Promise<string> {
const id = newId();
const now = nowIso();
await db.k
.insertInto("users")
.values({
id,
username,
display_name: username,
password_hash: "x",
role: "member",
preferences: "{}",
created_at: now,
updated_at: now,
})
.execute();
return id;
}
async function filesUnder(dir: string): Promise<string[]> {
const all = await readdir(dir, { recursive: true, withFileTypes: true });
return all.filter((e) => e.isFile()).map((e) => join(e.parentPath, e.name));
}
forEachDialect("attachments", (getDb) => {
let dataDir: string;
beforeEach(async () => {
dataDir = await mkdtemp(join(tmpdir(), "sttracker-att-"));
});
afterEach(async () => {
await rm(dataDir, { recursive: true, force: true });
});
const png = (s: string) => ({
bytes: new TextEncoder().encode(s),
mime: "image/png",
filename: "a.png",
});
test("same bytes twice: one file, two links", async () => {
const db = getDb();
const u = await addUser(db, "u1");
const a = await storeAttachment(db, dataDir, u, png("hello"), { entity: "work", entityId: "w1" });
const b = await storeAttachment(db, dataDir, u, png("hello"), { entity: "entry", entityId: "e1" });
expect(a.sha256).toBe(b.sha256);
expect(a.id).not.toBe(b.id);
const files = await filesUnder(join(dataDir, "attachments"));
expect(files).toHaveLength(1);
expect(files[0]).toBe(
join(dataDir, "attachments", a.sha256.slice(0, 2), a.sha256.slice(2, 4), a.sha256),
);
const links = await db.k.selectFrom("attachment_links").selectAll().execute();
expect(links).toHaveLength(2);
expect(await listAttachments(db, u, "work", "w1")).toHaveLength(1);
});
test("rejects disallowed mime and oversize", async () => {
const db = getDb();
const u = await addUser(db, "u1");
const link = { entity: "work", entityId: "w1" } as const;
await expect(
storeAttachment(db, dataDir, u, { ...png("x"), mime: "application/x-msdownload", filename: "a.exe" }, link),
).rejects.toBeInstanceOf(ValidationError);
await expect(
storeAttachment(db, dataDir, u, { ...png("x"), bytes: new Uint8Array(25 * 1024 * 1024 + 1) }, link),
).rejects.toBeInstanceOf(ValidationError);
});
test("reads own file; another user gets null", async () => {
const db = getDb();
const u1 = await addUser(db, "u1");
const u2 = await addUser(db, "u2");
const a = await storeAttachment(db, dataDir, u1, png("secret"), { entity: "work", entityId: "w1" });
const r = await readAttachment(db, dataDir, u1, a.id);
expect(r?.mime).toBe("image/png");
expect(await r?.file.text()).toBe("secret");
expect(await readAttachment(db, dataDir, u2, a.id)).toBeNull();
expect(await readAttachment(db, dataDir, u1, "nope")).toBeNull();
});
test("unlink removes link, orphan row and unshared file", async () => {
const db = getDb();
const u = await addUser(db, "u1");
const a = await storeAttachment(db, dataDir, u, png("bye"), { entity: "work", entityId: "w1" });
const b = await storeAttachment(db, dataDir, u, png("bye"), { entity: "work", entityId: "w2" });
await unlinkAttachment(db, dataDir, u, a.id, "work", "w1");
expect(await listAttachments(db, u, "work", "w1")).toHaveLength(0);
expect(await filesUnder(join(dataDir, "attachments"))).toHaveLength(1);
await unlinkAttachment(db, dataDir, u, b.id, "work", "w2");
expect(await filesUnder(join(dataDir, "attachments"))).toHaveLength(0);
expect(await db.k.selectFrom("attachments").selectAll().execute()).toHaveLength(0);
});
test("rejects svg and normalizes mime", async () => {
const db = getDb();
const u = await addUser(db, "u1");
const link = { entity: "work", entityId: "w1" } as const;
for (const mime of ["image/svg+xml", "IMAGE/SVG+XML; charset=utf-8"]) {
await expect(storeAttachment(db, dataDir, u, { ...png("x"), mime }, link)).rejects.toBeInstanceOf(
ValidationError,
);
}
const a = await storeAttachment(db, dataDir, u, { ...png("x"), mime: "Image/PNG; foo=bar" }, link);
expect(a.mime).toBe("image/png");
expect((await listAttachments(db, u, "work", "w1"))[0]?.mime).toBe("image/png");
});
test("shared bytes across users survive the other's unlink", async () => {
const db = getDb();
const ua = await addUser(db, "a");
const ub = await addUser(db, "b");
const a = await storeAttachment(db, dataDir, ua, png("same"), { entity: "work", entityId: "w1" });
const b = await storeAttachment(db, dataDir, ub, png("same"), { entity: "work", entityId: "w1" });
await unlinkAttachment(db, dataDir, ua, a.id, "work", "w1");
expect(await filesUnder(join(dataDir, "attachments"))).toHaveLength(1);
const r = await readAttachment(db, dataDir, ub, b.id);
expect(await r?.file.text()).toBe("same");
});
test("another user's unlink is a no-op", async () => {
const db = getDb();
const ua = await addUser(db, "a");
const ub = await addUser(db, "b");
const a = await storeAttachment(db, dataDir, ua, png("mine"), { entity: "work", entityId: "w1" });
await unlinkAttachment(db, dataDir, ub, a.id, "work", "w1");
expect(await listAttachments(db, ua, "work", "w1")).toHaveLength(1);
expect(await filesUnder(join(dataDir, "attachments"))).toHaveLength(1);
});
test("listAttachments is scoped to the user", async () => {
const db = getDb();
const ua = await addUser(db, "a");
const ub = await addUser(db, "b");
await storeAttachment(db, dataDir, ua, png("one"), { entity: "work", entityId: "w1" });
expect(await listAttachments(db, ub, "work", "w1")).toHaveLength(0);
expect(await listAttachments(db, ua, "work", "w1")).toHaveLength(1);
});
test("concurrent store and unlink never leave a row without its file", async () => {
const db = getDb();
const u = await addUser(db, "u1");
const first = await storeAttachment(db, dataDir, u, png("race"), { entity: "work", entityId: "w1" });
const [, second] = await Promise.all([
unlinkAttachment(db, dataDir, u, first.id, "work", "w1"),
storeAttachment(db, dataDir, u, png("race"), { entity: "work", entityId: "w2" }),
]);
const r = await readAttachment(db, dataDir, u, second.id);
expect(await r?.file.text()).toBe("race");
});
test("leaves no temp files behind", async () => {
const db = getDb();
const u = await addUser(db, "u1");
await storeAttachment(db, dataDir, u, png("clean"), { entity: "work", entityId: "w1" });
const files = await filesUnder(join(dataDir, "attachments"));
expect(files.some((f) => f.includes(".tmp-"))).toBe(false);
});
});
+155
View File
@@ -0,0 +1,155 @@
import { expect, test } from "bun:test";
import { forEachDialect } from "@sttracker/db/testing";
import {
createAppToken,
createInvite,
getSessionUser,
login,
logout,
registerUser,
revokeAppToken,
verifyAppToken,
} from "../src/auth.ts";
import { AuthError } from "../src/errors.ts";
async function code(p: Promise<unknown>): Promise<string | undefined> {
try {
await p;
} catch (e) {
return e instanceof AuthError ? e.code : "other";
}
return undefined;
}
forEachDialect("auth", (getDb) => {
test("first user is admin and username is lowercased", async () => {
const u = await registerUser(getDb(), { username: "Alice", password: "pw-one" });
expect(u.role).toBe("admin");
expect(u.username).toBe("alice");
expect(u.displayName).toBe("alice");
});
test("invalid and duplicate usernames are rejected", async () => {
const db = getDb();
expect(await code(registerUser(db, { username: "a b", password: "x" }))).toBe("invalid_username");
await registerUser(db, { username: "alice", password: "x" });
const inv = await createInvite(db, (await login(db, "alice", "x")).user.id);
expect(await code(registerUser(db, { username: "ALICE", password: "x", inviteCode: inv }))).toBe(
"username_taken",
);
});
test("second user needs a single-use invite", async () => {
const db = getDb();
const admin = await registerUser(db, { username: "alice", password: "x" });
expect(await code(registerUser(db, { username: "bob", password: "x" }))).toBe("invite_required");
expect(await code(registerUser(db, { username: "bob", password: "x", inviteCode: "nope" }))).toBe(
"invite_required",
);
const inv = await createInvite(db, admin.id);
const bob = await registerUser(db, { username: "bob", password: "x", inviteCode: inv });
expect(bob.role).toBe("member");
expect(await code(registerUser(db, { username: "carol", password: "x", inviteCode: inv }))).toBe(
"invite_required",
);
});
test("expired invite is rejected; members cannot create invites", async () => {
const db = getDb();
const admin = await registerUser(db, { username: "alice", password: "x" });
const inv = await createInvite(db, admin.id, -1);
expect(await code(registerUser(db, { username: "bob", password: "x", inviteCode: inv }))).toBe(
"invite_required",
);
const ok = await createInvite(db, admin.id);
const bob = await registerUser(db, { username: "bob", password: "x", inviteCode: ok });
expect(await code(createInvite(db, bob.id))).toBe("forbidden");
});
test("wrong password or unknown user -> invalid_credentials", async () => {
const db = getDb();
await registerUser(db, { username: "alice", password: "right" });
expect(await code(login(db, "alice", "wrong"))).toBe("invalid_credentials");
expect(await code(login(db, "nobody", "x"))).toBe("invalid_credentials");
});
test("session round-trip and logout", async () => {
const db = getDb();
await registerUser(db, { username: "alice", password: "pw" });
const { user, sessionToken } = await login(db, "Alice", "pw");
expect(await getSessionUser(db, sessionToken)).toEqual(user);
const stored = await db.k.selectFrom("sessions").selectAll().execute();
expect(stored[0]!.token_hash).not.toBe(sessionToken);
await logout(db, sessionToken);
expect(await getSessionUser(db, sessionToken)).toBeNull();
expect(await getSessionUser(db, "garbage")).toBeNull();
});
test("expired session returns null", async () => {
const db = getDb();
await registerUser(db, { username: "alice", password: "pw" });
const { sessionToken } = await login(db, "alice", "pw");
await db.k.updateTable("sessions").set({ expires_at: new Date(Date.now() - 1000).toISOString() }).execute();
expect(await getSessionUser(db, sessionToken)).toBeNull();
});
test("app tokens: scope check, last_used_at, revoke", async () => {
const db = getDb();
const u = await registerUser(db, { username: "alice", password: "pw" });
const { id, token } = await createAppToken(db, u.id, "cli", ["read"]);
expect(token.startsWith("stt_")).toBe(true);
expect(await verifyAppToken(db, token, "write")).toBeNull();
expect(await verifyAppToken(db, token, "read")).toEqual(u);
const row = await db.k.selectFrom("app_tokens").selectAll().executeTakeFirstOrThrow();
expect(row.last_used_at).not.toBeNull();
expect(await verifyAppToken(db, "stt_bogus", "read")).toBeNull();
await revokeAppToken(db, u.id, id);
expect(await verifyAppToken(db, token, "read")).toBeNull();
});
test("concurrent first registrations yield exactly one admin", async () => {
const db = getDb();
const results = await Promise.allSettled([
registerUser(db, { username: "alice", password: "x" }),
registerUser(db, { username: "bob", password: "x" }),
]);
const ok = results.filter((r) => r.status === "fulfilled");
const bad = results.filter((r) => r.status === "rejected");
expect(ok.length).toBe(1);
expect((ok[0] as PromiseFulfilledResult<{ role: string }>).value.role).toBe("admin");
expect(((bad[0] as PromiseRejectedResult).reason as AuthError).code).toBe("invite_required");
const admins = await db.k.selectFrom("users").select("id").where("role", "=", "admin").execute();
expect(admins.length).toBe(1);
});
test("no invite means invite_required even for a taken username", async () => {
const db = getDb();
await registerUser(db, { username: "alice", password: "x" });
expect(await code(registerUser(db, { username: "alice", password: "x" }))).toBe("invite_required");
});
test("only hashes are stored for invites and app tokens", async () => {
const db = getDb();
const u = await registerUser(db, { username: "alice", password: "x" });
const inv = await createInvite(db, u.id);
const { token } = await createAppToken(db, u.id, "cli", ["read"]);
const invRow = await db.k.selectFrom("invites").selectAll().executeTakeFirstOrThrow();
const tokRow = await db.k.selectFrom("app_tokens").selectAll().executeTakeFirstOrThrow();
expect(invRow.code_hash).not.toBe(inv);
expect(tokRow.token_hash).not.toBe(token);
expect(tokRow.token_hash).not.toContain(token);
});
test("revoking with another user's id leaves the token valid", async () => {
const db = getDb();
const a = await registerUser(db, { username: "alice", password: "x" });
const b = await registerUser(db, {
username: "bob",
password: "x",
inviteCode: await createInvite(db, a.id),
});
const { id, token } = await createAppToken(db, a.id, "cli", ["read"]);
await revokeAppToken(db, b.id, id);
expect(await verifyAppToken(db, token, "read")).toEqual(a);
});
});
+53
View File
@@ -0,0 +1,53 @@
import { afterEach, beforeEach, expect, test } from "bun:test";
import { mkdtemp, readdir, rm, stat } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { forEachDialect } from "@sttracker/db/testing";
import { backupHandler, runBackup } from "../src/backup.ts";
import { makeUser } from "./user-layer-fixtures.ts";
forEachDialect("backup", (getDb) => {
let dir: string;
beforeEach(async () => {
dir = await mkdtemp(join(tmpdir(), "sttracker-bak-"));
});
afterEach(async () => {
await rm(dir, { recursive: true, force: true });
});
test("backup writes a bundle per user (and the db on sqlite)", async () => {
const db = getDb();
await makeUser(db, "ada");
await makeUser(db, "bob");
const out = await runBackup(db, dir, { keep: 14 });
expect(out.startsWith(join(dir, "backups"))).toBe(true);
const files = (await readdir(out)).sort();
expect(files).toEqual(db.dialect === "sqlite" ? ["ada.zip", "bob.zip", "sttracker.db"] : ["ada.zip", "bob.zip"]);
});
test("backup prunes to keep", async () => {
const db = getDb();
await makeUser(db, "ada");
const made: string[] = [];
for (let i = 0; i < 4; i++) made.push(await runBackup(db, dir, { keep: 2 }));
const left = (await readdir(join(dir, "backups"))).sort();
expect(left).toHaveLength(2);
expect(left.map((n) => join(dir, "backups", n))).toEqual(made.slice(2));
});
test("backupHandler runs a backup", async () => {
const db = getDb();
await makeUser(db, "ada");
await backupHandler({ db, dataDir: dir, keep: 3 })();
expect(await readdir(join(dir, "backups"))).toHaveLength(1);
});
test("backup files are private and no partial directory remains", async () => {
const db = getDb();
await makeUser(db, "ada");
const out = await runBackup(db, dir, { keep: 5 });
expect((await stat(out)).mode & 0o777).toBe(0o700);
expect((await stat(join(out, "ada.zip"))).mode & 0o777).toBe(0o600);
expect((await readdir(join(dir, "backups"))).some((n) => n.endsWith(".tmp"))).toBe(false);
});
});
+165
View File
@@ -0,0 +1,165 @@
import { expect, spyOn, test } from "bun:test";
import { join } from "node:path";
import { forEachDialect } from "@sttracker/db/testing";
import { decryptJson } from "../src/crypto.ts";
import { DropJobError, NonRetryableError } from "../src/jobs.ts";
import { createConnection, deleteConnection, listConnections, makePluginContext, syncHandler, updateConnection } from "../src/connections.ts";
import { loadPlugins } from "../src/registry.ts";
import { makeUser } from "./sync/helpers.ts";
import { newId, nowIso } from "@sttracker/db/ids";
import type { Connection } from "../src/sync/run.ts";
const SECRET = "test-secret";
const warn = spyOn(console, "warn").mockImplementation(() => {});
const reg = await loadPlugins([join(import.meta.dir, "fixtures/plugins")]);
warn.mockRestore();
const echo = reg.get("echo")!;
forEachDialect("connections", (getDb) => {
const raw = (id: string) => getDb().k.selectFrom("source_connections").selectAll().where("id", "=", id).executeTakeFirstOrThrow();
const asConn = async (id: string): Promise<Connection> => {
const r = await raw(id);
return { id, userId: r.user_id, pluginId: r.plugin_id, name: r.name, config: JSON.parse(r.config), credentialsEnc: r.credentials_enc, scheduleMinutes: null, cursor: r.cursor, status: r.status, lastRunAt: null, createdAt: r.created_at };
};
test("secrets live only in the encrypted blob and never in list", async () => {
const db = getDb();
const u = await makeUser(db);
const id = await createConnection(db, SECRET, u, { pluginId: "echo", name: "E", config: { label: "hi", token: "s3cret" } }, echo.manifest);
const r = await raw(id);
expect(JSON.parse(r.config)).toEqual({ label: "hi" });
expect(r.credentials_enc).not.toContain("s3cret");
expect(await decryptJson(SECRET, r.credentials_enc!)).toEqual({ token: "s3cret" });
const [v] = await listConnections(db, u);
expect(JSON.stringify(v)).not.toContain("s3cret");
expect(v).toMatchObject({ id, config: { label: "hi" }, hasCredentials: true });
const ctx = await makePluginContext(db, SECRET, await asConn(id), "r", echo.manifest);
expect(ctx.config).toEqual({ label: "hi", token: "s3cret" });
expect(await ctx.credentials.get()).toEqual({ token: "s3cret" });
});
test("update keeps omitted secrets; delete pauses and clears credentials", async () => {
const db = getDb();
const u = await makeUser(db);
const id = await createConnection(db, SECRET, u, { pluginId: "echo", name: "E", config: { token: "t1" } }, echo.manifest);
await updateConnection(db, SECRET, u, id, { config: { label: "new" }, scheduleMinutes: 30 }, echo.manifest);
let r = await raw(id);
expect(JSON.parse(r.config)).toEqual({ label: "new" });
expect(r.schedule_minutes).toBe(30);
expect(await decryptJson(SECRET, r.credentials_enc!)).toEqual({ token: "t1" });
await updateConnection(db, SECRET, u, id, { config: { token: "t2" } }, echo.manifest);
expect(await decryptJson(SECRET, (await raw(id)).credentials_enc!)).toEqual({ token: "t2" });
const other = await makeUser(db, "bob");
await expect(deleteConnection(db, other, id)).rejects.toThrow();
await deleteConnection(db, u, id);
r = await raw(id);
expect(r.status).toBe("paused");
expect(r.credentials_enc).toBeNull();
expect(r.deleted_at).not.toBeNull();
expect(await listConnections(db, u)).toEqual([]);
});
test("cursor persists on set; http uses manifest rate limit", async () => {
const db = getDb();
const u = await makeUser(db);
const id = await createConnection(db, SECRET, u, { pluginId: "echo", name: "E", config: {} }, echo.manifest);
const ctx = await makePluginContext(db, SECRET, await asConn(id), "r", echo.manifest);
await ctx.cursor.set("c1");
expect((await raw(id)).cursor).toBe("c1");
expect(await ctx.cursor.get()).toBe("c1");
});
test("credentials.save persists encrypted before pull continues", async () => {
const db = getDb();
const u = await makeUser(db);
const id = await createConnection(db, SECRET, u, { pluginId: "echo", name: "E", config: { label: "explode", token: "t" } }, echo.manifest);
const handler = syncHandler({ db, secret: SECRET, registry: reg });
await expect(handler({ connectionId: id })).rejects.toThrow();
const r = await raw(id);
expect(await decryptJson(SECRET, r.credentials_enc!)).toEqual({ token: "t", refresh: "new-refresh" });
expect(r.status).toBe("error");
});
test("sync handler: full first, changes after a recent full, full when payload says so or full is stale", async () => {
const db = getDb();
const u = await makeUser(db);
const id = await createConnection(db, SECRET, u, { pluginId: "echo", name: "E", config: { label: "ok" } }, echo.manifest);
const handler = syncHandler({ db, secret: SECRET, registry: reg });
const modes = async () => (await db.k.selectFrom("source_runs").select(["mode", "status"]).where("connection_id", "=", id).orderBy("started_at").orderBy("id").execute()).map((r) => `${r.mode}:${r.status}`);
await handler({ connectionId: id });
await handler({ connectionId: id });
await handler({ connectionId: id, mode: "full" });
expect(await modes()).toEqual(["full:ok", "changes:ok", "full:ok"]);
await db.k.updateTable("source_runs").set({ started_at: new Date(Date.now() - 8 * 86400_000).toISOString() }).where("connection_id", "=", id).execute();
await handler({ connectionId: id });
expect((await modes()).at(-1)).toBe("full:ok");
await db.k.insertInto("source_runs").values({ id: newId(), connection_id: id, mode: "full", started_at: nowIso(), finished_at: null, status: "failed", counts: "{}", log: "", error: "x" }).execute();
await handler({ connectionId: id });
expect((await modes()).at(-1)).toBe("changes:ok");
});
const setup = async (config: Record<string, unknown>) => {
const db = getDb();
const u = await makeUser(db);
const id = await createConnection(db, SECRET, u, { pluginId: "echo", name: "E", config }, echo.manifest);
return { db, id };
};
const failedRuns = (id: string) => getDb().k.selectFrom("source_runs").selectAll().where("connection_id", "=", id).execute();
test("missing plugin: failed run, connection error, no retry", async () => {
const { db, id } = await setup({});
await db.k.updateTable("source_connections").set({ plugin_id: "gone" }).where("id", "=", id).execute();
await expect(syncHandler({ db, secret: SECRET, registry: reg })({ connectionId: id })).rejects.toBeInstanceOf(NonRetryableError);
const runs = await failedRuns(id);
expect(runs).toHaveLength(1);
expect(runs[0]).toMatchObject({ status: "failed", mode: "full", error: "Plugin 'gone' is not installed or failed to load." });
const c = await raw(id);
expect(c.status).toBe("error");
expect(c.last_run_at).not.toBeNull();
});
test("undecryptable credentials: failed run, connection error, no retry", async () => {
const { db, id } = await setup({ token: "t" });
await expect(syncHandler({ db, secret: "other-secret", registry: reg })({ connectionId: id })).rejects.toBeInstanceOf(NonRetryableError);
const runs = await failedRuns(id);
expect(runs[0]!.error).toBe("Cannot decrypt this source's credentials. Was STTRACKER_SECRET changed? Reconnect the source to fix it.");
expect((await raw(id)).status).toBe("error");
});
test("missing or paused connection drops the job", async () => {
const { db, id } = await setup({});
const h = syncHandler({ db, secret: SECRET, registry: reg });
await expect(h({ connectionId: newId() })).rejects.toBeInstanceOf(DropJobError);
await db.k.updateTable("source_connections").set({ status: "paused" }).where("id", "=", id).execute();
await expect(h({ connectionId: id })).rejects.toBeInstanceOf(DropJobError);
expect(await failedRuns(id)).toHaveLength(0);
});
test("ctx.log lands in the run's log", async () => {
const { db, id } = await setup({ label: "ok" });
const res = await syncHandler({ db, secret: SECRET, registry: reg })({ connectionId: id });
const run = await db.k.selectFrom("source_runs").selectAll().where("id", "=", res!.runId).executeTakeFirstOrThrow();
expect(run.log).toContain("echo pulled");
});
test("concurrent credentials.save calls keep every key", async () => {
const { db, id } = await setup({ token: "t" });
const conn = await asConn(id);
const ctx = await makePluginContext(db, SECRET, conn, "r", echo.manifest);
await Promise.all([ctx.credentials.save({ a: "1" }), ctx.credentials.save({ b: "2" })]);
expect(await decryptJson(SECRET, (await raw(id)).credentials_enc!)).toEqual({ token: "t", a: "1", b: "2" });
});
test("ctx.http is limited by manifest.rateLimit", async () => {
const { db, id } = await setup({});
let t = 0;
const sleeps: number[] = [];
const manifest = { ...echo.manifest, rateLimit: { perMinute: 60, burst: 1 } };
const ctx = await makePluginContext(db, SECRET, await asConn(id), "r", manifest, {
httpOptions: { fetchImpl: (async () => Response.json({})) as unknown as typeof fetch, now: () => t, sleep: async (ms) => { sleeps.push(ms); t += ms; } },
});
await ctx.http.json("https://example.test/a");
await ctx.http.json("https://example.test/b");
expect(sleeps).toEqual([1000]);
});
});
+76
View File
@@ -0,0 +1,76 @@
import { expect, test } from "bun:test";
import { forEachDialect } from "@sttracker/db/testing";
import { createCopy, listCopies, softDeleteCopy, updateCopy } from "../src/copies.ts";
import { defineCustomField } from "../src/custom-fields.ts";
import { NotFoundError, ValidationError } from "../src/errors.ts";
import { getHistory, restoreVersion } from "../src/history.ts";
import { makeUser, makeWork } from "./user-layer-fixtures.ts";
forEachDialect("copies", (getDb) => {
test("createCopy stores price in cents and partial acquired date", async () => {
const db = getDb();
const userId = await makeUser(db);
const workId = await makeWork(db);
const c = await createCopy(db, userId, {
workId,
format: "hardback",
acquiredFrom: "Waterstones",
acquiredAt: { value: "2019-03", precision: "month" },
priceCents: 1299,
currency: "GBP",
});
expect(c.priceCents).toBe(1299);
expect(c.acquiredAt).toEqual({ value: "2019-03", precision: "month" });
const row = await db.k.selectFrom("copies").selectAll().where("id", "=", c.id).executeTakeFirstOrThrow();
expect(row.price_cents).toBe(1299);
expect(row.acquired_precision).toBe("month");
await expect(createCopy(db, userId, { workId, priceCents: 12.5 })).rejects.toThrow(ValidationError);
await expect(createCopy(db, userId, { workId, currency: "pounds" })).rejects.toThrow(ValidationError);
await expect(createCopy(db, userId, {})).rejects.toThrow(ValidationError);
});
test("unmatched copy keeps its title hint; listCopies filters by work", async () => {
const db = getDb();
const userId = await makeUser(db);
const workId = await makeWork(db);
const steam = await createCopy(db, userId, { titleHint: "HALF-LIFE 2", format: "Steam", sourceRef: "steam:220" }, "plugin:steam:c1");
const book = await createCopy(db, userId, { workId });
expect(steam.workId).toBeNull();
expect(steam.titleHint).toBe("HALF-LIFE 2");
expect((await listCopies(db, userId, {})).map((c) => c.id).sort()).toEqual([steam.id, book.id].sort());
expect((await listCopies(db, userId, { workId })).map((c) => c.id)).toEqual([book.id]);
});
test("updateCopy writes history; soft delete and restore", async () => {
const db = getDb();
const userId = await makeUser(db);
const def = await defineCustomField(db, userId, { name: "Signed", type: "boolean", level: "copy" });
const c = await createCopy(db, userId, { workId: await makeWork(db), notes: "shelf 3" });
const up = await updateCopy(db, userId, c.id, { notes: "shelf 4", customFields: { [def.id]: true } });
expect(up.notes).toBe("shelf 4");
expect(up.customFields).toEqual({ [def.id]: true });
await expect(updateCopy(db, userId, c.id, { customFields: { [def.id]: "yes" } })).rejects.toThrow(ValidationError);
const [h] = await getHistory(db, userId, "copy", c.id);
expect(h!.before).toEqual({ notes: "shelf 3", customFields: {} });
await softDeleteCopy(db, userId, c.id);
expect(await listCopies(db, userId, {})).toEqual([]);
const [del] = await getHistory(db, userId, "copy", c.id);
await restoreVersion(db, userId, del!.id);
expect((await listCopies(db, userId, {})).map((x) => x.id)).toEqual([c.id]);
});
test("another user's copies are not found", async () => {
const db = getDb();
const ada = await makeUser(db);
const bob = await makeUser(db, "bob");
const c = await createCopy(db, ada, { titleHint: "x" });
await expect(updateCopy(db, bob, c.id, { notes: "y" })).rejects.toThrow(NotFoundError);
await expect(softDeleteCopy(db, bob, c.id)).rejects.toThrow(NotFoundError);
expect(await listCopies(db, bob, {})).toEqual([]);
await expect(createCopy(db, bob, { workId: await makeWork(db, { ownerUserId: ada }) })).rejects.toThrow(
NotFoundError,
);
});
});
+39
View File
@@ -0,0 +1,39 @@
import { describe, expect, test } from "bun:test";
import { decryptJson, encryptJson } from "../src/crypto.ts";
import { CryptoError } from "../src/errors.ts";
describe("crypto", () => {
test("round-trips JSON", async () => {
const value = { token: "abc", n: [1, 2, { x: null }] };
const blob = await encryptJson("s3cret", value);
expect(blob).toMatch(/^v1\.[A-Za-z0-9+/=]+\.[A-Za-z0-9+/=]+$/);
expect(await decryptJson("s3cret", blob)).toEqual(value);
});
test("uses a fresh IV each time", async () => {
expect(await encryptJson("k", 1)).not.toBe(await encryptJson("k", 1));
});
test("wrong secret throws CryptoError", async () => {
const blob = await encryptJson("right", { a: 1 });
await expect(decryptJson("wrong", blob)).rejects.toBeInstanceOf(CryptoError);
});
test("malformed blob throws CryptoError", async () => {
await expect(decryptJson("k", "garbage")).rejects.toBeInstanceOf(CryptoError);
await expect(decryptJson("k", "v2.AA==.AA==")).rejects.toBeInstanceOf(CryptoError);
});
test("IV decodes to 12 bytes", async () => {
const blob = await encryptJson("k", { a: 1 });
expect(Buffer.from(blob.split(".")[1]!, "base64")).toHaveLength(12);
});
test("flipping a ciphertext byte throws CryptoError", async () => {
const [v, iv, ct] = (await encryptJson("k", { a: 1 })).split(".");
const bytes = Buffer.from(ct!, "base64");
bytes[0] = bytes[0]! ^ 0xff;
const tampered = `${v}.${iv}.${bytes.toString("base64")}`;
await expect(decryptJson("k", tampered)).rejects.toBeInstanceOf(CryptoError);
});
});
+103
View File
@@ -0,0 +1,103 @@
import { expect, test } from "bun:test";
import { forEachDialect } from "@sttracker/db/testing";
import { ValidationError } from "../src/errors.ts";
import { defineCustomField, listCustomFields, validateCustomValues } from "../src/custom-fields.ts";
import { makeUser } from "./user-layer-fixtures.ts";
forEachDialect("custom fields", (getDb) => {
test("custom select field rejects unknown option", async () => {
const db = getDb();
const userId = await makeUser(db);
const mood = await defineCustomField(db, userId, {
name: "Mood",
type: "single_select",
options: ["cosy", "grim"],
});
expect(() => validateCustomValues([mood], { [mood.id]: "cosy" })).not.toThrow();
let err: unknown;
try {
validateCustomValues([mood], { [mood.id]: "chaotic" });
} catch (e) {
err = e;
}
expect(err).toBeInstanceOf(ValidationError);
expect((err as ValidationError).field).toBe("customFields.Mood");
expect((err as ValidationError).reason).toBe("unknown_option");
});
test("listCustomFields filters by media type and stays per user", async () => {
const db = getDb();
const userId = await makeUser(db);
const other = await makeUser(db, "bob");
await defineCustomField(db, userId, { name: "Translator", type: "text", appliesTo: ["book", "manga"] });
await defineCustomField(db, userId, { name: "Recommended by", type: "text" });
await defineCustomField(db, other, { name: "Bob field", type: "text" });
expect((await listCustomFields(db, userId)).map((d) => d.name).sort()).toEqual(["Recommended by", "Translator"]);
expect((await listCustomFields(db, userId, "game")).map((d) => d.name)).toEqual(["Recommended by"]);
expect((await listCustomFields(db, userId, "manga")).map((d) => d.name).sort()).toEqual([
"Recommended by",
"Translator",
]);
});
test("defineCustomField rejects bad input", async () => {
const db = getDb();
const userId = await makeUser(db);
await expect(defineCustomField(db, userId, { name: " ", type: "text" })).rejects.toThrow(ValidationError);
await expect(defineCustomField(db, userId, { name: "X", type: "colour" as never })).rejects.toThrow(
ValidationError,
);
await expect(defineCustomField(db, userId, { name: "S", type: "single_select" })).rejects.toThrow(
ValidationError,
);
await defineCustomField(db, userId, { name: "Mood", type: "text" });
await expect(defineCustomField(db, userId, { name: "mood", type: "text" })).rejects.toThrow(ValidationError);
});
});
test("validateCustomValues checks every type", () => {
const def = (id: string, type: string, options: string[] = []) => ({
id,
userId: "u",
name: id,
type: type as never,
options,
appliesTo: "all" as const,
level: "entry" as const,
createdAt: "",
});
const defs = [
def("t", "text"),
def("n", "number"),
def("d", "date"),
def("b", "boolean"),
def("m", "multi_select", ["a", "b"]),
def("u", "url"),
def("r", "rating"),
];
expect(() =>
validateCustomValues(defs, { t: "x", n: 1.5, d: "2019", b: true, m: ["a"], u: "https://x.org", r: 80 }),
).not.toThrow();
expect(() => validateCustomValues(defs, { t: null })).not.toThrow();
const bad: Array<[Record<string, unknown>, string]> = [
[{ nope: 1 }, "unknown_field"],
[{ t: 3 }, "not_text"],
[{ n: "3" }, "not_number"],
[{ d: "2019-13" }, "not_date"],
[{ b: 1 }, "not_boolean"],
[{ m: ["c"] }, "unknown_option"],
[{ m: "a" }, "not_list"],
[{ u: "ftp:x" }, "not_url"],
[{ r: 101 }, "out_of_range"],
];
for (const [values, reason] of bad) {
let err: unknown;
try {
validateCustomValues(defs, values);
} catch (e) {
err = e;
}
expect((err as ValidationError).reason).toBe(reason);
}
});
+147
View File
@@ -0,0 +1,147 @@
import { expect, test } from "bun:test";
import { forEachDialect } from "@sttracker/db/testing";
import { defineCustomField } from "../src/custom-fields.ts";
import {
addEngagement,
listEngagements,
softDeleteEngagement,
updateEngagement,
} from "../src/engagements.ts";
import { moveEntryToWork, softDeleteEntry, upsertEntry } from "../src/entries.ts";
import { NotFoundError, ValidationError } from "../src/errors.ts";
import { getHistory, restoreVersion } from "../src/history.ts";
import { makeUser, makeWork } from "./user-layer-fixtures.ts";
forEachDialect("engagements", (getDb) => {
async function setup(mediaType = "book") {
const db = getDb();
const userId = await makeUser(db);
const entry = await upsertEntry(db, userId, await makeWork(db, { mediaType }), { status: "completed" }, "user");
return { db, userId, entry };
}
test("engagement with year precision keeps '2019'", async () => {
const { db, userId, entry } = await setup();
const eg = await addEngagement(
db,
userId,
entry.id,
{ finishedAt: { value: "2019", precision: "year" }, platform: "Kindle" },
"user",
);
expect(eg.finishedAt).toEqual({ value: "2019", precision: "year" });
expect(eg.startedAt).toBeNull();
expect(eg.fieldSources).toEqual({ finishedAt: "user", platform: "user" });
const row = await db.k.selectFrom("engagements").selectAll().where("id", "=", eg.id).executeTakeFirstOrThrow();
expect(row.finished_at).toBe("2019");
expect(row.finished_precision).toBe("year");
expect(row.started_at).toBeNull();
expect(row.started_precision).toBe("unknown");
const [listed] = await listEngagements(db, userId, entry.id);
expect(listed!.finishedAt).toEqual({ value: "2019", precision: "year" });
});
test("partial dates must match their precision", async () => {
const { db, userId, entry } = await setup();
const bad = [
{ value: "2019-03", precision: "year" },
{ value: "2019-02-30", precision: "day" },
{ value: "", precision: "day" },
{ value: "2019", precision: "bogus" },
];
for (const d of bad) {
await expect(addEngagement(db, userId, entry.id, { startedAt: d as never }, "user")).rejects.toThrow(
ValidationError,
);
}
const ok = await addEngagement(
db,
userId,
entry.id,
{ startedAt: { value: "2019-03-04T20:15:00Z", precision: "day" }, finishedAt: { value: "", precision: "unknown" } },
"user",
);
expect(ok.startedAt).toEqual({ value: "2019-03-04T20:15:00Z", precision: "day" });
expect(ok.finishedAt).toBeNull();
});
test("updateEngagement touches only patched keys and writes history", async () => {
const { db, userId, entry } = await setup();
const eg = await addEngagement(db, userId, entry.id, { rating: 70, notes: "great" }, "user");
const up = await updateEngagement(db, userId, eg.id, { rating: 90, progress: { page: 120 } }, "plugin:hardcover:c1");
expect(up.notes).toBe("great");
expect(up.rating).toBe(90);
expect(up.progress).toEqual({ page: 120 });
expect(up.fieldSources).toEqual({ rating: "plugin:hardcover:c1", notes: "user", progress: "plugin:hardcover:c1" });
const [h] = await getHistory(db, userId, "engagement", eg.id);
expect(h!.before).toEqual({ rating: 70, progress: {} });
expect(h!.after).toEqual({ rating: 90, progress: { page: 120 } });
await restoreVersion(db, userId, h!.id);
const [back] = await listEngagements(db, userId, entry.id);
expect(back!.rating).toBe(70);
expect(back!.fieldSources.rating).toBe("user");
await expect(updateEngagement(db, userId, eg.id, { rating: -1 }, "user")).rejects.toThrow(ValidationError);
await expect(updateEngagement(db, userId, eg.id, { bogus: 1 } as never, "user")).rejects.toThrow(ValidationError);
});
test("engagement custom fields use engagement-level definitions", async () => {
const { db, userId, entry } = await setup();
const entryLevel = await defineCustomField(db, userId, { name: "Mood", type: "text" });
const def = await defineCustomField(db, userId, { name: "Format", type: "single_select", options: ["audio"], level: "engagement" });
const eg = await addEngagement(db, userId, entry.id, { customFields: { [def.id]: "audio" } }, "user");
expect(eg.customFields).toEqual({ [def.id]: "audio" });
await expect(
addEngagement(db, userId, entry.id, { customFields: { [entryLevel.id]: "x" } }, "user"),
).rejects.toThrow(ValidationError);
});
test("softDeleteEngagement hides it; restore brings it back", async () => {
const { db, userId, entry } = await setup();
const eg = await addEngagement(db, userId, entry.id, { platform: "Switch" }, "user");
await softDeleteEngagement(db, userId, eg.id, "user");
expect(await listEngagements(db, userId, entry.id)).toEqual([]);
await expect(updateEngagement(db, userId, eg.id, { platform: "PC" }, "user")).rejects.toThrow(NotFoundError);
const [del] = await getHistory(db, userId, "engagement", eg.id);
await restoreVersion(db, userId, del!.id);
expect((await listEngagements(db, userId, entry.id)).map((e) => e.id)).toEqual([eg.id]);
});
test("engagements sort by start date, undated last", async () => {
const { db, userId, entry } = await setup();
const undated = await addEngagement(db, userId, entry.id, {}, "user");
const later = await addEngagement(db, userId, entry.id, { startedAt: { value: "2021-05", precision: "month" } }, "user");
const early = await addEngagement(db, userId, entry.id, { startedAt: { value: "2019", precision: "year" } }, "user");
expect((await listEngagements(db, userId, entry.id)).map((e) => e.id)).toEqual([early.id, later.id, undated.id]);
});
test("engagements of a soft-deleted entry cannot be changed", async () => {
const { db, userId, entry } = await setup();
const eg = await addEngagement(db, userId, entry.id, {}, "user");
await softDeleteEntry(db, userId, entry.id, "user");
await expect(updateEngagement(db, userId, eg.id, { rating: 1 }, "user")).rejects.toThrow(NotFoundError);
await expect(softDeleteEngagement(db, userId, eg.id, "user")).rejects.toThrow(NotFoundError);
});
test("restoring a re-parenting onto a soft-deleted entry is rejected", async () => {
const { db, userId, entry } = await setup();
const other = await upsertEntry(db, userId, await makeWork(db, { title: "Other" }), {}, "user");
const eg = await addEngagement(db, userId, entry.id, {}, "user");
await moveEntryToWork(db, userId, entry.id, other.workId, "user");
const [h] = await getHistory(db, userId, "engagement", eg.id);
expect(h!.before).toEqual({ entryId: entry.id });
// the merged-away entry is soft-deleted, so the engagement cannot go back to it
const err = await restoreVersion(db, userId, h!.id).catch((e: unknown) => e);
expect(err).toBeInstanceOf(ValidationError);
expect(err).toMatchObject({ field: "entryId", reason: "target entry is deleted" });
});
test("another user's engagements are not found", async () => {
const { db, userId, entry } = await setup();
const bob = await makeUser(db, "bob");
const eg = await addEngagement(db, userId, entry.id, {}, "user");
await expect(addEngagement(db, bob, entry.id, {}, "user")).rejects.toThrow(NotFoundError);
await expect(updateEngagement(db, bob, eg.id, { rating: 1 }, "user")).rejects.toThrow(NotFoundError);
await expect(softDeleteEngagement(db, bob, eg.id, "user")).rejects.toThrow(NotFoundError);
await expect(listEngagements(db, bob, entry.id)).rejects.toThrow(NotFoundError);
});
});
+389
View File
@@ -0,0 +1,389 @@
import { expect, test } from "bun:test";
import { newId } from "@sttracker/db/ids";
import { forEachDialect } from "@sttracker/db/testing";
import { defineCustomField } from "../src/custom-fields.ts";
import { addEngagement, listEngagements } from "../src/engagements.ts";
import { NotFoundError, ValidationError } from "../src/errors.ts";
import {
getEntry,
listEntries,
moveEntryToWork,
quickAdd,
softDeleteEntry,
upsertEntry,
} from "../src/entries.ts";
import { getHistory } from "../src/history.ts";
import { makeUser, makeWork } from "./user-layer-fixtures.ts";
forEachDialect("entries", (getDb) => {
test("upsertEntry records field_sources and history", async () => {
const db = getDb();
const userId = await makeUser(db);
const workId = await makeWork(db);
const created = await upsertEntry(db, userId, workId, { status: "planned", tags: ["sci-fi"] }, "user");
expect(created.status).toBe("planned");
expect(created.visibility).toBe("private");
expect(created.fieldSources).toEqual({ status: "user", tags: "user" });
const updated = await upsertEntry(db, userId, workId, { rating: 80, status: "completed" }, "plugin:trakt:c1");
expect(updated.id).toBe(created.id);
expect(updated.rating).toBe(80);
expect(updated.tags).toEqual(["sci-fi"]);
expect(updated.fieldSources).toEqual({ status: "plugin:trakt:c1", tags: "user", rating: "plugin:trakt:c1" });
const history = await getHistory(db, userId, "entry", created.id);
expect(history).toHaveLength(2);
expect(history[0]!.before).toEqual({ status: "planned", rating: null });
expect(history[0]!.after).toEqual({ status: "completed", rating: 80 });
expect(history[0]!.actor).toBe("plugin:trakt:c1");
});
test("upsertEntry works inside a caller's transaction", async () => {
const db = getDb();
const userId = await makeUser(db);
const workId = await makeWork(db);
await expect(
db.k.transaction().execute(async (tx) => {
await upsertEntry(db, userId, workId, { status: "planned" }, "user", tx);
throw new Error("rollback");
}),
).rejects.toThrow("rollback");
expect((await listEntries(db, userId, {})).items).toHaveLength(0);
const e = await db.k.transaction().execute((tx) => upsertEntry(db, userId, workId, { favorite: true }, "user", tx));
expect(e.favorite).toBe(true);
expect(e.status).toBe("none");
});
test("visibility default comes from user preferences", async () => {
const db = getDb();
const userId = await makeUser(db, "pub", { defaultVisibility: "public" });
const e = await upsertEntry(db, userId, await makeWork(db), {}, "user");
expect(e.visibility).toBe("public");
expect(e.fieldSources).toEqual({});
});
test("rating 101 rejected", async () => {
const db = getDb();
const userId = await makeUser(db);
const workId = await makeWork(db);
await expect(upsertEntry(db, userId, workId, { rating: 101 }, "user")).rejects.toThrow(ValidationError);
await expect(upsertEntry(db, userId, workId, { rating: 7.5 }, "user")).rejects.toThrow(ValidationError);
await expect(upsertEntry(db, userId, workId, { status: "done" as never }, "user")).rejects.toThrow(
ValidationError,
);
expect((await listEntries(db, userId, {})).items).toHaveLength(0);
});
test("custom field values are validated against the user's entry definitions", async () => {
const db = getDb();
const userId = await makeUser(db);
const workId = await makeWork(db, { mediaType: "book" });
const def = await defineCustomField(db, userId, { name: "Mood", type: "single_select", options: ["cosy"] });
const gameOnly = await defineCustomField(db, userId, { name: "Hours", type: "number", appliesTo: ["game"] });
const e = await upsertEntry(db, userId, workId, { customFields: { [def.id]: "cosy" } }, "user");
expect(e.customFields).toEqual({ [def.id]: "cosy" });
await expect(upsertEntry(db, userId, workId, { customFields: { [def.id]: "grim" } }, "user")).rejects.toThrow(
ValidationError,
);
await expect(upsertEntry(db, userId, workId, { customFields: { [gameOnly.id]: 3 } }, "user")).rejects.toThrow(
ValidationError,
);
});
test("quickAdd creates draft entry", async () => {
const db = getDb();
const userId = await makeUser(db);
const view = await quickAdd(db, userId, { title: "Zine I found" });
expect(view.draft).toBe(true);
expect(view.work.title).toBe("Zine I found");
expect(view.work.mediaType).toBe("other");
expect(view.status).toBe("none");
const work = await db.k.selectFrom("works").selectAll().where("id", "=", view.workId).executeTakeFirstOrThrow();
expect(work.owner_user_id).toBe(userId);
const shared = await upsertEntry(db, userId, await makeWork(db, { externalIds: { isbn13: "1" } }), {}, "user");
expect((await getEntry(db, userId, shared.id)).draft).toBe(false);
expect((await listEntries(db, userId, { draft: true })).items.map((i) => i.id)).toEqual([view.id]);
await expect(quickAdd(db, userId, { title: " " })).rejects.toThrow(ValidationError);
await expect(quickAdd(db, userId, { title: "x", mediaType: "zine" as never })).rejects.toThrow(ValidationError);
});
test("listEntries rolls up children", async () => {
const db = getDb();
const userId = await makeUser(db);
const show = await makeWork(db, { title: "Severance", mediaType: "show" });
const showEntry = await upsertEntry(db, userId, show, { status: "in_progress" }, "user");
for (const status of ["completed", "completed", "in_progress"] as const) {
const ep = await makeWork(db, { title: "Episode", mediaType: "episode", parentId: show });
await upsertEntry(db, userId, ep, { status }, "user");
}
const lone = await upsertEntry(db, userId, await makeWork(db, { title: "Dune" }), {}, "user");
const top = await listEntries(db, userId, {});
expect(top.items.map((i) => i.id).sort()).toEqual([showEntry.id, lone.id].sort());
const showView = top.items.find((i) => i.id === showEntry.id)!;
expect(showView.childSummary).toEqual({ total: 3, completed: 2 });
expect(top.items.find((i) => i.id === lone.id)!.childSummary).toBeNull();
const children = await listEntries(db, userId, { parentId: show });
expect(children.items).toHaveLength(3);
expect(children.items.every((i) => i.work.mediaType === "episode")).toBe(true);
// grandchildren: show → season (no entry) → episodes still roll up to the show
const show2 = await makeWork(db, { title: "The Expanse", mediaType: "show" });
const show2Entry = await upsertEntry(db, userId, show2, {}, "user");
const season = await makeWork(db, { title: "Season 1", mediaType: "season", parentId: show2 });
for (const status of ["completed", "planned"] as const) {
const ep = await makeWork(db, { title: "Episode", mediaType: "episode", parentId: season });
await upsertEntry(db, userId, ep, { status }, "user");
}
const top2 = await listEntries(db, userId, {});
expect(top2.items.map((i) => i.id).sort()).toEqual([showEntry.id, lone.id, show2Entry.id].sort());
expect(top2.items.find((i) => i.id === show2Entry.id)!.childSummary).toEqual({ total: 2, completed: 1 });
// drilling into the show lists the episodes, since the season between them has no entry
const drill = await listEntries(db, userId, { parentId: show2 });
expect(drill.items.map((i) => i.work.mediaType)).toEqual(["episode", "episode"]);
expect(await listEntries(db, userId, { parentId: show2, q: "nothing matches" })).toEqual({ items: [], nextCursor: null });
// once the season has an entry, the show lists the season and the season lists the episodes
const seasonEntry = await upsertEntry(db, userId, season, { status: "in_progress" }, "user");
expect((await listEntries(db, userId, { parentId: show2 })).items.map((i) => i.id)).toEqual([seasonEntry.id]);
const seasonDrill = await listEntries(db, userId, { parentId: season });
expect(seasonDrill.items.map((i) => i.work.mediaType)).toEqual(["episode", "episode"]);
expect(seasonDrill.items.find((i) => i.id === seasonEntry.id)).toBeUndefined();
});
test("orphaned child entries are top level until an ancestor has an entry", async () => {
const db = getDb();
const userId = await makeUser(db);
const show = await makeWork(db, { title: "Severance", mediaType: "show" });
const season = await makeWork(db, { title: "Season 1", mediaType: "season", parentId: show });
const ep = await makeWork(db, { title: "Good News About Hell", mediaType: "episode", parentId: season });
const epEntry = await upsertEntry(db, userId, ep, { status: "completed" }, "user");
expect((await listEntries(db, userId, {})).items.map((i) => i.id)).toEqual([epEntry.id]);
// another user's show entry does not hide it
const bob = await makeUser(db, "bob");
await upsertEntry(db, bob, show, {}, "user");
expect((await listEntries(db, userId, {})).items.map((i) => i.id)).toEqual([epEntry.id]);
const showEntry = await upsertEntry(db, userId, show, {}, "user");
expect((await listEntries(db, userId, {})).items.map((i) => i.id)).toEqual([showEntry.id]);
// a soft-deleted ancestor entry does not hide it
await softDeleteEntry(db, userId, showEntry.id, "user");
expect((await listEntries(db, userId, {})).items.map((i) => i.id)).toEqual([epEntry.id]);
});
test("title pagination does not skip non-ASCII titles", async () => {
const db = getDb();
const userId = await makeUser(db);
const titles = ["Éclair", "Ölund", "Zeta", "alpha", "日本", "éclair 2", "BETA"];
for (const title of titles) await upsertEntry(db, userId, await makeWork(db, { title }), {}, "user");
const seen: string[] = [];
let cursor: string | undefined;
for (let i = 0; i < 20; i++) {
const page = await listEntries(db, userId, { sort: "title", limit: 1, cursor });
seen.push(...page.items.map((x) => x.work.title));
if (!page.nextCursor) break;
cursor = page.nextCursor;
}
expect(seen).toHaveLength(titles.length);
expect([...seen].sort()).toEqual([...titles].sort());
});
test("concurrent upserts of a new entry yield one entry with both patches", async () => {
const db = getDb();
const userId = await makeUser(db);
const workId = await makeWork(db);
const [a, b] = await Promise.all([
upsertEntry(db, userId, workId, { rating: 70 }, "plugin:trakt:c1"),
upsertEntry(db, userId, workId, { tags: ["cosy"] }, "user"),
]);
expect(a.id).toBe(b.id);
const rows = await db.k.selectFrom("entries").selectAll().where("user_id", "=", userId).execute();
expect(rows).toHaveLength(1);
const view = await getEntry(db, userId, a.id);
expect(view.rating).toBe(70);
expect(view.tags).toEqual(["cosy"]);
expect(view.fieldSources).toEqual({ rating: "plugin:trakt:c1", tags: "user" });
});
test("listEntries filters, sorts and paginates", async () => {
const db = getDb();
const userId = await makeUser(db);
const other = await makeUser(db, "bob");
const titles = ["Alpha", "bravo", "Charlie", "delta", "Echo"];
const ids: string[] = [];
for (const [i, title] of titles.entries()) {
const w = await makeWork(db, { title, mediaType: i % 2 ? "game" : "book" });
const e = await upsertEntry(
db,
userId,
w,
{ rating: i === 4 ? null : i * 10, tags: i < 2 ? ["Cosy"] : [], status: i === 0 ? "completed" : "planned" },
"user",
);
ids.push(e.id);
await upsertEntry(db, other, w, { status: "completed" }, "user");
}
// updated sort: newest first, keyset pagination
const p1 = await listEntries(db, userId, { limit: 2 });
expect(p1.items).toHaveLength(2);
expect(p1.nextCursor).not.toBeNull();
const p2 = await listEntries(db, userId, { limit: 2, cursor: p1.nextCursor! });
const p3 = await listEntries(db, userId, { limit: 2, cursor: p2.nextCursor! });
expect(p3.nextCursor).toBeNull();
expect([...p1.items, ...p2.items, ...p3.items].map((i) => i.id)).toEqual([...ids].reverse());
const byTitle = await listEntries(db, userId, { sort: "title", limit: 3 });
const byTitle2 = await listEntries(db, userId, { sort: "title", limit: 3, cursor: byTitle.nextCursor! });
expect([...byTitle.items, ...byTitle2.items].map((i) => i.work.title)).toEqual(titles);
const byRating = await listEntries(db, userId, { sort: "rating", limit: 2 });
const byRating2 = await listEntries(db, userId, { sort: "rating", limit: 10, cursor: byRating.nextCursor! });
expect([...byRating.items, ...byRating2.items].map((i) => i.rating)).toEqual([30, 20, 10, 0, null]);
expect((await listEntries(db, userId, { mediaType: "game" })).items).toHaveLength(2);
expect((await listEntries(db, userId, { status: "completed" })).items.map((i) => i.id)).toEqual([ids[0]!]);
expect((await listEntries(db, userId, { tag: "cosy" })).items).toHaveLength(2);
expect((await listEntries(db, userId, { q: "RAV" })).items.map((i) => i.id)).toEqual([ids[1]!]);
expect((await listEntries(db, userId, { q: "100%" })).items).toHaveLength(0);
await expect(listEntries(db, userId, { cursor: "garbage" })).rejects.toThrow(ValidationError);
await expect(listEntries(db, userId, { sort: "title", cursor: p1.nextCursor! })).rejects.toThrow(
ValidationError,
);
});
test("overrides apply to the work view", async () => {
const db = getDb();
const userId = await makeUser(db);
const workId = await makeWork(db, { title: "Dune", year: 1965 });
const e = await upsertEntry(db, userId, workId, { overrides: { title: "Dune (my edition)", year: 1990 } }, "user");
const view = await getEntry(db, userId, e.id);
expect(view.work.title).toBe("Dune (my edition)");
expect(view.work.year).toBe(1990);
expect(view.work.coverUrl).toBe("");
await expect(upsertEntry(db, userId, workId, { overrides: { colour: "red" } as never }, "user")).rejects.toThrow(
ValidationError,
);
});
test("other users' entries and private works are not found", async () => {
const db = getDb();
const ada = await makeUser(db);
const bob = await makeUser(db, "bob");
const draft = await quickAdd(db, ada, { title: "Secret" });
await expect(getEntry(db, bob, draft.id)).rejects.toThrow(NotFoundError);
await expect(upsertEntry(db, bob, draft.workId, {}, "user")).rejects.toThrow(NotFoundError);
await expect(softDeleteEntry(db, bob, draft.id, "user")).rejects.toThrow(NotFoundError);
await expect(moveEntryToWork(db, bob, draft.id, await makeWork(db), "user")).rejects.toThrow(NotFoundError);
await expect(getEntry(db, ada, newId())).rejects.toThrow(NotFoundError);
});
test("softDeleteEntry hides the entry and upsert revives it", async () => {
const db = getDb();
const userId = await makeUser(db);
const workId = await makeWork(db);
const e = await upsertEntry(db, userId, workId, { rating: 40 }, "user");
await softDeleteEntry(db, userId, e.id, "user");
await expect(getEntry(db, userId, e.id)).rejects.toThrow(NotFoundError);
expect((await listEntries(db, userId, {})).items).toHaveLength(0);
const again = await upsertEntry(db, userId, workId, { status: "planned" }, "user");
expect(again.id).toBe(e.id);
expect(again.deletedAt).toBeNull();
expect(again.rating).toBe(40);
});
test("moveEntryToWork leaves alias", async () => {
const db = getDb();
const userId = await makeUser(db);
const draft = await quickAdd(db, userId, { title: "Dune??", mediaType: "book" });
const shared = await makeWork(db, { title: "Dune", externalIds: { isbn13: "9780441013593" } });
const moved = await moveEntryToWork(db, userId, draft.id, shared, "user");
expect(moved.id).toBe(draft.id);
expect(moved.workId).toBe(shared);
const alias = await db.k
.selectFrom("work_aliases")
.selectAll()
.where("user_id", "=", userId)
.executeTakeFirstOrThrow();
expect(alias.from_work_id).toBe(draft.workId);
expect(alias.to_work_id).toBe(shared);
const old = await db.k.selectFrom("works").selectAll().where("id", "=", draft.workId).executeTakeFirstOrThrow();
expect(old.merged_into_id).toBe(shared);
expect((await getEntry(db, userId, draft.id)).draft).toBe(false);
const [h] = await getHistory(db, userId, "entry", draft.id);
expect(h!.after).toEqual({ workId: shared });
});
test("moveEntryToWork merges into an existing entry on the target", async () => {
const db = getDb();
const userId = await makeUser(db);
const a = await makeWork(db, { title: "A" });
const b = await makeWork(db, { title: "B" });
const src = await upsertEntry(db, userId, a, { rating: 70, tags: ["x"] }, "user");
const dst = await upsertEntry(db, userId, b, { status: "completed", tags: ["y"] }, "user");
const merged = await moveEntryToWork(db, userId, src.id, b, "user");
expect(merged.id).toBe(dst.id);
expect(merged.status).toBe("completed");
expect(merged.rating).toBe(70);
expect(merged.tags.sort()).toEqual(["x", "y"]);
await expect(getEntry(db, userId, src.id)).rejects.toThrow(NotFoundError);
});
test("merge keeps 'user' field_sources on combined fields", async () => {
const db = getDb();
const userId = await makeUser(db);
const a = await makeWork(db, { title: "A" });
const b = await makeWork(db, { title: "B" });
const src = await upsertEntry(db, userId, a, { tags: ["x"], overrides: { year: 1 } }, "plugin:trakt:c1");
await upsertEntry(db, userId, b, { tags: ["y"], status: "completed" }, "user");
await upsertEntry(db, userId, b, { overrides: { title: "B!" } }, "plugin:hardcover:c2");
const merged = await moveEntryToWork(db, userId, src.id, b, "plugin:steam:c3");
expect(merged.tags.sort()).toEqual(["x", "y"]);
expect(merged.overrides).toEqual({ year: 1, title: "B!" });
expect(merged.fieldSources).toEqual({ tags: "user", status: "user", overrides: "plugin:hardcover:c2" });
});
test("merge into a soft-deleted entry carries the moved field_sources as-is", async () => {
const db = getDb();
const userId = await makeUser(db);
const a = await makeWork(db, { title: "A" });
const b = await makeWork(db, { title: "B" });
const src = await upsertEntry(db, userId, a, { rating: 60, review: "mine" }, "user");
const dst = await upsertEntry(db, userId, b, { status: "dropped", tags: ["old"] }, "plugin:trakt:c1");
await softDeleteEntry(db, userId, dst.id, "user");
const merged = await moveEntryToWork(db, userId, src.id, b, "plugin:trakt:c2");
expect(merged.id).toBe(dst.id);
expect(merged.deletedAt).toBeNull();
expect(merged.rating).toBe(60);
expect(merged.status).toBe("none");
expect(merged.tags).toEqual([]);
expect(merged.fieldSources).toEqual({ rating: "user", review: "user" });
});
test("merge concatenates differing text, records the dropped rating and moves engagements with history", async () => {
const db = getDb();
const userId = await makeUser(db);
const a = await makeWork(db, { title: "A" });
const b = await makeWork(db, { title: "B" });
const src = await upsertEntry(db, userId, a, { review: "from A", privateNotes: "same", rating: 40 }, "user");
const dst = await upsertEntry(db, userId, b, { review: "from B", privateNotes: "same", rating: 90 }, "user");
const eg = await addEngagement(db, userId, src.id, { platform: "Kindle" }, "user");
const merged = await moveEntryToWork(db, userId, src.id, b, "user");
expect(merged.review).toBe("from B\n\n---\n\nfrom A");
expect(merged.privateNotes).toBe("same");
expect(merged.rating).toBe(90);
const [h] = await getHistory(db, userId, "entry", dst.id);
expect(h!.before.rating).toBe(40);
expect(h!.after.rating).toBe(90);
expect(h!.before.review).toBe("from B");
expect((await listEngagements(db, userId, dst.id)).map((e) => e.id)).toEqual([eg.id]);
const [egh] = await getHistory(db, userId, "engagement", eg.id);
expect(egh!.before).toEqual({ entryId: src.id });
expect(egh!.after).toEqual({ entryId: dst.id });
});
});
+224
View File
@@ -0,0 +1,224 @@
import { afterEach, beforeEach, expect, test } from "bun:test";
import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { strFromU8, unzipSync, zipSync, strToU8 } from "fflate";
import { forEachDialect } from "@sttracker/db/testing";
import { storeAttachment } from "../src/attachments.ts";
import { ImportError } from "../src/errors.ts";
import { moveEntryToWork, quickAdd, upsertEntry } from "../src/entries.ts";
import { exportUserBundle, exportUserCsv, importUserBundle } from "../src/export.ts";
import { makeUser, makeWork } from "./user-layer-fixtures.ts";
forEachDialect("export bundle", (getDb) => {
let dir: string;
let dir2: string;
beforeEach(async () => {
dir = await mkdtemp(join(tmpdir(), "sttracker-exp-"));
dir2 = await mkdtemp(join(tmpdir(), "sttracker-exp-"));
});
afterEach(async () => {
await rm(dir, { recursive: true, force: true });
await rm(dir2, { recursive: true, force: true });
});
async function seed(db: ReturnType<typeof getDb>, u: string) {
const shared = await makeWork(db, { title: "Dune", mediaType: "book", year: 1965, externalIds: { hardcover: "dune" } });
const parent = await makeWork(db, { title: "Show", mediaType: "tv", externalIds: { trakt: "show" } });
const child = await makeWork(db, { title: "Pilot", mediaType: "tv", parentId: parent, externalIds: { trakt: "pilot" } });
const priv = await makeWork(db, { title: "My Zine", mediaType: "book", ownerUserId: u });
const now = "2024-01-02T03:04:05.000Z";
const ent = async (id: string, workId: string, extra: object = {}) =>
db.k.insertInto("entries").values({
id, user_id: u, work_id: workId, status: "completed", rating: 85, review: "great", private_notes: null, favorite: 1,
tags: '["a","b"]', custom_fields: "{}", visibility: "private", overrides: "{}", media_type_override: null,
field_sources: '{"rating":"user"}', created_at: now, updated_at: now, deleted_at: null, ...extra,
}).execute();
const defId = crypto.randomUUID();
await db.k.insertInto("custom_field_definitions").values({ id: defId, user_id: u, name: "Shelf", type: "text", options: "[]", applies_to: '"all"', level: "entry", created_at: now }).execute();
const e1 = crypto.randomUUID(), e2 = crypto.randomUUID(), e3 = crypto.randomUUID(), e4 = crypto.randomUUID();
await ent(e1, shared, { overrides: '{"title":"Dune X"}', visibility: "public", custom_fields: JSON.stringify({ [defId]: "top shelf" }) });
await ent(e2, child);
await ent(e3, priv, { rating: null });
await ent(e4, parent, { deleted_at: now });
const eng = crypto.randomUUID();
await db.k.insertInto("engagements").values({
id: eng, entry_id: e1, started_at: "1999-05", started_precision: "month", finished_at: "2000", finished_precision: "year",
progress: "{}", platform: "kindle", rating: 70, notes: "n", custom_fields: "{}", field_sources: "{}", created_at: now, updated_at: now, deleted_at: null,
}).execute();
await db.k.insertInto("copies").values({ id: crypto.randomUUID(), user_id: u, work_id: shared, title_hint: null, format: "hardcover", acquired_from: "shop", acquired_at: "2020-03", acquired_precision: "month", price_cents: 1299, currency: "EUR", notes: null, custom_fields: "{}", source_ref: null, created_at: now, updated_at: now, deleted_at: null }).execute();
await db.k.insertInto("listens").values({ id: crypto.randomUUID(), user_id: u, listened_at: now, artist: "Artist", track: "Track", album: null, duration_ms: 1000, mbids: "{}", work_id: null, source: "lastfm", source_key: "k1", created_at: now }).execute();
const grp = crypto.randomUUID();
await db.k.insertInto("lists").values({ id: grp, user_id: u, name: "Grouped", description: "", kind: "grouping", visibility: "public", rating: 90, review: "nice set", created_at: now, updated_at: now, deleted_at: null }).execute();
const list = crypto.randomUUID();
await db.k.insertInto("lists").values({ id: list, user_id: u, name: "Faves", description: "d", kind: "list", visibility: "private", rating: null, review: null, created_at: now, updated_at: now, deleted_at: null }).execute();
await db.k.insertInto("list_items").values({ id: crypto.randomUUID(), list_id: list, work_id: shared, position: 0, note: "first" }).execute();
await db.k.insertInto("user_history").values({ id: crypto.randomUUID(), user_id: u, entity: "entry", entity_id: e1, before: "{}", after: '{"rating":85}', actor: "user", created_at: now }).execute();
await db.k.insertInto("user_history").values({ id: crypto.randomUUID(), user_id: u, entity: "entry", entity_id: e1, before: "{}", after: JSON.stringify({ workId: shared, note: "moved" }), actor: "user", created_at: now }).execute();
const att = await storeAttachment(db, dir, u, { bytes: new TextEncoder().encode("hello"), mime: "text/plain", filename: "n.txt", caption: "a caption" }, { entity: "entry", entityId: e1 });
await db.k.insertInto("attachment_links").values({ id: crypto.randomUUID(), attachment_id: att.id, entity: "engagement", entity_id: eng }).execute();
return { e1, shared, defId };
}
test("export then import into fresh user reproduces entries, ratings, engagements with precision, lists and attachments", async () => {
const db = getDb();
const u = await makeUser(db, "ada");
const { defId } = await seed(db, u);
// A second user's data must never leak into ada's bundle.
const other = await makeUser(db, "eve");
const otherWork = await makeWork(db, { title: "Other Zine", mediaType: "book", ownerUserId: other });
const oe = crypto.randomUUID();
await db.k.insertInto("entries").values({ id: oe, user_id: other, work_id: otherWork, status: "planned", rating: null, review: null, private_notes: "secret", favorite: 0, tags: "[]", custom_fields: "{}", visibility: "private", overrides: "{}", media_type_override: null, field_sources: "{}", created_at: "2024-01-01T00:00:00.000Z", updated_at: "2024-01-01T00:00:00.000Z", deleted_at: null }).execute();
await storeAttachment(db, dir, other, { bytes: new TextEncoder().encode("hello"), mime: "text/plain", filename: "eve.txt" }, { entity: "entry", entityId: oe });
const zip = await exportUserBundle(db, dir, u);
const files = unzipSync(zip);
const allText = Object.entries(files).filter(([n]) => n.endsWith(".jsonl")).map(([, b]) => strFromU8(b)).join("\n");
expect(allText).not.toContain("Other Zine");
expect(allText).not.toContain("secret");
expect(strFromU8(files["attachments.jsonl"]!).trim().split("\n")).toHaveLength(1);
const v = await makeUser(db, "bob");
expect(await importUserBundle(db, dir2, v, zip)).toEqual({ works: 4, entries: 4 });
expect(await db.k.selectFrom("entries").select("id").where("user_id", "=", other).execute()).toHaveLength(1);
const rows = await db.k.selectFrom("entries").innerJoin("works", "works.id", "entries.work_id").select(["works.title", "entries.rating", "entries.favorite", "entries.field_sources", "entries.deleted_at", "entries.created_at", "works.owner_user_id", "works.parent_id", "works.id as wid"]).where("entries.user_id", "=", v).execute();
const byTitle = Object.fromEntries(rows.map((r) => [r.title, r]));
expect(rows).toHaveLength(4);
expect(byTitle["Dune"]!.rating).toBe(85);
expect(byTitle["Dune"]!.favorite).toBe(1);
expect(byTitle["Dune"]!.field_sources).toBe('{"rating":"user"}');
expect(byTitle["Dune"]!.created_at).toBe("2024-01-02T03:04:05.000Z");
expect(byTitle["Show"]!.deleted_at).not.toBeNull();
expect(byTitle["My Zine"]!.owner_user_id).toBe(v);
expect(byTitle["Pilot"]!.parent_id).toBe(byTitle["Show"]!.wid);
// shared works were re-matched, not duplicated
expect(await db.k.selectFrom("works").select("id").where("title", "=", "Dune").execute()).toHaveLength(1);
const eng = await db.k.selectFrom("engagements").innerJoin("entries", "entries.id", "engagements.entry_id").selectAll("engagements").where("entries.user_id", "=", v).execute();
expect(eng).toHaveLength(1);
expect(eng[0]).toMatchObject({ started_at: "1999-05", started_precision: "month", finished_precision: "year", platform: "kindle" });
const lists = await db.k.selectFrom("lists").selectAll().where("user_id", "=", v).execute();
expect(lists).toHaveLength(2);
const items = await db.k.selectFrom("list_items").selectAll().where("list_id", "=", lists.find((l) => l.name === "Faves")!.id).execute();
expect(items).toMatchObject([{ note: "first", work_id: byTitle["Dune"]!.wid }]);
const hist = await db.k.selectFrom("user_history").selectAll().where("user_id", "=", v).execute();
expect(hist).toHaveLength(2);
const dune = await db.k.selectFrom("entries").select("id").where("user_id", "=", v).where("work_id", "=", byTitle["Dune"]!.wid).executeTakeFirstOrThrow();
expect(hist.every((h) => h.entity_id === dune.id)).toBe(true);
const atts = await db.k.selectFrom("attachments").innerJoin("attachment_links", "attachment_links.attachment_id", "attachments.id").selectAll("attachments").select("attachment_links.entity_id").where("attachments.user_id", "=", v).execute();
expect(atts).toHaveLength(2);
expect(atts.every((a) => a.caption === "a caption")).toBe(true);
const engRow = eng[0]!;
expect(atts.map((a) => a.entity_id).sort()).toEqual([dune.id, engRow.id].sort());
const full = await db.k.selectFrom("entries").selectAll().where("id", "=", dune.id).executeTakeFirstOrThrow();
expect(full).toMatchObject({ visibility: "public", overrides: '{"title":"Dune X"}' });
const newDef = await db.k.selectFrom("custom_field_definitions").selectAll().where("user_id", "=", v).executeTakeFirstOrThrow();
expect(newDef.id).not.toBe(defId);
expect(JSON.parse(full.custom_fields)).toEqual({ [newDef.id]: "top shelf" });
const copy = await db.k.selectFrom("copies").selectAll().where("user_id", "=", v).executeTakeFirstOrThrow();
expect(copy).toMatchObject({ price_cents: 1299, currency: "EUR", acquired_precision: "month", work_id: byTitle["Dune"]!.wid });
expect(await db.k.selectFrom("listens").select("id").where("user_id", "=", v).execute()).toHaveLength(1);
const grouping = await db.k.selectFrom("lists").selectAll().where("user_id", "=", v).where("kind", "=", "grouping").executeTakeFirstOrThrow();
expect(grouping).toMatchObject({ rating: 90, review: "nice set", visibility: "public" });
const remapped = (await db.k.selectFrom("user_history").selectAll().where("user_id", "=", v).execute()).find((h) => h.after.includes("moved"))!;
expect(JSON.parse(remapped.after).workId).toBe(byTitle["Dune"]!.wid);
expect(await Bun.file(join(dir2, "attachments", atts[0]!.sha256.slice(0, 2), atts[0]!.sha256.slice(2, 4), atts[0]!.sha256)).text()).toBe("hello");
});
test("bundle has no credential fields", async () => {
const db = getDb();
const u = await makeUser(db, "ada");
await seed(db, u);
const files = unzipSync(await exportUserBundle(db, dir, u));
expect(Object.keys(files)).toContain("manifest.json");
const keys = new Set<string>();
const walk = (v: unknown) => {
if (Array.isArray(v)) v.forEach(walk);
else if (v && typeof v === "object") for (const [k, x] of Object.entries(v)) (keys.add(k.toLowerCase().replaceAll("_", "")), walk(x));
};
for (const [n, b] of Object.entries(files)) {
if (n.endsWith(".jsonl")) strFromU8(b).split("\n").filter(Boolean).forEach((l) => walk(JSON.parse(l)));
}
expect(keys.size).toBeGreaterThan(10);
for (const bad of ["passwordhash", "tokenhash", "credentialsenc", "codehash"]) expect(keys.has(bad)).toBe(false);
expect(JSON.parse(strFromU8(files["manifest.json"]!)).format).toBe("sttracker-export/v1");
});
test("import into non-empty account refused", async () => {
const db = getDb();
const u = await makeUser(db, "ada");
await seed(db, u);
const zip = await exportUserBundle(db, dir, u);
const err = await importUserBundle(db, dir2, u, zip).catch((e) => e);
expect(err).toBeInstanceOf(ImportError);
expect(err.code).toBe("account_not_empty");
});
test("import of an unknown format is refused", async () => {
const db = getDb();
const v = await makeUser(db, "bob");
const zip = zipSync({ "manifest.json": strToU8('{"format":"other"}') });
const err = await importUserBundle(db, dir2, v, zip).catch((e) => e);
expect(err.code).toBe("unsupported_format");
});
test("csv export has entries, engagements and listens", async () => {
const db = getDb();
const u = await makeUser(db, "ada");
await seed(db, u);
const csv = await exportUserCsv(db, u);
expect(Object.keys(csv).sort()).toEqual(["engagements.csv", "entries.csv", "listens.csv"]);
expect(csv["entries.csv"]).toContain("Dune");
expect(csv["entries.csv"]).not.toContain("Show,");
expect(csv["engagements.csv"]).toContain("kindle");
});
test("a per-user move restores as alias plus tombstone and later upserts resolve to the moved entry", async () => {
const db = getDb();
const u = await makeUser(db, "ada");
const draft = await quickAdd(db, u, { title: "Dune??", mediaType: "book" });
const shared = await makeWork(db, { title: "Dune", mediaType: "book", externalIds: { isbn13: "9780441013593" } });
await moveEntryToWork(db, u, draft.id, shared, "user");
const zip = await exportUserBundle(db, dir, u);
const v = await makeUser(db, "bob");
await importUserBundle(db, dir2, v, zip);
const alias = await db.k.selectFrom("work_aliases").selectAll().where("user_id", "=", v).executeTakeFirstOrThrow();
expect(alias.to_work_id).toBe(shared);
const tomb = await db.k.selectFrom("works").selectAll().where("id", "=", alias.from_work_id).executeTakeFirstOrThrow();
expect(tomb).toMatchObject({ merged_into_id: shared, owner_user_id: v });
const moved = await upsertEntry(db, v, tomb.id, { rating: 10 }, "user");
expect(moved.workId).toBe(shared);
expect(await db.k.selectFrom("entries").select("id").where("user_id", "=", v).execute()).toHaveLength(1);
});
test("an invalid attachment aborts the import with nothing written", async () => {
const db = getDb();
const u = await makeUser(db, "ada");
await seed(db, u);
const files = unzipSync(await exportUserBundle(db, dir, u));
const key = Object.keys(files).find((n) => n.startsWith("attachments/"))!;
files[key] = new TextEncoder().encode("tampered");
const v = await makeUser(db, "bob");
const err = await importUserBundle(db, dir2, v, zipSync(files)).catch((e) => e);
expect(err.code).toBe("invalid_attachment");
expect(await db.k.selectFrom("entries").select("id").where("user_id", "=", v).execute()).toHaveLength(0);
expect(await db.k.selectFrom("attachments").select("id").where("user_id", "=", v).execute()).toHaveLength(0);
});
test("an out-of-range rating is a corrupt bundle", async () => {
const db = getDb();
const u = await makeUser(db, "ada");
await seed(db, u);
const files = unzipSync(await exportUserBundle(db, dir, u));
files["entries.jsonl"] = strToU8(strFromU8(files["entries.jsonl"]!).replaceAll('"rating":85', '"rating":185'));
const v = await makeUser(db, "bob");
const err = await importUserBundle(db, dir2, v, zipSync(files)).catch((e) => e);
expect(err.code).toBe("corrupt_bundle");
});
});
+1
View File
@@ -0,0 +1 @@
export default { nope: true };
+36
View File
@@ -0,0 +1,36 @@
import { definePlugin, type SourceRecord } from "@sttracker/plugin-sdk/types";
export default definePlugin({
manifest: {
id: "echo",
name: "Echo",
version: "0.0.1",
sdkVersion: 1,
mediaTypes: ["book"],
capabilities: ["pull", "pullChanges"],
config: [
{ key: "label", label: "Label", type: "string" },
{ key: "token", label: "Token", type: "secret" },
],
rateLimit: { perMinute: 30, burst: 2 },
},
async *pull(ctx) {
const cfg = ctx.config;
if (cfg.label === "explode") {
await ctx.credentials.save({ refresh: "new-refresh" });
throw new Error("boom after save");
}
ctx.log("echo pulled");
const rec: SourceRecord = {
kind: "entry",
key: "echo-1",
raw: { label: cfg.label ?? null },
work: { title: String(cfg.label ?? "Echo"), mediaType: "book", externalIds: { isbn13: "9780000000002" } },
entry: { status: "completed" },
};
yield rec;
},
async *pullChanges(ctx) {
yield* this.pull!(ctx);
},
});
+114
View File
@@ -0,0 +1,114 @@
import { expect, test } from "bun:test";
import { forEachDialect } from "@sttracker/db/testing";
import { defineCustomField } from "../src/custom-fields.ts";
import { NotFoundError } from "../src/errors.ts";
import { getEntry, moveEntryToWork, quickAdd, softDeleteEntry, upsertEntry } from "../src/entries.ts";
import { getHistory, recordHistory, restoreVersion } from "../src/history.ts";
import { makeUser, makeWork } from "./user-layer-fixtures.ts";
forEachDialect("history", (getDb) => {
test("recordHistory stores only the changed keys", async () => {
const db = getDb();
const userId = await makeUser(db);
const id = await recordHistory(db.k, {
userId,
entity: "entry",
entityId: "e1",
before: { status: "planned", rating: 50, tags: ["a"] },
after: { status: "completed", rating: 50, tags: ["a"] },
actor: "user",
});
expect(id).not.toBeNull();
const none = await recordHistory(db.k, {
userId,
entity: "entry",
entityId: "e1",
before: { rating: 1 },
after: { rating: 1 },
actor: "user",
});
expect(none).toBeNull();
const rows = await getHistory(db, userId, "entry", "e1");
expect(rows).toHaveLength(1);
expect(rows[0]!.before).toEqual({ status: "planned" });
expect(rows[0]!.after).toEqual({ status: "completed" });
expect(rows[0]!.actor).toBe("user");
});
test("restoreVersion brings back previous review", async () => {
const db = getDb();
const userId = await makeUser(db);
const workId = await makeWork(db);
const e = await upsertEntry(db, userId, workId, { review: "first take" }, "user");
await upsertEntry(db, userId, workId, { review: "second take", rating: 60 }, "plugin:trakt:c1");
const history = await getHistory(db, userId, "entry", e.id);
expect(history).toHaveLength(2);
const latest = history[0]!; // newest first
expect(latest.before).toEqual({ review: "first take", rating: null });
expect(latest.actor).toBe("plugin:trakt:c1");
await restoreVersion(db, userId, latest.id);
const view = await getEntry(db, userId, e.id);
expect(view.review).toBe("first take");
expect(view.rating).toBeNull();
expect(view.fieldSources.review).toBe("user");
expect(view.fieldSources.rating).toBe("user");
expect(await getHistory(db, userId, "entry", e.id)).toHaveLength(3);
});
test("restoreVersion undoes a soft delete", async () => {
const db = getDb();
const userId = await makeUser(db);
const e = await upsertEntry(db, userId, await makeWork(db), { status: "planned" }, "user");
await softDeleteEntry(db, userId, e.id, "user");
const [del] = await getHistory(db, userId, "entry", e.id);
expect(del!.after.deletedAt).toBeString();
await restoreVersion(db, userId, del!.id);
expect((await getEntry(db, userId, e.id)).status).toBe("planned");
});
test("restoreVersion undoes a move back to the draft work", async () => {
const db = getDb();
const userId = await makeUser(db);
const draft = await quickAdd(db, userId, { title: "Dune??" });
const shared = await makeWork(db, { title: "Dune", externalIds: { isbn13: "1" } });
await moveEntryToWork(db, userId, draft.id, shared, "user");
const [move] = await getHistory(db, userId, "entry", draft.id);
await restoreVersion(db, userId, move!.id);
const back = await getEntry(db, userId, draft.id);
expect(back.workId).toBe(draft.workId);
expect(back.draft).toBe(true);
const work = await db.k.selectFrom("works").selectAll().where("id", "=", draft.workId).executeTakeFirstOrThrow();
expect(work.merged_into_id).toBeNull();
// undoing removes the forward alias and writes no reverse alias
expect(await db.k.selectFrom("work_aliases").selectAll().where("user_id", "=", userId).execute()).toEqual([]);
// a later sync onto the shared work lands on the shared work, not on the draft
const synced = await upsertEntry(db, userId, shared, { rating: 80 }, "plugin:hardcover:c1");
expect(synced.workId).toBe(shared);
expect(synced.id).not.toBe(draft.id);
expect((await getEntry(db, userId, draft.id)).rating).toBeNull();
});
test("restoreVersion does not re-check custom fields against today's definitions", async () => {
const db = getDb();
const userId = await makeUser(db);
const def = await defineCustomField(db, userId, { name: "Mood", type: "text" });
const e = await upsertEntry(db, userId, await makeWork(db), { customFields: { [def.id]: "cosy" } }, "user");
await upsertEntry(db, userId, e.workId, { customFields: {} }, "user");
await db.k.deleteFrom("custom_field_definitions").where("id", "=", def.id).execute();
const [h] = await getHistory(db, userId, "entry", e.id);
await restoreVersion(db, userId, h!.id);
expect((await getEntry(db, userId, e.id)).customFields).toEqual({ [def.id]: "cosy" });
});
test("another user's history is invisible", async () => {
const db = getDb();
const ada = await makeUser(db);
const bob = await makeUser(db, "bob");
const e = await upsertEntry(db, ada, await makeWork(db), { review: "x" }, "user");
expect(await getHistory(db, bob, "entry", e.id)).toEqual([]);
const [h] = await getHistory(db, ada, "entry", e.id);
await expect(restoreVersion(db, bob, h!.id)).rejects.toThrow(NotFoundError);
});
});
+195
View File
@@ -0,0 +1,195 @@
import { expect, spyOn, test } from "bun:test";
import { forEachDialect } from "@sttracker/db/testing";
import { DropJobError, enqueue, NonRetryableError, startScheduler, startWorker } from "../src/jobs.ts";
import { makeConnection, makeUser } from "./sync/helpers.ts";
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
forEachDialect("jobs", (getDb) => {
test("enqueue with same lockKey returns null", async () => {
const db = getDb();
expect(await enqueue(db, "sync", {}, { lockKey: "sync:a" })).toBeString();
expect(await enqueue(db, "sync", {}, { lockKey: "sync:a" })).toBeNull();
expect(await enqueue(db, "sync", {}, { lockKey: "sync:b" })).toBeString();
expect(await enqueue(db, "x", {})).toBeString();
expect(await enqueue(db, "x", {})).toBeString();
});
test("concurrent enqueue with one lockKey admits exactly one", async () => {
const db = getDb();
const ids = await Promise.all(Array.from({ length: 6 }, () => enqueue(db, "sync", {}, { lockKey: "k" })));
expect(ids.filter((i) => i !== null)).toHaveLength(1);
});
test("worker runs a job and releases the lock", async () => {
const db = getDb();
const seen: unknown[] = [];
const id = await enqueue(db, "t", { n: 1 }, { lockKey: "l" });
const w = startWorker(db, { t: async (p) => void seen.push(p) }, { pollMs: 10 });
await sleep(150);
await w.stop();
expect(seen).toEqual([{ n: 1 }]);
expect((await db.k.selectFrom("jobs").selectAll().where("id", "=", id!).executeTakeFirstOrThrow()).status).toBe("done");
expect(await enqueue(db, "t", {}, { lockKey: "l" })).toBeString();
});
test("a job is claimed by only one of two workers", async () => {
const db = getDb();
let runs = 0;
const h = { t: async () => { runs++; await sleep(30); } };
await enqueue(db, "t", {});
const a = startWorker(db, h, { pollMs: 5 });
const b = startWorker(db, h, { pollMs: 5 });
await sleep(200);
await Promise.all([a.stop(), b.stop()]);
expect(runs).toBe(1);
});
test("worker retries then fails", async () => {
const db = getDb();
let calls = 0;
const id = (await enqueue(db, "bad", {}))!;
const w = startWorker(db, { bad: async () => { calls++; throw new Error("nope"); } }, { pollMs: 10 });
const row = () => db.k.selectFrom("jobs").selectAll().where("id", "=", id).executeTakeFirstOrThrow();
for (let attempt = 1; attempt <= 5; attempt++) {
for (let i = 0; i < 100 && (await row()).attempts < attempt; i++) await sleep(10);
const r = await row();
expect(r.attempts).toBe(attempt);
expect(r.last_error).toBe("nope");
if (attempt < 5) {
expect(r.status).toBe("queued");
const delay = Date.parse(r.run_at) - Date.now();
expect(delay).toBeGreaterThan(2 ** attempt * 60_000 - 5_000);
expect(delay).toBeLessThanOrEqual(2 ** attempt * 60_000);
await db.k.updateTable("jobs").set({ run_at: new Date(Date.now() - 1000).toISOString() }).where("id", "=", id).execute();
}
}
await w.stop();
expect((await row()).status).toBe("failed");
expect(calls).toBe(5);
});
test("scheduler enqueues due connections only", async () => {
const db = getDb();
const u = await makeUser(db);
const due = await makeConnection(db, u, "p");
const fresh = await makeConnection(db, u, "p");
const manual = await makeConnection(db, u, "p");
const errored = await makeConnection(db, u, "p");
const paused = await makeConnection(db, u, "p");
const old = new Date(Date.now() - 120 * 60_000).toISOString();
const recent = new Date(Date.now() - 5 * 60_000).toISOString();
const set = (id: string, v: object) => db.k.updateTable("source_connections").set(v).where("id", "=", id).execute();
await set(due, { schedule_minutes: 60, last_run_at: old });
await set(fresh, { schedule_minutes: 60, last_run_at: recent });
await set(errored, { schedule_minutes: 60, last_run_at: old, status: "error" });
await set(paused, { schedule_minutes: 60, last_run_at: old, status: "paused" });
void manual;
const s = startScheduler(db, { tickMs: 3_600_000 });
await s.tickNow();
await s.tickNow();
s.stop();
const jobs = await db.k.selectFrom("jobs").selectAll().execute();
expect(jobs.map((j) => j.lock_key).sort()).toEqual([`sync:${due}`, `sync:${errored}`].sort());
expect(JSON.parse(jobs.find((j) => j.lock_key === `sync:${due}`)!.payload)).toEqual({ connectionId: due });
});
const row = (db: ReturnType<typeof getDb>, id: string) => db.k.selectFrom("jobs").selectAll().where("id", "=", id).executeTakeFirstOrThrow();
async function until(fn: () => Promise<boolean>) {
for (let i = 0; i < 200 && !(await fn()); i++) await sleep(10);
}
test("NonRetryableError fails at once; DropJobError completes with last_error", async () => {
const db = getDb();
const a = (await enqueue(db, "a", {}))!;
const b = (await enqueue(db, "b", {}))!;
const w = startWorker(db, { a: async () => { throw new NonRetryableError("fatal"); }, b: async () => { throw new DropJobError("gone"); } }, { pollMs: 10 });
await until(async () => (await row(db, a)).status !== "queued" && (await row(db, b)).status !== "queued");
await w.stop();
expect(await row(db, a)).toMatchObject({ status: "failed", attempts: 1, last_error: "fatal" });
expect(await row(db, b)).toMatchObject({ status: "done", last_error: "gone" });
});
test("a stale running job is re-queued and then runs; a fresh one is left", async () => {
const db = getDb();
const stale = (await enqueue(db, "t", {}, { lockKey: "s" }))!;
const fresh = (await enqueue(db, "never", {}))!;
await db.k.updateTable("jobs").set({ status: "running", locked_at: new Date(Date.now() - 2 * 3600_000).toISOString(), attempts: 2 }).where("id", "=", stale).execute();
await db.k.updateTable("jobs").set({ status: "running", locked_at: nowStr() }).where("id", "=", fresh).execute();
let ran = 0;
const w = startWorker(db, { t: async () => void ran++ }, { pollMs: 10 });
await until(async () => (await row(db, stale)).status === "done");
await w.stop();
expect(ran).toBe(1);
expect((await row(db, stale)).attempts).toBe(2);
expect((await row(db, fresh)).status).toBe("running");
});
test("failing bookkeeping is logged and stop() does not reject", async () => {
const db = getDb();
const err = spyOn(console, "error").mockImplementation(() => {});
await enqueue(db, "t", {});
let updates = 0;
const k = new Proxy(db.k, {
get(t, p) {
const v = Reflect.get(t, p, t);
if (p === "updateTable") return (...a: unknown[]) => { if (++updates > 1) throw new Error("db down"); return (v as Function).apply(t, a); };
return typeof v === "function" ? v.bind(t) : v;
},
});
let called = false;
const w = startWorker({ ...db, k } as typeof db, { t: async () => void (called = true) }, { pollMs: 10 });
await until(async () => called && err.mock.calls.length > 0);
await expect(w.stop()).resolves.toBeUndefined();
expect(err.mock.calls.some((c) => c[0] === "[jobs]")).toBe(true);
err.mockRestore();
});
test("a job running past staleMs is not reaped by its own worker and runs once", async () => {
const db = getDb();
const id = (await enqueue(db, "t", {}, { lockKey: "slow" }))!;
let runs = 0;
const w = startWorker(db, { t: async () => { runs++; await sleep(300); } }, { pollMs: 10, staleMs: 50, heartbeatMs: 20 });
await until(async () => (await row(db, id)).status === "done");
await w.stop();
expect(runs).toBe(1);
});
test("heartbeat refreshes locked_at", async () => {
const db = getDb();
const id = (await enqueue(db, "t", {}))!;
let release!: () => void;
const gate = new Promise<void>((r) => (release = r));
const w = startWorker(db, { t: () => gate }, { pollMs: 10, heartbeatMs: 20 });
await until(async () => (await row(db, id)).status === "running");
const first = (await row(db, id)).locked_at!;
await until(async () => (await row(db, id)).locked_at !== first);
expect((await row(db, id)).locked_at).not.toBe(first);
release();
await w.stop();
});
test("stop() re-queues an aborted job without burning an attempt or backoff", async () => {
const db = getDb();
const id = (await enqueue(db, "t", {}))!;
const before = (await row(db, id)).run_at;
let started = false;
const w = startWorker(db, { t: (_p, { signal }) => new Promise<void>((_, rej) => { started = true; signal.addEventListener("abort", () => rej(new Error("aborted"))); }) }, { pollMs: 10 });
await until(async () => started);
await w.stop();
expect(await row(db, id)).toMatchObject({ status: "queued", attempts: 0, run_at: before, locked_at: null });
});
test("stop() aborts the signal handed to handlers", async () => {
const db = getDb();
await enqueue(db, "t", {});
let aborted = false;
let started = false;
const w = startWorker(db, { t: (_p, { signal }) => new Promise<void>((res) => { started = true; signal.addEventListener("abort", () => { aborted = true; res(); }); }) }, { pollMs: 10 });
await until(async () => started);
await w.stop();
expect(aborted).toBe(true);
});
});
const nowStr = () => new Date().toISOString();
+104
View File
@@ -0,0 +1,104 @@
import { expect, test } from "bun:test";
import { forEachDialect } from "@sttracker/db/testing";
import { NotFoundError, ValidationError } from "../src/errors.ts";
import { getHistory, restoreVersion } from "../src/history.ts";
import {
addListItem,
createList,
getList,
listLists,
moveListItem,
removeListItem,
softDeleteList,
updateList,
} from "../src/lists.ts";
import { makeUser, makeWork } from "./user-layer-fixtures.ts";
forEachDialect("lists", (getDb) => {
test("grouping list can be rated", async () => {
const db = getDb();
const userId = await makeUser(db);
const g = await createList(db, userId, { name: "Silent Hill, everything", kind: "grouping", rating: 90, review: "Dread." });
expect(g.kind).toBe("grouping");
expect(g.rating).toBe(90);
expect(g.review).toBe("Dread.");
expect(g.visibility).toBe("private");
const up = await updateList(db, userId, g.id, { rating: 95 });
expect(up.rating).toBe(95);
await expect(createList(db, userId, { name: "Plain", kind: "list", rating: 50 })).rejects.toThrow(ValidationError);
await expect(createList(db, userId, { name: "Bad", kind: "grouping", rating: 101 })).rejects.toThrow(
ValidationError,
);
await expect(updateList(db, userId, g.id, { kind: "thread" })).rejects.toThrow(ValidationError);
await expect(createList(db, userId, { name: " " })).rejects.toThrow(ValidationError);
await expect(createList(db, userId, { name: "x", kind: "box" as never })).rejects.toThrow(ValidationError);
});
test("items keep order through add, move and remove", async () => {
const db = getDb();
const userId = await makeUser(db);
const l = await createList(db, userId, { name: "Cosmic dread", kind: "thread" });
const [a, b, c] = [await makeWork(db, { title: "A" }), await makeWork(db, { title: "B" }), await makeWork(db, { title: "C" })];
const ia = await addListItem(db, userId, l.id, { workId: a, note: "start here" });
const ib = await addListItem(db, userId, l.id, { workId: b });
const ic = await addListItem(db, userId, l.id, { workId: c });
const titles = async () => (await getList(db, userId, l.id)).items.map((i) => i.work.title);
expect(await titles()).toEqual(["A", "B", "C"]);
expect((await getList(db, userId, l.id)).items[0]!.note).toBe("start here");
await moveListItem(db, userId, ic.id, 0);
expect(await titles()).toEqual(["C", "A", "B"]);
await moveListItem(db, userId, ic.id, 99);
expect(await titles()).toEqual(["A", "B", "C"]);
await removeListItem(db, userId, ib.id);
expect(await titles()).toEqual(["A", "C"]);
expect((await getList(db, userId, l.id)).items.map((i) => i.position)).toEqual([0, 1]);
const inserted = await addListItem(db, userId, l.id, { workId: b, position: 1 });
expect(await titles()).toEqual(["A", "B", "C"]);
expect(inserted.position).toBe(1);
expect(ia.position).toBe(0);
});
test("removing an item is recorded and restorable", async () => {
const db = getDb();
const userId = await makeUser(db);
const l = await createList(db, userId, { name: "Reading order" });
await addListItem(db, userId, l.id, { workId: await makeWork(db, { title: "A" }) });
const ib = await addListItem(db, userId, l.id, { workId: await makeWork(db, { title: "B" }) });
await removeListItem(db, userId, ib.id);
const [h] = await getHistory(db, userId, "list", l.id);
await restoreVersion(db, userId, h!.id);
expect((await getList(db, userId, l.id)).items.map((i) => i.work.title)).toEqual(["A", "B"]);
});
test("listLists and soft delete", async () => {
const db = getDb();
const userId = await makeUser(db);
const l1 = await createList(db, userId, { name: "One" });
const l2 = await createList(db, userId, { name: "Two", kind: "grouping" });
expect((await listLists(db, userId)).map((l) => l.id).sort()).toEqual([l1.id, l2.id].sort());
expect((await listLists(db, userId, { kind: "grouping" })).map((l) => l.id)).toEqual([l2.id]);
await softDeleteList(db, userId, l1.id);
expect((await listLists(db, userId)).map((l) => l.id)).toEqual([l2.id]);
await expect(getList(db, userId, l1.id)).rejects.toThrow(NotFoundError);
});
test("another user's lists and private works are not found", async () => {
const db = getDb();
const ada = await makeUser(db);
const bob = await makeUser(db, "bob");
const l = await createList(db, ada, { name: "Mine" });
const item = await addListItem(db, ada, l.id, { workId: await makeWork(db) });
await expect(getList(db, bob, l.id)).rejects.toThrow(NotFoundError);
await expect(updateList(db, bob, l.id, { name: "x" })).rejects.toThrow(NotFoundError);
await expect(addListItem(db, bob, l.id, { workId: await makeWork(db) })).rejects.toThrow(NotFoundError);
await expect(moveListItem(db, bob, item.id, 0)).rejects.toThrow(NotFoundError);
await expect(removeListItem(db, bob, item.id)).rejects.toThrow(NotFoundError);
expect(await listLists(db, bob)).toEqual([]);
const adaPrivate = await makeWork(db, { ownerUserId: ada });
const bobList = await createList(db, bob, { name: "Bob's" });
await expect(addListItem(db, bob, bobList.id, { workId: adaPrivate })).rejects.toThrow(NotFoundError);
});
});

Some files were not shown because too many files have changed in this diff Show More