Lucas WintherandClaude Opus 5.5
5fa59310f3
- Tags and custom fields update the cached entry optimistically and build from the latest cache; one entry's PATCHes run in order (mutation scope). - The review textarea follows server changes (Restore) unless edited. - friendlyError() maps validation reasons and 413 to plain copy everywhere. - Rating scale from me/preferences; /login and /register use auth/status; Attachments lists from the API. - Gelasio vendored (official OFL variable TTFs); minors from the review. - Docs: DESIGN (AA on orange, faint labels), DECISIONS, TODO, ARCHITECTURE. Co-Authored-By: Claude Opus 5.5 <[email protected]>
70 lines
6.6 KiB
Markdown
70 lines
6.6 KiB
Markdown
# Architecture
|
||
|
||
How STTracker is built today. Update in the same commit as any structural change.
|
||
|
||
## Overview
|
||
Status 2026-10-30: only the workspace skeleton exists (root `package.json`, `tsconfig.json`).
|
||
The target design is in `docs/specs/001-sttracker-v1-design.md` §11. Replace each "planned" marker below as the code lands.
|
||
|
||
One Bun process serves the REST API, the React web app and plugin ingest endpoints, and runs a
|
||
database-backed job worker. Source plugins pull data from platforms and return normalised `SourceRecord`s.
|
||
The core snapshots them, diffs against the previous run, matches them to works, and either applies them to the
|
||
user layer or files a Changes-inbox event.
|
||
|
||
## Map
|
||
| Path | Responsibility | State |
|
||
|---|---|---|
|
||
| `apps/server` | `main.ts` boots (config → `createDb` → `migrateToLatest` → `loadPlugins` → worker + scheduler + daily backup → `Bun.serve`). `app.ts` `createApp(deps)` = `/api/v1` route table + `/ingest/*` mount (404 in Phase 1), testable via `app.fetch`. `http.ts` = `route()`, auth (session cookie / Bearer app token), CSRF header, body caps, rate limit, error mapping. `routes/<area>.ts` one file per area; `openapi.ts` builds `/api/v1/openapi.json` from the table | exists |
|
||
| `apps/web` | React 19 app served by the server as an HTML import (`index.html` → `src/main.tsx`, `src/theme.css`). wouter routes in `main.tsx` (`/login`, `/register`, `/`, `/entries/:id`; other sections are placeholders until Task 18); `api.ts` = fetch client (`request`, `ApiError`, cookie auth, CSRF header, mirrored wire types); `format.ts` = pure display helpers and `friendlyError`; `edits.ts` = next-value builders for entry edits; `hooks.ts` = TanStack Query hooks incl. optimistic `useEntryPatch`; `settings.ts` = preferences; `components/` (shell, detail boxes, history drawer, rating control) and `routes/`. Never imports `packages/*` at runtime | exists |
|
||
| `packages/core` | domain services, one module per area (import by path, no barrel): `auth`, `crypto`, `works`, `matching`, `entries` (entries, library listing, quick add, per-user moves, match suggestions), `engagements`, `history` (`user_history`, restore), `lists`, `copies`, `custom-fields`, `attachments`, `preferences` (per-user rating scale and default visibility), `export` (per-user bundle, CSV, restore), `backup` (scheduled bundles + SQLite `VACUUM INTO`), `sync/` (the change engine: `snapshot` hashing and versions, `apply` per-record diff and apply, `run` = `runSync`, `changes` = Changes inbox `listChanges`/`resolveChange`, `listens` = `ingestListens`), `jobs` (queue, worker, scheduler), `registry` (`loadPlugins`), `connections` (CRUD, plugin context, `syncHandler`) | exists (partial) |
|
||
| `packages/db` | Kysely over one Bun.SQL dialect (SQLite + Postgres), ids, JSON helpers, migrations, `forEachDialect` test harness | exists (schema pending) |
|
||
| `packages/plugin-sdk` | plugin contract, `SourceRecord`, http/rate-limit helpers, fixture test harness | planned |
|
||
| `plugins/<id>` | one source integration each (hardcover, trakt, mangabaka, listenbrainz, lastfm, csv) | planned |
|
||
| `docs/design/mockups` | approved UI mockups (reference only) | exists |
|
||
|
||
## Data flow
|
||
HTTP request (`apps/server`):
|
||
1. `Bun.serve` routes `/api/*` and `/ingest/*` to `app.fetch`; everything else gets the web app (HTML import).
|
||
2. `apiHandler` matches method + path in the route table, applies the per-IP rate limit (login/register), authenticates
|
||
(`session` cookie `stt_session`, Bearer app token with a scope, or either), requires `X-Requested-With: sttracker` on
|
||
every non-Bearer write (login included), reads the body (JSON ≤ 1 MB with `Content-Type: application/json`,
|
||
multipart ≤ 50 MB) and validates it with zod.
|
||
3. The handler calls one core function with the caller's `userId` and actor (`token:<name>` for Bearer, else `"user"`;
|
||
`field_sources` stores both as `"user"`). Core enforces ownership
|
||
(`NotFoundError` → 404). Errors become `{ error: { code, message } }`; unknown errors are logged and return 500 `internal`.
|
||
4. `POST sources/:id/run|import` only enqueue a `sync` job (lockKey `sync:<id>`, 409 when held); the worker runs it.
|
||
|
||
Sync lifecycle (`packages/core/src/sync/`; steps 1–2 are `jobs.ts` and `connections.ts`):
|
||
1. The scheduler enqueues a `sync` job (lockKey `sync:<connectionId>`) for each non-paused connection that is due.
|
||
2. The worker claims the job; `syncHandler` loads the connection and plugin and calls `runSync(deps, plugin, connectionId, mode)`, injecting
|
||
`makeContext` (http client, credentials, cursor). `mode` picks `pull`, `pullChanges` or `importFile`.
|
||
3. `runSync` opens a `source_runs` row and iterates the plugin's `SourceRecord`s one at a time.
|
||
- `listen` records are batched into `ingestListens` (insert-or-ignore into `listens`; no snapshot, no events).
|
||
- Every other record goes through `applyRecord`: hash its normalised form; same hash as
|
||
`source_records.current_hash` → only `last_seen` moves. Otherwise a new `source_record_versions` row
|
||
(gzipped `{normalized, raw}`) is written, and:
|
||
- **new record**: `matchWork` (own transaction, result committed on the record row so retries reuse it),
|
||
then in one apply transaction create/patch the entry, engagement (de-duplicating plays across sources)
|
||
or list, and write an `added` `auto_applied` event;
|
||
- **changed record**: fields that differ from the previous version are decided against `field_sources`
|
||
→ applied (`changed` `auto_applied`) or filed as `pending` `changed`/`conflict` events. Mapped records
|
||
are updated through `source_records.target_*`, never re-matched;
|
||
- `deleted: true` → `pending` `removed`; a new `mergedIntoKey` → `pending` `merged`.
|
||
4. After a `full` pull that completed and applied every record, records not seen are marked removed
|
||
(`source_records.deleted_at` + `pending` `removed`).
|
||
5. The run's counts, log and error land in `source_runs`; the connection gets `last_run_at` and `status`.
|
||
6. The user resolves `pending` events with `resolveChange` (accept / ignore / keep_mine).
|
||
7. Every user-layer write appends to `user_history` with the plugin actor `plugin:<pluginId>:<connectionId>`.
|
||
|
||
## External dependencies
|
||
- Database: SQLite file under `DATA_DIR` (default) or Postgres via `DATABASE_URL`. If unavailable, nothing works.
|
||
- Platform APIs (Hardcover, Trakt, MangaBaka, ListenBrainz, Last.fm). If unavailable, only that source's sync fails; the run is logged and retried.
|
||
|
||
## Invariants
|
||
- A sync never overwrites or hard-deletes user-entered data (spec §8.3).
|
||
- Plugins never touch the database.
|
||
- All domain behaviour is identical on SQLite and Postgres.
|
||
|
||
## Known rough edges
|
||
None yet. See `docs/TODO.md`.
|