whisper_web/AGENTS.md

# whisper_web

Browser-first Whisper speech-to-text PWA. Bun 1.3 (runtime, bundler, test runner), React 19, TypeScript,
`bun:sqlite`, Passport.js; optional Python faster-whisper sidecar (`server-py/`) for the server engine.

`AGENTS.md` is the single source of truth for agents here. `CLAUDE.md`, `GEMINI.md`, and `docs/CLAUDE.md`
only point here. Deeper context on demand: `docs/Architecture.md`, `docs/Decisions.md` (`D-###` ids
cited throughout the code), `docs/WorkBreakdown.md` (milestones + backlog), `PRODUCT.md`, `DESIGN.md`.

## Commands

Install: `bun install --frozen-lockfile`
Dev: `bun dev`
Test one file: `bun test tests/export.test.ts` | One case: `bun test tests/models.test.ts -t "dtype"`
Test (fast, ~6 s): `bun test --path-ignore-patterns='tests/live-real-model.test.ts'`
Test all: `bun test` — ~17 min: `tests/live-real-model.test.ts` downloads and runs real Whisper
weights whenever huggingface.co is reachable. Run it only when touching the live pipeline or decoding.
Currently 1 known failure there (`en_alien` timeout — see WorkBreakdown "Noticed, not fixed").
Build: `bun run build` (rewrites `dist/`)
Opt-in real sidecar test (needs `pip install -r server-py/requirements.txt`): `FW_INTEGRATION=1 bun test tests/transcribe.test.ts`
Lint / typecheck: none configured (`typescript` is not installed). TODO(maintainer): confirm whether one is wanted.

Browser-only behaviour (offline PWA, mic, WebGPU) has no automated coverage and there is no browser in
the dev environment: follow `tests/offline.md`, `tests/live.md`, `tests/server-engine.md` and say
plainly when a manual check is still pending — never claim it was verified.

CI (`.gitlab-ci.yml`) only builds Docker images — nothing else runs your tests.

## Rules

- **Bun only.** No node/npm/npx/yarn/pnpm, vite/webpack, jest/vitest, express, `ws`, better-sqlite3,
  dotenv. Use `Bun.serve()` routes + HTML imports, `bun:sqlite`, built-in `WebSocket`, `Bun.file`,
  `Bun.$`. Bun API docs: `node_modules/bun-types/docs/`.
- **Two engines, two libraries (D-013).** Browser inference = Transformers.js, inside
  `src/worker/inference.worker.ts` only — never on the main thread. Server inference = faster-whisper
  via `server-py/sidecar.py` — never Transformers.js on the server.
- **`@huggingface/transformers` is patched and a devDependency on purpose.** It is pinned by
  `patches/@huggingface%2Ftransformers@4.2.0.patch` (D-027/D-028) and kept out of runtime deps so the
  Docker image stays small (D-031). Upgrading or moving it means redoing the patch: ask first.
- **No cross-origin subresources.** Every response carries COOP/COEP `require-corp` (D-023,
  `src/index.ts`); a CDN script/style/font will be blocked. Self-host it (onnxruntime is served from `/ort/`).
- **Selecting a model never downloads it (D-020).** Only a transcription start or the explicit Download
  button posts `load` to the worker.
- **Server modules take dependencies by injection** (`createTranscribeHandlers(deps)` pattern) so tests
  run against `tests/fixtures/fake_sidecar.py`; validate `language`/`task` with
  `parseLanguageAndTask` before anything reaches the sidecar (D-032). Every stateful server module has a test file.
- **Errors go through `classifyError` in `src/lib/errors.ts`** (pattern-matched, D-030) — don't add
  error class hierarchies.
- **Configuration is runtime env only (D-015)** — no build-time flags baked into the image. New
  variables go in README's env table.
- Anonymous users must keep every feature except the server engine; browser-engine audio never leaves
  the device (Architecture §5 invariants).

## Workflow

Required skills: `superpowers`, `impeccable`. Check at session start; install if missing.
Specs → `docs/specs/NNN-slug.md` (permanent). Plans → `docs/plans/NNN-slug.md` while in progress;
`git rm` them before merging — the spec is the record. (Overrides the superpowers default paths.)

Anything beyond a trivial fix: brainstorm → spec (sign-off) → plan (sign-off) → TDD → review.
TDD means failing test first, watch it fail, minimal code, watch it pass, commit.
Tick plan checkboxes as you go so an interrupted session is resumable.
Trivial changes (typo, version bump, one-line config): say "trivial, skipping spec" and proceed.

UI work: use impeccable — `craft` for new, `audit`/`polish` on existing, `harden` before shipping.
Design context: `PRODUCT.md`, `DESIGN.md`.

## Architecture

`src/index.ts` -> `Bun.serve` entry: routes, `/api/*`, isolation headers, dev worker bundling, prod `dist/` serving
`src/App.tsx`, `src/components/` -> React UI
`src/lib/` -> pure client logic (audio, live segmenter, model catalog, engines, errors) — most unit tests target this
`src/worker/` -> inference worker + typed postMessage protocol (shared with UI)
`src/pwa/` -> hand-rolled service worker (D-007); `src/server/` -> auth, db, sidecar supervisor, live WS
`server-py/` -> faster-whisper sidecar (JSON-lines over stdio)

Full picture: `docs/Architecture.md` — read it before your first change in a session.

## Boundaries

Never touch: `dist/` (build output), `src/pwa/icons/` (generated by `bun run icons`), `data/` (local SQLite).

Ask first: new runtime dependencies · changing or regenerating the transformers patch · auth/session
code (`src/server/auth.ts`, `src/server/db.ts`) · `Dockerfile` / `.gitlab-ci.yml` · anything
destructive · work the approved plan does not cover.

If the task turns out much larger than the plan assumed: stop, report, re-plan.

## Docs duties

Same commit as the code, not "later":
- `docs/Architecture.md` — when a module, boundary, or data flow changes.
- `docs/Decisions.md` — append at the **bottom** with the next `D-###` id; never rewrite an Accepted
  entry, supersede it. Cite the id in code comments where the decision bites.
- `docs/WorkBreakdown.md` — tick items as they land; anything noticed and deliberately not fixed goes
  under "Noticed, not fixed". No `TODO` comments in source.
- `docs/PERMISSIONS.md` — together with every tool permission file, in one commit.

## Git

Commit subjects: plain imperative sentence describing the change (e.g. "Show \"Loading model from
cache\" instead of \"Downloading\" when weights are already cached"). Small, each leaving tests green.
TODO(maintainer): confirm branch naming and whether work lands directly on `main`.

## Done means verified

Before saying done: relevant tests run *in this session* and seen passing · `bun run build` succeeds
if `src/` or `scripts/` changed · matches the approved spec · docs duties complete · `git status` clean
and the diff is only what you intended.