whisper_web/AGENTS.md

whisper_web

Browser-first Whisper speech-to-text PWA. Bun 1.3 (runtime, bundler, test runner), React 19, TypeScript, bun:sqlite, Passport.js; optional Python faster-whisper sidecar (server-py/) for the server engine.

AGENTS.md is the single source of truth for agents here. CLAUDE.md, GEMINI.md, and docs/CLAUDE.md only point here. Deeper context on demand: docs/Architecture.md, docs/Decisions.md (D-### ids cited throughout the code), docs/WorkBreakdown.md (milestones + backlog), PRODUCT.md, DESIGN.md.

Commands

Install: bun install --frozen-lockfile Dev: bun dev Test one file: bun test tests/export.test.ts | One case: bun test tests/models.test.ts -t "dtype" Test (fast, ~6 s): bun test --path-ignore-patterns='tests/live-real-model.test.ts' Test all: bun test — ~17 min: tests/live-real-model.test.ts downloads and runs real Whisper weights whenever huggingface.co is reachable. Run it only when touching the live pipeline or decoding. Currently 1 known failure there (en_alien timeout — see WorkBreakdown "Noticed, not fixed"). Build: bun run build (rewrites dist/) Opt-in real sidecar test (needs pip install -r server-py/requirements.txt): FW_INTEGRATION=1 bun test tests/transcribe.test.ts Lint / typecheck: none configured (typescript is not installed). TODO(maintainer): confirm whether one is wanted.

Browser-only behaviour (offline PWA, mic, WebGPU) has no automated coverage and there is no browser in the dev environment: follow tests/offline.md, tests/live.md, tests/server-engine.md and say plainly when a manual check is still pending — never claim it was verified.

CI (.gitlab-ci.yml) only builds Docker images — nothing else runs your tests.

Rules

• Bun only. No node/npm/npx/yarn/pnpm, vite/webpack, jest/vitest, express, ws, better-sqlite3, dotenv. Use Bun.serve() routes + HTML imports, bun:sqlite, built-in WebSocket, Bun.file, Bun.$. Bun API docs: node_modules/bun-types/docs/.
• Two engines, two libraries (D-013). Browser inference = Transformers.js, inside src/worker/inference.worker.ts only — never on the main thread. Server inference = faster-whisper via server-py/sidecar.py — never Transformers.js on the server.
• @huggingface/transformers is patched and a devDependency on purpose. It is pinned by patches/@huggingface%[email protected] (D-027/D-028) and kept out of runtime deps so the Docker image stays small (D-031). Upgrading or moving it means redoing the patch: ask first.
• No cross-origin subresources. Every response carries COOP/COEP require-corp (D-023, src/index.ts); a CDN script/style/font will be blocked. Self-host it (onnxruntime is served from /ort/).
• Selecting a model never downloads it (D-020). Only a transcription start or the explicit Download button posts load to the worker.
• Server modules take dependencies by injection (createTranscribeHandlers(deps) pattern) so tests run against tests/fixtures/fake_sidecar.py; validate language/task with parseLanguageAndTask before anything reaches the sidecar (D-032). Every stateful server module has a test file.
• Errors go through classifyError in src/lib/errors.ts (pattern-matched, D-030) — don't add error class hierarchies.
• Configuration is runtime env only (D-015) — no build-time flags baked into the image. New variables go in README's env table.
• Anonymous users must keep every feature except the server engine; browser-engine audio never leaves the device (Architecture §5 invariants).

Workflow

Required skills: superpowers, impeccable. Check at session start; install if missing. Specs → docs/specs/NNN-slug.md (permanent). Plans → docs/plans/NNN-slug.md while in progress; git rm them before merging — the spec is the record. (Overrides the superpowers default paths.)

Anything beyond a trivial fix: brainstorm → spec (sign-off) → plan (sign-off) → TDD → review. TDD means failing test first, watch it fail, minimal code, watch it pass, commit. Tick plan checkboxes as you go so an interrupted session is resumable. Trivial changes (typo, version bump, one-line config): say "trivial, skipping spec" and proceed.

UI work: use impeccable — craft for new, audit/polish on existing, harden before shipping. Design context: PRODUCT.md, DESIGN.md.

Architecture

src/index.ts -> Bun.serve entry: routes, /api/*, isolation headers, dev worker bundling, prod dist/ serving src/App.tsx, src/components/ -> React UI src/lib/ -> pure client logic (audio, live segmenter, model catalog, engines, errors) — most unit tests target this src/worker/ -> inference worker + typed postMessage protocol (shared with UI) src/pwa/ -> hand-rolled service worker (D-007); src/server/ -> auth, db, sidecar supervisor, live WS server-py/ -> faster-whisper sidecar (JSON-lines over stdio)

Full picture: docs/Architecture.md — read it before your first change in a session.

Boundaries

Never touch: dist/ (build output), src/pwa/icons/ (generated by bun run icons), data/ (local SQLite).

Ask first: new runtime dependencies · changing or regenerating the transformers patch · auth/session code (src/server/auth.ts, src/server/db.ts) · Dockerfile / .gitlab-ci.yml · anything destructive · work the approved plan does not cover.

If the task turns out much larger than the plan assumed: stop, report, re-plan.

Docs duties

Same commit as the code, not "later":

• docs/Architecture.md — when a module, boundary, or data flow changes.
• docs/Decisions.md — append at the bottom with the next D-### id; never rewrite an Accepted entry, supersede it. Cite the id in code comments where the decision bites.
• docs/WorkBreakdown.md — tick items as they land; anything noticed and deliberately not fixed goes under "Noticed, not fixed". No TODO comments in source.
• docs/PERMISSIONS.md — together with every tool permission file, in one commit.

Git

Commit subjects: plain imperative sentence describing the change (e.g. "Show "Loading model from cache" instead of "Downloading" when weights are already cached"). Small, each leaving tests green. TODO(maintainer): confirm branch naming and whether work lands directly on main.

Done means verified

Before saying done: relevant tests run in this session and seen passing · bun run build succeeds if src/ or scripts/ changed · matches the approved spec · docs duties complete · git status clean and the diff is only what you intended.