122 lines
7.4 KiB
Markdown
122 lines
7.4 KiB
Markdown
# whisper_web
|
||
|
||
Browser-first Whisper speech-to-text PWA. Bun 1.3 (runtime, bundler, test runner), React 19, TypeScript,
|
||
`bun:sqlite`, Passport.js; optional Python faster-whisper sidecar (`server-py/`) for the server engine.
|
||
|
||
`AGENTS.md` is the single source of truth for agents here. `CLAUDE.md`, `GEMINI.md`, and `docs/CLAUDE.md`
|
||
only point here. Deeper context on demand: `docs/Architecture.md`, `docs/Decisions.md` (`D-###` ids
|
||
cited throughout the code), `docs/WorkBreakdown.md` (milestones + backlog), `PRODUCT.md`, `DESIGN.md`.
|
||
|
||
## Commands
|
||
|
||
Install: `bun install --frozen-lockfile`
|
||
Dev: `bun dev`
|
||
Test one file: `bun test tests/export.test.ts` | One case: `bun test tests/models.test.ts -t "dtype"`
|
||
Test (fast, ~6 s): `bun test --path-ignore-patterns='tests/live-real-model.test.ts'`
|
||
Test all: `bun test` — ~19 min: `tests/live-real-model.test.ts` downloads and runs real Whisper
|
||
weights whenever huggingface.co is reachable. Run it when touching the live pipeline or decoding.
|
||
Build: `bun run build` (rewrites `dist/`)
|
||
Opt-in real sidecar test (needs `pip install -r server-py/requirements.txt`): `FW_INTEGRATION=1 bun test tests/transcribe.test.ts`
|
||
Opt-in real translation test (~3 min, downloads 2 × 113 MB Opus-MT): `MT_INTEGRATION=1 bun test tests/translate-real-model.test.ts`;
|
||
translation speed: `bun scripts/bench-translate.ts`
|
||
Typecheck: `bun run typecheck` (tsc 7, < 1 s). No linter configured.
|
||
|
||
Browser-only behaviour (offline PWA, mic, WebGPU) has no automated coverage: follow `tests/offline.md`,
|
||
`tests/live.md`, `tests/server-engine.md` and say plainly when a manual check is still pending — never
|
||
claim it was verified. Headless Chrome can screenshot the UI: `bun run build`, serve with
|
||
`PORT=3118 NODE_ENV=production bun src/index.ts`, then `google-chrome --headless=new --no-sandbox
|
||
--disable-gpu --virtual-time-budget=6000 --window-size=390,1100 --screenshot=/tmp/shot.png
|
||
http://localhost:3118/` (add `--force-dark-mode` for the dark theme).
|
||
|
||
CI (`.gitlab-ci.yml`) runs install, typecheck, and the fast suite, then builds Docker images. Keep it in
|
||
step with the commands above, in the same commit.
|
||
|
||
## Rules
|
||
|
||
- **Bun only.** No node/npm/npx/yarn/pnpm, vite/webpack, jest/vitest, express, `ws`, better-sqlite3,
|
||
dotenv. Use `Bun.serve()` routes + HTML imports, `bun:sqlite`, built-in `WebSocket`, `Bun.file`,
|
||
`Bun.$`. Bun API docs: `node_modules/bun-types/docs/`.
|
||
- **Two engines, two libraries (D-013).** Browser inference = Transformers.js, inside
|
||
`src/worker/inference.worker.ts` only (transcript translation: `src/worker/translate.worker.ts`,
|
||
D-WS4b) — never on the main thread. Server inference = faster-whisper
|
||
via `server-py/sidecar.py` — never Transformers.js on the server.
|
||
- **`@huggingface/transformers` is patched and a devDependency on purpose.** It is pinned by
|
||
`patches/@huggingface%[email protected]` (D-027/D-028) and kept out of runtime deps so the
|
||
Docker image stays small (D-031). Upgrading or moving it means redoing the patch: ask first.
|
||
- **No cross-origin subresources.** Every response carries COOP/COEP `require-corp` (D-023,
|
||
`src/index.ts`); a CDN script/style/font will be blocked. Self-host it (onnxruntime is served from `/ort/`).
|
||
- **Bun inlines every CSS `url()` asset as base64** in the shell build, ignoring `loader`. Anything
|
||
large goes `external` + copied into `dist/`, like the fonts (`scripts/fonts.ts`, D-037).
|
||
- **Colours are tokens.** `src/index.css` has a dark set and a `prefers-color-scheme: light` set; never
|
||
add a raw colour outside them (DESIGN.md).
|
||
- **Selecting a model never downloads it (D-020).** Only a transcription start or the explicit Download
|
||
button posts `load` to the worker.
|
||
- **Server modules take dependencies by injection** (`createTranscribeHandlers(deps)` pattern) so tests
|
||
run against `tests/fixtures/fake_sidecar.py`; validate `language`/`task` with
|
||
`parseLanguageAndTask` before anything reaches the sidecar (D-032). Every stateful server module has a test file.
|
||
- **Errors go through `classifyError` in `src/lib/errors.ts`** (pattern-matched, D-030) — don't add
|
||
error class hierarchies.
|
||
- **Configuration is runtime env only (D-015)** — no build-time flags baked into the image. New
|
||
variables go in README's env table.
|
||
- Anonymous users must keep every feature except the server engine; browser-engine audio never leaves
|
||
the device (Architecture §5 invariants).
|
||
|
||
## Workflow
|
||
|
||
Required skills: `superpowers`, `impeccable`. Check at session start; install if missing.
|
||
Specs → `docs/specs/NNN-slug.md` (permanent). Plans → `docs/plans/NNN-slug.md` while in progress;
|
||
`git rm` them before merging — the spec is the record. (Overrides the superpowers default paths.)
|
||
|
||
Anything beyond a trivial fix: brainstorm → spec (sign-off) → plan (sign-off) → TDD → review.
|
||
TDD means failing test first, watch it fail, minimal code, watch it pass, commit.
|
||
Tick plan checkboxes as you go so an interrupted session is resumable.
|
||
Trivial changes (typo, version bump, one-line config): say "trivial, skipping spec" and proceed.
|
||
|
||
UI work: use impeccable — `craft` for new, `audit`/`polish` on existing, `harden` before shipping.
|
||
Design context: `PRODUCT.md`, `DESIGN.md`.
|
||
|
||
## Architecture
|
||
|
||
`src/index.ts` -> `Bun.serve` entry: routes, `/api/*`, isolation headers, dev worker bundling, prod `dist/` serving
|
||
`src/App.tsx`, `src/components/` -> React UI
|
||
`src/lib/` -> pure client logic (audio, live segmenter, model catalog, engines, errors) — most unit tests target this
|
||
`src/worker/` -> inference worker + typed postMessage protocol (shared with UI)
|
||
`src/pwa/` -> hand-rolled service worker (D-007); `src/server/` -> auth, db, sidecar supervisor, live WS
|
||
`server-py/` -> faster-whisper sidecar (JSON-lines over stdio)
|
||
|
||
Full picture: `docs/Architecture.md` — read it before your first change in a session.
|
||
|
||
## Boundaries
|
||
|
||
Never touch: `dist/` (build output), `src/pwa/icons/` (generated by `bun run icons`), `data/` (local SQLite).
|
||
|
||
Ask first: new runtime dependencies · changing or regenerating the transformers patch · auth/session
|
||
code (`src/server/auth.ts`, `src/server/db.ts`) · `Dockerfile` / `.gitlab-ci.yml` · anything
|
||
destructive · work the approved plan does not cover.
|
||
|
||
If the task turns out much larger than the plan assumed: stop, report, re-plan.
|
||
|
||
## Docs duties
|
||
|
||
Same commit as the code, not "later":
|
||
- `docs/Architecture.md` — when a module, boundary, or data flow changes.
|
||
- `docs/Decisions.md` — append at the **bottom** with the next `D-###` id; never rewrite an Accepted
|
||
entry, supersede it. Cite the id in code comments where the decision bites.
|
||
- `docs/WorkBreakdown.md` — tick items as they land; anything noticed and deliberately not fixed goes
|
||
under "Noticed, not fixed". No `TODO` comments in source.
|
||
- `docs/PERMISSIONS.md` — together with every tool permission file, in one commit.
|
||
|
||
## Git
|
||
|
||
Branches: `feat/<slug>`, `fix/<slug>`, `chore/<slug>`, `docs/<slug>`, `refactor/<slug>`, `test/<slug>`
|
||
off `main`; never commit directly to `main`. Commit subjects: plain imperative sentence describing the
|
||
change (e.g. "Show \"Loading model from cache\" instead of \"Downloading\" when weights are already
|
||
cached"). Small, each leaving tests green.
|
||
|
||
## Done means verified
|
||
|
||
Before saying done: relevant tests run *in this session* and seen passing · `bun run typecheck` clean ·
|
||
`bun run build` succeeds
|
||
if `src/` or `scripts/` changed · matches the approved spec · docs duties complete · `git status` clean
|
||
and the diff is only what you intended.
|