119 lines
7.1 KiB
Markdown
119 lines
7.1 KiB
Markdown
# whisper_web
|
|
|
|
Browser-first Whisper speech-to-text PWA. Bun 1.3 (runtime, bundler, test runner), React 19, TypeScript,
|
|
`bun:sqlite`, Passport.js; optional Python faster-whisper sidecar (`server-py/`) for the server engine.
|
|
|
|
`AGENTS.md` is the single source of truth for agents here. `CLAUDE.md`, `GEMINI.md`, and `docs/CLAUDE.md`
|
|
only point here. Deeper context on demand: `docs/Architecture.md`, `docs/Decisions.md` (`D-###` ids
|
|
cited throughout the code), `docs/WorkBreakdown.md` (milestones + backlog), `PRODUCT.md`, `DESIGN.md`.
|
|
|
|
## Commands
|
|
|
|
Install: `bun install --frozen-lockfile`
|
|
Dev: `bun dev`
|
|
Test one file: `bun test tests/export.test.ts` | One case: `bun test tests/models.test.ts -t "dtype"`
|
|
Test (fast, ~6 s): `bun test --path-ignore-patterns='tests/live-real-model.test.ts'`
|
|
Test all: `bun test` — ~19 min: `tests/live-real-model.test.ts` downloads and runs real Whisper
|
|
weights whenever huggingface.co is reachable. Run it when touching the live pipeline or decoding.
|
|
Build: `bun run build` (rewrites `dist/`)
|
|
Opt-in real sidecar test (needs `pip install -r server-py/requirements.txt`): `FW_INTEGRATION=1 bun test tests/transcribe.test.ts`
|
|
Typecheck: `bun run typecheck` (tsc 7, < 1 s). No linter configured.
|
|
|
|
Browser-only behaviour (offline PWA, mic, WebGPU) has no automated coverage: follow `tests/offline.md`,
|
|
`tests/live.md`, `tests/server-engine.md` and say plainly when a manual check is still pending — never
|
|
claim it was verified. Headless Chrome can screenshot the UI: `bun run build`, serve with
|
|
`PORT=3118 NODE_ENV=production bun src/index.ts`, then `google-chrome --headless=new --no-sandbox
|
|
--disable-gpu --virtual-time-budget=6000 --window-size=390,1100 --screenshot=/tmp/shot.png
|
|
http://localhost:3118/` (add `--force-dark-mode` for the dark theme).
|
|
|
|
CI (`.gitlab-ci.yml`) runs install, typecheck, and the fast suite, then builds Docker images. Keep it in
|
|
step with the commands above, in the same commit.
|
|
|
|
## Rules
|
|
|
|
- **Bun only.** No node/npm/npx/yarn/pnpm, vite/webpack, jest/vitest, express, `ws`, better-sqlite3,
|
|
dotenv. Use `Bun.serve()` routes + HTML imports, `bun:sqlite`, built-in `WebSocket`, `Bun.file`,
|
|
`Bun.$`. Bun API docs: `node_modules/bun-types/docs/`.
|
|
- **Two engines, two libraries (D-013).** Browser inference = Transformers.js, inside
|
|
`src/worker/inference.worker.ts` only — never on the main thread. Server inference = faster-whisper
|
|
via `server-py/sidecar.py` — never Transformers.js on the server.
|
|
- **`@huggingface/transformers` is patched and a devDependency on purpose.** It is pinned by
|
|
`patches/@huggingface%[email protected]` (D-027/D-028, D-WS1a) and kept out of runtime deps so the
|
|
Docker image stays small (D-031). Upgrading or moving it means redoing the patch: ask first.
|
|
- **No cross-origin subresources.** Every response carries COOP/COEP `require-corp` (D-023,
|
|
`src/index.ts`); a CDN script/style/font will be blocked. Self-host it (onnxruntime is served from `/ort/`).
|
|
- **Bun inlines every CSS `url()` asset as base64** in the shell build, ignoring `loader`. Anything
|
|
large goes `external` + copied into `dist/`, like the fonts (`scripts/fonts.ts`, D-037).
|
|
- **Colours are tokens.** `src/index.css` has a dark set and a `prefers-color-scheme: light` set; never
|
|
add a raw colour outside them (DESIGN.md).
|
|
- **Selecting a model never downloads it (D-020).** Only a transcription start or the explicit Download
|
|
button posts `load` to the worker.
|
|
- **Server modules take dependencies by injection** (`createTranscribeHandlers(deps)` pattern) so tests
|
|
run against `tests/fixtures/fake_sidecar.py`; validate `language`/`task` with
|
|
`parseLanguageAndTask` before anything reaches the sidecar (D-032). Every stateful server module has a test file.
|
|
- **Errors go through `classifyError` in `src/lib/errors.ts`** (pattern-matched, D-030) — don't add
|
|
error class hierarchies.
|
|
- **Configuration is runtime env only (D-015)** — no build-time flags baked into the image. New
|
|
variables go in README's env table.
|
|
- Anonymous users must keep every feature except the server engine; browser-engine audio never leaves
|
|
the device (Architecture §5 invariants).
|
|
|
|
## Workflow
|
|
|
|
Required skills: `superpowers`, `impeccable`. Check at session start; install if missing.
|
|
Specs → `docs/specs/NNN-slug.md` (permanent). Plans → `docs/plans/NNN-slug.md` while in progress;
|
|
`git rm` them before merging — the spec is the record. (Overrides the superpowers default paths.)
|
|
|
|
Anything beyond a trivial fix: brainstorm → spec (sign-off) → plan (sign-off) → TDD → review.
|
|
TDD means failing test first, watch it fail, minimal code, watch it pass, commit.
|
|
Tick plan checkboxes as you go so an interrupted session is resumable.
|
|
Trivial changes (typo, version bump, one-line config): say "trivial, skipping spec" and proceed.
|
|
|
|
UI work: use impeccable — `craft` for new, `audit`/`polish` on existing, `harden` before shipping.
|
|
Design context: `PRODUCT.md`, `DESIGN.md`.
|
|
|
|
## Architecture
|
|
|
|
`src/index.ts` -> `Bun.serve` entry: routes, `/api/*`, isolation headers, dev worker bundling, prod `dist/` serving
|
|
`src/App.tsx`, `src/components/` -> React UI
|
|
`src/lib/` -> pure client logic (audio, live segmenter, model catalog, engines, errors) — most unit tests target this
|
|
`src/worker/` -> inference worker + typed postMessage protocol (shared with UI)
|
|
`src/pwa/` -> hand-rolled service worker (D-007); `src/server/` -> auth, db, sidecar supervisor, live WS
|
|
`server-py/` -> faster-whisper sidecar (JSON-lines over stdio)
|
|
|
|
Full picture: `docs/Architecture.md` — read it before your first change in a session.
|
|
|
|
## Boundaries
|
|
|
|
Never touch: `dist/` (build output), `src/pwa/icons/` (generated by `bun run icons`), `data/` (local SQLite).
|
|
|
|
Ask first: new runtime dependencies · changing or regenerating the transformers patch · auth/session
|
|
code (`src/server/auth.ts`, `src/server/db.ts`) · `Dockerfile` / `.gitlab-ci.yml` · anything
|
|
destructive · work the approved plan does not cover.
|
|
|
|
If the task turns out much larger than the plan assumed: stop, report, re-plan.
|
|
|
|
## Docs duties
|
|
|
|
Same commit as the code, not "later":
|
|
- `docs/Architecture.md` — when a module, boundary, or data flow changes.
|
|
- `docs/Decisions.md` — append at the **bottom** with the next `D-###` id; never rewrite an Accepted
|
|
entry, supersede it. Cite the id in code comments where the decision bites.
|
|
- `docs/WorkBreakdown.md` — tick items as they land; anything noticed and deliberately not fixed goes
|
|
under "Noticed, not fixed". No `TODO` comments in source.
|
|
- `docs/PERMISSIONS.md` — together with every tool permission file, in one commit.
|
|
|
|
## Git
|
|
|
|
Branches: `feat/<slug>`, `fix/<slug>`, `chore/<slug>`, `docs/<slug>`, `refactor/<slug>`, `test/<slug>`
|
|
off `main`; never commit directly to `main`. Commit subjects: plain imperative sentence describing the
|
|
change (e.g. "Show \"Loading model from cache\" instead of \"Downloading\" when weights are already
|
|
cached"). Small, each leaving tests green.
|
|
|
|
## Done means verified
|
|
|
|
Before saying done: relevant tests run *in this session* and seen passing · `bun run typecheck` clean ·
|
|
`bun run build` succeeds
|
|
if `src/` or `scripts/` changed · matches the approved spec · docs duties complete · `git status` clean
|
|
and the diff is only what you intended.
|