whisper_web/Decisions.md

Decisions

Lightweight ADR log. One entry per decision that shapes the codebase. Statuses: Accepted (act on it), Proposed (default choice, revisit before building on it), Superseded (kept for history, points to replacement).

Add new entries at the bottom with the next D-### id. Never rewrite an Accepted entry to mean something else — supersede it.

---

D-001 — Bun as the only runtime and toolchain

Status: Accepted (mandated by CLAUDE.md) Bun replaces Node, vite/webpack (via bun build + HTML imports), jest/vitest (bun test), express (Bun.serve), better-sqlite3 (bun:sqlite), and dotenv. Consequence: no vite plugin ecosystem; PWA assets (service worker, manifest) are wired by hand instead of via a plugin.

D-002 — Transformers.js (@huggingface/transformers) for all Whisper inference

Status: Accepted for browser inference; the server side is superseded by D-013 (faster-whisper). Same library in the browser (WebGPU/WASM) and on the server (ONNX Runtime under Bun). Why: one model catalog, one pipeline API, no native whisper.cpp build step. Trade-off: server CPU inference is slower than whisper.cpp; acceptable — the server engine is a convenience for logged-in users, not the primary path.

D-003 — Inference runs in a dedicated Web Worker

Status: Accepted Model load + transcription block for seconds-to-minutes; running them on the main thread would freeze the UI. Communication via a small typed postMessage protocol (src/worker/protocol.ts) shared between UI and worker.

D-004 — WebGPU with automatic WASM fallback

Status: Accepted Detect via navigator.gpu?.requestAdapter() inside the worker; pass device: "webgpu" | "wasm" to the pipeline. The active backend is surfaced in the UI but users don't have to configure anything. Encoder-heavy Whisper benefits ~10-40× from WebGPU where available (Chromium); WASM keeps Firefox/Safari working.

D-005 — Model weights cached via the browser Cache API (Transformers.js default)

Status: Accepted Transformers.js already persists Hub downloads in the Cache API (useBrowserCache). We do not re-cache weights in the service worker — the SW precaches only the app shell. Consequence: offline = installed PWA + at least one previously loaded model; the UI must show per-model "downloaded / not downloaded" state (query caches for hub URLs) so users can prepare for offline use.

D-006 — Quantized model catalog, curated not exhaustive

Status: Proposed Expose a curated list (src/lib/models.ts): whisper-tiny(.en), base(.en), small(.en), medium(.en), large-v3-turbo, each with sensible dtype defaults (e.g. q4/q8 on WASM, fp16/q4 on WebGPU) and a size estimate shown before download. Why not free-form model ids: predictable UX, tested combinations, honest size/VRAM warnings. Revisit if users ask for arbitrary Hub ids.

D-007 — PWA: hand-rolled service worker, app-shell precache

Status: Accepted No Workbox/vite-plugin-pwa (conflicts with D-001 tooling). bun build emits hashed assets to dist/; a small build step writes the precache manifest into sw.ts. Strategy: precache app shell, cache-first for static assets, network-only for /api/*.

D-008 — Passport.js via a connect-compat adapter over Bun.serve

Status: Accepted (Passport mandated by CLAUDE.md; adapter is the how) Passport expects connect/express-style (req, res, next) and req.login(). We are not allowed to use express (D-001), so src/server/auth.ts provides a thin adapter: wrap Bun's Request in a minimal req/res shim, invoke the passport authenticate chain, translate the outcome back to a Response. Local (username/password) strategy first; other strategies plug into the same adapter later. Alternative rejected: dropping Passport for hand-rolled auth — CLAUDE.md names Passport explicitly.

D-009 — Sessions + users in bun:sqlite

Status: Accepted Single-file SQLite DB (path from env, default ./data/app.db). Tables: users (id, username, password_hash — Argon2 via Bun.password), sessions (id, user_id, expires_at). Cookie: HttpOnly, SameSite=Lax, Secure in production. No Redis/Postgres until there's a scaling reason (would be Bun.redis / Bun.sql).

D-010 — Engine abstraction: BrowserEngine and ServerEngine behind one interface

Status: Accepted src/lib/engine.ts defines Engine { load(model), transcribe(audio), onProgress }. The UI never branches on engine type beyond the toggle. Keeps the server engine (milestone M5) from leaking into UI code built in M1–M4.

D-011 — Server engine transport: streaming HTTP response first, WebSocket if needed

Status: Proposed POST /api/transcribe returns a streamed (chunked JSON-lines) response with progress and partial transcripts. Simpler than WebSockets (no connection lifecycle, works with plain fetch + ReadableStream). Switch to Bun.serve websockets only if we need bidirectional control (e.g. cancel mid-run) that abort signals can't cover.

D-012 — Audio decoding on the main thread via Web Audio API

Status: Accepted decodeAudioData (handles most audio and audio-in-video containers) → mono downmix → resample to 16 kHz → transfer Float32Array to worker (zero-copy). No ffmpeg.wasm unless we hit real container-support gaps; note in src/lib/audio.ts if we do.

D-013 — Server engine runs faster-whisper (Python sidecar)

Status: Accepted (user requirement, 2026-07-10; supersedes the server half of D-002) Server-side inference (M5) uses faster-whisper (CTranslate2) instead of Transformers.js under Bun. Browser inference is untouched — faster-whisper cannot run in a browser (checked 2026-07): it is Python over native C++ CTranslate2, which has no WASM build (OpenNMT/CTranslate2#528 open since 2021, no progress) and relies on native SIMD/threading kernels. So D-002 stays Accepted for the browser, where Transformers.js remains the engine. Shape: Bun stays the only web runtime. src/server/transcribe.ts talks to a small Python sidecar (server-py/, faster-whisper behind a minimal stdio or localhost-HTTP interface) that Bun.spawn starts and supervises. The POST /api/transcribe contract (D-011: streamed JSON-lines) is unchanged — faster-whisper's segment generator maps naturally onto it. Consequence: the "one inference codebase" invariant narrows to "one browser inference codebase + one shared model catalog"; src/lib/models.ts needs a mapping from catalog ids to faster-whisper/CTranslate2 model names (Systran/faster-whisper-*); deployment gains a Python ≥3.9 dependency (server engine only — the anonymous/browser path still deploys with Bun alone, and the app must degrade gracefully when the sidecar is absent). Why: CTranslate2 CPU inference is several times faster and more memory-efficient than ONNX Runtime under Bun; the server engine becomes genuinely useful rather than a slow convenience.