au_whisper_web/README.md

AU Whisper Web

Browser-first speech-to-text powered by OpenAI Whisper. Anonymous users run Whisper entirely in their browser via Transformers.js (WebNN → WebGPU → WASM, installable PWA, works offline once a model is cached). Logged-in users can optionally run inference on the server instead, backed by faster-whisper (CTranslate2) through a small Python sidecar (server-py/, Decisions.md D-013).

See Architecture.md for the design, Decisions.md for the decision log, and CLAUDE.md for the toolchain rules (Bun everywhere — no npm/node).

Development

bun install
bun dev          # dev server with HMR at http://localhost:3000
bun test         # unit tests

Production build (bare metal)

bun install
bun run build    # emits self-contained dist/ (app shell, worker, sw.js, /ort/*)
bun start        # NODE_ENV=production bun src/index.ts — serves dist/ + /api/*

To enable the server engine on bare metal, additionally install Python ≥ 3.9 and the sidecar dependencies: python3 -m pip install -r server-py/requirements.txt. Nothing else changes — the sidecar is spawned lazily on the first server-engine request.

Deployment (Docker)

The Dockerfile is multi-stage and produces two image flavors:

Flavor	Build command	Contents
Bun-only (default)	docker build -t whisper-web .	Bun runtime + built dist/. No Python.
With server engine	docker build --target with-server-engine -t whisper-web:server-engine .	Everything above + Debian python3 and server-py/requirements.txt (faster-whisper, CPU-only) in a virtualenv at /opt/faster-whisper.

--target was chosen over a build-arg because the optional layer is a strict superset of the runtime stage: with-server-engine extends runtime, so the Bun layers are shared byte-for-byte between both flavors and there is no conditional logic inside the Dockerfile. A plain docker build . (what .gitlab-ci.yml runs) intentionally resolves to the Bun-only flavor.

Run either flavor the same way:

docker run -d -p 3000:3000 -v whisper-data:/app/data whisper-web
# or, with the server engine:
docker run -d -p 3000:3000 -v whisper-data:/app/data whisper-web:server-engine

docker compose up --build builds and runs the Bun-only flavor with the whisper-data volume already wired up (see docker-compose.yml).

Graceful degradation (why the Bun-only image is safe to deploy)

The server engine is optional by design (D-013):

• Bun-only image: everything anonymous users see works in full — browser inference, model downloads, PWA/offline, and (if AUTH_ENABLED stays on) accounts. The first server-engine probe finds no python3, the supervisor records that as a permanent condition (no respawn storms), and GET /api/transcribe/status answers {"available": false, "reason": ...}; POST /api/transcribe answers 503 with the same reason. The UI simply doesn't offer the server engine.
• with-server-engine image: status reports available: true and logged-in users get the server/device toggle. The first request for a model downloads its CTranslate2 weights from the Hugging Face Hub into FASTER_WHISPER_DOWNLOAD_ROOT (inside the data volume, so restarts don't re-download); the last-used model stays loaded between requests.

Runtime environment variables

All configuration is runtime-only (D-015) — one image serves every configuration; flip flags with docker run -e / compose environment:.

Variable	Default	Meaning
PORT	3000	Port the Bun server listens on (also honored by the image's EXPOSE/healthcheck).
NODE_ENV	production (set in the image)	production serves the prebuilt dist/; anything else runs the dev server. Leave as-is in Docker.
AUTH_ENABLED	true	D-015 feature flag. false/0/no/off disables login/register: /api/auth/* and /api/transcribe* return 404 (the server engine is session-gated, so it disappears with auth) and the client hides the Sign in UI.
DB_PATH	/app/data/app.db (image) / ./data/app.db (bare metal)	SQLite database (users + sessions, D-009). Keep it inside the mounted volume.
TRANSCRIBE_MAX_UPLOAD_BYTES	536870912 (512 MiB ≈ 2¼ h of audio)	Upload cap for POST /api/transcribe (raw float32 mono 16 kHz PCM), enforced via Content-Length and again while streaming.
SIDECAR_PYTHON	python3 (with-server-engine image: /opt/faster-whisper/bin/python3)	Python interpreter used to spawn server-py/sidecar.py. Only override for exotic setups.
FASTER_WHISPER_DEVICE	auto	CTranslate2 device for the sidecar (auto, cpu, cuda). In the stock image auto resolves to CPU.
FASTER_WHISPER_COMPUTE	int8	CTranslate2 compute type (int8 is the fast, low-memory CPU choice; e.g. float16 on GPU).
FASTER_WHISPER_DOWNLOAD_ROOT	unset (with-server-engine image: /app/data/faster-whisper-models)	Where the sidecar caches downloaded model weights.

There are no session-secret style variables: sessions are random 256-bit ids stored server-side in SQLite; the cookie is HttpOnly, SameSite=Lax, and Secure whenever NODE_ENV=production.

Persistence

Mount a volume at /app/data. It holds:

• app.db — the SQLite database (users + sessions). Without the volume every redeploy logs everyone out and drops accounts.
• faster-whisper-models/ — (server-engine flavor) downloaded Whisper weights, from ~75 MB (tiny) to ~1.6 GB (large-v3-turbo).

Browser-engine model weights never touch the server — they live in each visitor's browser cache.

HTTPS / reverse proxy

Run the container behind a TLS-terminating reverse proxy (Caddy, nginx, Traefik…). HTTPS is functionally required in production, not just good practice: service workers (PWA/offline), microphone capture (getUserMedia for live transcription), WebGPU, and the Secure session cookie all require a secure context. Plain HTTP only works on localhost. The app uses relative URLs and also tolerates being served under a subpath.

GPU note

The with-server-engine target is CPU-only on purpose (small-ish image, no CUDA runtime). For GPU inference you would build your own image on an nvidia/cuda base with cuDNN 9 + cuBLAS, install Bun and server-py/requirements.txt the same way, run with docker run --gpus all, and set FASTER_WHISPER_DEVICE=cuda (plus e.g. FASTER_WHISPER_COMPUTE=float16). Not shipped here — CPU int8 already makes the server engine several times faster than real time on modest hardware.