148 lines
8.1 KiB
Markdown
148 lines
8.1 KiB
Markdown
# AU Whisper Web
|
||
|
||
Browser-first speech-to-text powered by OpenAI Whisper. Anonymous users run
|
||
Whisper entirely in their browser via
|
||
[Transformers.js](https://github.com/huggingface/transformers) (WebNN → WebGPU
|
||
→ WASM, installable PWA, works offline once a model is cached). Logged-in
|
||
users can optionally run inference on the **server** instead, backed by
|
||
[faster-whisper](https://github.com/SYSTRAN/faster-whisper) (CTranslate2)
|
||
through a small Python sidecar (`server-py/`, Decisions.md D-013).
|
||
|
||
See `Architecture.md` for the design, `Decisions.md` for the decision log, and
|
||
`CLAUDE.md` for the toolchain rules (Bun everywhere — no npm/node).
|
||
|
||
## Development
|
||
|
||
```sh
|
||
bun install
|
||
bun dev # dev server with HMR at http://localhost:3000
|
||
bun test # unit tests
|
||
```
|
||
|
||
## Production build (bare metal)
|
||
|
||
```sh
|
||
bun install
|
||
bun run build # emits self-contained dist/ (app shell, worker, sw.js, /ort/*)
|
||
bun start # NODE_ENV=production bun src/index.ts — serves dist/ + /api/*
|
||
```
|
||
|
||
To enable the server engine on bare metal, additionally install Python ≥ 3.9
|
||
and the sidecar dependencies: `python3 -m pip install -r server-py/requirements.txt`.
|
||
Nothing else changes — the sidecar is spawned lazily on the first server-engine
|
||
request.
|
||
|
||
## Deployment (Docker)
|
||
|
||
The `Dockerfile` is multi-stage and produces **two image flavors**:
|
||
|
||
| Flavor | Build command | Contents |
|
||
| --- | --- | --- |
|
||
| **Bun-only** (default) | `docker build -t whisper-web .` | Bun runtime + built `dist/`. No Python. |
|
||
| **With server engine** | `docker build --target with-server-engine -t whisper-web:server-engine .` | Everything above + Debian `python3` and `server-py/requirements.txt` (faster-whisper, CPU-only) in a virtualenv at `/opt/faster-whisper`. |
|
||
|
||
`--target` was chosen over a build-arg because the optional layer is a strict
|
||
superset of the runtime stage: `with-server-engine` extends `runtime`, so the
|
||
Bun layers are shared byte-for-byte between both flavors and there is no
|
||
conditional logic inside the Dockerfile. A plain `docker build .` (what
|
||
`.gitlab-ci.yml` runs) intentionally resolves to the Bun-only flavor.
|
||
|
||
Run either flavor the same way:
|
||
|
||
```sh
|
||
docker run -d -p 3000:3000 -v whisper-data:/app/data whisper-web
|
||
# or, with the server engine:
|
||
docker run -d -p 3000:3000 -v whisper-data:/app/data whisper-web:server-engine
|
||
```
|
||
|
||
`docker compose up --build` builds and runs the Bun-only flavor with the
|
||
`whisper-data` volume already wired up (see `docker-compose.yml`).
|
||
|
||
### Graceful degradation (why the Bun-only image is safe to deploy)
|
||
|
||
The server engine is optional by design (D-013):
|
||
|
||
- **Bun-only image:** everything anonymous users see works in full — browser
|
||
inference, model downloads, PWA/offline, and (if `AUTH_ENABLED` stays on)
|
||
accounts. The first server-engine probe finds no `python3`, the supervisor
|
||
records that as a permanent condition (no respawn storms), and
|
||
`GET /api/transcribe/status` answers `{"available": false, "reason": ...}`;
|
||
`POST /api/transcribe` answers `503` with the same reason. The UI simply
|
||
doesn't offer the server engine.
|
||
- **with-server-engine image:** `status` reports `available: true` and
|
||
logged-in users get the server/device toggle. The first request for a model
|
||
downloads its CTranslate2 weights from the Hugging Face Hub into
|
||
`FASTER_WHISPER_DOWNLOAD_ROOT` (inside the data volume, so restarts don't
|
||
re-download); the last-used model stays loaded between requests.
|
||
|
||
### Runtime environment variables
|
||
|
||
All configuration is runtime-only (D-015) — one image serves every
|
||
configuration; flip flags with `docker run -e` / compose `environment:`.
|
||
|
||
| Variable | Default | Meaning |
|
||
| --- | --- | --- |
|
||
| `PORT` | `3000` | Port the Bun server listens on (also honored by the image's `EXPOSE`/healthcheck). |
|
||
| `NODE_ENV` | `production` (set in the image) | `production` serves the prebuilt `dist/`; anything else runs the dev server. Leave as-is in Docker. |
|
||
| `AUTH_ENABLED` | `true` | D-015 feature flag. `false`/`0`/`no`/`off` disables login/register: `/api/auth/*` **and** `/api/transcribe*` return 404 (the server engine is session-gated, so it disappears with auth) and the client hides the Sign in UI. |
|
||
| `DB_PATH` | `/app/data/app.db` (image) / `./data/app.db` (bare metal) | SQLite database (users + sessions, D-009). Keep it inside the mounted volume. |
|
||
| `TRANSCRIBE_MAX_UPLOAD_BYTES` | `536870912` (512 MiB ≈ 2¼ h of audio) | Upload cap for `POST /api/transcribe` (raw float32 mono 16 kHz PCM), enforced via Content-Length and again while streaming. |
|
||
| `SIDECAR_PYTHON` | `python3` (with-server-engine image: `/opt/faster-whisper/bin/python3`) | Python interpreter used to spawn `server-py/sidecar.py`. Only override for exotic setups. |
|
||
| `FASTER_WHISPER_DEVICE` | `auto` | CTranslate2 device for the sidecar (`auto`, `cpu`, `cuda`). In the stock image `auto` resolves to CPU. |
|
||
| `FASTER_WHISPER_COMPUTE` | `int8` | CTranslate2 compute type (`int8` is the fast, low-memory CPU choice; e.g. `float16` on GPU). |
|
||
| `FASTER_WHISPER_DOWNLOAD_ROOT` | unset (with-server-engine image: `/opt/faster-whisper/models`) | Where the sidecar caches downloaded model weights. The with-server-engine image pre-downloads both server-selectable models (`large-v3-turbo`, `medium.en`) into this path **at build time**, deliberately outside the `/app/data` volume (D-031) — a fresh container never has a cold-download wait, and the models survive volume-less redeploys since they're baked into the image layer. Override to a path under `/app/data` only if you want runtime downloads instead (e.g. a custom image with more server-selectable models than the two baked in). |
|
||
|
||
There are no session-secret style variables: sessions are random 256-bit ids
|
||
stored server-side in SQLite; the cookie is `HttpOnly`, `SameSite=Lax`, and
|
||
`Secure` whenever `NODE_ENV=production`.
|
||
|
||
### Persistence
|
||
|
||
Mount a volume at **`/app/data`**. It holds:
|
||
|
||
- `app.db` — the SQLite database (users + sessions). Without the volume every
|
||
redeploy logs everyone out and drops accounts.
|
||
|
||
The server-engine flavor's Whisper weights (`large-v3-turbo`, `medium.en` —
|
||
the only two models the UI offers on the server engine) are **not** in this
|
||
volume: they're pre-downloaded into the image itself at
|
||
`/opt/faster-whisper/models` during `docker build --target with-server-engine`
|
||
(D-031), so they survive redeploys with no volume needed and there's no
|
||
first-request download wait. This is also why that image is several GB — see
|
||
"Image sizes" below.
|
||
|
||
Browser-engine model weights never touch the server — they live in each
|
||
visitor's browser cache.
|
||
|
||
### Image sizes
|
||
|
||
- **Bun-only** (`runtime`, the default): roughly 400–450 MB — Debian +
|
||
Bun + the built `dist/` + production `node_modules` (react/react-dom/passport
|
||
only; `@huggingface/transformers` and its `onnxruntime-node`/`onnxruntime-web`
|
||
dependencies are a devDependency, D-031 — they're needed to *build* the
|
||
browser worker bundle, never to *serve* it).
|
||
- **with-server-engine**: several GB (>7 GB) — the Debian/venv/faster-whisper
|
||
layer adds a few hundred MB, but the two pre-downloaded models
|
||
(`large-v3-turbo` + `medium.en`) dominate. Budget disk space and pull time
|
||
accordingly; this is a deliberate build-time-vs-runtime tradeoff (see above),
|
||
not bloat to fix.
|
||
|
||
### HTTPS / reverse proxy
|
||
|
||
Run the container behind a TLS-terminating reverse proxy (Caddy, nginx,
|
||
Traefik…). HTTPS is **functionally required** in production, not just good
|
||
practice: service workers (PWA/offline), microphone capture
|
||
(`getUserMedia` for live transcription), WebGPU, and the `Secure` session
|
||
cookie all require a secure context. Plain HTTP only works on `localhost`.
|
||
The app uses relative URLs and also tolerates being served under a subpath.
|
||
|
||
### GPU note
|
||
|
||
The `with-server-engine` target is CPU-only on purpose (small-ish image, no
|
||
CUDA runtime). For GPU inference you would build your own image on an
|
||
`nvidia/cuda` base with cuDNN 9 + cuBLAS, install Bun and
|
||
`server-py/requirements.txt` the same way, run with `docker run --gpus all`,
|
||
and set `FASTER_WHISPER_DEVICE=cuda` (plus e.g.
|
||
`FASTER_WHISPER_COMPUTE=float16`). Not shipped here — CPU int8 already makes
|
||
the server engine several times faster than real time on modest hardware.
|