129 lines
6.6 KiB
Markdown
129 lines
6.6 KiB
Markdown
# AU Whisper Web
|
|
|
|
Browser-first speech-to-text powered by OpenAI Whisper. Anonymous users run
|
|
Whisper entirely in their browser via
|
|
[Transformers.js](https://github.com/huggingface/transformers) (WebNN → WebGPU
|
|
→ WASM, installable PWA, works offline once a model is cached). Logged-in
|
|
users can optionally run inference on the **server** instead, backed by
|
|
[faster-whisper](https://github.com/SYSTRAN/faster-whisper) (CTranslate2)
|
|
through a small Python sidecar (`server-py/`, Decisions.md D-013).
|
|
|
|
See `Architecture.md` for the design, `Decisions.md` for the decision log, and
|
|
`CLAUDE.md` for the toolchain rules (Bun everywhere — no npm/node).
|
|
|
|
## Development
|
|
|
|
```sh
|
|
bun install
|
|
bun dev # dev server with HMR at http://localhost:3000
|
|
bun test # unit tests
|
|
```
|
|
|
|
## Production build (bare metal)
|
|
|
|
```sh
|
|
bun install
|
|
bun run build # emits self-contained dist/ (app shell, worker, sw.js, /ort/*)
|
|
bun start # NODE_ENV=production bun src/index.ts — serves dist/ + /api/*
|
|
```
|
|
|
|
To enable the server engine on bare metal, additionally install Python ≥ 3.9
|
|
and the sidecar dependencies: `python3 -m pip install -r server-py/requirements.txt`.
|
|
Nothing else changes — the sidecar is spawned lazily on the first server-engine
|
|
request.
|
|
|
|
## Deployment (Docker)
|
|
|
|
The `Dockerfile` is multi-stage and produces **two image flavors**:
|
|
|
|
| Flavor | Build command | Contents |
|
|
| --- | --- | --- |
|
|
| **Bun-only** (default) | `docker build -t whisper-web .` | Bun runtime + built `dist/`. No Python. |
|
|
| **With server engine** | `docker build --target with-server-engine -t whisper-web:server-engine .` | Everything above + Debian `python3` and `server-py/requirements.txt` (faster-whisper, CPU-only) in a virtualenv at `/opt/faster-whisper`. |
|
|
|
|
`--target` was chosen over a build-arg because the optional layer is a strict
|
|
superset of the runtime stage: `with-server-engine` extends `runtime`, so the
|
|
Bun layers are shared byte-for-byte between both flavors and there is no
|
|
conditional logic inside the Dockerfile. A plain `docker build .` (what
|
|
`.gitlab-ci.yml` runs) intentionally resolves to the Bun-only flavor.
|
|
|
|
Run either flavor the same way:
|
|
|
|
```sh
|
|
docker run -d -p 3000:3000 -v whisper-data:/app/data whisper-web
|
|
# or, with the server engine:
|
|
docker run -d -p 3000:3000 -v whisper-data:/app/data whisper-web:server-engine
|
|
```
|
|
|
|
`docker compose up --build` builds and runs the Bun-only flavor with the
|
|
`whisper-data` volume already wired up (see `docker-compose.yml`).
|
|
|
|
### Graceful degradation (why the Bun-only image is safe to deploy)
|
|
|
|
The server engine is optional by design (D-013):
|
|
|
|
- **Bun-only image:** everything anonymous users see works in full — browser
|
|
inference, model downloads, PWA/offline, and (if `AUTH_ENABLED` stays on)
|
|
accounts. The first server-engine probe finds no `python3`, the supervisor
|
|
records that as a permanent condition (no respawn storms), and
|
|
`GET /api/transcribe/status` answers `{"available": false, "reason": ...}`;
|
|
`POST /api/transcribe` answers `503` with the same reason. The UI simply
|
|
doesn't offer the server engine.
|
|
- **with-server-engine image:** `status` reports `available: true` and
|
|
logged-in users get the server/device toggle. The first request for a model
|
|
downloads its CTranslate2 weights from the Hugging Face Hub into
|
|
`FASTER_WHISPER_DOWNLOAD_ROOT` (inside the data volume, so restarts don't
|
|
re-download); the last-used model stays loaded between requests.
|
|
|
|
### Runtime environment variables
|
|
|
|
All configuration is runtime-only (D-015) — one image serves every
|
|
configuration; flip flags with `docker run -e` / compose `environment:`.
|
|
|
|
| Variable | Default | Meaning |
|
|
| --- | --- | --- |
|
|
| `PORT` | `3000` | Port the Bun server listens on (also honored by the image's `EXPOSE`/healthcheck). |
|
|
| `NODE_ENV` | `production` (set in the image) | `production` serves the prebuilt `dist/`; anything else runs the dev server. Leave as-is in Docker. |
|
|
| `AUTH_ENABLED` | `true` | D-015 feature flag. `false`/`0`/`no`/`off` disables login/register: `/api/auth/*` **and** `/api/transcribe*` return 404 (the server engine is session-gated, so it disappears with auth) and the client hides the Sign in UI. |
|
|
| `DB_PATH` | `/app/data/app.db` (image) / `./data/app.db` (bare metal) | SQLite database (users + sessions, D-009). Keep it inside the mounted volume. |
|
|
| `TRANSCRIBE_MAX_UPLOAD_BYTES` | `536870912` (512 MiB ≈ 2¼ h of audio) | Upload cap for `POST /api/transcribe` (raw float32 mono 16 kHz PCM), enforced via Content-Length and again while streaming. |
|
|
| `SIDECAR_PYTHON` | `python3` (with-server-engine image: `/opt/faster-whisper/bin/python3`) | Python interpreter used to spawn `server-py/sidecar.py`. Only override for exotic setups. |
|
|
| `FASTER_WHISPER_DEVICE` | `auto` | CTranslate2 device for the sidecar (`auto`, `cpu`, `cuda`). In the stock image `auto` resolves to CPU. |
|
|
| `FASTER_WHISPER_COMPUTE` | `int8` | CTranslate2 compute type (`int8` is the fast, low-memory CPU choice; e.g. `float16` on GPU). |
|
|
| `FASTER_WHISPER_DOWNLOAD_ROOT` | unset (with-server-engine image: `/app/data/faster-whisper-models`) | Where the sidecar caches downloaded model weights. |
|
|
|
|
There are no session-secret style variables: sessions are random 256-bit ids
|
|
stored server-side in SQLite; the cookie is `HttpOnly`, `SameSite=Lax`, and
|
|
`Secure` whenever `NODE_ENV=production`.
|
|
|
|
### Persistence
|
|
|
|
Mount a volume at **`/app/data`**. It holds:
|
|
|
|
- `app.db` — the SQLite database (users + sessions). Without the volume every
|
|
redeploy logs everyone out and drops accounts.
|
|
- `faster-whisper-models/` — (server-engine flavor) downloaded Whisper
|
|
weights, from ~75 MB (`tiny`) to ~1.6 GB (`large-v3-turbo`).
|
|
|
|
Browser-engine model weights never touch the server — they live in each
|
|
visitor's browser cache.
|
|
|
|
### HTTPS / reverse proxy
|
|
|
|
Run the container behind a TLS-terminating reverse proxy (Caddy, nginx,
|
|
Traefik…). HTTPS is **functionally required** in production, not just good
|
|
practice: service workers (PWA/offline), microphone capture
|
|
(`getUserMedia` for live transcription), WebGPU, and the `Secure` session
|
|
cookie all require a secure context. Plain HTTP only works on `localhost`.
|
|
The app uses relative URLs and also tolerates being served under a subpath.
|
|
|
|
### GPU note
|
|
|
|
The `with-server-engine` target is CPU-only on purpose (small-ish image, no
|
|
CUDA runtime). For GPU inference you would build your own image on an
|
|
`nvidia/cuda` base with cuDNN 9 + cuBLAS, install Bun and
|
|
`server-py/requirements.txt` the same way, run with `docker run --gpus all`,
|
|
and set `FASTER_WHISPER_DEVICE=cuda` (plus e.g.
|
|
`FASTER_WHISPER_COMPUTE=float16`). Not shipped here — CPU int8 already makes
|
|
the server engine several times faster than real time on modest hardware.
|