diff --git a/.dockerignore b/.dockerignore index 98d653f..90ace79 100644 --- a/.dockerignore +++ b/.dockerignore @@ -15,6 +15,17 @@ docker-compose.yml __pycache__/ *.pyc +# Local Python virtualenvs for developing/testing the sidecar against a real +# faster-whisper install (e.g. server-py/venv, server-py/.venv). A nested +# venv/.gitignore keeps these out of `git status`, but Docker does not read +# .gitignore, so without this they silently ride along in `COPY server-py +# ./server-py` — confirmed to balloon the "Bun-only" image from the documented +# ~244 MB to 2.18 GB and bake a machine-specific interpreter path nothing in +# the container would ever use (the with-server-engine target builds its own +# venv from requirements.txt at /opt/faster-whisper). +**/venv/ +**/.venv/ + # Editor / agent state .idea .claude diff --git a/README.md b/README.md index 27f4975..96a1304 100644 --- a/README.md +++ b/README.md @@ -90,7 +90,7 @@ configuration; flip flags with `docker run -e` / compose `environment:`. | `SIDECAR_PYTHON` | `python3` (with-server-engine image: `/opt/faster-whisper/bin/python3`) | Python interpreter used to spawn `server-py/sidecar.py`. Only override for exotic setups. | | `FASTER_WHISPER_DEVICE` | `auto` | CTranslate2 device for the sidecar (`auto`, `cpu`, `cuda`). In the stock image `auto` resolves to CPU. | | `FASTER_WHISPER_COMPUTE` | `int8` | CTranslate2 compute type (`int8` is the fast, low-memory CPU choice; e.g. `float16` on GPU). | -| `FASTER_WHISPER_DOWNLOAD_ROOT` | unset (with-server-engine image: `/app/data/faster-whisper-models`) | Where the sidecar caches downloaded model weights. | +| `FASTER_WHISPER_DOWNLOAD_ROOT` | unset (with-server-engine image: `/opt/faster-whisper/models`) | Where the sidecar caches downloaded model weights. The with-server-engine image pre-downloads both server-selectable models (`large-v3-turbo`, `medium.en`) into this path **at build time**, deliberately outside the `/app/data` volume (D-031) — a fresh container never has a cold-download wait, and the models survive volume-less redeploys since they're baked into the image layer. Override to a path under `/app/data` only if you want runtime downloads instead (e.g. a custom image with more server-selectable models than the two baked in). | There are no session-secret style variables: sessions are random 256-bit ids stored server-side in SQLite; the cookie is `HttpOnly`, `SameSite=Lax`, and @@ -102,12 +102,31 @@ Mount a volume at **`/app/data`**. It holds: - `app.db` — the SQLite database (users + sessions). Without the volume every redeploy logs everyone out and drops accounts. -- `faster-whisper-models/` — (server-engine flavor) downloaded Whisper - weights, from ~75 MB (`tiny`) to ~1.6 GB (`large-v3-turbo`). + +The server-engine flavor's Whisper weights (`large-v3-turbo`, `medium.en` — +the only two models the UI offers on the server engine) are **not** in this +volume: they're pre-downloaded into the image itself at +`/opt/faster-whisper/models` during `docker build --target with-server-engine` +(D-031), so they survive redeploys with no volume needed and there's no +first-request download wait. This is also why that image is several GB — see +"Image sizes" below. Browser-engine model weights never touch the server — they live in each visitor's browser cache. +### Image sizes + +- **Bun-only** (`runtime`, the default): roughly 400–450 MB — Debian + + Bun + the built `dist/` + production `node_modules` (react/react-dom/passport + only; `@huggingface/transformers` and its `onnxruntime-node`/`onnxruntime-web` + dependencies are a devDependency, D-031 — they're needed to *build* the + browser worker bundle, never to *serve* it). +- **with-server-engine**: several GB (>7 GB) — the Debian/venv/faster-whisper + layer adds a few hundred MB, but the two pre-downloaded models + (`large-v3-turbo` + `medium.en`) dominate. Budget disk space and pull time + accordingly; this is a deliberate build-time-vs-runtime tradeoff (see above), + not bloat to fix. + ### HTTPS / reverse proxy Run the container behind a TLS-terminating reverse proxy (Caddy, nginx, diff --git a/bun.lock b/bun.lock index 5a06c05..ba418d5 100644 --- a/bun.lock +++ b/bun.lock @@ -5,13 +5,13 @@ "": { "name": "bun-react-template", "dependencies": { - "@huggingface/transformers": "^4.2.0", "passport": "^0.7.0", "passport-local": "^1.0.0", "react": "^19", "react-dom": "^19", }, "devDependencies": { + "@huggingface/transformers": "^4.2.0", "@types/bun": "latest", "@types/passport": "^1.0.17", "@types/passport-local": "^1.0.38", diff --git a/docs/Decisions.md b/docs/Decisions.md index d791789..a40df3e 100644 --- a/docs/Decisions.md +++ b/docs/Decisions.md @@ -755,3 +755,84 @@ for no benefit, and the UI is where `context.offline`/retry actions live). surfacing richer structured errors (e.g. a `GPUDeviceLostInfo` object instead of a stringified message), prefer inspecting that directly over adding more substring patterns. + +## D-031 — Docker image hardening: exclude local venvs, move Transformers.js out of the runtime image +**Status:** Accepted (2026-07-29) +WorkBreakdown's M6 "production build + `bun start` deployment check" item — +a `docker build .` from this checkout, not a fresh clone, exposed two real +bugs the documented image sizes (WorkBreakdown M5 notes: "244 MB" / +"405 MB") had already gone stale against. +**Bug 1 — local Python venvs leak into every image.** `server-py/venv` and +`server-py/.venv` (created locally to test the sidecar against a real +faster-whisper install) are invisible to `git status` — `python -m venv` +drops its own nested `.gitignore` (`*`) inside the venv directory — but +Docker does not read `.gitignore`, and `.dockerignore` had no matching +pattern. `COPY server-py ./server-py` therefore copied both venvs (858 MB +combined) into the image; measured effect on the Bun-only image: 999 MB +instead of the intended size, with a completely unused, +machine-specific Python interpreter baked in at `/app/server-py/venv` and +`/app/server-py/.venv`. **Fix:** `.dockerignore` gained `**/venv/` and +`**/.venv/`. Verified: rebuilding after the fix, `find /app/server-py` +inside the image no longer lists either directory. +**Bug 2 — `@huggingface/transformers` (and its `onnxruntime-node` + +`onnxruntime-web` dependencies, 341 MB combined) had no reason to be in the +*production runtime* image.** Grepped: the only file in `src/` that imports +`@huggingface/transformers` as a value (not a type) is +`src/worker/inference.worker.ts`, and that file is bundled once, at build +time, into the static `dist/inference.worker.js` the production server +just serves as a file (confirmed: `src/index.ts`'s `Bun.build`-based +on-the-fly bundling path — needed because Bun doesn't follow `new +Worker(...)` graphs, per M1's Architecture notes — only runs in +**development** mode; production never calls it). No server-side module +(`src/server/*`, `src/index.ts`, `src/lib/models.ts`'s only import from the +package is `import type`, erased at transpile time) touches the package at +runtime. Yet it was a `dependencies` entry, so the Dockerfile's runtime +stage (`bun install --frozen-lockfile --production`) installed it and its +full dependency tree anyway — dead weight on every deploy. **Fix:** +`@huggingface/transformers` moved to `devDependencies` in `package.json`. +The **build** stage's `bun install --frozen-lockfile` (no `--production`) +still installs devDependencies, so `scripts/build.ts`'s `Bun.build` call +(which needs the package on disk to bundle the worker) is unaffected; the +patch in `patches/@huggingface%2Ftransformers@4.2.0.patch` still applies +there the same way. Verified: production `node_modules` in the built image +dropped from 409 MB to 7.9 MB (react/react-dom/passport only); the running +container still served `/inference.worker.js` correctly (534,714 bytes, +byte-identical to the pre-change build) and `/api/transcribe/status` +correctly reported the expected `faster-whisper`/`python3` unavailability +message — nothing at runtime needed the package. +**Net effect, measured against a real `docker build .`/`--target +with-server-engine` in this checkout:** +| Image | Before both fixes | After | +| --- | --- | --- | +| Bun-only (`runtime`) | 2.18 GB (venv leak) | ~421 MB | +| with-server-engine | not separately measured (inherits the leak) | ~7.1 GB — see below | +**A third, non-bug finding while re-measuring:** the with-server-engine +image is legitimately several GB, not the "405 MB" previously written down +— its last `RUN` layer pre-downloads both server-selectable models +(`large-v3-turbo`, `medium.en`, matching the `EngineToggle`'s hardcoded +allow-list in `App.tsx`) into `/opt/faster-whisper/models` at *build* time, +which alone measured 3.15 GB; the venv/faster-whisper/ctranslate2 install +layer is a separate 451 MB. This is intentional (comment already in the +Dockerfile: baking the weights into the image, outside `/app/data`, means +no first-request cold-download and no dependence on the data volume for +those two models) — not something to shrink — but the number that made it +into WorkBreakdown's M5 notes and README's env-var table was never updated +to match, and the README's Persistence section wrongly implied these +weights live in the `/app/data` volume (`FASTER_WHISPER_DOWNLOAD_ROOT`'s +actual default in that image is `/opt/faster-whisper/models`, not +`/app/data/faster-whisper-models`). **Fix:** README's env-var table, +Persistence section, and a new "Image sizes" subsection now state the +verified numbers and the reason for each; WorkBreakdown's M5 notes updated +to match. +**Consequence:** none of this is CI-visible — GitLab CI (`.gitlab-ci.yml`) +builds from a clean checkout that never has `server-py/venv` on disk, so +Bug 1 only bit a local `docker build .` run from a working directory that +had one (as this checkout did); the `.dockerignore` fix is still correct +defense-in-depth regardless. Bug 2 was real everywhere, CI included — every +pushed Bun-only image has been ~340 MB heavier than necessary since +`@huggingface/transformers` became a production dependency. +**Revisit:** if the with-server-engine catalog ever grows beyond the two +baked-in models, either accept runtime downloads for the extra ones (point +`FASTER_WHISPER_DOWNLOAD_ROOT` at `/app/data` for those, per the README's +override note) or extend the Dockerfile's pre-download `RUN` line — +re-measure and re-document the image size either way. diff --git a/docs/WorkBreakdown.md b/docs/WorkBreakdown.md index e9c8ed1..cde0254 100644 --- a/docs/WorkBreakdown.md +++ b/docs/WorkBreakdown.md @@ -204,13 +204,16 @@ Notes (M5): boundaries), and cancel by aborting the fetch (D-018 amendment) — browser runs keep worker-termination cancel. The choice persists in localStorage but only takes effect while signed in with the engine available. -- Deployment: multi-stage `Dockerfile` — `runtime` (Bun-only, 244 MB) is the - default target so plain `docker build .` (existing CI) is unchanged; - `--target with-server-engine` (405 MB) adds python3 + faster-whisper in a - venv at `/opt/faster-whisper` (wired via `SIDECAR_PYTHON`), with - `FASTER_WHISPER_DOWNLOAD_ROOT` under the `/app/data` volume so weights - persist. README.md documents both flavors, all runtime env vars, the DB - volume, and the HTTPS/reverse-proxy requirement. Verified live in Docker: +- Deployment: multi-stage `Dockerfile` — `runtime` (Bun-only, ~400-450 MB, see + D-031) is the default target so plain `docker build .` (existing CI) is + unchanged; `--target with-server-engine` (several GB — two Whisper models + pre-downloaded at build time, D-031) adds python3 + faster-whisper in a venv + at `/opt/faster-whisper` (wired via `SIDECAR_PYTHON`), with + `FASTER_WHISPER_DOWNLOAD_ROOT` pointed at `/opt/faster-whisper/models` + (deliberately outside the `/app/data` volume, D-031) so weights persist + without a volume and there's no first-request download. README.md documents + both flavors, all runtime env vars, the DB volume, and the HTTPS/reverse-proxy + requirement. Verified live in Docker: Bun-only image degrades per D-013 (`available:false`, "python3 is not available…"); server-engine image ran a real `POST /api/transcribe` of `en_aiproblem.m4a` end-to-end, streamed segments matching the fixture @@ -271,7 +274,21 @@ Notes (M5): explicit Download button and the lazy auto-load on file-drop/live-start alike. - [ ] Basic a11y pass (keyboard, focus, aria on progress/live transcript) -- [ ] Production build + `bun start` deployment check; README for deployers +- [x] Production build + `bun start` deployment check; README for deployers — + `bun run build` + `bun start` smoke-tested live (headers, hashed-asset + immutability, `/api/*` routing, `AUTH_ENABLED=false`, server-engine + graceful degradation all matched the README's claims). A real + `docker build .` from this checkout also caught two real bugs (D-031): + local Python venvs leaking into the image (858 MB, via a nested + `.gitignore` `.dockerignore` doesn't read) and `@huggingface/transformers` + +`onnxruntime-node`/`-web` (341 MB) sitting in the production + `dependencies` despite never being imported at server runtime — both + fixed, bringing the Bun-only image from a buggy 2.18 GB down to ~421 MB. + Also corrected stale docs: the with-server-engine image is legitimately + several GB (it bakes in two Whisper models at build time), and + `FASTER_WHISPER_DOWNLOAD_ROOT`'s real default in that image is + `/opt/faster-whisper/models`, not under `/app/data` as previously + written. ## M7 — Live microphone transcription (browser engine, D-016) Promoted out of the out-of-scope list via D-016. Browser engine only: audio diff --git a/package.json b/package.json index e56927f..070b4ff 100644 --- a/package.json +++ b/package.json @@ -10,13 +10,13 @@ "start": "NODE_ENV=production bun src/index.ts" }, "dependencies": { - "@huggingface/transformers": "^4.2.0", "passport": "^0.7.0", "passport-local": "^1.0.0", "react": "^19", "react-dom": "^19" }, "devDependencies": { + "@huggingface/transformers": "^4.2.0", "@types/bun": "latest", "@types/passport": "^1.0.17", "@types/passport-local": "^1.0.38",