Fix Docker image bloat: exclude local venvs from build context, move transformers.js to devDependencies.
This commit is contained in:
Lucas Winther
@@ -15,6 +15,17 @@ docker-compose.yml
|
||||
__pycache__/
|
||||
*.pyc
|
||||
|
||||
# Local Python virtualenvs for developing/testing the sidecar against a real
|
||||
# faster-whisper install (e.g. server-py/venv, server-py/.venv). A nested
|
||||
# venv/.gitignore keeps these out of `git status`, but Docker does not read
|
||||
# .gitignore, so without this they silently ride along in `COPY server-py
|
||||
# ./server-py` — confirmed to balloon the "Bun-only" image from the documented
|
||||
# ~244 MB to 2.18 GB and bake a machine-specific interpreter path nothing in
|
||||
# the container would ever use (the with-server-engine target builds its own
|
||||
# venv from requirements.txt at /opt/faster-whisper).
|
||||
**/venv/
|
||||
**/.venv/
|
||||
|
||||
# Editor / agent state
|
||||
.idea
|
||||
.claude
|
||||
|
||||
@@ -90,7 +90,7 @@ configuration; flip flags with `docker run -e` / compose `environment:`.
|
||||
| `SIDECAR_PYTHON` | `python3` (with-server-engine image: `/opt/faster-whisper/bin/python3`) | Python interpreter used to spawn `server-py/sidecar.py`. Only override for exotic setups. |
|
||||
| `FASTER_WHISPER_DEVICE` | `auto` | CTranslate2 device for the sidecar (`auto`, `cpu`, `cuda`). In the stock image `auto` resolves to CPU. |
|
||||
| `FASTER_WHISPER_COMPUTE` | `int8` | CTranslate2 compute type (`int8` is the fast, low-memory CPU choice; e.g. `float16` on GPU). |
|
||||
| `FASTER_WHISPER_DOWNLOAD_ROOT` | unset (with-server-engine image: `/app/data/faster-whisper-models`) | Where the sidecar caches downloaded model weights. |
|
||||
| `FASTER_WHISPER_DOWNLOAD_ROOT` | unset (with-server-engine image: `/opt/faster-whisper/models`) | Where the sidecar caches downloaded model weights. The with-server-engine image pre-downloads both server-selectable models (`large-v3-turbo`, `medium.en`) into this path **at build time**, deliberately outside the `/app/data` volume (D-031) — a fresh container never has a cold-download wait, and the models survive volume-less redeploys since they're baked into the image layer. Override to a path under `/app/data` only if you want runtime downloads instead (e.g. a custom image with more server-selectable models than the two baked in). |
|
||||
|
||||
There are no session-secret style variables: sessions are random 256-bit ids
|
||||
stored server-side in SQLite; the cookie is `HttpOnly`, `SameSite=Lax`, and
|
||||
@@ -102,12 +102,31 @@ Mount a volume at **`/app/data`**. It holds:
|
||||
|
||||
- `app.db` — the SQLite database (users + sessions). Without the volume every
|
||||
redeploy logs everyone out and drops accounts.
|
||||
- `faster-whisper-models/` — (server-engine flavor) downloaded Whisper
|
||||
weights, from ~75 MB (`tiny`) to ~1.6 GB (`large-v3-turbo`).
|
||||
|
||||
The server-engine flavor's Whisper weights (`large-v3-turbo`, `medium.en` —
|
||||
the only two models the UI offers on the server engine) are **not** in this
|
||||
volume: they're pre-downloaded into the image itself at
|
||||
`/opt/faster-whisper/models` during `docker build --target with-server-engine`
|
||||
(D-031), so they survive redeploys with no volume needed and there's no
|
||||
first-request download wait. This is also why that image is several GB — see
|
||||
"Image sizes" below.
|
||||
|
||||
Browser-engine model weights never touch the server — they live in each
|
||||
visitor's browser cache.
|
||||
|
||||
### Image sizes
|
||||
|
||||
- **Bun-only** (`runtime`, the default): roughly 400–450 MB — Debian +
|
||||
Bun + the built `dist/` + production `node_modules` (react/react-dom/passport
|
||||
only; `@huggingface/transformers` and its `onnxruntime-node`/`onnxruntime-web`
|
||||
dependencies are a devDependency, D-031 — they're needed to *build* the
|
||||
browser worker bundle, never to *serve* it).
|
||||
- **with-server-engine**: several GB (>7 GB) — the Debian/venv/faster-whisper
|
||||
layer adds a few hundred MB, but the two pre-downloaded models
|
||||
(`large-v3-turbo` + `medium.en`) dominate. Budget disk space and pull time
|
||||
accordingly; this is a deliberate build-time-vs-runtime tradeoff (see above),
|
||||
not bloat to fix.
|
||||
|
||||
### HTTPS / reverse proxy
|
||||
|
||||
Run the container behind a TLS-terminating reverse proxy (Caddy, nginx,
|
||||
|
||||
@@ -5,13 +5,13 @@
|
||||
"": {
|
||||
"name": "bun-react-template",
|
||||
"dependencies": {
|
||||
"@huggingface/transformers": "^4.2.0",
|
||||
"passport": "^0.7.0",
|
||||
"passport-local": "^1.0.0",
|
||||
"react": "^19",
|
||||
"react-dom": "^19",
|
||||
},
|
||||
"devDependencies": {
|
||||
"@huggingface/transformers": "^4.2.0",
|
||||
"@types/bun": "latest",
|
||||
"@types/passport": "^1.0.17",
|
||||
"@types/passport-local": "^1.0.38",
|
||||
|
||||
@@ -755,3 +755,84 @@ for no benefit, and the UI is where `context.offline`/retry actions live).
|
||||
surfacing richer structured errors (e.g. a `GPUDeviceLostInfo` object
|
||||
instead of a stringified message), prefer inspecting that directly over
|
||||
adding more substring patterns.
|
||||
|
||||
## D-031 — Docker image hardening: exclude local venvs, move Transformers.js out of the runtime image
|
||||
**Status:** Accepted (2026-07-29)
|
||||
WorkBreakdown's M6 "production build + `bun start` deployment check" item —
|
||||
a `docker build .` from this checkout, not a fresh clone, exposed two real
|
||||
bugs the documented image sizes (WorkBreakdown M5 notes: "244 MB" /
|
||||
"405 MB") had already gone stale against.
|
||||
**Bug 1 — local Python venvs leak into every image.** `server-py/venv` and
|
||||
`server-py/.venv` (created locally to test the sidecar against a real
|
||||
faster-whisper install) are invisible to `git status` — `python -m venv`
|
||||
drops its own nested `.gitignore` (`*`) inside the venv directory — but
|
||||
Docker does not read `.gitignore`, and `.dockerignore` had no matching
|
||||
pattern. `COPY server-py ./server-py` therefore copied both venvs (858 MB
|
||||
combined) into the image; measured effect on the Bun-only image: 999 MB
|
||||
instead of the intended size, with a completely unused,
|
||||
machine-specific Python interpreter baked in at `/app/server-py/venv` and
|
||||
`/app/server-py/.venv`. **Fix:** `.dockerignore` gained `**/venv/` and
|
||||
`**/.venv/`. Verified: rebuilding after the fix, `find /app/server-py`
|
||||
inside the image no longer lists either directory.
|
||||
**Bug 2 — `@huggingface/transformers` (and its `onnxruntime-node` +
|
||||
`onnxruntime-web` dependencies, 341 MB combined) had no reason to be in the
|
||||
*production runtime* image.** Grepped: the only file in `src/` that imports
|
||||
`@huggingface/transformers` as a value (not a type) is
|
||||
`src/worker/inference.worker.ts`, and that file is bundled once, at build
|
||||
time, into the static `dist/inference.worker.js` the production server
|
||||
just serves as a file (confirmed: `src/index.ts`'s `Bun.build`-based
|
||||
on-the-fly bundling path — needed because Bun doesn't follow `new
|
||||
Worker(...)` graphs, per M1's Architecture notes — only runs in
|
||||
**development** mode; production never calls it). No server-side module
|
||||
(`src/server/*`, `src/index.ts`, `src/lib/models.ts`'s only import from the
|
||||
package is `import type`, erased at transpile time) touches the package at
|
||||
runtime. Yet it was a `dependencies` entry, so the Dockerfile's runtime
|
||||
stage (`bun install --frozen-lockfile --production`) installed it and its
|
||||
full dependency tree anyway — dead weight on every deploy. **Fix:**
|
||||
`@huggingface/transformers` moved to `devDependencies` in `package.json`.
|
||||
The **build** stage's `bun install --frozen-lockfile` (no `--production`)
|
||||
still installs devDependencies, so `scripts/build.ts`'s `Bun.build` call
|
||||
(which needs the package on disk to bundle the worker) is unaffected; the
|
||||
patch in `patches/@huggingface%[email protected]` still applies
|
||||
there the same way. Verified: production `node_modules` in the built image
|
||||
dropped from 409 MB to 7.9 MB (react/react-dom/passport only); the running
|
||||
container still served `/inference.worker.js` correctly (534,714 bytes,
|
||||
byte-identical to the pre-change build) and `/api/transcribe/status`
|
||||
correctly reported the expected `faster-whisper`/`python3` unavailability
|
||||
message — nothing at runtime needed the package.
|
||||
**Net effect, measured against a real `docker build .`/`--target
|
||||
with-server-engine` in this checkout:**
|
||||
| Image | Before both fixes | After |
|
||||
| --- | --- | --- |
|
||||
| Bun-only (`runtime`) | 2.18 GB (venv leak) | ~421 MB |
|
||||
| with-server-engine | not separately measured (inherits the leak) | ~7.1 GB — see below |
|
||||
**A third, non-bug finding while re-measuring:** the with-server-engine
|
||||
image is legitimately several GB, not the "405 MB" previously written down
|
||||
— its last `RUN` layer pre-downloads both server-selectable models
|
||||
(`large-v3-turbo`, `medium.en`, matching the `EngineToggle`'s hardcoded
|
||||
allow-list in `App.tsx`) into `/opt/faster-whisper/models` at *build* time,
|
||||
which alone measured 3.15 GB; the venv/faster-whisper/ctranslate2 install
|
||||
layer is a separate 451 MB. This is intentional (comment already in the
|
||||
Dockerfile: baking the weights into the image, outside `/app/data`, means
|
||||
no first-request cold-download and no dependence on the data volume for
|
||||
those two models) — not something to shrink — but the number that made it
|
||||
into WorkBreakdown's M5 notes and README's env-var table was never updated
|
||||
to match, and the README's Persistence section wrongly implied these
|
||||
weights live in the `/app/data` volume (`FASTER_WHISPER_DOWNLOAD_ROOT`'s
|
||||
actual default in that image is `/opt/faster-whisper/models`, not
|
||||
`/app/data/faster-whisper-models`). **Fix:** README's env-var table,
|
||||
Persistence section, and a new "Image sizes" subsection now state the
|
||||
verified numbers and the reason for each; WorkBreakdown's M5 notes updated
|
||||
to match.
|
||||
**Consequence:** none of this is CI-visible — GitLab CI (`.gitlab-ci.yml`)
|
||||
builds from a clean checkout that never has `server-py/venv` on disk, so
|
||||
Bug 1 only bit a local `docker build .` run from a working directory that
|
||||
had one (as this checkout did); the `.dockerignore` fix is still correct
|
||||
defense-in-depth regardless. Bug 2 was real everywhere, CI included — every
|
||||
pushed Bun-only image has been ~340 MB heavier than necessary since
|
||||
`@huggingface/transformers` became a production dependency.
|
||||
**Revisit:** if the with-server-engine catalog ever grows beyond the two
|
||||
baked-in models, either accept runtime downloads for the extra ones (point
|
||||
`FASTER_WHISPER_DOWNLOAD_ROOT` at `/app/data` for those, per the README's
|
||||
override note) or extend the Dockerfile's pre-download `RUN` line —
|
||||
re-measure and re-document the image size either way.
|
||||
|
||||
+25
-8
@@ -204,13 +204,16 @@ Notes (M5):
|
||||
boundaries), and cancel by aborting the fetch (D-018 amendment) — browser
|
||||
runs keep worker-termination cancel. The choice persists in localStorage
|
||||
but only takes effect while signed in with the engine available.
|
||||
- Deployment: multi-stage `Dockerfile` — `runtime` (Bun-only, 244 MB) is the
|
||||
default target so plain `docker build .` (existing CI) is unchanged;
|
||||
`--target with-server-engine` (405 MB) adds python3 + faster-whisper in a
|
||||
venv at `/opt/faster-whisper` (wired via `SIDECAR_PYTHON`), with
|
||||
`FASTER_WHISPER_DOWNLOAD_ROOT` under the `/app/data` volume so weights
|
||||
persist. README.md documents both flavors, all runtime env vars, the DB
|
||||
volume, and the HTTPS/reverse-proxy requirement. Verified live in Docker:
|
||||
- Deployment: multi-stage `Dockerfile` — `runtime` (Bun-only, ~400-450 MB, see
|
||||
D-031) is the default target so plain `docker build .` (existing CI) is
|
||||
unchanged; `--target with-server-engine` (several GB — two Whisper models
|
||||
pre-downloaded at build time, D-031) adds python3 + faster-whisper in a venv
|
||||
at `/opt/faster-whisper` (wired via `SIDECAR_PYTHON`), with
|
||||
`FASTER_WHISPER_DOWNLOAD_ROOT` pointed at `/opt/faster-whisper/models`
|
||||
(deliberately outside the `/app/data` volume, D-031) so weights persist
|
||||
without a volume and there's no first-request download. README.md documents
|
||||
both flavors, all runtime env vars, the DB volume, and the HTTPS/reverse-proxy
|
||||
requirement. Verified live in Docker:
|
||||
Bun-only image degrades per D-013 (`available:false`, "python3 is not
|
||||
available…"); server-engine image ran a real `POST /api/transcribe` of
|
||||
`en_aiproblem.m4a` end-to-end, streamed segments matching the fixture
|
||||
@@ -271,7 +274,21 @@ Notes (M5):
|
||||
explicit Download button and the lazy auto-load on file-drop/live-start
|
||||
alike.
|
||||
- [ ] Basic a11y pass (keyboard, focus, aria on progress/live transcript)
|
||||
- [ ] Production build + `bun start` deployment check; README for deployers
|
||||
- [x] Production build + `bun start` deployment check; README for deployers —
|
||||
`bun run build` + `bun start` smoke-tested live (headers, hashed-asset
|
||||
immutability, `/api/*` routing, `AUTH_ENABLED=false`, server-engine
|
||||
graceful degradation all matched the README's claims). A real
|
||||
`docker build .` from this checkout also caught two real bugs (D-031):
|
||||
local Python venvs leaking into the image (858 MB, via a nested
|
||||
`.gitignore` `.dockerignore` doesn't read) and `@huggingface/transformers`
|
||||
+`onnxruntime-node`/`-web` (341 MB) sitting in the production
|
||||
`dependencies` despite never being imported at server runtime — both
|
||||
fixed, bringing the Bun-only image from a buggy 2.18 GB down to ~421 MB.
|
||||
Also corrected stale docs: the with-server-engine image is legitimately
|
||||
several GB (it bakes in two Whisper models at build time), and
|
||||
`FASTER_WHISPER_DOWNLOAD_ROOT`'s real default in that image is
|
||||
`/opt/faster-whisper/models`, not under `/app/data` as previously
|
||||
written.
|
||||
|
||||
## M7 — Live microphone transcription (browser engine, D-016)
|
||||
Promoted out of the out-of-scope list via D-016. Browser engine only: audio
|
||||
|
||||
+1
-1
@@ -10,13 +10,13 @@
|
||||
"start": "NODE_ENV=production bun src/index.ts"
|
||||
},
|
||||
"dependencies": {
|
||||
"@huggingface/transformers": "^4.2.0",
|
||||
"passport": "^0.7.0",
|
||||
"passport-local": "^1.0.0",
|
||||
"react": "^19",
|
||||
"react-dom": "^19"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@huggingface/transformers": "^4.2.0",
|
||||
"@types/bun": "latest",
|
||||
"@types/passport": "^1.0.17",
|
||||
"@types/passport-local": "^1.0.38",
|
||||
|
||||
Reference in New Issue
Block a user