Show "Loading model from cache" instead of "Downloading" when weights are already cached

Transformers.js's progress_callback always fires status:"download" regardless
of whether the file is actually served from the Cache API — there's no
cacheHit signal for the worker to forward. Since App.tsx already tracks which
models are downloaded (for ModelPicker's badges), it now checks that before
starting a load and threads a loadedFromCache flag into ModelLoadProgress, so
the heading/hint reflect reality for the common already-cached case.

Adds Decisions.md D-033.
This commit is contained in:
Lucas Winther
2026-07-30 08:30:19 +02:00
parent bb77185e3f
commit c5d193600a
3 changed files with 85 additions and 4 deletions
+53
View File
@@ -905,3 +905,56 @@ through a real spawned sidecar (matches `createSidecar`'s own integration
tests, but that suite already covers the sidecar process itself — the gap
was specifically the WebSocket/`LiveTranscriber` glue, which needs no Python
process to exercise).
## D-033 — "Downloading model…" said that even when the weights came straight from cache
**Status:** Accepted (2026-07-30)
Reported symptom: with a model's weights already fully cached in the browser
(Cache API, D-005), starting a load still showed `ModelLoadProgress`'s
heading, "Downloading model `<id>`", and its "this happens once" hint — as
if bytes were coming over the network, when nothing was.
**Cause.** `node_modules/@huggingface/transformers/src/utils/hub.js`'s
`loadResourceFile` computes a local `cacheHit` boolean but its
`progress_callback` dispatch always fires `{ status: 'download', ... }`
unconditionally — `cacheHit` only reaches the callback on a Firefox-specific
branch that skips progress entirely. So `inference.worker.ts`'s
`forwardProgress()` has no reliable "this came from cache" signal to forward;
it just relays `initiate`/`download`/`progress`/`done` verbatim, and
`ModelLoadProgress`'s heading was hardcoded to the download copy regardless.
**Fix.** Rather than patch the vendored library for a cosmetic flag (the
existing `bun patch` at `patches/@huggingface%2Ftransformers@4.2.0.patch` is
reserved for the two structural D-027/D-028 session-loading changes — adding
an unrelated third patch just for this is disproportionate and fragile
against future upstream bumps), the app answers the question itself, before
the load ever starts, from information it already tracks: `App.tsx` already
queries `lib/modelCache.ts`'s `getDownloadedModels` into a `downloaded`
`Set<string>` for `ModelPicker`'s "available offline" badges. A new
`loadedFromCache` state is set from `downloaded.has(nextModelId)` at the top
of `startLoad()` — the single function every load path (file-drop lazy load,
live-start lazy load, the explicit Download button, and the error box's
Retry) already funnels through — and reset to `false` everywhere `loadStarted`
/`loadProgress` are reset (model switch, device-preference switch, cancel,
the server-engine forced model switch), so a stale flag never survives into
an unrelated load. It is threaded as a new required prop into
`ModelLoadProgress`, whose heading/hint now read "Loading model from cache…"
/ "This model is already downloaded — just loading it into memory." when
true, unchanged otherwise. No worker protocol change: the decision is made
entirely from state the UI has before it ever posts the `load` message.
**Partial-cache handling.** `matchCachedModels` (the pure matcher backing
`getDownloadedModels`) flags a model "downloaded" on any single cached
`.onnx` file, not all of them — a genuinely partial download (e.g. encoder
cached, decoder not) would misreport as fully cached here. This is the same
imprecision `ModelPicker`'s existing "available offline" badge already
lives with; per this fix's scope, no new finer-grained per-file cache
probing was added — `modelCache.ts` stays the single source of truth for
"is this downloaded," reused as-is rather than duplicated at a different
granularity.
**Consequence:** the loading UI now matches reality for the common
already-cached case (reopening the app, switching back to a previously used
model); first-time downloads are unaffected. `ModelPicker`'s own "Download"
button never needed a matching change — it is only rendered for
`!isDownloaded` models, so it can never show "Downloading…" for an
already-cached one.
**Revisit:** if `modelCache.ts` ever gains a per-file/fully-cached check
(e.g. for the storage-management panel's exact byte accounting), reuse it
here too so a genuinely partial cache falls back to the "Downloading" copy
instead of "Loading from cache."