Fix output text not working.

This commit is contained in:
Lucas Winther
2026-07-15 14:13:59 +02:00
parent 0bd5d326a5
commit 742053144e
3 changed files with 61 additions and 2 deletions
+31
View File
@@ -325,3 +325,34 @@ on both engines. Three independent causes, all length-dependent:
**Revisit:** when Transformers.js changes the seek-loop condition (currently
`return_timestamps && !max_new_tokens`, `src/models/whisper/modeling_whisper.js`),
retest whether the explicit `max_new_tokens` is still needed.
## D-022 — Medium models never default to fp16 in the browser
**Status:** Accepted (2026-07-15)
Reported symptom: transcribing `en_aiproblem` in-browser with Whisper Medium
printed just "the". The medium catalog entries defaulted to fp16 on WebNN
(whole model) and WebGPU (encoder) — and D-017's "auto" order now tries WebNN
first, so browsers exposing `navigator.ml` silently moved medium onto fp16.
Two problems with those fp16 conversions
(`onnx-community/whisper-medium(.en_timestamped|-ONNX)`):
1. The fp16 ONNX files crash onnxruntime's graph optimizer at session
creation (`SimplifiedLayerNormFusion` fails on their
`InsertedPrecisionFreeCast_*` nodes) — reproduced on both onnxruntime-node
and onnxruntime-web 1.26.0-dev (wasm EP). EPs whose optimizer pipeline
skips that fusion can load the model and hit problem 2 instead.
2. Whisper medium/large encoder activations are the classic fp16 overflow
case — degenerate output with no error raised.
Meanwhile q4 medium was verified correct end-to-end on the real browser
runtime (onnxruntime-web wasm EP driven under Bun via the
`Symbol.for("onnxruntime")` override, `graphOptimizationLevel: "basic"` as in
D-014, `max_new_tokens: 440` as in D-021; both the `.en` and multilingual
conversions, auto language detect included). **Fix:** medium's `webgpu` and
`webnn` defaults are q4 (`ACCEL_Q4` in `src/lib/models.ts`). A backend that
cannot execute q4 (`MatMulNBits`) throws at pipeline creation and falls down
the D-017 device chain — strictly better than silently emitting garbage.
**Scope:** medium only — that's where load-failure evidence exists.
tiny/base/small keep fp16 on WebNN and small keeps fp16+q4 on WebGPU;
large-v3-turbo keeps fp16+q4 on WebGPU (the widely-used Transformers.js demo
configuration). If a one-word/garbage transcript ever shows up on those, suspect
their fp16 encoders and extend this decision.
**Revisit:** if the onnx-community medium conversions are re-exported with
working fp16 files, or once WebNN/WebGPU runs can be exercised in CI.