diff --git a/Decisions.md b/Decisions.md index 1913e2a..e25351c 100644 --- a/Decisions.md +++ b/Decisions.md @@ -117,3 +117,23 @@ sidecar is absent). **Why:** CTranslate2 CPU inference is several times faster and more memory-efficient than ONNX Runtime under Bun; the server engine becomes genuinely useful rather than a slow convenience. + +## D-014 — WASM sessions run at graphOptimizationLevel "basic" +**Status:** Accepted (bug workaround, 2026-07-10) +On the WASM backend, the inference worker passes +`session_options: { graphOptimizationLevel: "basic" }` to the Transformers.js +pipeline. The onnxruntime-web build bundled with Transformers.js 4.2 +(1.26.0-dev.20260416) has a broken level-2 QDQ graph transform +(`qdq_actions.cc` `TransposeDQWeightsForMatMulNBits`) that fails session +creation for the 8-bit Whisper decoders (`_quantized`/`_int8`/`_uint8`), whose +tied `embed_tokens` weight is both gathered (embedding) and matmul'd (lm_head): +"Missing required scale: model.decoder.embed_tokens.weight_merged_0_scale". +Reproduced outside the browser against `decoder_model_merged_quantized.onnx` +of `onnx-community/whisper-tiny`; levels "all"/"extended" fail, "basic"/ +"disabled" load fine, and onnxruntime-node 1.24.3 loads the same file at every +level (regression in the ort-web dev build, not the model). +**Scope:** WASM only — WebGPU keeps the default level. The q4 conversions use +native `MatMulNBits` ops (no QDQ), so skipping level-2 costs the larger WASM +models nothing. +**Revisit:** when Transformers.js bumps its onnxruntime-web dependency past +1.26.0-dev.20260416, retest q8-on-WASM at the default level and drop this. diff --git a/src/worker/inference.worker.ts b/src/worker/inference.worker.ts index dfb16c6..405268b 100644 --- a/src/worker/inference.worker.ts +++ b/src/worker/inference.worker.ts @@ -136,6 +136,14 @@ async function createPipeline( const asr = await pipeline("automatic-speech-recognition", message.modelId, { device, dtype, + // WASM sessions cap graph optimization at "basic" (D-014): the level-2 + // QDQ transform in the bundled onnxruntime-web build rejects the 8-bit + // decoders' tied embed_tokens weight ("Missing required scale … + // DequantizeLinear"). q4 models use native MatMulNBits ops, so the + // lower level costs them nothing. WebGPU keeps the default level. + ...(device === "wasm" && { + session_options: { graphOptimizationLevel: "basic" as const }, + }), progress_callback: forwardProgress, }); post({ type: "load-progress", status: "ready", device }); diff --git a/tests/offline.md b/tests/offline.md index 3af2fc6..c1f0912 100644 --- a/tests/offline.md +++ b/tests/offline.md @@ -61,6 +61,14 @@ model.decoder.embed_tokens.weight_merged_0_scale for node: model.decoder.embed_tokens.weight_transposed_DequantizeLinear ``` +**Diagnosed & fixed (2026-07-10, D-014):** a level-2 graph-optimizer bug in +the onnxruntime-web build bundled with Transformers.js 4.2, triggered by the +8-bit decoders' tied `embed_tokens` weight — not a model or cache problem. +The worker now caps WASM sessions at `graphOptimizationLevel: "basic"` +(`src/worker/inference.worker.ts`), verified against the same ort-web build +outside the browser. **Re-run this step with Backend = WASM to confirm in a +real browser.** + ## 3. Go offline and transcribe 1. DevTools → Network → throttling **Offline** (or OS airplane mode — stronger,