Run WASM sessions at the default graph optimization level now that ort-web 1.31 loads the 8-bit decoders

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Lucas WintherandClaude Opus 5.5 committed 2026-10-30 00:45:46 +01:00
1 parent 9f1d58c0b7
commit cc8e031d6b
6 files changed
+70 -11

No files matched your search

+42 -1
View File
@@ -125,7 +125,8 @@ than ONNX Runtime under Bun; the server engine becomes genuinely useful rather t
slow convenience.
## D-014 — WASM sessions run at graphOptimizationLevel "basic"
**Status:** Accepted (bug workaround, 2026-07-10)
**Status:** **Superseded by D-WS1c** (bug workaround, 2026-07-10; the ort-web
build in Transformers.js 4.3.2 no longer has the bug, so the cap is gone)
On the WASM backend, the inference worker passes
`session_options: { graphOptimizationLevel: "basic" }` to the Transformers.js
pipeline. The onnxruntime-web build bundled with Transformers.js 4.2
@@ -1319,3 +1320,43 @@ share the CPU), so read it only as "the seek loop is slower", not as a
version-to-version speed comparison.
**Decision:** keep `max_new_tokens: 440` in `src/worker/inference.worker.ts`
and in `tests/live-real-model.test.ts`.
## D-WS1c — WASM sessions back at the default graph optimization level (D-014's "basic" cap dropped)
**Status:** Accepted (2026-10-30) — supersedes D-014
**Re-test** of D-014's revisit condition after the Transformers.js 4.3.2 bump
(D-WS1a), which brings onnxruntime-web `1.31.0-dev.20260914-8d85527a0`.
Measured under Bun with onnxruntime-web's own wasm EP (not onnxruntime-node),
1 thread, AMD Ryzen 9 9950X shared with other workstreams.
1. **Session creation**, `onnx-community/whisper-tiny`
`decoder_model_merged_quantized.onnx` (D-014's repro file), every level:
| onnxruntime-web | disabled | basic | extended | all |
|---|---|---|---|---|
| 1.26.0-dev.20260416 (control) | OK | OK | FAIL | FAIL |
| 1.31.0-dev.20260914 | OK | OK | OK | OK |
The control still fails with D-014's exact error (`qdq_actions.cc:137
TransposeDQWeightsForMatMulNBits Missing required scale:
model.decoder.embed_tokens.weight_merged_0_scale`); 1.31 loads at all four.
2. **Transcription**, Transformers.js 4.3.2 pipeline on that ort-web build,
`whisper-tiny.en` q8 (catalog WASM default), `en_aiproblem` (14.4 s),
worker options (30 s chunks, 5 s stride, timestamps, `max_new_tokens:
440`). Both pipelines in one process, 1 warm-up each, then 8 interleaved
runs (A/B order alternating) so machine-load drift hits both equally:
| level | warm runs (s) | median | WER | text |
|---|---|---|---|---|
| `basic` (D-014) | 1.49 1.44 1.36 1.39 1.41 1.47 1.48 1.60 | 1.47 s | 0.000 | identical |
| default (`all`) | 1.38 1.47 1.34 1.39 1.37 1.42 1.43 1.43 | 1.42 s | 0.000 | identical |
Same transcript byte-for-byte at both levels and across every run; the
default level is ~3 % faster here (within noise; certainly not slower).
**Decision:** `src/worker/inference.worker.ts` no longer passes
`session_options` — WASM and WebGPU both use onnxruntime's default level.
`tests/transformers-runtime.test.ts` pins the onnxruntime-web version this
was measured on and that the worker sets no `graphOptimizationLevel`; a
different ort-web build fails the fast suite until this session-creation
check is re-run.
**Not covered here:** a real browser's ort-web wasm build (the same
`ort-wasm-simd-threaded.*` files, but multithreaded under COOP/COEP) —
`tests/offline.md` §2's WASM step is the owner's browser check.