Run WASM sessions at the default graph optimization level now that ort-web 1.31 loads the 8-bit decoders
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
9f1d58c0b7
commit
cc8e031d6b
6 files changed
+70
-11
No files matched your search
+42
-1
@@ -125,7 +125,8 @@ than ONNX Runtime under Bun; the server engine becomes genuinely useful rather t
|
||||
slow convenience.
|
||||
|
||||
## D-014 — WASM sessions run at graphOptimizationLevel "basic"
|
||||
**Status:** Accepted (bug workaround, 2026-07-10)
|
||||
**Status:** **Superseded by D-WS1c** (bug workaround, 2026-07-10; the ort-web
|
||||
build in Transformers.js 4.3.2 no longer has the bug, so the cap is gone)
|
||||
On the WASM backend, the inference worker passes
|
||||
`session_options: { graphOptimizationLevel: "basic" }` to the Transformers.js
|
||||
pipeline. The onnxruntime-web build bundled with Transformers.js 4.2
|
||||
@@ -1319,3 +1320,43 @@ share the CPU), so read it only as "the seek loop is slower", not as a
|
||||
version-to-version speed comparison.
|
||||
**Decision:** keep `max_new_tokens: 440` in `src/worker/inference.worker.ts`
|
||||
and in `tests/live-real-model.test.ts`.
|
||||
|
||||
## D-WS1c — WASM sessions back at the default graph optimization level (D-014's "basic" cap dropped)
|
||||
**Status:** Accepted (2026-10-30) — supersedes D-014
|
||||
**Re-test** of D-014's revisit condition after the Transformers.js 4.3.2 bump
|
||||
(D-WS1a), which brings onnxruntime-web `1.31.0-dev.20260914-8d85527a0`.
|
||||
Measured under Bun with onnxruntime-web's own wasm EP (not onnxruntime-node),
|
||||
1 thread, AMD Ryzen 9 9950X shared with other workstreams.
|
||||
1. **Session creation**, `onnx-community/whisper-tiny`
|
||||
`decoder_model_merged_quantized.onnx` (D-014's repro file), every level:
|
||||
|
||||
| onnxruntime-web | disabled | basic | extended | all |
|
||||
|---|---|---|---|---|
|
||||
| 1.26.0-dev.20260416 (control) | OK | OK | FAIL | FAIL |
|
||||
| 1.31.0-dev.20260914 | OK | OK | OK | OK |
|
||||
|
||||
The control still fails with D-014's exact error (`qdq_actions.cc:137
|
||||
TransposeDQWeightsForMatMulNBits Missing required scale:
|
||||
model.decoder.embed_tokens.weight_merged_0_scale`); 1.31 loads at all four.
|
||||
2. **Transcription**, Transformers.js 4.3.2 pipeline on that ort-web build,
|
||||
`whisper-tiny.en` q8 (catalog WASM default), `en_aiproblem` (14.4 s),
|
||||
worker options (30 s chunks, 5 s stride, timestamps, `max_new_tokens:
|
||||
440`). Both pipelines in one process, 1 warm-up each, then 8 interleaved
|
||||
runs (A/B order alternating) so machine-load drift hits both equally:
|
||||
|
||||
| level | warm runs (s) | median | WER | text |
|
||||
|---|---|---|---|---|
|
||||
| `basic` (D-014) | 1.49 1.44 1.36 1.39 1.41 1.47 1.48 1.60 | 1.47 s | 0.000 | identical |
|
||||
| default (`all`) | 1.38 1.47 1.34 1.39 1.37 1.42 1.43 1.43 | 1.42 s | 0.000 | identical |
|
||||
|
||||
Same transcript byte-for-byte at both levels and across every run; the
|
||||
default level is ~3 % faster here (within noise; certainly not slower).
|
||||
**Decision:** `src/worker/inference.worker.ts` no longer passes
|
||||
`session_options` — WASM and WebGPU both use onnxruntime's default level.
|
||||
`tests/transformers-runtime.test.ts` pins the onnxruntime-web version this
|
||||
was measured on and that the worker sets no `graphOptimizationLevel`; a
|
||||
different ort-web build fails the fast suite until this session-creation
|
||||
check is re-run.
|
||||
**Not covered here:** a real browser's ort-web wasm build (the same
|
||||
`ort-wasm-simd-threaded.*` files, but multithreaded under COOP/COEP) —
|
||||
`tests/offline.md` §2's WASM step is the owner's browser check.
|
||||
Reference in new issue
Block a user