diff --git a/docs/plans/001-1-transformers-upgrade.md b/docs/plans/001-1-transformers-upgrade.md new file mode 100644 index 0000000..851c621 --- /dev/null +++ b/docs/plans/001-1-transformers-upgrade.md @@ -0,0 +1,122 @@ +--- +status: in-progress +--- + +# 001-1 — Transformers.js 4.2.0 → 4.3.2 Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:executing-plans (inline, no nested +> subagents — coordination plan rule). Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** run the browser engine on `@huggingface/transformers` 4.3.2 with the D-027/D-028 +sequential-session patch carried over, and re-test the decisions the upgrade may obsolete. + +**Architecture:** dependency bump + `bun patch` regeneration; a small runtime-guard test file pins +the installed version, the patch, and the upstream seek-loop condition D-021 depends on, so a future +bump that silently drops the patch or changes the condition fails the fast suite. Re-tests run as +scratch measurement scripts outside the repo (`/tmp/ws1/`); their numbers go into `D-WS1*` entries. + +**Tech Stack:** Bun 1.3, `@huggingface/transformers` 4.3.2 (onnxruntime-web 1.31.0-dev.20260914, +onnxruntime-node 1.30.0), bun:test. + +**Spec:** `docs/specs/001-transcription-quality-and-speed.md` (Goal 1); coordination rules in +`docs/plans/001-0-coordination.md`. + +## Global Constraints + +- Bun only; no new runtime dependency; transformers stays a devDependency (D-031). +- New decisions are `## D-WS1 — …` appended at the bottom of `docs/Decisions.md`; Accepted + entries are superseded, never rewritten. +- Heavy commands (real-model runs, benchmarks, model downloads, full `bun test`) under + `flock /tmp/whisper-heavy.lock`. Ports 3110–3119. No push, no publishing. +- Measurements are CPU numbers (onnxruntime-node, or onnxruntime-web wasm EP under Bun); say which. +- No WebGPU-only default change ships on WASM numbers; WebGPU items → "needs owner's browser check". + +## Review Focus + +1. **Patch silently lost on reinstall** (lockfile/`patchedDependencies` key mismatch) → the app + loads fine but mobile OOM returns. Pinned by Task 1's patch-applied test. +2. **onnxruntime-web file names changed** (new version may ship different `.mjs/.wasm` names) → + `/ort/` serving and `dist/ort` copy must still contain what ort-web requests. Task 1: build and + check `dist/ort/` contains `ort-wasm-simd-threaded.jsep.{mjs,wasm}` (the WebGPU bundle's files). +3. **Upstream changes the seek-loop condition** → D-021's `max_new_tokens` could become wrong or + unneeded without notice. Pinned by Task 2's source-tripwire test. +4. **onnxruntime-node 1.30 drifts real-model WER** → `tests/live-real-model.test.ts` under the lock + (Task 5). +5. **q8 decoders on WASM at default optimization level** (D-014's failure) — Task 3 measures every + level before any change to the worker's session options. + +--- + +### Task 1: Upgrade to 4.3.2 and carry the patch + +**Files:** +- Create: `tests/transformers-runtime.test.ts` +- Modify: `package.json`, `bun.lock`, `AGENTS.md` (patch path), `docs/WorkBreakdown.md` +- Create: `patches/@huggingface%2Ftransformers@4.3.2.patch`; Delete: the 4.2.0 patch + +**Interfaces:** +- Produces: `tests/transformers-runtime.test.ts` (Task 2 adds a case to it). + +- [ ] **Step 1: Write the failing test** `describe("@huggingface/transformers runtime")`: + - `"installed version is the pinned 4.3.2"` — `node_modules/@huggingface/transformers/package.json` + `.version === "4.3.2"`. + - `"patchedDependencies points at an existing patch for the installed version"` — root + `package.json` `patchedDependencies["@huggingface/transformers@4.3.2"]` is a path that exists. + - `"web bundle's constructSessions carries the sequential-load patch (D-027/D-028)"` — the + `constructSessions` function text sliced from `dist/transformers.web.js` contains + `options?.sequentialSessionLoad` and both `Promise.all(` and `for (const name of Object.keys(names))`. +- [ ] **Step 2: Run** `bun test tests/transformers-runtime.test.ts` → FAIL on version (4.2.0) and key. +- [ ] **Step 3:** `bun add -d @huggingface/transformers@4.3.2` (remove the 4.2.0 + `patchedDependencies` entry first if bun refuses), `bun patch @huggingface/transformers`, re-apply + the 4.2.0 hunk to `constructSessions` in `dist/transformers.web.js` (upstream is unchanged: + still a bare `Promise.all`), `bun patch --commit node_modules/@huggingface/transformers`. + `git rm` the 4.2.0 patch. Re-link the shared `.cache` symlink if the reinstall removed it. +- [ ] **Step 4: Run** the test → PASS; `bun install --frozen-lockfile` clean; typecheck; fast + suite; `bun run build`; `ls dist/ort` shows the files ort-web 1.31 requests. +- [ ] **Step 5:** docs: AGENTS.md patch filename; `D-WS1a` (upgrade, patch carried, why upstream + doesn't obsolete it, ort versions, Parakeet = CTC only); WorkBreakdown tick. Commit + "Upgrade @huggingface/transformers to 4.3.2 and carry the sequential-session patch". + +### Task 2: D-021 re-test (seek loop) + +**Files:** Modify `tests/transformers-runtime.test.ts`, `docs/Decisions.md`, +`src/worker/inference.worker.ts` (comment only: "4.2" → "4.2–4.3", cite D-WS1b). + +- [ ] **Step 1: Test** `"Whisper generate still takes the seek loop when max_new_tokens is unset (D-021)"` + — `src/models/whisper/modeling_whisper.js` contains + `if (generation_config.return_timestamps && !kwargs.max_new_tokens)`. Run → PASS expected + (tripwire; it encodes the revisit condition, verified against the 4.3.2 source). +- [ ] **Step 2: Measure** (`/tmp/ws1/d021.ts`, onnxruntime-node, under the lock): tiny.en q8 on + `en_alien.opus` (ffmpeg-static → 16 kHz f32), pipeline options as the worker + (`chunk_length_s 30, stride_length_s 5, return_timestamps true`), with and without + `max_new_tokens: 440`; record wall-clock, generate-pass count (wrap `model.generate`), window + count, WER vs `en_alien.txt` (`tests/dsp.ts` `wordErrorRate`). +- [ ] **Step 3:** `D-WS1b` with the numbers; keep `max_new_tokens: 440` unless the seek path is now + both faster and not worse on WER. Commit. + +### Task 3: D-014 re-test (WASM graphOptimizationLevel) + +**Files:** `docs/Decisions.md`; `src/worker/inference.worker.ts` only if the outcome is "drop". + +- [ ] **Step 1: Measure** (`/tmp/ws1/d014.ts`): `globalThis[Symbol.for("onnxruntime")] = + await import("onnxruntime-web")` before importing transformers, wasm EP, 1 thread; for + `onnx-community/whisper-tiny` `decoder_model_merged_quantized.onnx` create a session at + `disabled/basic/extended/all`; record success/error. If all load: transcribe `en_aiproblem` with + tiny.en q8 at `basic` vs default (`all`), 3 warm runs each, wall-clock + text identity. +- [ ] **Step 2:** If default fails → keep, `D-WS1c` "still needed on ort-web 1.31.0-dev". If it + loads and is not slower → remove the override (worker comment cites D-WS1c), run a q8 transcribe + again through the harness with the worker's new options. Commit. + +### Task 4: WebGPU-affecting changes (D-022, D-025/D-026) — inspection only + +- [ ] **Step 1:** Read 4.3.0/4.3.1/4.3.2 release notes and diff 4.2.0↔4.3.2 `src/` for + WebGPU/fp16/Firefox/Safari changes; check ort-web 1.31 dist for Gecko/fp16 handling. +- [ ] **Step 2:** `D-WS1d` listing each item as "needs owner's browser check" with what to try + (explicit WebGPU on Firefox for D-026; medium fp16 for D-022; Safari 26 WebGPU). No code change. + Commit. + +### Task 5: Verify and finish + +- [ ] typecheck, fast suite, build green; `flock /tmp/whisper-heavy.lock bun test + tests/live-real-model.test.ts` green; record WER per clip in D-WS1a if changed from the bars. +- [ ] `git status` clean; last commit `git rm docs/plans/001-1-transformers-upgrade.md`.