Enhanced transcription usability.

This commit is contained in:
Lucas Winther committed 2026-07-17 15:18:15 +02:00
1 parent 770cfc9ff7
commit 74858f63fc
8 files changed
+244 -21

No files matched your search

+41
View File
@@ -396,3 +396,44 @@ validation failed. Buffer with '' label is invalid". The worker now sets
fixes its thread pool at first-session init, so the first attempt in a
worker decides; a wasm fallback after a failed accelerator attempt may run
single-threaded until the next worker respawn — correct over fast.
## D-024 — Live mode batches queued final passes into merged passes
**Status:** Accepted (2026-07-17)
Reported symptom: stopping a live session got stuck on "Finishing up…" while
the machine sat at full CPU — indistinguishable from a hang, and the saturated
cores (D-023 uses most of them) made the whole browser feel locked up.
**Cause — a scheduling backlog, not a deadlock.** Whisper pads every inference
pass to a full 30 s encoder window, so a pass costs nearly the same whether it
carries 1 s or 29 s of audio (measured on the browser runtime, ort-web WASM EP
medium q4: ~4 s per pass at 12 threads, ~9–12 s at 4 threads, for 1–9 s
segments alike). Segments therefore commit faster than their final passes
complete whenever the model is slower than speech (medium/large on WASM), the
committed-segment queue grows all session, and `stop()` used to drain it **one
pass per segment** — a 60 s session on 4 threads measured a 52 s drain; longer
sessions scale it into minutes. All the while the UI showed only a static
"Finishing up…".
**Fix (two parts):**
1. `LiveTranscriber.pump()` batches consecutive queued segments into **one
merged final pass**, capped at the segmenter's `maxSegmentSeconds` so a
merged pass stays a single Whisper window. Segment audio is concatenated
(segments end on trailing silence or a soft cut, so splices land in natural
gaps); the result's chunk timestamps are mapped piecewise back to absolute
session time via each segment's own start offset. The backlog — and the
stop drain — is now bounded by ~one pass per 29 s of speech instead of one
pass per segment (the same 60 s/4-thread scenario drains in ~2 passes).
2. `App.tsx` surfaces `LiveUpdate.pendingFinalPasses` while stopping:
"Finishing up… transcribing N remaining segments" counts down instead of
freezing, on both the browser and the server-live (D-019) paths.
**Alternatives rejected:** skipping final passes for segments that already had
tentative text (during a backlog new segments get *no* tentative pass, so
skipping loses their words entirely); a "stop now" escape hatch (same data
loss, silently); merging across the 30 s window (impossible — Whisper's
context is one window).
**Consequence:** a merged pass attributes its whole text to the batch's first
segment (per-segment text attribution is meaningless after joining anyway) and
inter-segment silence is not re-inserted, so chunk timestamps inside a merged
pass are exact per segment region but a chunk spanning a splice maps to its
start's region. Covered by `tests/live.test.ts` (batching + cap) and the WER
gates (`tests/live-accuracy.test.ts`, `tests/live-real-model.test.ts` — the
real-model test's fast feed makes real inference lag, so merging is exercised
there with real weights).