Focus on speedup and not hanging on large audio files.

This commit is contained in:
Lucas Winther committed 2026-07-15 13:01:22 +02:00
1 parent cba27fc728
commit 0bd5d326a5
7 files changed
+366 -2

No files matched your search

+31
View File
@@ -294,3 +294,34 @@ download is a full pipeline load, so it also warms up inference and surfaces the
available), `ModelPicker` states explicitly that nothing is downloaded to the
device; the Download button stays (with a matching tooltip) for on-device /
offline preparation.
## D-021 — Long-file hardening: no idle timeout on the transcribe stream, heartbeats, single-pass Whisper generate
**Status:** Accepted (2026-07-15)
Long uploads (tested: 15-min Opus, `tests/audio_files/en_alien.opus`) hung or died
on both engines. Three independent causes, all length-dependent:
1. **Bun's default 10 s `idleTimeout` reset `/api/transcribe` mid-stream.** Segment
gaps legitimately exceed 10 s (sidecar spawn + model load/download before the
first segment; slow decode windows — 30 s+ gaps observed with large-v3-turbo on
CPU). The stream then ended without a `result` line, so the client reported
"connection was lost" after minutes of apparent hang. Fix: `src/index.ts` calls
`server.timeout(request, 0)` for `POST /api/transcribe` (and 90 s for
`GET /api/transcribe/status`, whose first probe waits up to 60 s for the sidecar
import), and `src/server/transcribe.ts` enqueues a blank ndjson line every 5 s as
a heartbeat — reverse proxies get liveness, dead connections surface as enqueue
failures, and `ServerEngine` already skips blank lines.
2. **Bun's default 128 MB `maxRequestBodySize` rejected uploads > ~35 min** (raw
f32 PCM is 3.84 MB/min) before the route's own 512 MB limit ever applied. Fix:
`maxRequestBodySize` is set to the shared `MAX_UPLOAD_BYTES` export.
3. **Browser engine: Transformers.js 4.2's Whisper `generate()` takes a seek-loop
path when `return_timestamps` is set without `max_new_tokens`** — 1–3 full decode
passes per window, redundant under the pipeline's own 30 s/5 s chunking. Measured
on the 15-min clip (tiny.en, onnxruntime-node): 80 generate passes for 44 windows,
1.4× slower, WER 44.7 % vs 31.5 % (duplicated boundary text), and the streamer's
`on_finalize` fired per pass instead of per window, so `TranscribeProgressTracker`
saturated the progress bar at ~60 % of the run and the UI looked frozen for the
remainder (on WASM: many minutes). A window whose last boundary timestamp is 0 can
even loop forever. Fix: the worker passes `max_new_tokens: 440`, which selects the
classic single-pass-per-window path (440 ≈ 448 decoder positions minus prompt).
**Revisit:** when Transformers.js changes the seek-loop condition (currently
`return_timestamps && !max_new_tokens`, `src/models/whisper/modeling_whisper.js`),
retest whether the explicit `max_new_tokens` is still needed.