lucasw89/whisper_web
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Detect the language once for auto-detect file passes on the chunked path (D-053)

Browse Source 
Transformers.js has no Whisper language detection and silently decoded
Danish as English whenever "Skip silence" was off or unavailable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Lucas Winther
2026-10-30 14:35:09 +01:00
co-authored by Claude Opus 5.5
parent 3583d60847
commit dd9dfc628b
6 changed files with 112 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/Decisions.md
+25
View File
@@ -1821,3 +1821,28 @@ chunked path. Live mode is unaffected.
**Caveat:** the WER numbers behind this use reference transcripts generated
with Whisper (owner, 2026-10-30), so they favour Whisper-like output; re-check
once correct transcripts are available.
## D-053 — "Auto" language on the chunked file path: detect once on the first 30 s, then pin it
**Status:** Accepted (2026-10-30)
Transformers.js 4.3.2 still has no Whisper language detection
(`modeling_whisper.js`: `// TODO: Implement language detection` → "No
language specified - defaulting to English"). With "Auto-detect" selected,
a file transcribed on the chunked path — the fallback whenever "Skip
silence" (D-052) is off or its VAD model is unavailable — was decoded as
English: Danish `da_festuge` came out as a rough English translation (WER
0.94 vs 0.14 with `da`, found in D-049's measurements).
**Decision:** `src/worker/language.ts` `detectFileLanguage` reuses the
long-form pipeline's one-step detector (D-049: encoder pass + one decoder
step restricted to language tokens) on the first 30 s of the file, and the
worker pins the result for the whole chunked pass. Only file passes on
multilingual models without an explicit language; `.en` models and an
explicit language are unchanged.
**Verified:** `tests/live-real-model.test.ts` "auto language on the chunked
file path": da_festuge → `da` (WER within its bar), en_aiproblem and
en_alien → `en` (Large v3 Turbo, onnxruntime-node).
**Cost:** one extra encoder pass per file (the chunked pipeline can't reuse
it). **Not covered:** live passes with "auto" still decode as English —
detecting on every short live pass would add an encoder run per pass; it
needs a once-per-session design (WorkBreakdown).
**Caveat:** leading silence or music in the first 30 s can mislead
detection; long-form detects on the first *speech* chunk instead.