Offer Whisper's Translate to English only on models trained for it (canTranslate)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
036635e854
commit
02a5eab119
8 files changed
+151
-15
No files matched your search
@@ -141,7 +141,7 @@ environment). See `WorkBreakdown.md` for the itemized checklist.)
|
||||
`AudioContext.decodeAudioData`, downmixes to mono, resamples to 16 kHz,
|
||||
transfers the `Float32Array` to the worker. For multilingual models the
|
||||
`transcribe` message also carries `language` (or auto-detect) and `task`
|
||||
(transcribe | translate).
|
||||
(transcribe | translate — translate only for models with `canTranslate`, D-WS4a).
|
||||
5. Worker runs the pipeline (chunked, with timestamps), streaming partial results
|
||||
back via `{type:"partial"}` messages; final result as `{type:"result"}`.
|
||||
|
||||
|
||||
@@ -1224,3 +1224,20 @@ whole-file output is equally bad, which is exactly what the absolute bar
|
||||
exists to catch.
|
||||
**Consequence:** full `bun test` ≈ 19 min on CPU (en_alien ≈ 20 min of it
|
||||
on a cold run); the fast suite excludes this file.
|
||||
|
||||
## D-WS4a — Whisper's "Translate to English" only on models trained for it (`canTranslate`)
|
||||
**Status:** Accepted (2026-10-29) · spec 001 Goal 4a
|
||||
`WhisperModel` gains `canTranslate: boolean` (`src/lib/models.ts`): **true** for multilingual
|
||||
tiny / base / small / medium, **false** for every `.en` model (no task token) and for
|
||||
`whisper-large-v3-turbo` — OpenAI's model card says turbo was fine-tuned on transcription data only
|
||||
and is not trained for translation, so its "translate" output is unreliable. Any future turbo-based
|
||||
fine-tune defaults to false until tested. `TranscriptionOptions` renders the Task control only where
|
||||
true (language stays). **Stale task:** the `task` state is kept, but `taskForModel(modelId, task)`
|
||||
maps "translate" to "transcribe" for a model that can't translate; App applies it on model switch
|
||||
(picker and the server-engine auto-switch to turbo) and again when building every file / live run's
|
||||
options, so a leftover "translate" can never reach the worker or the server. **Not changed:** the
|
||||
server's `parseLanguageAndTask` (D-032) still accepts "translate" for any model — the browser never
|
||||
sends it for turbo now, and a direct API caller asking faster-whisper to translate with turbo gets
|
||||
what faster-whisper does; rejecting it server-side would be a behaviour change for API users with no
|
||||
UI benefit. Speech in another language than English can still end up in English (or Danish) through
|
||||
text translation of the finished transcript (D-WS4b), for every model.
|
||||
@@ -65,6 +65,8 @@ Notes (M1):
|
||||
surface active backend in UI
|
||||
- [x] Handle model switching (dispose old pipeline, load new)
|
||||
- [x] Language / task options for multilingual models (transcribe vs translate)
|
||||
- [x] Whisper translate only on models trained for it (`canTranslate`; hidden for
|
||||
large-v3-turbo and `.en`, stale "translate" resets to transcribe — D-WS4a)
|
||||
- [~] Verify quantized variants (q4/q8/fp16) on both backends; record working
|
||||
combinations in `models.ts`
|
||||
|
||||
|
||||
Reference in new issue
Block a user