Add workstream 4 plan: canTranslate and Opus-MT transcript translation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Lucas Winther
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
9159e2ab5e
commit
036635e854
@@ -0,0 +1,137 @@
|
||||
---
|
||||
status: in-progress
|
||||
---
|
||||
|
||||
# 001-4 — Translate: implementation plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:executing-plans (inline, no subagents —
|
||||
> coordination plan). Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** (4a) a per-model `canTranslate` flag that hides Whisper's "Translate to English" where it is
|
||||
unsupported and resets a stale `task: "translate"`; (4b) on-device Opus-MT text translation of any
|
||||
finished transcript (da→en, en→da), with timestamps kept for `.srt` / timestamped `.txt` exports.
|
||||
|
||||
**Architecture:** `canTranslate` lives in the catalog (`src/lib/models.ts`) with a pure
|
||||
`taskForModel()` guard used by App on model switch and on every run. Text MT runs in a **dedicated
|
||||
worker** (`src/worker/translate.worker.ts`) — its own onnxruntime instance with full WASM threads,
|
||||
independent of the Whisper worker's lifetime (cancel/D-018 terminate, D-025 restart) and of its
|
||||
accelerator (an ort runtime first initialised for WebGPU is pinned single-threaded, D-023). Pure
|
||||
planning code (`src/lib/translation.ts`) groups transcript chunks into sentence-ending units, splits
|
||||
units into sentence pieces under a token budget, and reassembles translated units with the merged
|
||||
timestamps. UI: a secondary `<details>` disclosure "Translate transcript…" under the transcript.
|
||||
|
||||
**Tech Stack:** Bun, React 19, Transformers.js 4.2.0 (`translation` pipeline, MarianMT), onnxruntime-web
|
||||
(WASM) in the browser, onnxruntime-node for measurement.
|
||||
|
||||
**Spec:** `docs/specs/001-transcription-quality-and-speed.md` (Goal 4, call "Text translation (4b)
|
||||
scope"); coordination: `docs/plans/001-0-coordination.md`.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- AGENTS.md rules: Bun only; no cross-origin subresources (models come from huggingface.co via
|
||||
Transformers.js like Whisper; ort from `/ort/`); colours as tokens; errors via `classifyError`.
|
||||
- D-020 / anti-goal: the MT model is downloaded only when the user clicks Translate; the panel states
|
||||
the one-time size first. Cached via Transformers.js Cache API (D-005); offline works once cached.
|
||||
- Nothing leaves the device: translation runs in the browser for every engine (server results too).
|
||||
- Copy: Whisper's option stays **"Translate to English"**; the new entry is **"Translate transcript…"**.
|
||||
- `canTranslate`: false for `onnx-community/whisper-large-v3-turbo` and every `.en` model; true for
|
||||
multilingual tiny/base/small/medium.
|
||||
- Decision ids: `D-WS4a` (canTranslate), `D-WS4b` (text MT). Ports 3140–3149. Heavy jobs under
|
||||
`flock /tmp/whisper-heavy.lock`.
|
||||
|
||||
## Review Focus
|
||||
|
||||
1. **Transcript > 512 tokens** (Opus-MT input limit): split on sentence boundaries, order preserved,
|
||||
nothing dropped → Task 3 test `splitForTranslation keeps every word of en_alien in order with every
|
||||
piece under budget` + real-model test (Task 4) on all of `en_alien.txt`.
|
||||
2. **Multi-sentence input to Opus-MT drops later sentences** (observed in the probe: "…Copenhagen.
|
||||
How are you?" → only the first sentence) → each sentence is its own piece (Task 3 test).
|
||||
3. **Unpunctuated text** (live output, or a 2-minute run-on): hard-wrap on commas, then words, so no
|
||||
piece exceeds the budget (Task 3 test).
|
||||
4. **Stale `task: "translate"` after switching to turbo / a `.en` model / the server auto-switch to
|
||||
turbo** → reset to transcribe (Task 1 test on `taskForModel`, App wired at every switch + run).
|
||||
5. **Offline / model not cached** → the Translate button still offers the download; a failed download
|
||||
surfaces a classified network error with Retry (Task 5, errors test).
|
||||
|
||||
---
|
||||
|
||||
### Task 1: `canTranslate` + stale-task guard (4a)
|
||||
|
||||
**Files:** Modify `src/lib/models.ts`, `src/components/TranscriptionOptions.tsx`, `src/App.tsx`;
|
||||
Test `tests/models.test.ts`, new `tests/transcriptionOptions.test.tsx`.
|
||||
|
||||
**Interfaces — Produces:** `WhisperModel.canTranslate: boolean`;
|
||||
`taskForModel(modelId: string, task: WhisperTask): WhisperTask`;
|
||||
`TranscriptionOptions` prop `canTranslate: boolean`.
|
||||
|
||||
- [ ] Failing tests: `canTranslate` false for turbo and every `.en`, true for the other multilingual
|
||||
ids; `taskForModel("onnx-community/whisper-large-v3-turbo", "translate") === "transcribe"`,
|
||||
unchanged for `whisper-small`; unknown id → `"transcribe"`. `renderToStaticMarkup` of
|
||||
`TranscriptionOptions` with `canTranslate={false}` contains no "Translate to English" and no Task
|
||||
select; with `true` contains it.
|
||||
- [ ] Implement; App: `setTask(t => taskForModel(next, t))` in `handleModelSelect` and the server
|
||||
auto-switch effect; runs pass `task: taskForModel(modelId, task)`.
|
||||
- [ ] Typecheck, fast suite; D-WS4a entry; commit.
|
||||
|
||||
### Task 2: error stage for translation
|
||||
|
||||
**Files:** `src/lib/errors.ts`, `tests/errors.test.ts`.
|
||||
- [ ] Failing test: `classifyError("Out of memory", { stage: "translate" })` → suggestion mentions a
|
||||
shorter transcript / closing tabs, not "smaller model"; network on `stage: "translate"` retryable.
|
||||
- [ ] Add `"translate"` to `ErrorStage`; commit with Task 3.
|
||||
|
||||
### Task 3: pure translation planning (`src/lib/translation.ts`)
|
||||
|
||||
**Interfaces — Produces:**
|
||||
- `TRANSLATION_MODELS: readonly TranslationModel[]` — `{ id, source: "da"|"en", target, label,
|
||||
approxBytes }` for `Xenova/opus-mt-da-en`, `Xenova/opus-mt-en-da` (q8, ~113 MB each).
|
||||
- `getTranslationModel(source, target)`, `guessSourceLanguage(text, hint?): "da" | "en"`.
|
||||
- `buildTranslationUnits(result: { text; chunks }): TranslationUnit[]` —
|
||||
`{ text: string; timestamp: [number, number | null] | null }`; consecutive chunks merged until the
|
||||
text ends a sentence (cap 8 chunks); no chunks → one unit per non-empty line.
|
||||
- `splitForTranslation(text, { maxTokens = 200, countTokens = estimateTokens }): string[]` —
|
||||
Intl.Segmenter sentences (abbreviation-only fragments merge forward), then comma, then word wrap.
|
||||
- `assembleTranslation(units, translated: string[]): { text; chunks }`.
|
||||
- Tests (`tests/translation.test.ts`): unit merging & timestamps, null end, no-chunk fallback, sentence
|
||||
splitting, Review Focus 1–3, assemble length mismatch throws, `guessSourceLanguage`.
|
||||
- [ ] Red → green → commit.
|
||||
|
||||
### Task 4: translate worker + protocol + measurement
|
||||
|
||||
**Files:** Create `src/worker/translate.worker.ts`, `src/worker/translateProtocol.ts`,
|
||||
`tests/translateProtocol.test.ts`, `tests/translate-real-model.test.ts` (skips when the Hub is
|
||||
unreachable and the model is not cached; run under the lock), `scripts/bench-translate.ts`.
|
||||
- Protocol: UI→worker `{ type: "translate"; requestId; modelId; units: string[] }`;
|
||||
worker→UI `translate-load-progress` (file/loaded/total/status), `translate-progress
|
||||
{ done; total }` (pieces), `translate-result { texts: string[] }`, `translate-error
|
||||
{ stage: "load" | "translate"; message }`; guards `isTranslateRequest` / `isTranslateWorkerMessage`.
|
||||
- Worker: `pipeline("translation", id, { dtype: "q8", device: "wasm",
|
||||
session_options: { graphOptimizationLevel: "basic" } })` (D-014 — Marian also ties embeddings),
|
||||
WASM threads as D-023, `wasmPaths "./ort/"`; one cached pipeline per model id; pieces via
|
||||
`splitForTranslation` with the real tokenizer count; batches of `BATCH_SIZE` (measured).
|
||||
- Bundle + serve: `src/index.ts` dev route `/translate.worker.js`, `scripts/build.ts` emits and
|
||||
precaches it.
|
||||
- [ ] Measure en_alien.txt → da and da_festuge → en on onnxruntime-node (batch 1 vs 4 vs 8); record.
|
||||
- [ ] Commit.
|
||||
|
||||
### Task 5: UI — `TranslatePanel`
|
||||
|
||||
**Files:** Create `src/components/TranslatePanel.tsx`; modify `src/App.tsx`,
|
||||
`src/components/TranscriptView.tsx` (`title` prop + `exportSuffix`), `src/components/StorageManager.tsx`
|
||||
(list MT models), `src/index.css`.
|
||||
- `<details className="translate-panel"><summary>Translate transcript…</summary>` with direction
|
||||
select ("Danish → English", "English → Danish", default from `guessSourceLanguage`), size line when
|
||||
not cached ("Downloads a translation model (~113 MB, one time)"), Translate / Cancel buttons,
|
||||
progress ("Downloading translation model… 40%", "Translating… 12 of 80 sentences"), error box via
|
||||
`classifyError(…, { stage: "translate" | "load" })` with Retry when retryable.
|
||||
- Result rendered by `TranscriptView` with title "Translation (English)" and exports named
|
||||
`<file>.en.txt` / `.en.srt`.
|
||||
- State resets whenever `result` changes; Cancel terminates the translate worker.
|
||||
- [ ] impeccable polish pass; screenshots light/dark × desktop/mobile; real headless-Chrome run of a
|
||||
translation (WASM) if feasible.
|
||||
- [ ] Commit.
|
||||
|
||||
### Task 6: docs + wrap-up
|
||||
- [ ] D-WS4b entry (models, licences, numbers); Architecture.md (translate worker), AGENTS.md rule
|
||||
wording (two workers), WorkBreakdown, README if needed, `tests/offline.md` step.
|
||||
- [ ] typecheck, fast suite, build; `git rm` this plan; commit.
|
||||
Reference in New Issue
Block a user