Renumber spec 001 decision placeholders to D-040–D-051; record the Danish publication
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
30969e86b7
commit
0632f7843b
71 files changed
+244
-236
No files matched your search
+20
-20
@@ -27,7 +27,7 @@ Logged-in users may optionally route inference to the server.
|
||||
│ ├─ Device selection: WebNN → WebGPU → WASM fallback │
|
||||
│ └─ Model weights cached via browser Cache API │
|
||||
│ │
|
||||
│ Translate Web Worker (D-WS4b) │
|
||||
│ Translate Web Worker (D-045) │
|
||||
│ └─ Transformers.js Opus-MT (MarianMT), WASM, on user request │
|
||||
│ │
|
||||
│ Service Worker (PWA) │
|
||||
@@ -76,19 +76,19 @@ src/
|
||||
parakeet.ts # pure Parakeet logic: ≤30 s window plan, words→chunks, vocab tokenizer, pinned URLs
|
||||
modelCache.ts # "downloaded?" per model via Cache Storage (D-005; reused by M3 offline indicator)
|
||||
engine.ts # engine abstraction: BrowserEngine | ServerEngine
|
||||
translation.ts # transcript translation planning: Opus-MT catalog, units, sentence pieces (pure, D-WS4b)
|
||||
hint.ts # "Names and terms" hint: normalize, localStorage, prompt truncation rule (D-WS2a)
|
||||
translation.ts # transcript translation planning: Opus-MT catalog, units, sentence pieces (pure, D-045)
|
||||
hint.ts # "Names and terms" hint: normalize, localStorage, prompt truncation rule (D-048)
|
||||
worker/
|
||||
inference.worker.ts # Transformers.js pipeline, WebNN/WebGPU/WASM selection; dispatches on catalog `family`
|
||||
parakeetEngine.ts # Parakeet TDT path: parakeet.js ParakeetModel on the shared onnxruntime-web (D-WS7a)
|
||||
parakeetEngine.ts # Parakeet TDT path: parakeet.js ParakeetModel on the shared onnxruntime-web (D-051)
|
||||
modelFiles.ts # Cache-API weight download for non-Transformers.js paths (same cache, same progress msgs)
|
||||
ortShim.ts # re-exports onnxruntime-web/webgpu; parakeet.js's own ort import is aliased here
|
||||
protocol.ts # typed postMessage messages (shared with UI)
|
||||
prompt.ts # decoder prompt (<|startofprev|> hint) via a generate() wrapper (D-WS2a)
|
||||
translate.worker.ts # transcript translation (Opus-MT), its own worker + ort instance (D-WS4b)
|
||||
prompt.ts # decoder prompt (<|startofprev|> hint) via a generate() wrapper (D-048)
|
||||
translate.worker.ts # transcript translation (Opus-MT), its own worker + ort instance (D-045)
|
||||
translateProtocol.ts # typed messages for the translate worker
|
||||
opusMt.ts # Opus-MT loader shared by the worker, bench script and real-model test
|
||||
longform/ # long-form file pipeline (D-WS3a): pure vad.ts / quality.ts / pipeline.ts
|
||||
longform/ # long-form file pipeline (D-049): pure vad.ts / quality.ts / pipeline.ts
|
||||
# behind transcribeLongForm(audio, deps, options); transformers.ts implements
|
||||
# the deps (Silero VAD + Whisper decodes) on the same Transformers.js runtime
|
||||
pwa/
|
||||
@@ -158,17 +158,17 @@ environment). See `WorkBreakdown.md` for the itemized checklist.)
|
||||
`AudioContext.decodeAudioData`, downmixes to mono, resamples to 16 kHz,
|
||||
transfers the `Float32Array` to the worker. For multilingual models the
|
||||
`transcribe` message also carries `language` (or auto-detect) and `task`
|
||||
(transcribe | translate — translate only for models with `canTranslate`, D-WS4a).
|
||||
(transcribe | translate — translate only for models with `canTranslate`, D-044).
|
||||
For every model it may carry `prompt`: the optional "Names and terms" hint
|
||||
(`VocabularyHint`, stored in localStorage by `lib/hint.ts`), raw as typed (D-WS2a).
|
||||
(`VocabularyHint`, stored in localStorage by `lib/hint.ts`), raw as typed (D-048).
|
||||
5. Worker runs the pipeline (chunked, with timestamps), streaming partial results
|
||||
back via `{type:"partial"}` messages; final result as `{type:"result"}`.
|
||||
A hint becomes `<|startofprev|> hint.` once per pass (`worker/prompt.ts`,
|
||||
cut to a 120-token prompt budget at a whole word) and is prepended
|
||||
to the decoder input of every 30 s window by a wrapper around the model's
|
||||
`generate()`, which also strips it from the output again (D-WS2a).
|
||||
`generate()`, which also strips it from the output again (D-048).
|
||||
Live passes (4.1b) carry the same `prompt`.
|
||||
6. **Long-form branch (D-WS3a/D-WS3b, "Advanced → Skip silence", off by default).**
|
||||
6. **Long-form branch (D-049/D-050, "Advanced → Skip silence", off by default).**
|
||||
With the option on, every load posted for a file run or the Download button carries
|
||||
`vad: true`; once the Whisper pipeline is ready the worker loads Silero VAD
|
||||
(`onnx-community/silero-vad`, 2.2 MB, Cache API like the weights; offline it is
|
||||
@@ -183,7 +183,7 @@ environment). See `WorkBreakdown.md` for the itemized checklist.)
|
||||
chunked path whenever the VAD model isn't loaded, and live passes never take
|
||||
this branch (live keeps its RMS segmenter, D-016/D-024). A finished transcript
|
||||
with no words shows "No speech was found in this recording." The names-and-terms
|
||||
hint (`prompt`) is applied to every chunk's decode the same way (D-WS2a).
|
||||
hint (`prompt`) is applied to every chunk's decode the same way (D-048).
|
||||
|
||||
### 4.1b Live microphone transcription (anonymous, browser engine — D-016)
|
||||
1. `LiveControls` starts a session: `getUserMedia` → `AudioContext` at the
|
||||
@@ -216,12 +216,12 @@ environment). See `WorkBreakdown.md` for the itemized checklist.)
|
||||
only when signed in with the server engine available (`EngineToggle`);
|
||||
anonymous/offline users always get the browser-engine path above.
|
||||
|
||||
### 4.1c Transcript translation (any engine — D-WS4b)
|
||||
### 4.1c Transcript translation (any engine — D-045)
|
||||
|
||||
1. Any finished transcript (browser file, browser live, server engine) shows a
|
||||
"Translate transcript…" disclosure (`TranslatePanel`). Whisper's own
|
||||
"Translate to English" task stays in `TranscriptionOptions`, only for
|
||||
models with `canTranslate` (D-WS4a).
|
||||
models with `canTranslate` (D-044).
|
||||
2. On **Translate**, the UI groups the transcript's chunks into sentence-ending
|
||||
units (`buildTranslationUnits`, merged time spans) and posts their texts to
|
||||
`translate.worker.js` — a separate worker with its own onnxruntime instance.
|
||||
@@ -248,7 +248,7 @@ environment). See `WorkBreakdown.md` for the itemized checklist.)
|
||||
- Model weights live in the Cache API (`transformers-cache`) via Transformers.js —
|
||||
independent of the SW, which never touches that cache (D-005). `/api/*` is
|
||||
network-only. The Silero VAD model of the long-form branch lives there too
|
||||
(D-WS3b); offline, the worker checks that cache before loading it and never
|
||||
(D-050); offline, the worker checks that cache before loading it and never
|
||||
fetches, falling back to the chunked path if it is missing.
|
||||
- Updates: `pwa/register.ts` surfaces a waiting SW as a toast; accepting posts
|
||||
`SKIP_WAITING`, activation drops old `whisper-*` caches, and the page reloads.
|
||||
@@ -266,11 +266,11 @@ environment). See `WorkBreakdown.md` for the itemized checklist.)
|
||||
`src/server/transcribe.ts` bridges the sidecar's segment stream to the streamed
|
||||
JSON-lines HTTP response (D-011). `language`, `task` and the vocabulary hint
|
||||
(`prompt`) travel as query params (also on the live WebSocket) and are
|
||||
validated by `parseLanguageAndTask` (D-032, D-WS2a); the sidecar cuts the
|
||||
validated by `parseLanguageAndTask` (D-032, D-048); the sidecar cuts the
|
||||
hint with the same rule as the browser and passes it to faster-whisper as
|
||||
`initial_prompt` (D-WS2a). File requests (`mode: "file"`) decode with
|
||||
`initial_prompt` (D-048). File requests (`mode: "file"`) decode with
|
||||
faster-whisper's batched pipeline over Silero-VAD speech chunks; server-live
|
||||
passes send `mode: "live"` and stay sequential (D-WS5a). The shared catalog maps model ids to
|
||||
passes send `mode: "live"` and stay sequential (D-047). The shared catalog maps model ids to
|
||||
`Systran/faster-whisper-*` CTranslate2 conversions. If the sidecar is absent,
|
||||
the server engine reports unavailable; the browser/anonymous flow is unaffected.
|
||||
4. Both engines implement the same `Engine` interface in `src/lib/engine.ts`, so the
|
||||
@@ -283,7 +283,7 @@ environment). See `WorkBreakdown.md` for the itemized checklist.)
|
||||
- Anonymous users are never blocked from browser inference — auth only unlocks the
|
||||
server engine.
|
||||
|
||||
### 4.1c Parakeet (English-only, desktop-only browser model — D-WS7a)
|
||||
### 4.1c Parakeet (English-only, desktop-only browser model — D-051)
|
||||
Catalog entries carry `family?: "whisper" | "parakeet"` (absent ⇒ whisper). For
|
||||
`family: "parakeet"` the worker skips the Transformers.js pipeline:
|
||||
1. **Load:** `modelFiles.ts` fetches `vocab.txt`, the decoder_joint and the
|
||||
@@ -316,7 +316,7 @@ Catalog entries carry `family?: "whisper" | "parakeet"` (absent ⇒ whisper). Fo
|
||||
- **Anonymous-first:** every feature except the server engine works logged out.
|
||||
- **Offline-first:** app shell + previously-downloaded models must work offline.
|
||||
- **One browser inference runtime:** onnxruntime-web, driven by Transformers.js for
|
||||
Whisper and by parakeet.js for Parakeet (D-WS7a) — the same ort instance, never a
|
||||
Whisper and by parakeet.js for Parakeet (D-051) — the same ort instance, never a
|
||||
second runtime; the server engine runs faster-whisper (D-013). The model catalog
|
||||
stays shared across both engines (browser-only entries have no faster-whisper name).
|
||||
- **Worker isolation:** inference never runs on the main thread.
|
||||
+26
-18
@@ -125,7 +125,7 @@ than ONNX Runtime under Bun; the server engine becomes genuinely useful rather t
|
||||
slow convenience.
|
||||
|
||||
## D-014 — WASM sessions run at graphOptimizationLevel "basic"
|
||||
**Status:** **Superseded by D-WS1c** (bug workaround, 2026-07-10; the ort-web
|
||||
**Status:** **Superseded by D-043** (bug workaround, 2026-07-10; the ort-web
|
||||
build in Transformers.js 4.3.2 no longer has the bug, so the cap is gone)
|
||||
On the WASM backend, the inference worker passes
|
||||
`session_options: { graphOptimizationLevel: "basic" }` to the Transformers.js
|
||||
@@ -1226,7 +1226,7 @@ exists to catch.
|
||||
**Consequence:** full `bun test` ≈ 19 min on CPU (en_alien ≈ 20 min of it
|
||||
on a cold run); the fast suite excludes this file.
|
||||
|
||||
## D-WS1a — Transformers.js 4.2.0 → 4.3.2, sequential-session patch carried over
|
||||
## D-040 — Transformers.js 4.2.0 → 4.3.2, sequential-session patch carried over
|
||||
**Status:** Accepted (2026-10-29, spec 001 Goal 1)
|
||||
`@huggingface/transformers` is now pinned to exactly `4.3.2` (latest; 4.3.0
|
||||
added Safari 26+ WebGPU). It brings onnxruntime-web
|
||||
@@ -1254,7 +1254,7 @@ TDT/RNN-T (transducer) decoder or joint-network loop anywhere in the
|
||||
package, so **Parakeet TDT 0.6B v3 is not supported natively**; per the
|
||||
spec, that path uses `parakeet.js`.
|
||||
|
||||
## D-WS1d — WebGPU-side effects of the 4.3.2 upgrade: inspected, not measured (needs the owner's browser check)
|
||||
## D-041 — WebGPU-side effects of the 4.3.2 upgrade: inspected, not measured (needs the owner's browser check)
|
||||
**Status:** Open — inspection only (2026-10-29)
|
||||
This machine has no GPU and no browser WebGPU, so D-022, D-025 and D-026
|
||||
cannot be re-measured here. What the upgrade changes on that side, from the
|
||||
@@ -1294,7 +1294,7 @@ cannot be re-measured here. What the upgrade changes on that side, from the
|
||||
5. **Safari 26 (macOS/iOS):** what the footer reports for "auto"; if WebGPU,
|
||||
check speed and output.
|
||||
|
||||
## D-WS1b — D-021's `max_new_tokens: 440` is still needed on Transformers.js 4.3.2
|
||||
## D-042 — D-021's `max_new_tokens: 440` is still needed on Transformers.js 4.3.2
|
||||
**Status:** Accepted (2026-10-29) — re-test of D-021 part 3, which stands unchanged
|
||||
**Source:** 4.3.2's `WhisperForConditionalGeneration.generate()` still takes
|
||||
the seek loop on exactly D-021's condition,
|
||||
@@ -1321,10 +1321,10 @@ version-to-version speed comparison.
|
||||
**Decision:** keep `max_new_tokens: 440` in `src/worker/inference.worker.ts`
|
||||
and in `tests/live-real-model.test.ts`.
|
||||
|
||||
## D-WS1c — WASM sessions back at the default graph optimization level (D-014's "basic" cap dropped)
|
||||
## D-043 — WASM sessions back at the default graph optimization level (D-014's "basic" cap dropped)
|
||||
**Status:** Accepted (2026-10-30) — supersedes D-014
|
||||
**Re-test** of D-014's revisit condition after the Transformers.js 4.3.2 bump
|
||||
(D-WS1a), which brings onnxruntime-web `1.31.0-dev.20260914-8d85527a0`.
|
||||
(D-040), which brings onnxruntime-web `1.31.0-dev.20260914-8d85527a0`.
|
||||
Measured under Bun with onnxruntime-web's own wasm EP (not onnxruntime-node),
|
||||
1 thread, AMD Ryzen 9 9950X shared with other workstreams.
|
||||
1. **Session creation**, `onnx-community/whisper-tiny`
|
||||
@@ -1361,7 +1361,7 @@ check is re-run.
|
||||
`ort-wasm-simd-threaded.*` files, but multithreaded under COOP/COEP) —
|
||||
`tests/offline.md` §2's WASM step is the owner's browser check.
|
||||
|
||||
## D-WS4a — Whisper's "Translate to English" only on models trained for it (`canTranslate`)
|
||||
## D-044 — Whisper's "Translate to English" only on models trained for it (`canTranslate`)
|
||||
**Status:** Accepted (2026-10-29) · spec 001 Goal 4a
|
||||
`WhisperModel` gains `canTranslate: boolean` (`src/lib/models.ts`): **true** for multilingual
|
||||
tiny / base / small / medium, **false** for every `.en` model (no task token) and for
|
||||
@@ -1376,13 +1376,13 @@ server's `parseLanguageAndTask` (D-032) still accepts "translate" for any model
|
||||
sends it for turbo now, and a direct API caller asking faster-whisper to translate with turbo gets
|
||||
what faster-whisper does; rejecting it server-side would be a behaviour change for API users with no
|
||||
UI benefit. Speech in another language than English can still end up in English (or Danish) through
|
||||
text translation of the finished transcript (D-WS4b), for every model.
|
||||
text translation of the finished transcript (D-045), for every model.
|
||||
|
||||
## D-WS4b — Text translation of finished transcripts with Opus-MT, in its own worker
|
||||
## D-045 — Text translation of finished transcripts with Opus-MT, in its own worker
|
||||
**Status:** Accepted (2026-10-29) · spec 001 Goal 4b
|
||||
**What:** any finished transcript — browser file, browser live, server engine — can be translated
|
||||
da→en or en→da from a secondary "Translate transcript…" disclosure under the transcript
|
||||
(`TranslatePanel`). Whisper's own task keeps its label "Translate to English" (D-WS4a). The
|
||||
(`TranslatePanel`). Whisper's own task keeps its label "Translate to English" (D-044). The
|
||||
translation renders as a second transcript card ("Translation (English)") with its own
|
||||
`.txt` / `.srt` / `.txt+ts` exports named `<file>.<en|da>.<ext>`.
|
||||
**Models:** `Xenova/opus-mt-da-en` and `Xenova/opus-mt-en-da` — Transformers.js ONNX exports of
|
||||
@@ -1432,7 +1432,7 @@ terminates the translate worker only.
|
||||
**Not done / follow-ups:** WebGPU for Opus-MT (unmeasurable here); more language pairs (the catalog
|
||||
is two entries, `TRANSLATION_MODELS`); a translation of live text *while* recording.
|
||||
|
||||
## D-WS6a — Danish fine-tuned Whisper models: our own ONNX + CTranslate2 conversions, unpublished until the owner says go
|
||||
## D-046 — Danish fine-tuned Whisper models: our own ONNX + CTranslate2 conversions; only whisper-small-danish published
|
||||
**Status:** Proposed (2026-10-29, spec 001 Goal 6)
|
||||
**Candidates checked (licence + architecture, from each repo's model card, config and notices):**
|
||||
- `thorhojhus/whisper-small-danish` — MIT; plain `WhisperForConditionalGeneration`, whisper-small
|
||||
@@ -1484,7 +1484,15 @@ use beam 5.
|
||||
cards credit the original authors and carry MIT, or Apache-2.0 + NOTICE), then set
|
||||
`DANISH_MODELS_HF_OWNER`. The integrator sets `canTranslate: false` on these entries at merge.
|
||||
|
||||
## D-WS5a — Server engine: batched faster-whisper + tuned Silero VAD for files; live passes stay sequential
|
||||
**Publication (2026-10-30, owner's decision):** only the small fine-tune is
|
||||
published — `lucasw89/whisper-small-danish-ONNX` (browser) and
|
||||
`lucasw89/whisper-small-danish-ct2-int8` (server), public, model cards
|
||||
crediting the `thorhojhus` account and carrying MIT. The turbo-danish and
|
||||
edda-v0.2 conversions showed no clear gain over stock turbo on `da_festuge`, so
|
||||
they stay unpublished (folders kept locally in `/home/user/hf-publish/`) and
|
||||
their catalog entries were removed. `DANISH_MODELS_HF_OWNER = "lucasw89"`.
|
||||
|
||||
## D-047 — Server engine: batched faster-whisper + tuned Silero VAD for files; live passes stay sequential
|
||||
**Status:** Accepted (2026-10-30, spec 001 Goal 5)
|
||||
`server-py/sidecar.py` decoded every request with the sequential
|
||||
`WhisperModel.transcribe`, no VAD.
|
||||
@@ -1529,7 +1537,7 @@ long. Segments arrive in bursts of 8 chunks (≈ 4 min of speech) and the first
|
||||
burst can take ~15–25 s on a long file; the D-021 heartbeat covers it, and
|
||||
cancel lands at the next burst boundary. No new env vars.
|
||||
|
||||
## D-WS2a — "Names and terms" vocabulary hint as a Whisper decoder prompt, cut to 120 tokens
|
||||
## D-048 — "Names and terms" vocabulary hint as a Whisper decoder prompt, cut to 120 tokens
|
||||
**Status:** Accepted (2026-10-29) — spec 001, Goal 2. Placeholder id; the
|
||||
integrator renumbers it.
|
||||
**Context:** users want names and jargon spelled their way. Whisper takes a
|
||||
@@ -1613,14 +1621,14 @@ test; revisit if a larger list is wanted on turbo).
|
||||
long hint. WebGPU behaviour of the prompt path is unmeasured here (CPU
|
||||
only) — needs the owner's browser check.
|
||||
|
||||
**Integration note (2026-10-30, merge with D-WS5a):** faster-whisper's
|
||||
**Integration note (2026-10-30, merge with D-047):** faster-whisper's
|
||||
`BatchedInferencePipeline` (server file mode) re-encodes `initial_prompt`
|
||||
itself and accepts only text, so the sidecar passes it the budget-cut ids
|
||||
decoded back to text (re-encoding gives the same ids, checked with the real
|
||||
tokenizer incl. "Århus, Ærø"); server-live passes keep the ids. Pinned by the
|
||||
stubbed-sidecar test "the vocabulary hint reaches the batched pipeline as text".
|
||||
|
||||
## D-WS3a — Long-form file pipeline (Silero VAD chunks, language pinned once) ships as an Advanced option, off by default
|
||||
## D-049 — Long-form file pipeline (Silero VAD chunks, language pinned once) ships as an Advanced option, off by default
|
||||
**Status:** Accepted (2026-10-30) — spec 001 Goal 3
|
||||
Browser file transcription cuts audio into fixed 30 s windows with 5 s
|
||||
overlap (Transformers.js's chunked pipeline), decoding silence and
|
||||
@@ -1678,14 +1686,14 @@ bars. Whisper's `generate()` drops kwargs not in `forward_params`, so
|
||||
instance; a Transformers.js upgrade must keep the "one encoder pass per
|
||||
chunk" assertion green. WebGPU timings are unmeasured.
|
||||
|
||||
**Integration note (2026-10-30, merge with D-WS2a):** the long-form adapter
|
||||
**Integration note (2026-10-30, merge with D-048):** the long-form adapter
|
||||
(`worker/longform/transformers.ts`) applies the names-and-terms hint to every
|
||||
chunk's decode with the same decoder-prompt prefix as the chunked path
|
||||
(tokenized once per pass); before the merge the `prompt` option reached the
|
||||
adapter but was ignored. Pinned by a unit test in
|
||||
`tests/longform-transformers.test.ts`.
|
||||
|
||||
## D-WS3b — Silero VAD model: onnx-community/silero-vad from the Hub, cached like Whisper weights, never fetched offline
|
||||
## D-050 — Silero VAD model: onnx-community/silero-vad from the Hub, cached like Whisper weights, never fetched offline
|
||||
**Status:** Accepted (2026-10-30) — spec 001 Goal 3
|
||||
**Decision:** the long-form pipeline's VAD is Silero VAD v5 as exported by
|
||||
`onnx-community/silero-vad` (`onnx/model.onnx`, fp32, 2 243 022 bytes, MIT;
|
||||
@@ -1704,7 +1712,7 @@ path (`choosePipeline`). Live mode keeps its energy gate (D-016).
|
||||
off-by-default option); running it on the main thread (D-013).
|
||||
**Consequence:** the first "Skip silence" run online downloads 2.2 MB.
|
||||
|
||||
## D-WS7a — Parakeet TDT 0.6B v3 as an English-only, desktop-only browser model via parakeet.js on the shared onnxruntime-web
|
||||
## D-051 — Parakeet TDT 0.6B v3 as an English-only, desktop-only browser model via parakeet.js on the shared onnxruntime-web
|
||||
**Status:** Accepted (2026-10-29, spec 001 Goal 7)
|
||||
**Runtime decision (evidence).** `@huggingface/transformers` 4.2.0 (ours) and
|
||||
4.3.2 (npm `latest`) were inspected from their published tarballs: both ship
|
||||
|
||||
+16
-16
@@ -66,10 +66,10 @@ Notes (M1):
|
||||
- [x] Handle model switching (dispose old pipeline, load new)
|
||||
- [x] Language / task options for multilingual models (transcribe vs translate)
|
||||
- [x] Whisper translate only on models trained for it (`canTranslate`; hidden for
|
||||
large-v3-turbo and `.en`, stale "translate" resets to transcribe — D-WS4a)
|
||||
large-v3-turbo and `.en`, stale "translate" resets to transcribe — D-044)
|
||||
- [~] Verify quantized variants (q4/q8/fp16) on both backends; record working
|
||||
combinations in `models.ts`
|
||||
- [~] Danish fine-tunes (spec 001 Goal 6, D-WS6a): `whisper-small-danish`,
|
||||
- [~] Danish fine-tunes (spec 001 Goal 6, D-046): `whisper-small-danish`,
|
||||
`whisper-large-v3-turbo-danish`, `edda-v0.2`. We converted them to ONNX
|
||||
(fp32/fp16/q8/q4) and CT2 int8 and verified them on CPU. The catalog
|
||||
entries hang off `DANISH_MODELS_HF_OWNER`. Still pending: the owner's go
|
||||
@@ -154,7 +154,7 @@ Notes (M4):
|
||||
Bun (D-013 supersedes the Transformers.js-under-Bun plan); map the shared
|
||||
model catalog to faster-whisper model ids in `src/lib/models.ts`
|
||||
- [x] Batched inference + tuned Silero VAD for server-engine file
|
||||
transcriptions, live passes sequential (spec 001 Goal 5, D-WS5a):
|
||||
transcriptions, live passes sequential (spec 001 Goal 5, D-047):
|
||||
≈ 2× faster on CPU int8 at equal-or-better WER; segments ≤ 30 s chunks
|
||||
- [x] Degrade gracefully when Python/the sidecar is absent (server engine
|
||||
unavailable; browser engine untouched — anonymous path deploys with Bun alone)
|
||||
@@ -352,10 +352,10 @@ cached.
|
||||
|
||||
## M8 — Transcription quality and speed (spec 001)
|
||||
- [x] Transformers.js 4.2.0 → 4.3.2, sequential-session patch regenerated
|
||||
for 4.3.2 and guarded by `tests/transformers-runtime.test.ts` (D-WS1a).
|
||||
for 4.3.2 and guarded by `tests/transformers-runtime.test.ts` (D-040).
|
||||
- [x] WASM sessions back at the default graph optimization level: ort-web
|
||||
1.31.0-dev loads the q8 decoders at every level (D-WS1c supersedes D-014).
|
||||
- [x] Vocabulary hint "Names and terms" (D-WS2a) — optional field for every
|
||||
1.31.0-dev loads the q8 decoders at every level (D-043 supersedes D-014).
|
||||
- [x] Vocabulary hint "Names and terms" (D-048) — optional field for every
|
||||
model, remembered in localStorage, sent as `prompt` on file and live
|
||||
passes (browser and server engines); cut to 120 tokens at a whole word.
|
||||
faster-whisper gets `initial_prompt` only (hotwords measured, no gain).
|
||||
@@ -364,15 +364,15 @@ cached.
|
||||
|
||||
## Spec 001 — Translate (workstream 4)
|
||||
- [x] `canTranslate` per model; Task control hidden where false; stale
|
||||
"translate" resets to transcribe (D-WS4a)
|
||||
"translate" resets to transcribe (D-044)
|
||||
- [x] Text translation of any finished transcript, da→en / en→da, Opus-MT
|
||||
in its own worker, sentence-chunked, timestamps kept for exports (D-WS4b)
|
||||
in its own worker, sentence-chunked, timestamps kept for exports (D-045)
|
||||
- [~] Real-browser checks: offline translation (`tests/offline.md` §5),
|
||||
WebGPU-machine speed (owner)
|
||||
|
||||
---
|
||||
|
||||
## Spec 001 Goal 7 — Parakeet TDT 0.6B v3 (English, desktop, D-WS7a)
|
||||
## Spec 001 Goal 7 — Parakeet TDT 0.6B v3 (English, desktop, D-051)
|
||||
- [x] Runtime decision: Transformers.js 4.2.0/4.3.2 have CTC-only Parakeet → parakeet.js on the
|
||||
shared onnxruntime-web (one runtime in the bundle, `tests/workerBundle.test.ts`).
|
||||
- [x] Catalog `family` + exact sizes; worker engine path (≤ 30 s windows, word → chunk
|
||||
@@ -419,27 +419,27 @@ Found while bootstrapping agent docs (D-036); all fixed on 2026-10-29.
|
||||
— Light theme via tokens + self-hosted Inter (D-037). Also fixed: the
|
||||
Backend select overflowed its panel on 390 px-wide phones.
|
||||
- [ ] An English "Names and terms" hint can pull a short Danish live window
|
||||
toward English words (one 2 s da_festuge snippet on turbo, D-WS2a).
|
||||
toward English words (one 2 s da_festuge snippet on turbo, D-048).
|
||||
Not addressed: suggest same-language hints if it shows up in use.
|
||||
|
||||
Found while building transcript translation (D-WS4b), 2026-10-29:
|
||||
Found while building transcript translation (D-045), 2026-10-29:
|
||||
- [ ] Transformers.js 4.2 batched Marian generation returns row 0's output for
|
||||
every row of a padded batch, so translation runs one sentence per call
|
||||
(~4× slower than batch 8 would be). Re-check after the 4.3 upgrade with
|
||||
`bun scripts/bench-translate.ts` ("output identical" across batch sizes).
|
||||
- [ ] The server API still accepts `task: "translate"` with large-v3-turbo
|
||||
(only the UI hides it, D-WS4a).
|
||||
(only the UI hides it, D-044).
|
||||
|
||||
Found in spec 001 workstream 6 (D-WS6a), open:
|
||||
Found in spec 001 workstream 6 (D-046), open:
|
||||
- [ ] Transformers.js 4.2 has no beam search: `num_beams > 1` in a model's
|
||||
`generation_config.json` (edda-v0.2 ships 5) silently decodes greedily
|
||||
and does a full-vocabulary top-k per step (D-WS6a). Not fixed: it would
|
||||
and does a full-vocabulary top-k per step (D-046). Not fixed: it would
|
||||
mean overriding `num_beams: 1` in the worker, or a patch.
|
||||
- [ ] On `da_festuge`, neither Danish turbo fine-tune beats the stock turbo
|
||||
(D-WS6a). Judging them needs a longer, harder Danish reference clip than
|
||||
(D-046). Judging them needs a longer, harder Danish reference clip than
|
||||
this repo has.
|
||||
|
||||
Found while building the long-form pipeline (spec 001 Goal 3, D-WS3a), 2026-10-29:
|
||||
Found while building the long-form pipeline (spec 001 Goal 3, D-049), 2026-10-29:
|
||||
- [ ] "Auto-detect" on the default (chunked) file path does not detect: with no
|
||||
`language`, Transformers.js 4.2 logs "No language specified - defaulting to
|
||||
English" and decodes Danish audio as an English translation (da_festuge WER
|
||||
|
||||
@@ -53,7 +53,7 @@ The integrator merges in this order, rebasing/fixing up each onto the result of
|
||||
- `AGENTS.md` rules apply in full (Bun only, COEP/no CDN, colour tokens, DI on server modules,
|
||||
`classifyError`, runtime-env config, TDD, docs duties).
|
||||
- **Decision ids:** don't take real `D-###` numbers. Write new entries as `## D-WS<n><letter> — …`
|
||||
(e.g. `D-WS3a`) and cite the same placeholder in code comments; the integrator renumbers at merge.
|
||||
(e.g. `D-049`) and cite the same placeholder in code comments; the integrator renumbers at merge.
|
||||
- **Heavy jobs take turns.** This machine has 16 cores / 18 GB RAM shared by all workstreams. Wrap
|
||||
every heavy command — real-model tests, the full `bun test`, benchmarks, model downloads,
|
||||
conversions, `pip install` — in `flock /tmp/whisper-heavy.lock <command>`. Light commands
|
||||
|
||||
Reference in new issue
Block a user