Merge feat/danish-models: Danish fine-tune catalog entries (spec 001 workstream 6)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> # Conflicts: # docs/Decisions.md # docs/WorkBreakdown.md
This commit is contained in:
commit
439d099279
4 files changed
+202
-1
No files matched your search
@@ -1431,3 +1431,55 @@ tabs instead of a smaller Whisper model; a failed download is a retryable networ
|
||||
terminates the translate worker only.
|
||||
**Not done / follow-ups:** WebGPU for Opus-MT (unmeasurable here); more language pairs (the catalog
|
||||
is two entries, `TRANSLATION_MODELS`); a translation of live text *while* recording.
|
||||
|
||||
## D-WS6a — Danish fine-tuned Whisper models: our own ONNX + CTranslate2 conversions, unpublished until the owner says go
|
||||
**Status:** Proposed (2026-10-29, spec 001 Goal 6)
|
||||
**Candidates checked (licence + architecture, from each repo's model card, config and notices):**
|
||||
- `thorhojhus/whisper-small-danish` — MIT; plain `WhisperForConditionalGeneration`, whisper-small
|
||||
shapes and tokenizer unchanged (80 mel bins, 12/12 layers, vocab 51865). **Kept.**
|
||||
- `thorhojhus/whisper-large-v3-turbo-danish` — MIT; plain large-v3-turbo (128 mel, 32/4 layers,
|
||||
vocab 51866). **Kept.**
|
||||
- `danish-foundation-models/edda-v0.2` — **Apache-2.0 confirmed** (licensor: the Alexandra
|
||||
Institute; base model MIT); plain large-v3-turbo architecture, so Transformers.js loads it like the
|
||||
stock turbo. Its `generation_config.json` pins `language: "da"`, `task: "transcribe"` and
|
||||
`num_beams: 5`. **Kept.**
|
||||
**Catalog:** three entries with `verified: false` everywhere, ids built from one exported constant
|
||||
`DANISH_MODELS_HF_OWNER` (placeholder `"PENDING-OWNER"`): `<owner>/<name>-ONNX` for the browser
|
||||
engine and `<owner>/<name>-ct2-int8` as `fasterWhisperModel` (a Hub repo id — faster-whisper accepts
|
||||
those as well as its aliases). Publishing under the owner's account is a one-line change.
|
||||
**Conversions (outside the repo, in `/home/user/hf-publish/<repo>/`; nothing uploaded):** optimum 2.1
|
||||
`main_export` (fp32, KV-cached merged decoder) → onnxruntime fp16 / dynamic q8 / MatMulNBits q4 with
|
||||
the stock onnx-community settings; `ct2-transformers-converter --quantization int8`. File names, sizes
|
||||
and `transformers.js_config.use_external_data_format` match the stock `onnx-community/whisper-small`
|
||||
and `-large-v3-turbo` repos to within 0.2 MB per file, with one exception: our q8 turbo decoder
|
||||
(173 MB) quantizes the token embedding, while the stock one (440 MB) keeps it in fp32. Two export
|
||||
traps, now fixed in the tooling and described in each model card: optimum needs `accelerate`
|
||||
installed, or the tied `embed_tokens`/`proj_out` weight is stored twice (+159 MB on small); and
|
||||
onnxruntime's fp16 converter makes the merged decoder's If branches cyclic unless their outputs get
|
||||
branch-local names first. `generation_config.json` (`<|da|>` = 50285, `task_to_id`) and the tokenizer
|
||||
are copied unchanged and asserted equal to the source.
|
||||
**Measured** (WER on `da_festuge` with `language: "da"`; ONNX loaded from the local folder through
|
||||
Transformers.js 4.2.0 on onnxruntime-node, CT2 through faster-whisper 1.2.1 int8 beam 5; CPU only, one
|
||||
21.5 s clip, so the results are a conversion sanity check, not a benchmark):
|
||||
|
||||
| model | q8 | q4 | fp16 enc + q4 dec | fp16 | CT2 int8 |
|
||||
|---|---|---|---|---|---|
|
||||
| whisper-small-danish | 0.109 | 0.094 | 0.094 | 0.094 | 0.078 |
|
||||
| stock whisper-small | 0.297 | 0.281 | — | — | 0.234 |
|
||||
| whisper-large-v3-turbo-danish | 0.141 | 0.141 | 0.094 | 0.094 | 0.094 |
|
||||
| edda-v0.2 | 0.109 | 0.063 | 0.094 | 0.094 | 0.078 |
|
||||
| stock whisper-large-v3-turbo | 0.125 | 0.141 | — | — | 0.094 |
|
||||
|
||||
The small fine-tune cuts WER by about a factor of three. Neither turbo fine-tune separates from the
|
||||
stock turbo on this easy clip, so a longer, harder Danish test is needed to judge them. **D-022
|
||||
check:** the turbo fp16 encoders do not overflow on onnxruntime-node; whole-model fp16 gives the
|
||||
same transcript as fp16 + q4. WebGPU/WebNN behaviour still needs a real browser.
|
||||
**Dtype defaults:** WASM q4 for all three, including small (q4 matched or beat q8 and was 1.4–1.6×
|
||||
faster on CPU); WebGPU fp16 encoder + q4 decoder; WebNN fp16. All stay `verified: false`.
|
||||
**edda's `num_beams: 5`:** Transformers.js 4.2 has no beam search (its `BeamSearchSampler` returns
|
||||
`num_beams` candidates but generation keeps only the top-1), so browser decoding is greedy. It does
|
||||
not crash; the only cost is a full-vocabulary top-k per step. The server engine (faster-whisper) does
|
||||
use beam 5.
|
||||
**Pending the owner:** the Hugging Face account name, then upload the six prepared folders (model
|
||||
cards credit the original authors and carry MIT, or Apache-2.0 + NOTICE), then set
|
||||
`DANISH_MODELS_HF_OWNER`. The integrator sets `canTranslate: false` on these entries at merge.
|
||||
Reference in new issue
Block a user