From 498e361b8a86fe1b0ea9a5eed6a5da9472a03399 Mon Sep 17 00:00:00 2001 From: Lucas Winther Date: Fri, 30 Oct 2026 01:13:27 +0100 Subject: [PATCH] Remove the workstream 6 plan now that the Danish models work is done Co-Authored-By: Claude Opus 5.5 --- docs/plans/001-6-danish-models.md | 115 ------------------------------ 1 file changed, 115 deletions(-) delete mode 100644 docs/plans/001-6-danish-models.md diff --git a/docs/plans/001-6-danish-models.md b/docs/plans/001-6-danish-models.md deleted file mode 100644 index 9844a69..0000000 --- a/docs/plans/001-6-danish-models.md +++ /dev/null @@ -1,115 +0,0 @@ -# 001-6 Danish fine-tuned Whisper models Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans (inline, no subagents) to -> implement this plan task-by-task. Steps use checkbox (`- [x]`) syntax for tracking. - -**Goal:** convert the Danish Whisper fine-tunes to the onnx-community (Transformers.js) and CTranslate2 -int8 layouts, verify them locally, prepare (not upload) one publication folder per would-be Hugging Face -repo, and add `verified: false` catalog entries whose ids hang off one owner constant. - -**Architecture:** all conversion work happens outside the repo (`/home/user/venvs/convert`, sources in -`/home/user/hf-src/`, outputs in `/home/user/hf-publish//`, helper scripts in -`/home/user/hf-tools/`). The repo only gains catalog data in `src/lib/models.ts`, its tests, and docs. - -**Tech Stack:** Python 3.13 venv (optimum[onnxruntime], transformers, onnx, onnxruntime, ctranslate2), -faster-whisper venv for CT2 checks, Bun + Transformers.js 4.2.0 (onnxruntime-node) for WER. - -**Spec:** `docs/specs/001-transcription-quality-and-speed.md` Goal 6; `docs/plans/001-0-coordination.md`. - -## Global Constraints - -- `flock /tmp/whisper-heavy.lock` around every download, `pip install`, conversion, WER/benchmark run. -- Venvs only in `/home/user/venvs/`. Nothing leaves the machine: no HF upload, no repo creation, no push. -- Decision placeholder `D-WS6a` (not a real `D-###`); cite it in code comments. -- Do **not** add `canTranslate` (integrator sets it to `false` for the new entries at merge). -- New entries `verified: false` on every device; ids `` `${DANISH_MODELS_HF_OWNER}/` ``. -- "Show the owner first": the final report lists repos, files, sizes, cards and WER; uploading waits for go. -- Measurements are CPU (onnxruntime-node / CTranslate2) numbers on one 21.5 s clip; say so. - -## Review Focus - -1. **Owner constant unchanged at merge** — ids must still resolve: catalog tests assert ids are built from - `DANISH_MODELS_HF_OWNER`, so changing that one line moves every id + `fasterWhisperModel` together. -2. **Language tokens lost in export** — `generation_config.json` must keep `lang_to_id["<|da|>"]` and - `task_to_id`; checked per output folder in Task 3 (script asserts). -3. **fp16 encoder overflow on large models (D-022)** — turbo fine-tunes' fp16 encoders are WER-checked on - onnxruntime-node; if garbage, the catalog's WebGPU/WebNN defaults avoid fp16 encoder (Task 5). -4. **edda's `num_beams: 5` generation config** — Transformers.js 4.2 must not crash or silently - misbehave; WER run in Task 4 uses the shipped `generation_config.json` unmodified. -5. **Server engine with a Hub-id model name** — `fasterWhisperModel` is a repo id, not an alias; existing - test that derives aliases from ids must exempt the Danish entries explicitly, not be loosened globally. - ---- - -### Task 1: Licence + architecture check (done during planning; record it) - -- [x] `thorhojhus/whisper-small-danish`: MIT (card + THIRD_PARTY_NOTICES), `WhisperForConditionalGeneration`, - unchanged whisper-small architecture/tokenizer (80 mel, 12/12 layers, vocab 51865). -- [x] `thorhojhus/whisper-large-v3-turbo-danish`: MIT, plain large-v3-turbo (128 mel, 32/4 layers, vocab 51866). -- [x] `danish-foundation-models/edda-v0.2`: Apache-2.0 (licensor: Alexandra Institute), plain - large-v3-turbo; generation config pins `language: da`, `num_beams: 5`. Kept. - -### Task 2: Conversion venv + export/quantize tooling - -**Files:** Create `/home/user/hf-tools/export_onnx.py`, `/home/user/hf-tools/quantize.py`, -`/home/user/hf-tools/check_layout.py` (outside the repo). - -- [x] `python3 -m venv /home/user/venvs/convert`; `flock … pip install "optimum[onnxruntime]" transformers onnx onnxruntime ctranslate2 onnxconverter-common`. -- [x] Download each source repo snapshot under the lock to `/home/user/hf-src/` (`huggingface_hub.snapshot_download`, read-only). -- [x] `export_onnx.py `: optimum `main_export(task="automatic-speech-recognition-with-past", dtype fp32)` - → keep only `encoder_model.onnx(+_data)` and `decoder_model_merged.onnx(+_data)` under `/onnx/`; - copy config/tokenizer/generation/preprocessor/normalizer/added_tokens/special_tokens/vocab/merges. -- [x] `quantize.py `: for `encoder_model`, `decoder_model_merged` write `_fp16` (onnxruntime - `float16.convert_float_to_float16`, keep_io_types), `_quantized` (q8: `quantize_dynamic`, QUInt8, - op_types from stock `onnx-community/whisper-small/quantize_config.json`), `_q4` - (`MatMul4BitsQuantizer`, block 32, symmetric). Encoder > 2 GB → external data + `transformers.js_config.use_external_data_format` in config.json, as the stock turbo repo. -- [x] `check_layout.py `: asserts every file the stock repo has for those dtypes - exists, `<|da|>` id in `generation_config.json` equals the source, and onnxruntime can create a session - for each `.onnx` (ORT_ENABLE_BASIC, D-014). - -### Task 3: Convert all three (ONNX + CT2) - -- [x] For each kept model: `flock … export_onnx.py`, `flock … quantize.py`, `check_layout.py` → pass. -- [x] `flock … ct2-transformers-converter --model --output_dir /home/user/hf-publish/-ct2-int8 --quantization int8 --copy_files tokenizer.json preprocessor_config.json` (+ generation config / normalizer if present). - -### Task 4: Verify (WER) - -**Files:** Create `/home/user/hf-tools/wer.ts` (Bun; imports the worktree's `tests/dsp.ts` -`wordErrorRate`; same normalisation and worker options as `tests/live-real-model.test.ts`: -`chunk_length_s 30, stride 5, return_timestamps, max_new_tokens 440, language "da"`). -Create `/home/user/hf-tools/ct2_wer.py`. - -- [x] `wer.ts --local --dtype q8|q4` with `env.localModelPath`, `env.allowRemoteModels = false`; - run under the lock for every conversion at q8 and q4, plus fp16-encoder+q4-decoder for the WebGPU default. -- [x] Same for stock `onnx-community/whisper-small` (q8, q4) and `onnx-community/whisper-large-v3-turbo` (q4, q8). -- [x] `ct2_wer.py ` with faster-whisper (language da, beam 5, CPU int8) vs stock `small` / `large-v3-turbo`. -- [x] Record wall-clock + WER table (goes into D-WS6a and the model cards). - -### Task 5: Catalog entries (TDD) - -**Files:** Modify `src/lib/models.ts`, `tests/models.test.ts`. - -**Interfaces — Produces:** `export const DANISH_MODELS_HF_OWNER = "PENDING-OWNER"`; entries with -ids `` `${DANISH_MODELS_HF_OWNER}/whisper-small-danish-ONNX` `` etc. (exact names per Task 6), -`fasterWhisperModel` = `` `${DANISH_MODELS_HF_OWNER}/-ct2-int8` ``, `multilingual: true`. - -- [x] Failing tests: lineup includes the Danish ids (built from the constant); every Danish entry's id and - `fasterWhisperModel` start with `${DANISH_MODELS_HF_OWNER}/`; none `verified`; `getFasterWhisperModel` - returns the CT2 repo; dtype defaults equal the measured choice; the existing alias test skips only ids - under the owner constant. -- [x] Run → fail. Implement entries. Run → pass; typecheck; commit. - -### Task 6: Publication folders + model cards - -- [x] Proposed names: `whisper-small-danish-ONNX`, `whisper-small-danish-ct2-int8`, - `whisper-large-v3-turbo-danish-ONNX`, `whisper-large-v3-turbo-danish-ct2-int8`, `edda-v0.2-ONNX`, - `edda-v0.2-ct2-int8` under ``. -- [x] Each folder gets README.md (front matter: licence carried, `base_model`, `library_name`; credit to the - original authors, conversion commands, measured WER, intended use with whisper_web), LICENSE/ - THIRD_PARTY_NOTICES from the source, and a `SIZES.txt` (not uploaded — listed for the owner). - -### Task 7: Docs + finish - -- [x] `docs/Decisions.md`: append `## D-WS6a — …` (licences, layout, measurements, dtype choice, pending owner go). -- [x] `docs/WorkBreakdown.md`: add/tick the item; "Noticed, not fixed" for anything left. -- [x] `bun run typecheck`, fast suite, `bun run build` green; `git rm` this plan as the last commit.