Remove the workstream 6 plan now that the Danish models work is done
Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Lucas Winther
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
19228297f1
commit
498e361b8a
@@ -1,115 +0,0 @@
|
||||
# 001-6 Danish fine-tuned Whisper models Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans (inline, no subagents) to
|
||||
> implement this plan task-by-task. Steps use checkbox (`- [x]`) syntax for tracking.
|
||||
|
||||
**Goal:** convert the Danish Whisper fine-tunes to the onnx-community (Transformers.js) and CTranslate2
|
||||
int8 layouts, verify them locally, prepare (not upload) one publication folder per would-be Hugging Face
|
||||
repo, and add `verified: false` catalog entries whose ids hang off one owner constant.
|
||||
|
||||
**Architecture:** all conversion work happens outside the repo (`/home/user/venvs/convert`, sources in
|
||||
`/home/user/hf-src/<repo>`, outputs in `/home/user/hf-publish/<repo-name>/`, helper scripts in
|
||||
`/home/user/hf-tools/`). The repo only gains catalog data in `src/lib/models.ts`, its tests, and docs.
|
||||
|
||||
**Tech Stack:** Python 3.13 venv (optimum[onnxruntime], transformers, onnx, onnxruntime, ctranslate2),
|
||||
faster-whisper venv for CT2 checks, Bun + Transformers.js 4.2.0 (onnxruntime-node) for WER.
|
||||
|
||||
**Spec:** `docs/specs/001-transcription-quality-and-speed.md` Goal 6; `docs/plans/001-0-coordination.md`.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- `flock /tmp/whisper-heavy.lock` around every download, `pip install`, conversion, WER/benchmark run.
|
||||
- Venvs only in `/home/user/venvs/<name>`. Nothing leaves the machine: no HF upload, no repo creation, no push.
|
||||
- Decision placeholder `D-WS6a` (not a real `D-###`); cite it in code comments.
|
||||
- Do **not** add `canTranslate` (integrator sets it to `false` for the new entries at merge).
|
||||
- New entries `verified: false` on every device; ids `` `${DANISH_MODELS_HF_OWNER}/<repo>` ``.
|
||||
- "Show the owner first": the final report lists repos, files, sizes, cards and WER; uploading waits for go.
|
||||
- Measurements are CPU (onnxruntime-node / CTranslate2) numbers on one 21.5 s clip; say so.
|
||||
|
||||
## Review Focus
|
||||
|
||||
1. **Owner constant unchanged at merge** — ids must still resolve: catalog tests assert ids are built from
|
||||
`DANISH_MODELS_HF_OWNER`, so changing that one line moves every id + `fasterWhisperModel` together.
|
||||
2. **Language tokens lost in export** — `generation_config.json` must keep `lang_to_id["<|da|>"]` and
|
||||
`task_to_id`; checked per output folder in Task 3 (script asserts).
|
||||
3. **fp16 encoder overflow on large models (D-022)** — turbo fine-tunes' fp16 encoders are WER-checked on
|
||||
onnxruntime-node; if garbage, the catalog's WebGPU/WebNN defaults avoid fp16 encoder (Task 5).
|
||||
4. **edda's `num_beams: 5` generation config** — Transformers.js 4.2 must not crash or silently
|
||||
misbehave; WER run in Task 4 uses the shipped `generation_config.json` unmodified.
|
||||
5. **Server engine with a Hub-id model name** — `fasterWhisperModel` is a repo id, not an alias; existing
|
||||
test that derives aliases from ids must exempt the Danish entries explicitly, not be loosened globally.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Licence + architecture check (done during planning; record it)
|
||||
|
||||
- [x] `thorhojhus/whisper-small-danish`: MIT (card + THIRD_PARTY_NOTICES), `WhisperForConditionalGeneration`,
|
||||
unchanged whisper-small architecture/tokenizer (80 mel, 12/12 layers, vocab 51865).
|
||||
- [x] `thorhojhus/whisper-large-v3-turbo-danish`: MIT, plain large-v3-turbo (128 mel, 32/4 layers, vocab 51866).
|
||||
- [x] `danish-foundation-models/edda-v0.2`: Apache-2.0 (licensor: Alexandra Institute), plain
|
||||
large-v3-turbo; generation config pins `language: da`, `num_beams: 5`. Kept.
|
||||
|
||||
### Task 2: Conversion venv + export/quantize tooling
|
||||
|
||||
**Files:** Create `/home/user/hf-tools/export_onnx.py`, `/home/user/hf-tools/quantize.py`,
|
||||
`/home/user/hf-tools/check_layout.py` (outside the repo).
|
||||
|
||||
- [x] `python3 -m venv /home/user/venvs/convert`; `flock … pip install "optimum[onnxruntime]" transformers onnx onnxruntime ctranslate2 onnxconverter-common`.
|
||||
- [x] Download each source repo snapshot under the lock to `/home/user/hf-src/<repo>` (`huggingface_hub.snapshot_download`, read-only).
|
||||
- [x] `export_onnx.py <src> <out>`: optimum `main_export(task="automatic-speech-recognition-with-past", dtype fp32)`
|
||||
→ keep only `encoder_model.onnx(+_data)` and `decoder_model_merged.onnx(+_data)` under `<out>/onnx/`;
|
||||
copy config/tokenizer/generation/preprocessor/normalizer/added_tokens/special_tokens/vocab/merges.
|
||||
- [x] `quantize.py <out>`: for `encoder_model`, `decoder_model_merged` write `_fp16` (onnxruntime
|
||||
`float16.convert_float_to_float16`, keep_io_types), `_quantized` (q8: `quantize_dynamic`, QUInt8,
|
||||
op_types from stock `onnx-community/whisper-small/quantize_config.json`), `_q4`
|
||||
(`MatMul4BitsQuantizer`, block 32, symmetric). Encoder > 2 GB → external data + `transformers.js_config.use_external_data_format` in config.json, as the stock turbo repo.
|
||||
- [x] `check_layout.py <out> <stock-file-list>`: asserts every file the stock repo has for those dtypes
|
||||
exists, `<|da|>` id in `generation_config.json` equals the source, and onnxruntime can create a session
|
||||
for each `.onnx` (ORT_ENABLE_BASIC, D-014).
|
||||
|
||||
### Task 3: Convert all three (ONNX + CT2)
|
||||
|
||||
- [x] For each kept model: `flock … export_onnx.py`, `flock … quantize.py`, `check_layout.py` → pass.
|
||||
- [x] `flock … ct2-transformers-converter --model <src> --output_dir /home/user/hf-publish/<name>-ct2-int8 --quantization int8 --copy_files tokenizer.json preprocessor_config.json` (+ generation config / normalizer if present).
|
||||
|
||||
### Task 4: Verify (WER)
|
||||
|
||||
**Files:** Create `/home/user/hf-tools/wer.ts` (Bun; imports the worktree's `tests/dsp.ts`
|
||||
`wordErrorRate`; same normalisation and worker options as `tests/live-real-model.test.ts`:
|
||||
`chunk_length_s 30, stride 5, return_timestamps, max_new_tokens 440, language "da"`).
|
||||
Create `/home/user/hf-tools/ct2_wer.py`.
|
||||
|
||||
- [x] `wer.ts --local <dir> --dtype q8|q4` with `env.localModelPath`, `env.allowRemoteModels = false`;
|
||||
run under the lock for every conversion at q8 and q4, plus fp16-encoder+q4-decoder for the WebGPU default.
|
||||
- [x] Same for stock `onnx-community/whisper-small` (q8, q4) and `onnx-community/whisper-large-v3-turbo` (q4, q8).
|
||||
- [x] `ct2_wer.py <ct2 dir>` with faster-whisper (language da, beam 5, CPU int8) vs stock `small` / `large-v3-turbo`.
|
||||
- [x] Record wall-clock + WER table (goes into D-WS6a and the model cards).
|
||||
|
||||
### Task 5: Catalog entries (TDD)
|
||||
|
||||
**Files:** Modify `src/lib/models.ts`, `tests/models.test.ts`.
|
||||
|
||||
**Interfaces — Produces:** `export const DANISH_MODELS_HF_OWNER = "PENDING-OWNER"`; entries with
|
||||
ids `` `${DANISH_MODELS_HF_OWNER}/whisper-small-danish-ONNX` `` etc. (exact names per Task 6),
|
||||
`fasterWhisperModel` = `` `${DANISH_MODELS_HF_OWNER}/<name>-ct2-int8` ``, `multilingual: true`.
|
||||
|
||||
- [x] Failing tests: lineup includes the Danish ids (built from the constant); every Danish entry's id and
|
||||
`fasterWhisperModel` start with `${DANISH_MODELS_HF_OWNER}/`; none `verified`; `getFasterWhisperModel`
|
||||
returns the CT2 repo; dtype defaults equal the measured choice; the existing alias test skips only ids
|
||||
under the owner constant.
|
||||
- [x] Run → fail. Implement entries. Run → pass; typecheck; commit.
|
||||
|
||||
### Task 6: Publication folders + model cards
|
||||
|
||||
- [x] Proposed names: `whisper-small-danish-ONNX`, `whisper-small-danish-ct2-int8`,
|
||||
`whisper-large-v3-turbo-danish-ONNX`, `whisper-large-v3-turbo-danish-ct2-int8`, `edda-v0.2-ONNX`,
|
||||
`edda-v0.2-ct2-int8` under `<HF_OWNER>`.
|
||||
- [x] Each folder gets README.md (front matter: licence carried, `base_model`, `library_name`; credit to the
|
||||
original authors, conversion commands, measured WER, intended use with whisper_web), LICENSE/
|
||||
THIRD_PARTY_NOTICES from the source, and a `SIZES.txt` (not uploaded — listed for the owner).
|
||||
|
||||
### Task 7: Docs + finish
|
||||
|
||||
- [x] `docs/Decisions.md`: append `## D-WS6a — …` (licences, layout, measurements, dtype choice, pending owner go).
|
||||
- [x] `docs/WorkBreakdown.md`: add/tick the item; "Noticed, not fixed" for anything left.
|
||||
- [x] `bun run typecheck`, fast suite, `bun run build` green; `git rm` this plan as the last commit.
|
||||
Reference in New Issue
Block a user