whisper_web/docs/plans/001-0-coordination.md

status

status
in-progress

001 — Transcription quality and speed: coordination plan

For agentic workers: this file coordinates seven parallel workstreams. Each workstream writes its own task plan (docs/plans/001-<n>-<slug>.md, superpowers:writing-plans format) in its own worktree and executes it with TDD (superpowers:executing-plans, inline — no nested subagents).

Goal: implement spec 001 as seven independently mergeable branches.

Spec: docs/specs/001-transcription-quality-and-speed.md (approved). Read it, AGENTS.md, PRODUCT.md, and the docs/Decisions.md entries your workstream touches before planning.

Workstreams

#	Branch	Plan file	Owns
1	chore/transformers-4.3	001-1-transformers-upgrade.md	package.json, bun.lock, patches/, re-tests of D-014/D-021/D-022/D-026
2	feat/vocabulary-hint	001-2-vocabulary-hint.md	hint UI + storage, prompt through protocol/worker/live/server
3	feat/longform-pipeline	001-3-longform-pipeline.md	src/worker/longform/ (VAD, guard, fallback, language pinning), progress tracker
4	feat/translate	001-4-translate.md	canTranslate, translate UI, Opus-MT text translation
5	feat/server-batched-vad	001-5-server-batched-vad.md	server-py/sidecar.py
6	feat/danish-models	001-6-danish-models.md	conversions (outside repo), catalog entries
7	feat/parakeet-engine	001-7-parakeet.md	Parakeet engine path + catalog entry

All branch from docs/transcription-improvements-spec (spec commit) on Transformers.js 4.2.0. The integrator merges in this order, rebasing/fixing up each onto the result of the previous: 1 → 4 → 6 → 5 → 2 → 3 → 7.

Shared interfaces (fixed here so parallel branches agree)

• src/lib/models.ts — WhisperModel gains (by workstream 4) canTranslate: boolean. Workstream 6 does not add it (it doesn't exist on its base); the integrator sets it to false for the new Danish entries at merge.
• src/worker/protocol.ts — the transcribe message gains (workstream 2) prompt?: string (the raw user hint; the worker tokenizes and truncates it). Live and server paths carry the same field name.
• src/worker/longform/ (workstream 3) exposes transcribeLongForm(audio: Float32Array, deps: LongFormDeps, options: LongFormOptions): Promise<LongFormResult> where LongFormOptions includes language?: string, task?: WhisperTask, and prompt?: string (accepted and passed to every decode even before workstream 2 lands — a no-op when undefined).
• Synthetic meeting stand-in (workstream 3 owns it): a test helper that builds it at test time from the existing clips — tests/meetingFixture.ts exporting buildMeetingFixture(): Promise<{ audio: Float32Array; reference: string }> (16 kHz mono; clips joined with 2–8 s silences, en_alien excerpts included). Other workstreams may import it after merge; before that they measure on the plain clips.
• Parakeet (workstream 7) selects its engine from a catalog field it adds: family?: "whisper" | "parakeet" (absent ⇒ "whisper"), so existing entries need no edit.

Global constraints (every workstream)

• AGENTS.md rules apply in full (Bun only, COEP/no CDN, colour tokens, DI on server modules, classifyError, runtime-env config, TDD, docs duties).
• Decision ids: don't take real D-### numbers. Write new entries as ## D-WS<n><letter> — … (e.g. D-WS3a) and cite the same placeholder in code comments; the integrator renumbers at merge.
• Heavy jobs take turns. This machine has 16 cores / 18 GB RAM shared by all workstreams. Wrap every heavy command — real-model tests, the full bun test, benchmarks, model downloads, conversions, pip install — in flock /tmp/whisper-heavy.lock <command>. Light commands (typecheck, the fast suite, single unit tests, build) run without the lock.
• Shared model cache. After bun install --frozen-lockfile in your worktree, replace node_modules/@huggingface/transformers/.cache with a symlink to /home/user/whisper_web/node_modules/@huggingface/transformers/.cache (727 MB of turbo weights already there). Never delete anything in it.
• Ports: dev/prod servers use PORT=31<n>0–31<n>9 for workstream <n>.
• Python: virtualenvs live in /home/user/venvs/<name>, never in the repo.
• Nothing leaves this machine: no git push, no Hugging Face uploads, no publishing.
• Measurements (wall-clock, WER vs tests/audio_files/*.txt, generated-token counts) are CPU/WASM or onnxruntime-node numbers; say so. Anything synthetic is labelled synthetic. WebGPU-dependent effects go in the final report as "needs owner's browser check".
• Done per workstream: bun run typecheck, fast suite, bun run build green; affected real-model tests green (under the lock); docs duties done; plan file removed with git rm as the last commit; branch left committed in its worktree, not merged.

Review Focus (inputs the spec implies but no single test obviously covers)

1. A 90-minute file (the owner's real case): no full-length copies of the audio per stage, no per-window state that grows unbounded. Owner: 3 (pipeline), 2 (prompt repeated per window).
2. A very long hint (more than Whisper's ~224-token prompt budget): truncated predictably, never an error or a garbled first window. Owner: 2.
3. Offline / VAD model never downloaded: file transcription still works on the old path, with no network request. Owner: 3.
4. Silence-only or music-only audio: empty transcript with a clear message, no hang, no hallucinated text. Owner: 3.
5. Translating a long transcript (beyond Opus-MT's 512-token input): split on sentence boundaries, order preserved, nothing dropped. Owner: 4.

Integration (integrator only)

• Merge 1, 4, 6, 5, 2, 3, 7 in order; renumber D-WS* placeholders to the next free D-###.
• After each merge: typecheck, fast suite, build; after the last: full bun test under the lock.
• Workstream 6: present the would-be Hugging Face publication to the owner; publish only on go.
• Final report to the owner: decisions made, measurements, items needing a browser check.