From 7fb8dde423272c7a5b3afc85dbd36c9cb622fd43 Mon Sep 17 00:00:00 2001 From: Lucas Winther Date: Fri, 30 Oct 2026 00:24:06 +0100 Subject: [PATCH] Remove the finished server batched-VAD plan; spec 001 and D-WS5a are the record Co-Authored-By: Claude Opus 5.5 --- docs/plans/001-5-server-batched-vad.md | 102 ------------------------- 1 file changed, 102 deletions(-) delete mode 100644 docs/plans/001-5-server-batched-vad.md diff --git a/docs/plans/001-5-server-batched-vad.md b/docs/plans/001-5-server-batched-vad.md deleted file mode 100644 index bf92712..0000000 --- a/docs/plans/001-5-server-batched-vad.md +++ /dev/null @@ -1,102 +0,0 @@ -# Server engine: batched + VAD faster-whisper Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan -> task-by-task (inline, no subagents — coordination plan). Steps use checkbox (`- [ ]`) syntax. - -**Goal:** file transcriptions on the server engine run through faster-whisper's -`BatchedInferencePipeline` with Silero VAD, without breaking segment streaming, cancel, the D-021 -heartbeat, or the one-loaded-model cache. - -**Architecture:** the sidecar request gains an optional `mode: "file" | "live"` (absent ⇒ `"file"`). -`"file"` → `BatchedInferencePipeline(model).transcribe(audio, vad_filter=True, batch_size=8)`; -`"live"` → the existing sequential `WhisperModel.transcribe` (short ≤ 29 s passes gain nothing from -batching; decided from measurements in Task 1). The pipeline wrapper is cached with the model. -Bun side: `SidecarTranscribeRequest.mode`, `createSidecar` forwards it, `live.ts` sends `"live"`. - -**Tech Stack:** Python 3 + faster-whisper 1.2.1 (venv `/home/user/venvs/faster-whisper`), Bun test. - -**Spec:** `docs/specs/001-transcription-quality-and-speed.md` Goal 5; `docs/plans/001-0-coordination.md`. - -## Global Constraints - -- Only `server-py/sidecar.py` is this workstream's core; Bun edits limited to the protocol field. -- Heavy jobs (pip, benchmarks, `FW_INTEGRATION=1`) under `flock /tmp/whisper-heavy.lock`. -- Decision entry `## D-WS5a — …` at the bottom of `docs/Decisions.md`; cite `D-WS5a` in code. -- `requirements.txt` stays `faster-whisper>=1.1,<2` (BatchedInferencePipeline exists since 1.1.0). -- No new env vars unless measurement demands one (then README env table). -- Measurements: CPU int8, this machine (16 cores, shared), labelled as such. - -## Review Focus - -1. **Cancel during a batch** — batched segments arrive in bursts (one forward per 8 VAD chunks); a - cancel must still end the run with `done {cancelled: true}` and no further segments. Test: stub - sidecar "slow" model, abort after first segment (Task 2). -2. **Long silent stretch before the first segment** (VAD + feature extraction over the whole file - happen eagerly in `transcribe()`, then a full batch decodes) — heartbeat lines must keep flowing. - Test: injectable heartbeat interval, fake sidecar that waits, assert blank lines (Task 3). -3. **Silence-only audio** — VAD finds no speech: must answer `done` with zero segments, not error. - Test: stub "silence" model yields nothing → `done` (Task 2); real run on tone in FW_INTEGRATION. -4. **Model switch** — still one model resident; the batched wrapper must be rebuilt for the new - model, not reused. Test: stub reports load count and which model the pipeline wraps (Task 2). -5. **Old Bun ↔ new sidecar** — request without `mode` must behave as `"file"`. Test (Task 2). - ---- - -### Task 1: Measure (outside the repo) - -**Files:** `/home/user/ws5-bench/bench.py` (not committed). - -- [x] Venv `/home/user/venvs/faster-whisper` with requirements + jiwer (under lock); confirm API. -- [x] large-v3-turbo, both clips: `base` (current sidecar), `vad`, `batched` (vad+bs8), - `batched_b1` (beam 1). Record wall, WER (repo normalize), first-segment time, max gap. - Done 2026-10-29 (`/home/user/ws5-bench/turbo-*.jsonl`); VAD-parameter follow-up in - `diag2.out` / `round3-*.jsonl` (threshold 0.2 + pad 800 ms recovers most deletions). -- [x] Live-style passes (clip sliced into 6 s pieces): `live_seq` vs `live_batched` vs `live_vad`. -- [x] medium.en on en_alien: `base` vs `batched` if time allows. -- [x] Decide: beam size, live mode routing. - -### Task 2: Sidecar `mode` routing, tested against a stubbed faster-whisper - -**Files:** -- Create: `tests/fixtures/stub_fw/faster_whisper/__init__.py`, `tests/fixtures/stub_fw/numpy/__init__.py` -- Modify: `server-py/sidecar.py`, `tests/transcribe.test.ts` - -**Interfaces:** -- Produces: sidecar stdin `transcribe` message accepts `"mode": "file" | "live"` (default `"file"`). -- Stub: `WhisperModel(name, **kw)` counts loads; `.transcribe(audio, **kw)` yields segments whose - text reports `seq` + kwargs; `BatchedInferencePipeline(model).transcribe(audio, **kw)` yields - `batched model= loads=` + kwargs. Model `"slow"` yields 20 segments 0.1 s apart; - `"silence"` yields none. Stub numpy: `float32`, `fromfile(path, dtype)` → bytes. -- Test runs `createSidecar(["python3", "-S", REAL_SIDECAR])` with `PYTHONPATH=stub_fw` (createSidecar - inherits env; set `process.env.PYTHONPATH` around the test or pass `env` — add optional - `env` arg only if needed). - -- [x] Failing tests: default/`file` → batched with `vad_filter=True batch_size=8`; `live` → - sequential, no `vad_filter`; model switch reloads and pipeline wraps the new model, same model - doesn't reload; `slow` + abort → `cancelled: true`, < 20 segments; `silence` → done, 0 segments. -- [x] Implement in `sidecar.py`; update docstring protocol; cite D-WS5a. -- [x] Green; commit. - -### Task 3: Bun side — `mode` field, live sends `"live"`, heartbeat test - -**Files:** `src/server/transcribe.ts`, `src/server/live.ts`, `tests/fixtures/fake_sidecar.py`, -`tests/transcribe.test.ts`, `tests/server-live.test.ts` - -- [x] Failing tests: fake sidecar "ok" echoes `mode=`; `createSidecar` default sends `file`, explicit - `live` forwarded; server-live test asserts the request has `mode: "live"`; HTTP handler passes - no mode (⇒ file) — assert `requests[0].mode` undefined or `"file"`. -- [x] Heartbeat: `TranscribeDeps.heartbeatIntervalMs?` (default 5000); test with 20 ms and a fake - sidecar that waits ~120 ms before the first segment; the response text contains ≥ 2 blank - lines before the first segment line. -- [x] Implement; typecheck; commit. - -### Task 4: Real-model verification + docs - -- [x] `FW_INTEGRATION=1 SIDECAR_PYTHON=/home/user/venvs/faster-whisper/bin/python bun test - tests/transcribe.test.ts` under the lock; extend the integration block with a real speech - run (first 60 s of `en_alien`, mode file, asserts ≥ 1 segment and a key word) and the tone - (silence-ish) run finishing `done`. -- [x] `docs/Decisions.md` D-WS5a with the before/after table; `tests/server-engine.md` updated; - `docs/Architecture.md` sidecar protocol line if it describes the message; WorkBreakdown tick. -- [x] typecheck, fast suite, build; commit. -- [x] `git rm` this plan; commit.