diff --git a/docs/plans/001-5-server-batched-vad.md b/docs/plans/001-5-server-batched-vad.md new file mode 100644 index 0000000..65b5bd9 --- /dev/null +++ b/docs/plans/001-5-server-batched-vad.md @@ -0,0 +1,100 @@ +# Server engine: batched + VAD faster-whisper Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan +> task-by-task (inline, no subagents — coordination plan). Steps use checkbox (`- [ ]`) syntax. + +**Goal:** file transcriptions on the server engine run through faster-whisper's +`BatchedInferencePipeline` with Silero VAD, without breaking segment streaming, cancel, the D-021 +heartbeat, or the one-loaded-model cache. + +**Architecture:** the sidecar request gains an optional `mode: "file" | "live"` (absent ⇒ `"file"`). +`"file"` → `BatchedInferencePipeline(model).transcribe(audio, vad_filter=True, batch_size=8)`; +`"live"` → the existing sequential `WhisperModel.transcribe` (short ≤ 29 s passes gain nothing from +batching; decided from measurements in Task 1). The pipeline wrapper is cached with the model. +Bun side: `SidecarTranscribeRequest.mode`, `createSidecar` forwards it, `live.ts` sends `"live"`. + +**Tech Stack:** Python 3 + faster-whisper 1.2.1 (venv `/home/user/venvs/faster-whisper`), Bun test. + +**Spec:** `docs/specs/001-transcription-quality-and-speed.md` Goal 5; `docs/plans/001-0-coordination.md`. + +## Global Constraints + +- Only `server-py/sidecar.py` is this workstream's core; Bun edits limited to the protocol field. +- Heavy jobs (pip, benchmarks, `FW_INTEGRATION=1`) under `flock /tmp/whisper-heavy.lock`. +- Decision entry `## D-WS5a — …` at the bottom of `docs/Decisions.md`; cite `D-WS5a` in code. +- `requirements.txt` stays `faster-whisper>=1.1,<2` (BatchedInferencePipeline exists since 1.1.0). +- No new env vars unless measurement demands one (then README env table). +- Measurements: CPU int8, this machine (16 cores, shared), labelled as such. + +## Review Focus + +1. **Cancel during a batch** — batched segments arrive in bursts (one forward per 8 VAD chunks); a + cancel must still end the run with `done {cancelled: true}` and no further segments. Test: stub + sidecar "slow" model, abort after first segment (Task 2). +2. **Long silent stretch before the first segment** (VAD + feature extraction over the whole file + happen eagerly in `transcribe()`, then a full batch decodes) — heartbeat lines must keep flowing. + Test: injectable heartbeat interval, fake sidecar that waits, assert blank lines (Task 3). +3. **Silence-only audio** — VAD finds no speech: must answer `done` with zero segments, not error. + Test: stub "silence" model yields nothing → `done` (Task 2); real run on tone in FW_INTEGRATION. +4. **Model switch** — still one model resident; the batched wrapper must be rebuilt for the new + model, not reused. Test: stub reports load count and which model the pipeline wraps (Task 2). +5. **Old Bun ↔ new sidecar** — request without `mode` must behave as `"file"`. Test (Task 2). + +--- + +### Task 1: Measure (outside the repo) + +**Files:** `/home/user/ws5-bench/bench.py` (not committed). + +- [x] Venv `/home/user/venvs/faster-whisper` with requirements + jiwer (under lock); confirm API. +- [ ] large-v3-turbo, both clips: `base` (current sidecar), `vad`, `batched` (vad+bs8), + `batched_b1` (beam 1). Record wall, WER (repo normalize), first-segment time, max gap. +- [ ] Live-style passes (clip sliced into 6 s pieces): `live_seq` vs `live_batched` vs `live_vad`. +- [ ] medium.en on en_alien: `base` vs `batched` if time allows. +- [ ] Decide: beam size, live mode routing. + +### Task 2: Sidecar `mode` routing, tested against a stubbed faster-whisper + +**Files:** +- Create: `tests/fixtures/stub_fw/faster_whisper/__init__.py`, `tests/fixtures/stub_fw/numpy/__init__.py` +- Modify: `server-py/sidecar.py`, `tests/transcribe.test.ts` + +**Interfaces:** +- Produces: sidecar stdin `transcribe` message accepts `"mode": "file" | "live"` (default `"file"`). +- Stub: `WhisperModel(name, **kw)` counts loads; `.transcribe(audio, **kw)` yields segments whose + text reports `seq` + kwargs; `BatchedInferencePipeline(model).transcribe(audio, **kw)` yields + `batched model= loads=` + kwargs. Model `"slow"` yields 20 segments 0.1 s apart; + `"silence"` yields none. Stub numpy: `float32`, `fromfile(path, dtype)` → bytes. +- Test runs `createSidecar(["python3", "-S", REAL_SIDECAR])` with `PYTHONPATH=stub_fw` (createSidecar + inherits env; set `process.env.PYTHONPATH` around the test or pass `env` — add optional + `env` arg only if needed). + +- [ ] Failing tests: default/`file` → batched with `vad_filter=True batch_size=8`; `live` → + sequential, no `vad_filter`; model switch reloads and pipeline wraps the new model, same model + doesn't reload; `slow` + abort → `cancelled: true`, < 20 segments; `silence` → done, 0 segments. +- [ ] Implement in `sidecar.py`; update docstring protocol; cite D-WS5a. +- [ ] Green; commit. + +### Task 3: Bun side — `mode` field, live sends `"live"`, heartbeat test + +**Files:** `src/server/transcribe.ts`, `src/server/live.ts`, `tests/fixtures/fake_sidecar.py`, +`tests/transcribe.test.ts`, `tests/server-live.test.ts` + +- [ ] Failing tests: fake sidecar "ok" echoes `mode=`; `createSidecar` default sends `file`, explicit + `live` forwarded; server-live test asserts the request has `mode: "live"`; HTTP handler passes + no mode (⇒ file) — assert `requests[0].mode` undefined or `"file"`. +- [ ] Heartbeat: `TranscribeDeps.heartbeatIntervalMs?` (default 5000); test with 20 ms and a fake + sidecar that waits ~120 ms before the first segment; the response text contains ≥ 2 blank + lines before the first segment line. +- [ ] Implement; typecheck; commit. + +### Task 4: Real-model verification + docs + +- [ ] `FW_INTEGRATION=1 SIDECAR_PYTHON=/home/user/venvs/faster-whisper/bin/python bun test + tests/transcribe.test.ts` under the lock; extend the integration block with a real speech + run (first 60 s of `en_alien`, mode file, asserts ≥ 1 segment and a key word) and the tone + (silence-ish) run finishing `done`. +- [ ] `docs/Decisions.md` D-WS5a with the before/after table; `tests/server-engine.md` updated; + `docs/Architecture.md` sidecar protocol line if it describes the message; WorkBreakdown tick. +- [ ] typecheck, fast suite, build; commit. +- [ ] `git rm` this plan; commit.