# Server engine: batched + VAD faster-whisper Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan
> task-by-task (inline, no subagents — coordination plan). Steps use checkbox (`- [ ]`) syntax.

**Goal:** file transcriptions on the server engine run through faster-whisper's
`BatchedInferencePipeline` with Silero VAD, without breaking segment streaming, cancel, the D-021
heartbeat, or the one-loaded-model cache.

**Architecture:** the sidecar request gains an optional `mode: "file" | "live"` (absent ⇒ `"file"`).
`"file"` → `BatchedInferencePipeline(model).transcribe(audio, vad_filter=True, batch_size=8)`;
`"live"` → the existing sequential `WhisperModel.transcribe` (short ≤ 29 s passes gain nothing from
batching; decided from measurements in Task 1). The pipeline wrapper is cached with the model.
Bun side: `SidecarTranscribeRequest.mode`, `createSidecar` forwards it, `live.ts` sends `"live"`.

**Tech Stack:** Python 3 + faster-whisper 1.2.1 (venv `/home/user/venvs/faster-whisper`), Bun test.

**Spec:** `docs/specs/001-transcription-quality-and-speed.md` Goal 5; `docs/plans/001-0-coordination.md`.

## Global Constraints

- Only `server-py/sidecar.py` is this workstream's core; Bun edits limited to the protocol field.
- Heavy jobs (pip, benchmarks, `FW_INTEGRATION=1`) under `flock /tmp/whisper-heavy.lock`.
- Decision entry `## D-WS5a — …` at the bottom of `docs/Decisions.md`; cite `D-WS5a` in code.
- `requirements.txt` stays `faster-whisper>=1.1,<2` (BatchedInferencePipeline exists since 1.1.0).
- No new env vars unless measurement demands one (then README env table).
- Measurements: CPU int8, this machine (16 cores, shared), labelled as such.

## Review Focus

1. **Cancel during a batch** — batched segments arrive in bursts (one forward per 8 VAD chunks); a
   cancel must still end the run with `done {cancelled: true}` and no further segments. Test: stub
   sidecar "slow" model, abort after first segment (Task 2).
2. **Long silent stretch before the first segment** (VAD + feature extraction over the whole file
   happen eagerly in `transcribe()`, then a full batch decodes) — heartbeat lines must keep flowing.
   Test: injectable heartbeat interval, fake sidecar that waits, assert blank lines (Task 3).
3. **Silence-only audio** — VAD finds no speech: must answer `done` with zero segments, not error.
   Test: stub "silence" model yields nothing → `done` (Task 2); real run on tone in FW_INTEGRATION.
4. **Model switch** — still one model resident; the batched wrapper must be rebuilt for the new
   model, not reused. Test: stub reports load count and which model the pipeline wraps (Task 2).
5. **Old Bun ↔ new sidecar** — request without `mode` must behave as `"file"`. Test (Task 2).

---

### Task 1: Measure (outside the repo)

**Files:** `/home/user/ws5-bench/bench.py` (not committed).

- [x] Venv `/home/user/venvs/faster-whisper` with requirements + jiwer (under lock); confirm API.
- [x] large-v3-turbo, both clips: `base` (current sidecar), `vad`, `batched` (vad+bs8),
      `batched_b1` (beam 1). Record wall, WER (repo normalize), first-segment time, max gap.
      Done 2026-10-29 (`/home/user/ws5-bench/turbo-*.jsonl`); VAD-parameter follow-up in
      `diag2.out` / `round3-*.jsonl` (threshold 0.2 + pad 800 ms recovers most deletions).
- [ ] Live-style passes (clip sliced into 6 s pieces): `live_seq` vs `live_batched` vs `live_vad`.
- [ ] medium.en on en_alien: `base` vs `batched` if time allows.
- [ ] Decide: beam size, live mode routing.

### Task 2: Sidecar `mode` routing, tested against a stubbed faster-whisper

**Files:**
- Create: `tests/fixtures/stub_fw/faster_whisper/__init__.py`, `tests/fixtures/stub_fw/numpy/__init__.py`
- Modify: `server-py/sidecar.py`, `tests/transcribe.test.ts`

**Interfaces:**
- Produces: sidecar stdin `transcribe` message accepts `"mode": "file" | "live"` (default `"file"`).
- Stub: `WhisperModel(name, **kw)` counts loads; `.transcribe(audio, **kw)` yields segments whose
  text reports `seq` + kwargs; `BatchedInferencePipeline(model).transcribe(audio, **kw)` yields
  `batched model=<name> loads=<n>` + kwargs. Model `"slow"` yields 20 segments 0.1 s apart;
  `"silence"` yields none. Stub numpy: `float32`, `fromfile(path, dtype)` → bytes.
- Test runs `createSidecar(["python3", "-S", REAL_SIDECAR])` with `PYTHONPATH=stub_fw` (createSidecar
  inherits env; set `process.env.PYTHONPATH` around the test or pass `env` — add optional
  `env` arg only if needed).

- [x] Failing tests: default/`file` → batched with `vad_filter=True batch_size=8`; `live` →
      sequential, no `vad_filter`; model switch reloads and pipeline wraps the new model, same model
      doesn't reload; `slow` + abort → `cancelled: true`, < 20 segments; `silence` → done, 0 segments.
- [x] Implement in `sidecar.py`; update docstring protocol; cite D-WS5a.
- [x] Green; commit.

### Task 3: Bun side — `mode` field, live sends `"live"`, heartbeat test

**Files:** `src/server/transcribe.ts`, `src/server/live.ts`, `tests/fixtures/fake_sidecar.py`,
`tests/transcribe.test.ts`, `tests/server-live.test.ts`

- [x] Failing tests: fake sidecar "ok" echoes `mode=`; `createSidecar` default sends `file`, explicit
      `live` forwarded; server-live test asserts the request has `mode: "live"`; HTTP handler passes
      no mode (⇒ file) — assert `requests[0].mode` undefined or `"file"`.
- [x] Heartbeat: `TranscribeDeps.heartbeatIntervalMs?` (default 5000); test with 20 ms and a fake
      sidecar that waits ~120 ms before the first segment; the response text contains ≥ 2 blank
      lines before the first segment line.
- [x] Implement; typecheck; commit.

### Task 4: Real-model verification + docs

- [ ] `FW_INTEGRATION=1 SIDECAR_PYTHON=/home/user/venvs/faster-whisper/bin/python bun test
      tests/transcribe.test.ts` under the lock; extend the integration block with a real speech
      run (first 60 s of `en_alien`, mode file, asserts ≥ 1 segment and a key word) and the tone
      (silence-ish) run finishing `done`.
- [ ] `docs/Decisions.md` D-WS5a with the before/after table; `tests/server-engine.md` updated;
      `docs/Architecture.md` sidecar protocol line if it describes the message; WorkBreakdown tick.
- [ ] typecheck, fast suite, build; commit.
- [ ] `git rm` this plan; commit.