whisper_web/docs/plans/001-5-server-batched-vad.md

Server engine: batched + VAD faster-whisper Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task (inline, no subagents — coordination plan). Steps use checkbox (- [ ]) syntax.

Goal: file transcriptions on the server engine run through faster-whisper's BatchedInferencePipeline with Silero VAD, without breaking segment streaming, cancel, the D-021 heartbeat, or the one-loaded-model cache.

Architecture: the sidecar request gains an optional mode: "file" | "live" (absent ⇒ "file"). "file" → BatchedInferencePipeline(model).transcribe(audio, vad_filter=True, batch_size=8); "live" → the existing sequential WhisperModel.transcribe (short ≤ 29 s passes gain nothing from batching; decided from measurements in Task 1). The pipeline wrapper is cached with the model. Bun side: SidecarTranscribeRequest.mode, createSidecar forwards it, live.ts sends "live".

Tech Stack: Python 3 + faster-whisper 1.2.1 (venv /home/user/venvs/faster-whisper), Bun test.

Spec: docs/specs/001-transcription-quality-and-speed.md Goal 5; docs/plans/001-0-coordination.md.

Global Constraints

• Only server-py/sidecar.py is this workstream's core; Bun edits limited to the protocol field.
• Heavy jobs (pip, benchmarks, FW_INTEGRATION=1) under flock /tmp/whisper-heavy.lock.
• Decision entry ## D-WS5a — … at the bottom of docs/Decisions.md; cite D-WS5a in code.
• requirements.txt stays faster-whisper>=1.1,<2 (BatchedInferencePipeline exists since 1.1.0).
• No new env vars unless measurement demands one (then README env table).
• Measurements: CPU int8, this machine (16 cores, shared), labelled as such.

Review Focus

1. Cancel during a batch — batched segments arrive in bursts (one forward per 8 VAD chunks); a cancel must still end the run with done {cancelled: true} and no further segments. Test: stub sidecar "slow" model, abort after first segment (Task 2).
2. Long silent stretch before the first segment (VAD + feature extraction over the whole file happen eagerly in transcribe(), then a full batch decodes) — heartbeat lines must keep flowing. Test: injectable heartbeat interval, fake sidecar that waits, assert blank lines (Task 3).
3. Silence-only audio — VAD finds no speech: must answer done with zero segments, not error. Test: stub "silence" model yields nothing → done (Task 2); real run on tone in FW_INTEGRATION.
4. Model switch — still one model resident; the batched wrapper must be rebuilt for the new model, not reused. Test: stub reports load count and which model the pipeline wraps (Task 2).
5. Old Bun ↔ new sidecar — request without mode must behave as "file". Test (Task 2).

---

Task 1: Measure (outside the repo)

Files: /home/user/ws5-bench/bench.py (not committed).

• Venv /home/user/venvs/faster-whisper with requirements + jiwer (under lock); confirm API.
• large-v3-turbo, both clips: base (current sidecar), vad, batched (vad+bs8), batched_b1 (beam 1). Record wall, WER (repo normalize), first-segment time, max gap. Done 2026-10-29 (/home/user/ws5-bench/turbo-*.jsonl); VAD-parameter follow-up in diag2.out / round3-*.jsonl (threshold 0.2 + pad 800 ms recovers most deletions).
• Live-style passes (clip sliced into 6 s pieces): live_seq vs live_batched vs live_vad.
• medium.en on en_alien: base vs batched if time allows.
• Decide: beam size, live mode routing.

Task 2: Sidecar mode routing, tested against a stubbed faster-whisper

Files:

• Create: tests/fixtures/stub_fw/faster_whisper/__init__.py, tests/fixtures/stub_fw/numpy/__init__.py
• Modify: server-py/sidecar.py, tests/transcribe.test.ts

Interfaces:

• Produces: sidecar stdin transcribe message accepts "mode": "file" | "live" (default "file").

• Stub: WhisperModel(name, **kw) counts loads; .transcribe(audio, **kw) yields segments whose text reports seq + kwargs; BatchedInferencePipeline(model).transcribe(audio, **kw) yields batched model=<name> loads=<n> + kwargs. Model "slow" yields 20 segments 0.1 s apart; "silence" yields none. Stub numpy: float32, fromfile(path, dtype) → bytes.

• Test runs createSidecar(["python3", "-S", REAL_SIDECAR]) with PYTHONPATH=stub_fw (createSidecar inherits env; set process.env.PYTHONPATH around the test or pass env — add optional env arg only if needed).

• Failing tests: default/file → batched with vad_filter=True batch_size=8; live → sequential, no vad_filter; model switch reloads and pipeline wraps the new model, same model doesn't reload; slow + abort → cancelled: true, < 20 segments; silence → done, 0 segments.

• Implement in sidecar.py; update docstring protocol; cite D-WS5a.

• Green; commit.

Task 3: Bun side — mode field, live sends "live", heartbeat test

Files: src/server/transcribe.ts, src/server/live.ts, tests/fixtures/fake_sidecar.py, tests/transcribe.test.ts, tests/server-live.test.ts

• Failing tests: fake sidecar "ok" echoes mode=; createSidecar default sends file, explicit live forwarded; server-live test asserts the request has mode: "live"; HTTP handler passes no mode (⇒ file) — assert requests[0].mode undefined or "file".
• Heartbeat: TranscribeDeps.heartbeatIntervalMs? (default 5000); test with 20 ms and a fake sidecar that waits ~120 ms before the first segment; the response text contains ≥ 2 blank lines before the first segment line.
• Implement; typecheck; commit.

Task 4: Real-model verification + docs

• FW_INTEGRATION=1 SIDECAR_PYTHON=/home/user/venvs/faster-whisper/bin/python bun test tests/transcribe.test.ts under the lock; extend the integration block with a real speech run (first 60 s of en_alien, mode file, asserts ≥ 1 segment and a key word) and the tone (silence-ish) run finishing done.
• docs/Decisions.md D-WS5a with the before/after table; tests/server-engine.md updated; docs/Architecture.md sidecar protocol line if it describes the message; WorkBreakdown tick.
• typecheck, fast suite, build; commit.
• git rm this plan; commit.