lucasw89/whisper_web
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Record the batched-VAD measurements and check heartbeat, bursts and cancel against the real sidecar

Browse Source 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Lucas Winther
2026-10-30 00:23:55 +01:00
co-authored by Claude Opus 5.5
parent 1c1806840e
commit 104c1c4e4d
5 changed files with 143 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/Decisions.md
+45
View File
@@ -1224,3 +1224,48 @@ whole-file output is equally bad, which is exactly what the absolute bar
exists to catch.
**Consequence:** full `bun test` ≈ 19 min on CPU (en_alien ≈ 20 min of it
on a cold run); the fast suite excludes this file.
## D-WS5a — Server engine: batched faster-whisper + tuned Silero VAD for files; live passes stay sequential
**Status:** Accepted (2026-10-30, spec 001 Goal 5)
`server-py/sidecar.py` decoded every request with the sequential
`WhisperModel.transcribe`, no VAD.
**Decision:** the sidecar request gains `mode: "file" | "live"` (absent ⇒
`"file"`, so an older Bun still gets the new path). `"file"` (`POST
/api/transcribe`) runs `BatchedInferencePipeline(model).transcribe(audio,
vad_filter=True, vad_parameters={threshold: 0.2, speech_pad_ms: 800},
batch_size=8)`; the pipeline is a weightless wrapper cached and rebuilt with
the one resident model. `"live"` (server-live passes, `src/server/live.ts`)
keeps the sequential decoder. Beam size stays at the default 5.
**Measurements** (CPU int8, AMD Ryzen 9 9950X, 16 cores shared with other
jobs — load average 7–24 during runs, so wall-clock is ±20 %; WER via jiwer
on the `tests/live-real-model.test.ts` normalisation vs `tests/audio_files/*.txt`):
| model | audio | config | wall-clock | WER |
|---|---|---|---|---|
| large-v3-turbo | en_alien (888 s) | sequential (before) | 97 s / 130 s (two runs) | 0.097 / 0.067 |
| large-v3-turbo | en_alien | sequential + VAD | 79 s | 0.136 |
| large-v3-turbo | en_alien | batched, default VAD (0.5 / 400 ms) | 47 s / 59 s | 0.117 |
| large-v3-turbo | en_alien | batched, beam 1 | 44 s | 0.114 |
| large-v3-turbo | en_alien | **batched, VAD 0.2 / 800 ms (after)** | **51 s / 58 s** | **0.086** |
| large-v3-turbo | en_alien | batched, VAD 0.2 / 800 ms, word-level timestamps | 56 s | 0.144 |
| large-v3-turbo | da_festuge (21.5 s, da) | sequential (before) | 2.9 s | 0.094 |
| large-v3-turbo | da_festuge | batched, default VAD | 2.9 s | 0.141 |
| large-v3-turbo | da_festuge | **batched, VAD 0.2 / 800 ms (after)** | **2.7 s** | **0.109** |
| medium.en | en_alien | sequential (before) | 114 s | 0.126 |
| medium.en | en_alien | **batched, VAD 0.2 / 800 ms (after)** | **67 s** | **0.111** |
The sequential decoder is not deterministic run to run on en_alien (temperature
fallback; 0.067–0.097 over three runs), while the batched result repeated
exactly. Default-VAD errors were deletions (121 vs 64 sequential): Silero at
0.5 dropped quiet speech under music; 0.2 / 800 ms recovered most of them.
da_festuge's gap is one word of 64. Live-style 6 s passes over the first
180 s of en_alien: sequential 2.45 s/pass, batched 2.12, sequential + VAD
1.88 — within the load noise, each losing a few words, and real live
passes are already speech-gated client-side (D-016), so live stays as it was.
**Consequences:** file transcriptions are ≈ 1.7–2.5× faster on CPU at
equal-or-better WER on en_alien (one word worse on the 21 s Danish clip). Segments are coarser — one per VAD chunk (≤ 30 s) instead of one per
sentence — because word-level timestamps (`without_timestamps=False`) cost
WER here (0.144); SRT/VTT cues from the server engine are correspondingly
long. Segments arrive in bursts of 8 chunks (≈ 4 min of speech) and the first
burst can take ~15–25 s on a long file; the D-021 heartbeat covers it, and
cancel lands at the next burst boundary. No new env vars.