whisper_web/tests/live.md

# Manual live-transcription verification (M7, D-016)

The segment loop (`src/lib/live.ts`) and the protocol changes are covered by
unit tests, and transcript *accuracy* of the pipeline (segmentation, pass
scheduling, mic-path resampling) is regression-tested as WER by
`tests/live-accuracy.test.ts` + `tests/resampler.test.ts`. End-to-end quality
with real Whisper weights and real speech is regression-tested too:
`tests/live-real-model.test.ts` runs **every** recording in
`tests/audio_files/` through the full mic path with `whisper-large-v3-turbo`
(q4) and holds live WER against the reference transcript (≤ 0.25; measured
≈ 0.11 for `da_festuge`, ≈ 0.06 for `en_aiproblem`; the noisy 15-min
`en_alien` stream has its own measured bar of 0.35 in `MAX_WER_BY_CLIP`,
D-039) and against whole-file mode (it skips itself when the model is
neither cached nor downloadable). Each clip's time budget scales with its
length, so `en_alien` alone takes ~20 min on CPU.
To cover a new recording, drop `<lang>_<name>.<m4a|mp3|wav|…>` plus its
reference transcript `<lang>_<name>.txt` into `tests/audio_files/` — the test
discovers it, decodes it with ffmpeg-static, pins `<lang>` as the Whisper
language, and fails loudly on unpaired or unprefixed files. Microphone capture,
the AudioWorklet path, and WebGPU/WASM execution in an actual browser can
only be exercised manually. This is that procedure.

Known model behavior (not a pipeline bug): with language **auto**, Whisper
misdetects the Festuge clip as English and returns a rough English
translation — in whole-file mode as well as live. Danish (and likely other
non-English) users should pick their language explicitly; the worker forwards
it and accuracy is then as tested above.
Re-run it after any change to `src/lib/live.ts`, `src/lib/mic.ts`,
`src/components/LiveControls.tsx`, or the live wiring in `src/App.tsx`.

## Prerequisites

- A machine with a working microphone.
- Chrome/Edge (WebGPU) and ideally also Firefox and Safari — the three differ
  exactly where mic capture is fragile (context sample rates, AudioWorklet
  Blob-URL modules; Safari exercises the ScriptProcessorNode fallback if the
  worklet path fails there).
- `getUserMedia` requires a secure context; `http://localhost` qualifies:
  ```sh
  bun dev            # or: bun run build && bun run start
  ```

## 1. Permission flow

1. Open the app, pick `whisper-tiny.en`, click **Start live transcription**.
2. The browser prompts for the microphone. **Deny** it: the button must return
   to idle and the error "Microphone access was denied…" must appear; the app
   stays usable (file flow unaffected).
3. Click again and **allow**: the browser's mic indicator turns on immediately.
   If the model was already loaded, the button becomes **Stop** with a pulsing
   red dot right away. If not (first use, or a model/device switch), the
   button instead shows **"Loading model…"** (disabled, no live dot, no
   "Listening…" hint) while the download/progress panel appears, and only
   flips to **Stop**/pulsing dot once the pipeline actually reports ready
   (D-034) — it must never claim "Listening" before that.

## 2. Live transcription behavior

1. Speak a few sentences with natural pauses. Expect:
   - While speaking: dimmed italic *tentative* text appears within a couple of
     seconds (tiny.en; slower models lag proportionally) and refreshes.
   - After a ~1 s pause: the sentence commits — the text turns from italic to
     regular ("committed") and no longer changes.
   - Silence alone (say nothing for 30 s) must produce **no** text — the
     energy gate drops silent segments, so no hallucinated "thank you."-style
     output.
2. DevTools → Network: **zero** requests while speaking (weights may download
   at the start; the audio itself never leaves the device).
3. During a live session, ModelPicker / options / dropzone are disabled.
4. Model choice sanity: repeat briefly with a multilingual model + a fixed
   language and confirm the language/task options are honored.

## 3. Stop & result

1. Click **Stop** mid-sentence. The button shows "Finishing…" while the last
   segment gets its final pass — if several segments are still queued (slow
   model), the transcript heading counts them down ("Finishing up…
   transcribing N remaining segments", D-024) — then the normal
   **Transcript** panel appears:
   full text, Copy button, and a Timestamps section whose segment times are
   **absolute** (a sentence spoken ~10 s in must not be timestamped 0:00).
2. The browser's mic indicator must turn off immediately on Stop.
3. Start a new live session afterwards, then also transcribe a file — both
   must still work (session cleanup, request correlation).

## 4. Quality/latency knobs (only if needed)

Production tuning lives in `LIVE_TUNING` (`src/lib/live.ts`) and is shared by
`src/App.tsx` and the accuracy harness (`tests/live-accuracy.test.ts`), so a
knob change is automatically re-validated for WER under `bun test`. If commits
are too eager/lazy in real rooms, tune `LIVE_TUNING.segmenter`
(`commitSilenceSeconds` 1.2, `maxSegmentSeconds` 29) or the
`LiveSegmenterOptions` defaults (`speechRms` gate 0.015, `sustainRmsRatio`
0.5, `softCutWindowSeconds` 3, `minSpeechSeconds` 0.25) and the throttle
(`tentativeIntervalMs` 2000, `minTentativeSeconds` 1). Record any tuned
values in D-016.

## 5. Offline live transcription

With the PWA installed and a model cached (see `tests/offline.md`), go
offline (airplane mode) and run a live session — it must work identically:
capture, worklet (Blob URL, no network), inference, stop/transcript.

## Recording the result

When this passes in a real browser, flip the `[~]` manual-verification item
in `WorkBreakdown.md` M7 to `[x]` and note browser + version here.

- [ ] Verified on: _browser/version, date_