71 lines
3.5 KiB
Markdown
71 lines
3.5 KiB
Markdown
# Manual verification: server engine (M5, D-013)
|
|
|
|
The dev environment has no Python package manager and no browser, so — like
|
|
`tests/offline.md` and `tests/live.md` — the real-inference path needs a
|
|
manual run on a machine with Python ≥ 3.9. Everything protocol- and
|
|
HTTP-level is already covered by `bun test` (`tests/transcribe.test.ts`
|
|
exercises the real `Bun.spawn` supervisor against
|
|
`tests/fixtures/fake_sidecar.py`, plus the degradation paths against the real
|
|
`server-py/sidecar.py`).
|
|
|
|
## 1. Install the sidecar dependencies
|
|
|
|
```sh
|
|
python3 -m pip install -r server-py/requirements.txt
|
|
```
|
|
|
|
## 2. Opt-in automated pass (no browser needed)
|
|
|
|
```sh
|
|
FW_INTEGRATION=1 bun test tests/transcribe.test.ts
|
|
```
|
|
|
|
Runs the real sidecar end to end (spawn → ready handshake → `tiny.en` load →
|
|
transcribe → done), then the first 60 s of `tests/audio_files/en_alien.opus`
|
|
(needs `ffmpeg`) in both decoding modes (D-WS5a): `file` (faster-whisper's
|
|
`BatchedInferencePipeline` + Silero VAD, what `POST /api/transcribe` uses) and
|
|
`live` (sequential, what server-live passes use). First run downloads ~75 MB
|
|
of weights from the Hub. Point `SIDECAR_PYTHON` at a virtualenv's python if
|
|
faster-whisper isn't installed for `python3` itself.
|
|
|
|
Without faster-whisper at all, `bun test tests/transcribe.test.ts` still runs
|
|
the real `server-py/sidecar.py` against a stub (`tests/fixtures/stub_fw`) to
|
|
check the mode routing, the one-model cache and cancel between segments.
|
|
With `FW_INTEGRATION=1` it also pushes the whole 15-min `en_alien` clip
|
|
(`tiny.en`) through `POST /api/transcribe` — heartbeat lines before the first
|
|
burst, segments in bursts — and cancels a batched run after its first burst.
|
|
|
|
- [ ] The `real faster-whisper sidecar` describe block passes.
|
|
|
|
## 3. In-browser run (both engines, same audio)
|
|
|
|
1. `bun run dev`, open the app, register/sign in.
|
|
2. `GET /api/transcribe/status` (with the session cookie) returns
|
|
`{"available": true}` — e.g. check the Network tab.
|
|
3. Pick `Whisper Tiny (English)`; transcribe a short clean English clip
|
|
(e.g. the JFK "ask not…" sample) **on device**. Note the transcript.
|
|
4. Switch the engine toggle to **Server** (visible only while signed in;
|
|
requires the M5 UI wiring — until that lands, drive it manually:
|
|
`curl -X POST --data-binary @audio.f32 -b "sid=…" \
|
|
"http://localhost:3000/api/transcribe?model=onnx-community/whisper-tiny.en"`
|
|
where `audio.f32` is raw float32 mono 16 kHz PCM).
|
|
5. Transcribe the same clip through the server engine.
|
|
|
|
- [ ] Segments stream in as JSON lines (not one blob at the end). With the
|
|
batched pipeline (D-WS5a) they arrive in bursts — one burst per 8
|
|
speech chunks (≈ up to 4 min of audio) — with blank heartbeat lines
|
|
every 5 s in between; on a clip under ~30 s of speech, everything
|
|
arrives as a single segment at the end. Each segment covers one VAD
|
|
speech chunk (up to 30 s), not one sentence — expected (D-WS5a).
|
|
- [ ] Cancelling a long server transcription (abort / page refresh) stops
|
|
the sidecar within one burst; the next request then starts normally.
|
|
- [ ] Both transcripts agree in content (minor punctuation/casing drift
|
|
between Transformers.js and faster-whisper is expected).
|
|
- [ ] Signed out, the toggle is gone and `POST /api/transcribe` answers 401.
|
|
- [ ] Second request on the same model is visibly faster (no reload —
|
|
warm-up strategy).
|
|
- [ ] `AUTH_ENABLED=false bun run dev`: `/api/transcribe*` answers 404 and
|
|
the anonymous browser flow is unaffected.
|
|
|
|
When this passes, check off the M5 integration item in `WorkBreakdown.md`.
|