Usability and test infrastructure updates.

This commit is contained in:
Lucas Winther committed 2026-07-29 09:20:43 +02:00
1 parent 99b1732003
commit 7040f031a6
12 files changed
+549 -127

No files matched your search

+27 -15
View File
@@ -84,24 +84,26 @@ src/
auth.ts # Passport.js wiring + session middleware adapter
db.ts # bun:sqlite schema + queries
transcribe.ts # server-side ASR endpoint (bridges the sidecar, M5)
live.ts # server-engine live mic over WebSocket (D-019, M7)
server-py/ # faster-whisper sidecar (Python, D-013; M5)
tests/ # bun test
```
(Current state: M0–M3 built — browser transcription with the curated model
(Current state: M0–M6 built — browser transcription with the curated model
catalog, WebNN→WebGPU→WASM auto-selection (D-017), model switching,
language/task options,
and the installable offline PWA (`pwa/`, `scripts/build.ts` production build,
`dist/` serving in `src/index.ts`; browser-level offline behavior pending the
manual run in `tests/offline.md`). M4 accounts are built (`server/db.ts`,
`server/auth.ts`, `/api/auth/*`, `AuthForms`). M7 live microphone
transcription is built (`lib/live.ts`, `lib/mic.ts`, `LiveControls`, D-016;
in-browser run pending per `tests/live.md`). M5's server side is built —
`server-py/sidecar.py`, `server/transcribe.ts` (supervisor +
`/api/transcribe` + `/status`), `lib/engine.ts` (both engines, D-010), and
the catalog mapping in `lib/models.ts` — with graceful degradation when
Python/faster-whisper is absent; the `EngineToggle` component and App wiring
are the remaining M5 work; see `WorkBreakdown.md`.)
language/task options, the installable offline PWA (`pwa/`,
`scripts/build.ts` production build, `dist/` serving in `src/index.ts`;
browser-level offline behavior pending the manual run in `tests/offline.md`),
M4 accounts (`server/db.ts`, `server/auth.ts`, `/api/auth/*`, `AuthForms`),
and the M5 server engine end to end — `server-py/sidecar.py`,
`server/transcribe.ts` (supervisor + `/api/transcribe` + `/status`),
`lib/engine.ts` (both engines, D-010), `EngineToggle`, and the catalog
mapping in `lib/models.ts` — with graceful degradation when
Python/faster-whisper is absent. M7 live microphone transcription is built
for both engines: browser (`lib/live.ts`, `lib/mic.ts`, `LiveControls`,
D-016) and server (`server/live.ts`, D-019); in-browser manual verification
of both is still pending per `tests/live.md` (no real browser in the dev
environment). See `WorkBreakdown.md` for the itemized checklist.)
## 4. Key flows
@@ -160,8 +162,18 @@ are the remaining M5 work; see `WorkBreakdown.md`.)
4. Stopping flushes the segmenter, drains pending final passes (merged, with
a "N remaining segments" countdown in the UI), and yields a normal
transcript (text + absolute-timestamped chunks) in `TranscriptView`.
Everything runs on-device; works offline like the file flow. Server-engine
live input is a recorded future want (WorkBreakdown M7, D-016).
Everything runs on-device; works offline like the file flow.
5. **Server-engine live (logged in, D-019).** Same `LiveControls` UI, but
`App.tsx` opens a `GET /api/transcribe/live` WebSocket instead of running
the segmenter locally: the browser only captures/resamples mic audio
(`lib/mic.ts`) and streams raw PCM frames over the socket; the *same* pure
`LiveTranscriber`/segmenter (`lib/live.ts`) runs server-side
(`server/live.ts`), writing each segment to a temp file and bridging it to
the faster-whisper sidecar (D-013) the same way `POST /api/transcribe`
does. `{type:"update"}` frames carry committed/tentative text back;
`{type:"stop"}` drains and closes with a `{type:"result"}` frame. Selected
only when signed in with the server engine available (`EngineToggle`);
anonymous/offline users always get the browser-engine path above.
### 4.2 Offline (PWA)
- Service worker (`pwa/sw.ts`, hand-rolled — D-007) precaches the built app shell
+69
View File
@@ -836,3 +836,72 @@ baked-in models, either accept runtime downloads for the extra ones (point
`FASTER_WHISPER_DOWNLOAD_ROOT` at `/app/data` for those, per the README's
override note) or extend the Dockerfile's pre-download `RUN` line —
re-measure and re-document the image size either way.
## D-032 — Server-live WebSocket brought in line with the HTTP engine's DI/validation/test conventions
**Status:** Accepted (2026-07-29)
D-019's server-engine live transcription (`GET /api/transcribe/live` →
`src/server/live.ts`) had landed and was wired end-to-end in `App.tsx`, but
three things about it lagged the standard the rest of M5 set with
`src/server/transcribe.ts`:
1. **Not injectable.** `live.ts` imported the real `sidecar` singleton
directly and called it from inside the `LiveTranscriber`'s `transcribe`
closure, unlike `createTranscribeHandlers(deps)`, whose whole point (per
its own comment) is that `Sidecar` is "injectable — tests use a fake".
There was no way to unit-test the WebSocket bridge without a real Python
sidecar.
2. **Unvalidated query params.** `POST /api/transcribe`'s `parseParams`
validates `language` against `LANGUAGE_RE` and `task` against
`"transcribe" | "translate"` before it ever reaches the sidecar protocol.
The WS upgrade route in `src/index.ts` skipped both checks —
`url.searchParams.get("task") || undefined` forwarded an arbitrary string
straight into `LiveSessionData.task` and on to the Python sidecar's
JSON-lines protocol. Also skipped: the upfront `sidecar.status()` check the HTTP
route makes before doing any work (D-013's "degrade gracefully" —
opening a socket that can only report unavailability after the first
segment reaches the sidecar is a worse failure mode than a prompt 503).
3. **Zero test coverage.** No `tests/server-live.*` file existed; every
other stateful server module in the app (`transcribe.ts`, `auth.ts`,
`db.ts`) has one.
**Fix:**
1. `src/server/transcribe.ts` exports `parseLanguageAndTask(url)` (extracted
from `parseParams`, same validation, same 400-text-on-string-return
contract); `src/index.ts`'s `GET /api/transcribe/live` case now calls it
and 400s on an invalid value, then calls `sidecar.status()` and 503s with
the reason before upgrading — matching `POST /api/transcribe` exactly.
2. `src/server/live.ts` exports `createLiveWebSocket(deps: { sidecar:
Sidecar })`; `liveWebSocket` (the instance `src/index.ts` wires into
`Bun.serve`) is just `createLiveWebSocket({ sidecar })` against the real
singleton. `LiveSessionData.user`/`task` are now typed `SessionUser`/
`WhisperTask` instead of `any`/`string | undefined` (task always has a
value now that the WS route defaults it like the HTTP route does), which
let the pre-existing `options.websocket = liveWebSocket as any` cast in
`src/index.ts` drop to a plain assignment — it was papering over the
untyped `user`/`task` fields, not a real incompatibility with `Bun.serve`'s
generic `websocket` option.
3. `tests/server-live.test.ts` exercises `createLiveWebSocket` against a fake
`Sidecar` (same shape as `tests/transcribe.test.ts`'s): a committed
segment writes exactly its own PCM bytes to the temp file and cleans up
after, model/language/task pass through, a sidecar error surfaces via the
`update` message's `error` field instead of crashing the session, a
`{"type":"stop"}` frame drains and closes with a `result` message, and a
malformed text frame is ignored. `tests/transcribe.test.ts` gained direct
cases for `parseLanguageAndTask`.
**Incidental correctness fix found while rewriting the `transcribe` closure:**
the original code wrote `new Uint8Array(audio.buffer as ArrayBuffer)` to the
temp file — the whole underlying `ArrayBuffer`, ignoring `audio.byteOffset`/
`audio.byteLength`. Every real caller today happens to pass a fresh
`Float32Array` from `lib/live.ts`'s `concat()` helper (always offset 0, full
length), so this never manifested, but `LiveTranscribeFn`'s type signature
(`(audio: Float32Array) => ...`) makes no such promise to callers. Restored
the original's `audio.byteOffset, audio.byteLength` slice (which the
pre-refactor code already had — it would have been an actual regression to
drop it during the DI rewrite).
**Consequence:** the server-live path now has the same testability and
input-validation posture as the HTTP path; no behavior change for a valid
client, since `App.tsx` already only ever sends catalog model ids and
`"transcribe" | "translate"` task values.
**Alternatives rejected:** leaving `live.ts` untested and testing only
through a real spawned sidecar (matches `createSidecar`'s own integration
tests, but that suite already covers the sidecar process itself — the gap
was specifically the WebSocket/`LiveTranscriber` glue, which needs no Python
process to exercise).
+23 -5
View File
@@ -273,7 +273,19 @@ Notes (M5):
(`estimatePeakLoadBytes`) is risky on a phone/tablet, covering the
explicit Download button and the lazy auto-load on file-drop/live-start
alike.
- [ ] Basic a11y pass (keyboard, focus, aria on progress/live transcript)
- [x] Basic a11y pass (keyboard, focus, aria on progress/live transcript) —
the model-download and transcription `<progress>` bars gained
`aria-label`s (previously announced as an unnamed percentage);
`TranscriptView`'s heading is now a programmatic focus target
(`tabIndex={-1}`) that receives focus the moment a final result lands,
so keyboard/screen-reader users land on new content instead of hunting
for it; the Copy button's "Copied!" state and `LiveControls`' status
transitions (requesting mic / listening / finishing) now also announce
via a visually-hidden `role="status"` region for users not focused on
the button. Keyboard operability was already in place everywhere in
scope (native buttons/inputs, `FileDropzone`'s `role="button"` +
`onKeyDown`) and needed no changes. `bun test` unaffected (233 pass, 1
pre-existing unrelated WER-gate failure, see Decisions.md).
- [x] Production build + `bun start` deployment check; README for deployers —
`bun run build` + `bun start` smoke-tested live (headers, hashed-asset
immutability, `/api/*` routing, `AUTH_ENABLED=false`, server-engine
@@ -316,10 +328,16 @@ cached.
- [~] Manual in-browser verification (mic permission, live text, offline run)
— procedure in `tests/live.md`, pending a real browser (same dev-env
caveat as M2/M3)
- [ ] **Future want:** live transcription on the **server engine** — stream
mic audio over a WebSocket (D-011's escape hatch) to faster-whisper
(D-013) for logged-in users on weak devices. Needs its own decision
entry; blocked on M5 landing first.
- [x] Live transcription on the **server engine** (D-019) — `GET
/api/transcribe/live` upgrades to a WebSocket (`src/server/live.ts`,
wired in `src/index.ts`); the client streams mic PCM and runs the
browser-engine UI unchanged otherwise (`App.tsx`'s `startLive`/`stopLive`
branch on `useServer`). Landed already but was undocumented here and
untested; brought up to the rest of the codebase's bar (D-032):
`createLiveWebSocket` now takes the sidecar as an injected dependency
like `createTranscribeHandlers`, the WS route validates
`language`/`task` the same way the HTTP route does, and
`tests/server-live.test.ts` covers the bridge against a fake sidecar.
---