Usability and test infrastructure updates.
This commit is contained in:
1 parent
99b1732003
commit
7040f031a6
12 files changed
+549
-127
No files matched your search
+27
-15
@@ -84,24 +84,26 @@ src/
|
||||
auth.ts # Passport.js wiring + session middleware adapter
|
||||
db.ts # bun:sqlite schema + queries
|
||||
transcribe.ts # server-side ASR endpoint (bridges the sidecar, M5)
|
||||
live.ts # server-engine live mic over WebSocket (D-019, M7)
|
||||
server-py/ # faster-whisper sidecar (Python, D-013; M5)
|
||||
tests/ # bun test
|
||||
```
|
||||
|
||||
(Current state: M0–M3 built — browser transcription with the curated model
|
||||
(Current state: M0–M6 built — browser transcription with the curated model
|
||||
catalog, WebNN→WebGPU→WASM auto-selection (D-017), model switching,
|
||||
language/task options,
|
||||
and the installable offline PWA (`pwa/`, `scripts/build.ts` production build,
|
||||
`dist/` serving in `src/index.ts`; browser-level offline behavior pending the
|
||||
manual run in `tests/offline.md`). M4 accounts are built (`server/db.ts`,
|
||||
`server/auth.ts`, `/api/auth/*`, `AuthForms`). M7 live microphone
|
||||
transcription is built (`lib/live.ts`, `lib/mic.ts`, `LiveControls`, D-016;
|
||||
in-browser run pending per `tests/live.md`). M5's server side is built —
|
||||
`server-py/sidecar.py`, `server/transcribe.ts` (supervisor +
|
||||
`/api/transcribe` + `/status`), `lib/engine.ts` (both engines, D-010), and
|
||||
the catalog mapping in `lib/models.ts` — with graceful degradation when
|
||||
Python/faster-whisper is absent; the `EngineToggle` component and App wiring
|
||||
are the remaining M5 work; see `WorkBreakdown.md`.)
|
||||
language/task options, the installable offline PWA (`pwa/`,
|
||||
`scripts/build.ts` production build, `dist/` serving in `src/index.ts`;
|
||||
browser-level offline behavior pending the manual run in `tests/offline.md`),
|
||||
M4 accounts (`server/db.ts`, `server/auth.ts`, `/api/auth/*`, `AuthForms`),
|
||||
and the M5 server engine end to end — `server-py/sidecar.py`,
|
||||
`server/transcribe.ts` (supervisor + `/api/transcribe` + `/status`),
|
||||
`lib/engine.ts` (both engines, D-010), `EngineToggle`, and the catalog
|
||||
mapping in `lib/models.ts` — with graceful degradation when
|
||||
Python/faster-whisper is absent. M7 live microphone transcription is built
|
||||
for both engines: browser (`lib/live.ts`, `lib/mic.ts`, `LiveControls`,
|
||||
D-016) and server (`server/live.ts`, D-019); in-browser manual verification
|
||||
of both is still pending per `tests/live.md` (no real browser in the dev
|
||||
environment). See `WorkBreakdown.md` for the itemized checklist.)
|
||||
|
||||
## 4. Key flows
|
||||
|
||||
@@ -160,8 +162,18 @@ are the remaining M5 work; see `WorkBreakdown.md`.)
|
||||
4. Stopping flushes the segmenter, drains pending final passes (merged, with
|
||||
a "N remaining segments" countdown in the UI), and yields a normal
|
||||
transcript (text + absolute-timestamped chunks) in `TranscriptView`.
|
||||
Everything runs on-device; works offline like the file flow. Server-engine
|
||||
live input is a recorded future want (WorkBreakdown M7, D-016).
|
||||
Everything runs on-device; works offline like the file flow.
|
||||
5. **Server-engine live (logged in, D-019).** Same `LiveControls` UI, but
|
||||
`App.tsx` opens a `GET /api/transcribe/live` WebSocket instead of running
|
||||
the segmenter locally: the browser only captures/resamples mic audio
|
||||
(`lib/mic.ts`) and streams raw PCM frames over the socket; the *same* pure
|
||||
`LiveTranscriber`/segmenter (`lib/live.ts`) runs server-side
|
||||
(`server/live.ts`), writing each segment to a temp file and bridging it to
|
||||
the faster-whisper sidecar (D-013) the same way `POST /api/transcribe`
|
||||
does. `{type:"update"}` frames carry committed/tentative text back;
|
||||
`{type:"stop"}` drains and closes with a `{type:"result"}` frame. Selected
|
||||
only when signed in with the server engine available (`EngineToggle`);
|
||||
anonymous/offline users always get the browser-engine path above.
|
||||
|
||||
### 4.2 Offline (PWA)
|
||||
- Service worker (`pwa/sw.ts`, hand-rolled — D-007) precaches the built app shell
|
||||
|
||||
@@ -836,3 +836,72 @@ baked-in models, either accept runtime downloads for the extra ones (point
|
||||
`FASTER_WHISPER_DOWNLOAD_ROOT` at `/app/data` for those, per the README's
|
||||
override note) or extend the Dockerfile's pre-download `RUN` line —
|
||||
re-measure and re-document the image size either way.
|
||||
|
||||
## D-032 — Server-live WebSocket brought in line with the HTTP engine's DI/validation/test conventions
|
||||
**Status:** Accepted (2026-07-29)
|
||||
D-019's server-engine live transcription (`GET /api/transcribe/live` →
|
||||
`src/server/live.ts`) had landed and was wired end-to-end in `App.tsx`, but
|
||||
three things about it lagged the standard the rest of M5 set with
|
||||
`src/server/transcribe.ts`:
|
||||
1. **Not injectable.** `live.ts` imported the real `sidecar` singleton
|
||||
directly and called it from inside the `LiveTranscriber`'s `transcribe`
|
||||
closure, unlike `createTranscribeHandlers(deps)`, whose whole point (per
|
||||
its own comment) is that `Sidecar` is "injectable — tests use a fake".
|
||||
There was no way to unit-test the WebSocket bridge without a real Python
|
||||
sidecar.
|
||||
2. **Unvalidated query params.** `POST /api/transcribe`'s `parseParams`
|
||||
validates `language` against `LANGUAGE_RE` and `task` against
|
||||
`"transcribe" | "translate"` before it ever reaches the sidecar protocol.
|
||||
The WS upgrade route in `src/index.ts` skipped both checks —
|
||||
`url.searchParams.get("task") || undefined` forwarded an arbitrary string
|
||||
straight into `LiveSessionData.task` and on to the Python sidecar's
|
||||
JSON-lines protocol. Also skipped: the upfront `sidecar.status()` check the HTTP
|
||||
route makes before doing any work (D-013's "degrade gracefully" —
|
||||
opening a socket that can only report unavailability after the first
|
||||
segment reaches the sidecar is a worse failure mode than a prompt 503).
|
||||
3. **Zero test coverage.** No `tests/server-live.*` file existed; every
|
||||
other stateful server module in the app (`transcribe.ts`, `auth.ts`,
|
||||
`db.ts`) has one.
|
||||
**Fix:**
|
||||
1. `src/server/transcribe.ts` exports `parseLanguageAndTask(url)` (extracted
|
||||
from `parseParams`, same validation, same 400-text-on-string-return
|
||||
contract); `src/index.ts`'s `GET /api/transcribe/live` case now calls it
|
||||
and 400s on an invalid value, then calls `sidecar.status()` and 503s with
|
||||
the reason before upgrading — matching `POST /api/transcribe` exactly.
|
||||
2. `src/server/live.ts` exports `createLiveWebSocket(deps: { sidecar:
|
||||
Sidecar })`; `liveWebSocket` (the instance `src/index.ts` wires into
|
||||
`Bun.serve`) is just `createLiveWebSocket({ sidecar })` against the real
|
||||
singleton. `LiveSessionData.user`/`task` are now typed `SessionUser`/
|
||||
`WhisperTask` instead of `any`/`string | undefined` (task always has a
|
||||
value now that the WS route defaults it like the HTTP route does), which
|
||||
let the pre-existing `options.websocket = liveWebSocket as any` cast in
|
||||
`src/index.ts` drop to a plain assignment — it was papering over the
|
||||
untyped `user`/`task` fields, not a real incompatibility with `Bun.serve`'s
|
||||
generic `websocket` option.
|
||||
3. `tests/server-live.test.ts` exercises `createLiveWebSocket` against a fake
|
||||
`Sidecar` (same shape as `tests/transcribe.test.ts`'s): a committed
|
||||
segment writes exactly its own PCM bytes to the temp file and cleans up
|
||||
after, model/language/task pass through, a sidecar error surfaces via the
|
||||
`update` message's `error` field instead of crashing the session, a
|
||||
`{"type":"stop"}` frame drains and closes with a `result` message, and a
|
||||
malformed text frame is ignored. `tests/transcribe.test.ts` gained direct
|
||||
cases for `parseLanguageAndTask`.
|
||||
**Incidental correctness fix found while rewriting the `transcribe` closure:**
|
||||
the original code wrote `new Uint8Array(audio.buffer as ArrayBuffer)` to the
|
||||
temp file — the whole underlying `ArrayBuffer`, ignoring `audio.byteOffset`/
|
||||
`audio.byteLength`. Every real caller today happens to pass a fresh
|
||||
`Float32Array` from `lib/live.ts`'s `concat()` helper (always offset 0, full
|
||||
length), so this never manifested, but `LiveTranscribeFn`'s type signature
|
||||
(`(audio: Float32Array) => ...`) makes no such promise to callers. Restored
|
||||
the original's `audio.byteOffset, audio.byteLength` slice (which the
|
||||
pre-refactor code already had — it would have been an actual regression to
|
||||
drop it during the DI rewrite).
|
||||
**Consequence:** the server-live path now has the same testability and
|
||||
input-validation posture as the HTTP path; no behavior change for a valid
|
||||
client, since `App.tsx` already only ever sends catalog model ids and
|
||||
`"transcribe" | "translate"` task values.
|
||||
**Alternatives rejected:** leaving `live.ts` untested and testing only
|
||||
through a real spawned sidecar (matches `createSidecar`'s own integration
|
||||
tests, but that suite already covers the sidecar process itself — the gap
|
||||
was specifically the WebSocket/`LiveTranscriber` glue, which needs no Python
|
||||
process to exercise).
|
||||
+23
-5
@@ -273,7 +273,19 @@ Notes (M5):
|
||||
(`estimatePeakLoadBytes`) is risky on a phone/tablet, covering the
|
||||
explicit Download button and the lazy auto-load on file-drop/live-start
|
||||
alike.
|
||||
- [ ] Basic a11y pass (keyboard, focus, aria on progress/live transcript)
|
||||
- [x] Basic a11y pass (keyboard, focus, aria on progress/live transcript) —
|
||||
the model-download and transcription `<progress>` bars gained
|
||||
`aria-label`s (previously announced as an unnamed percentage);
|
||||
`TranscriptView`'s heading is now a programmatic focus target
|
||||
(`tabIndex={-1}`) that receives focus the moment a final result lands,
|
||||
so keyboard/screen-reader users land on new content instead of hunting
|
||||
for it; the Copy button's "Copied!" state and `LiveControls`' status
|
||||
transitions (requesting mic / listening / finishing) now also announce
|
||||
via a visually-hidden `role="status"` region for users not focused on
|
||||
the button. Keyboard operability was already in place everywhere in
|
||||
scope (native buttons/inputs, `FileDropzone`'s `role="button"` +
|
||||
`onKeyDown`) and needed no changes. `bun test` unaffected (233 pass, 1
|
||||
pre-existing unrelated WER-gate failure, see Decisions.md).
|
||||
- [x] Production build + `bun start` deployment check; README for deployers —
|
||||
`bun run build` + `bun start` smoke-tested live (headers, hashed-asset
|
||||
immutability, `/api/*` routing, `AUTH_ENABLED=false`, server-engine
|
||||
@@ -316,10 +328,16 @@ cached.
|
||||
- [~] Manual in-browser verification (mic permission, live text, offline run)
|
||||
— procedure in `tests/live.md`, pending a real browser (same dev-env
|
||||
caveat as M2/M3)
|
||||
- [ ] **Future want:** live transcription on the **server engine** — stream
|
||||
mic audio over a WebSocket (D-011's escape hatch) to faster-whisper
|
||||
(D-013) for logged-in users on weak devices. Needs its own decision
|
||||
entry; blocked on M5 landing first.
|
||||
- [x] Live transcription on the **server engine** (D-019) — `GET
|
||||
/api/transcribe/live` upgrades to a WebSocket (`src/server/live.ts`,
|
||||
wired in `src/index.ts`); the client streams mic PCM and runs the
|
||||
browser-engine UI unchanged otherwise (`App.tsx`'s `startLive`/`stopLive`
|
||||
branch on `useServer`). Landed already but was undocumented here and
|
||||
untested; brought up to the rest of the codebase's bar (D-032):
|
||||
`createLiveWebSocket` now takes the sidecar as an injected dependency
|
||||
like `createTranscribeHandlers`, the WS route validates
|
||||
`language`/`task` the same way the HTTP route does, and
|
||||
`tests/server-live.test.ts` covers the bridge against a fake sidecar.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in new issue
Block a user