lucasw89/au_whisper_web
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Usability and test infrastructure updates.

Browse Source
This commit is contained in:
Lucas Winther
2026-07-29 09:20:43 +02:00
parent 99b1732003
commit 7040f031a6
12 changed files with 549 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/Architecture.md
+27 -15
View File
@@ -84,24 +84,26 @@ src/
auth.ts # Passport.js wiring + session middleware adapter
db.ts # bun:sqlite schema + queries
transcribe.ts # server-side ASR endpoint (bridges the sidecar, M5)
live.ts # server-engine live mic over WebSocket (D-019, M7)
server-py/ # faster-whisper sidecar (Python, D-013; M5)
tests/ # bun test
```
(Current state: M0–M3 built — browser transcription with the curated model
(Current state: M0–M6 built — browser transcription with the curated model
catalog, WebNN→WebGPU→WASM auto-selection (D-017), model switching,
language/task options,
and the installable offline PWA (`pwa/`, `scripts/build.ts` production build,
`dist/` serving in `src/index.ts`; browser-level offline behavior pending the
manual run in `tests/offline.md`). M4 accounts are built (`server/db.ts`,
`server/auth.ts`, `/api/auth/*`, `AuthForms`). M7 live microphone
transcription is built (`lib/live.ts`, `lib/mic.ts`, `LiveControls`, D-016;
in-browser run pending per `tests/live.md`). M5's server side is built —
`server-py/sidecar.py`, `server/transcribe.ts` (supervisor +
`/api/transcribe` + `/status`), `lib/engine.ts` (both engines, D-010), and
the catalog mapping in `lib/models.ts` — with graceful degradation when
Python/faster-whisper is absent; the `EngineToggle` component and App wiring
are the remaining M5 work; see `WorkBreakdown.md`.)
language/task options, the installable offline PWA (`pwa/`,
`scripts/build.ts` production build, `dist/` serving in `src/index.ts`;
browser-level offline behavior pending the manual run in `tests/offline.md`),
M4 accounts (`server/db.ts`, `server/auth.ts`, `/api/auth/*`, `AuthForms`),
and the M5 server engine end to end — `server-py/sidecar.py`,
`server/transcribe.ts` (supervisor + `/api/transcribe` + `/status`),
`lib/engine.ts` (both engines, D-010), `EngineToggle`, and the catalog
mapping in `lib/models.ts` — with graceful degradation when
Python/faster-whisper is absent. M7 live microphone transcription is built
for both engines: browser (`lib/live.ts`, `lib/mic.ts`, `LiveControls`,
D-016) and server (`server/live.ts`, D-019); in-browser manual verification
of both is still pending per `tests/live.md` (no real browser in the dev
environment). See `WorkBreakdown.md` for the itemized checklist.)
## 4. Key flows
@@ -160,8 +162,18 @@ are the remaining M5 work; see `WorkBreakdown.md`.)
4. Stopping flushes the segmenter, drains pending final passes (merged, with
a "N remaining segments" countdown in the UI), and yields a normal
transcript (text + absolute-timestamped chunks) in `TranscriptView`.
Everything runs on-device; works offline like the file flow. Server-engine
live input is a recorded future want (WorkBreakdown M7, D-016).
Everything runs on-device; works offline like the file flow.
5. **Server-engine live (logged in, D-019).** Same `LiveControls` UI, but
`App.tsx` opens a `GET /api/transcribe/live` WebSocket instead of running
the segmenter locally: the browser only captures/resamples mic audio
(`lib/mic.ts`) and streams raw PCM frames over the socket; the *same* pure
`LiveTranscriber`/segmenter (`lib/live.ts`) runs server-side
(`server/live.ts`), writing each segment to a temp file and bridging it to
the faster-whisper sidecar (D-013) the same way `POST /api/transcribe`
does. `{type:"update"}` frames carry committed/tentative text back;
`{type:"stop"}` drains and closes with a `{type:"result"}` frame. Selected
only when signed in with the server engine available (`EngineToggle`);
anonymous/offline users always get the browser-engine path above.
### 4.2 Offline (PWA)
- Service worker (`pwa/sw.ts`, hand-rolled — D-007) precaches the built app shell
docs/Decisions.md
+69
View File
@@ -836,3 +836,72 @@ baked-in models, either accept runtime downloads for the extra ones (point
`FASTER_WHISPER_DOWNLOAD_ROOT` at `/app/data` for those, per the README's
override note) or extend the Dockerfile's pre-download `RUN` line —
re-measure and re-document the image size either way.
## D-032 — Server-live WebSocket brought in line with the HTTP engine's DI/validation/test conventions
**Status:** Accepted (2026-07-29)
D-019's server-engine live transcription (`GET /api/transcribe/live` →
`src/server/live.ts`) had landed and was wired end-to-end in `App.tsx`, but
three things about it lagged the standard the rest of M5 set with
`src/server/transcribe.ts`:
1. **Not injectable.** `live.ts` imported the real `sidecar` singleton
directly and called it from inside the `LiveTranscriber`'s `transcribe`
closure, unlike `createTranscribeHandlers(deps)`, whose whole point (per
its own comment) is that `Sidecar` is "injectable — tests use a fake".
There was no way to unit-test the WebSocket bridge without a real Python
sidecar.
2. **Unvalidated query params.** `POST /api/transcribe`'s `parseParams`
validates `language` against `LANGUAGE_RE` and `task` against
`"transcribe" | "translate"` before it ever reaches the sidecar protocol.
The WS upgrade route in `src/index.ts` skipped both checks —
`url.searchParams.get("task") || undefined` forwarded an arbitrary string
straight into `LiveSessionData.task` and on to the Python sidecar's
JSON-lines protocol. Also skipped: the upfront `sidecar.status()` check the HTTP
route makes before doing any work (D-013's "degrade gracefully" —
opening a socket that can only report unavailability after the first
segment reaches the sidecar is a worse failure mode than a prompt 503).
3. **Zero test coverage.** No `tests/server-live.*` file existed; every
other stateful server module in the app (`transcribe.ts`, `auth.ts`,
`db.ts`) has one.
**Fix:**
1. `src/server/transcribe.ts` exports `parseLanguageAndTask(url)` (extracted
from `parseParams`, same validation, same 400-text-on-string-return
contract); `src/index.ts`'s `GET /api/transcribe/live` case now calls it
and 400s on an invalid value, then calls `sidecar.status()` and 503s with
the reason before upgrading — matching `POST /api/transcribe` exactly.
2. `src/server/live.ts` exports `createLiveWebSocket(deps: { sidecar:
Sidecar })`; `liveWebSocket` (the instance `src/index.ts` wires into
`Bun.serve`) is just `createLiveWebSocket({ sidecar })` against the real
singleton. `LiveSessionData.user`/`task` are now typed `SessionUser`/
`WhisperTask` instead of `any`/`string | undefined` (task always has a
value now that the WS route defaults it like the HTTP route does), which
let the pre-existing `options.websocket = liveWebSocket as any` cast in
`src/index.ts` drop to a plain assignment — it was papering over the
untyped `user`/`task` fields, not a real incompatibility with `Bun.serve`'s
generic `websocket` option.
3. `tests/server-live.test.ts` exercises `createLiveWebSocket` against a fake
`Sidecar` (same shape as `tests/transcribe.test.ts`'s): a committed
segment writes exactly its own PCM bytes to the temp file and cleans up
after, model/language/task pass through, a sidecar error surfaces via the
`update` message's `error` field instead of crashing the session, a
`{"type":"stop"}` frame drains and closes with a `result` message, and a
malformed text frame is ignored. `tests/transcribe.test.ts` gained direct
cases for `parseLanguageAndTask`.
**Incidental correctness fix found while rewriting the `transcribe` closure:**
the original code wrote `new Uint8Array(audio.buffer as ArrayBuffer)` to the
temp file — the whole underlying `ArrayBuffer`, ignoring `audio.byteOffset`/
`audio.byteLength`. Every real caller today happens to pass a fresh
`Float32Array` from `lib/live.ts`'s `concat()` helper (always offset 0, full
length), so this never manifested, but `LiveTranscribeFn`'s type signature
(`(audio: Float32Array) => ...`) makes no such promise to callers. Restored
the original's `audio.byteOffset, audio.byteLength` slice (which the
pre-refactor code already had — it would have been an actual regression to
drop it during the DI rewrite).
**Consequence:** the server-live path now has the same testability and
input-validation posture as the HTTP path; no behavior change for a valid
client, since `App.tsx` already only ever sends catalog model ids and
`"transcribe" | "translate"` task values.
**Alternatives rejected:** leaving `live.ts` untested and testing only
through a real spawned sidecar (matches `createSidecar`'s own integration
tests, but that suite already covers the sidecar process itself — the gap
was specifically the WebSocket/`LiveTranscriber` glue, which needs no Python
process to exercise).
docs/WorkBreakdown.md
+23 -5
View File
@@ -273,7 +273,19 @@ Notes (M5):
(`estimatePeakLoadBytes`) is risky on a phone/tablet, covering the
explicit Download button and the lazy auto-load on file-drop/live-start
alike.
- [ ] Basic a11y pass (keyboard, focus, aria on progress/live transcript)
- [x] Basic a11y pass (keyboard, focus, aria on progress/live transcript) —
the model-download and transcription `<progress>` bars gained
`aria-label`s (previously announced as an unnamed percentage);
`TranscriptView`'s heading is now a programmatic focus target
(`tabIndex={-1}`) that receives focus the moment a final result lands,
so keyboard/screen-reader users land on new content instead of hunting
for it; the Copy button's "Copied!" state and `LiveControls`' status
transitions (requesting mic / listening / finishing) now also announce
via a visually-hidden `role="status"` region for users not focused on
the button. Keyboard operability was already in place everywhere in
scope (native buttons/inputs, `FileDropzone`'s `role="button"` +
`onKeyDown`) and needed no changes. `bun test` unaffected (233 pass, 1
pre-existing unrelated WER-gate failure, see Decisions.md).
- [x] Production build + `bun start` deployment check; README for deployers —
`bun run build` + `bun start` smoke-tested live (headers, hashed-asset
immutability, `/api/*` routing, `AUTH_ENABLED=false`, server-engine
@@ -316,10 +328,16 @@ cached.
- [~] Manual in-browser verification (mic permission, live text, offline run)
— procedure in `tests/live.md`, pending a real browser (same dev-env
caveat as M2/M3)
- [ ] **Future want:** live transcription on the **server engine** — stream
mic audio over a WebSocket (D-011's escape hatch) to faster-whisper
(D-013) for logged-in users on weak devices. Needs its own decision
entry; blocked on M5 landing first.
- [x] Live transcription on the **server engine** (D-019) — `GET
/api/transcribe/live` upgrades to a WebSocket (`src/server/live.ts`,
wired in `src/index.ts`); the client streams mic PCM and runs the
browser-engine UI unchanged otherwise (`App.tsx`'s `startLive`/`stopLive`
branch on `useServer`). Landed already but was undocumented here and
untested; brought up to the rest of the codebase's bar (D-032):
`createLiveWebSocket` now takes the sidecar as an injected dependency
like `createTranscribeHandlers`, the WS route validates
`language`/`task` the same way the HTTP route does, and
`tests/server-live.test.ts` covers the bridge against a fake sidecar.
---
src/components/LiveControls.tsx
+8
View File
@@ -41,6 +41,14 @@ export function LiveControls({
? "Listening — like everything else, your audio stays on this device."
: "…or transcribe live from your microphone."}
</p>
{/* Announce mode transitions for screen-reader users not focused on the
button — the button's own label change is only reliably heard by
whoever triggered it. */}
<span className="visually-hidden" role="status">
{status === "starting" && "Requesting microphone access"}
{status === "listening" && "Live transcription started"}
{status === "stopping" && "Finishing live transcription"}
</span>
</div>
);
}
src/components/ModelLoadProgress.tsx
+5 -1
View File
@@ -39,7 +39,11 @@ export function ModelLoadProgress({ modelId, progress }: ModelLoadProgressProps)
: "…"}
</span>
</div>
<progress max={100} value={percent} />
<progress
max={100}
value={percent}
aria-label={`Download progress for ${file.file}`}
/>
</li>
);
})}
src/components/TranscriptView.tsx
+18 -2
View File
@@ -1,4 +1,4 @@
import { useEffect, useState } from "react";
import { useEffect, useRef, useState } from "react";
import {
exportFilename,
transcriptToSrt,
@@ -64,6 +64,8 @@ export function TranscriptView({
exportName = null,
}: TranscriptViewProps) {
const [copied, setCopied] = useState(false);
const headingRef = useRef<HTMLHeadingElement>(null);
const prevResultRef = useRef<typeof result>(null);
const text = result?.text ?? partialText;
const copyText = `${text}${tentativeText}`.trim();
const showProgress =
@@ -78,6 +80,16 @@ export function TranscriptView({
return () => clearTimeout(timer);
}, [copied]);
// Move focus to the transcript heading the moment a final result lands, so
// keyboard/screen-reader users land on the new content instead of having to
// hunt for it after a file or live run finishes.
useEffect(() => {
if (result && !prevResultRef.current) {
headingRef.current?.focus();
}
prevResultRef.current = result;
}, [result]);
if (!transcribing && !text && !tentativeText) return null;
async function copy() {
@@ -89,7 +101,7 @@ export function TranscriptView({
return (
<section className="transcript">
<div className="transcript-header">
<h2>
<h2 ref={headingRef} tabIndex={-1}>
{result ? "Transcript" : activeLabel}
{percent !== null && (
<span className="transcript-percent"> {percent}%</span>
@@ -150,6 +162,9 @@ export function TranscriptView({
>
{copied ? "Copied!" : "Copy"}
</button>
<span className="visually-hidden" role="status">
{copied ? "Copied to clipboard" : ""}
</span>
</div>
</div>
{showProgress && (
@@ -157,6 +172,7 @@ export function TranscriptView({
<progress
max={progress.totalSeconds}
value={Math.min(progress.processedSeconds, progress.totalSeconds)}
aria-label="Transcription progress"
/>
<span className="transcript-progress-time">
{formatTimestamp(progress.processedSeconds)} /{" "}
src/index.css
+12
View File
@@ -16,6 +16,18 @@ body {
min-height: 100vh;
}
.visually-hidden {
position: absolute;
width: 1px;
height: 1px;
padding: 0;
margin: -1px;
overflow: hidden;
clip: rect(0, 0, 0, 0);
white-space: nowrap;
border: 0;
}
.app {
max-width: 720px;
margin: 0 auto;
src/index.ts
+19 -6
View File
@@ -1,10 +1,11 @@
import { serve } from "bun";
import { tmpdir } from "node:os";
import { join, normalize, sep } from "node:path";
import { ensureIcons, ICONS_DIR } from "../scripts/generate-icons.ts";
import { injectPrecacheManifest } from "./pwa/precache.ts";
import { auth } from "./server/auth.ts";
import { AUTH_ENABLED } from "./server/flags.ts";
import { MAX_UPLOAD_BYTES, transcribe } from "./server/transcribe.ts";
import { MAX_UPLOAD_BYTES, parseLanguageAndTask, sidecar, transcribe } from "./server/transcribe.ts";
import { liveWebSocket } from "./server/live.ts";
import { getFasterWhisperModel } from "./lib/models.ts";
@@ -132,15 +133,27 @@ async function handleApiRequest(request: Request, server: any): Promise<Response
if (!modelId) return new Response("Missing model", { status: 400 });
const fasterWhisperModel = getFasterWhisperModel(modelId);
if (!fasterWhisperModel) return new Response("Unknown model", { status: 400 });
// Same validation as POST /api/transcribe (D-030-adjacent: never forward
// an unvalidated task/language string into the sidecar protocol just
// because this entry point is a WebSocket, not JSON over HTTP).
const options = parseLanguageAndTask(url);
if (typeof options === "string") return new Response(options, { status: 400 });
// Fail fast like the HTTP route (D-013 "degrade gracefully") instead of
// opening a socket that can only report unavailability once the first
// segment reaches the sidecar.
const status = await sidecar.status();
if (!status.available) {
return new Response(status.reason ?? "server engine unavailable", { status: 503 });
}
const success = server.upgrade(request, {
data: {
user,
model: fasterWhisperModel,
language: url.searchParams.get("language") || undefined,
task: url.searchParams.get("task") || undefined,
tempDir: require("node:os").tmpdir()
}
language: options.language,
task: options.task,
tempDir: tmpdir(),
},
});
if (success) return undefined;
return new Response("WebSocket upgrade failed", { status: 500 });
@@ -310,7 +323,7 @@ options.port = process.env.PORT || 3000;
// Bun's default cap is 128 MB, which would 4xx a /api/transcribe upload of
// more than ~35 minutes of raw PCM before the route's own limit applies.
options.maxRequestBodySize = MAX_UPLOAD_BYTES;
options.websocket = liveWebSocket as any;
options.websocket = liveWebSocket;
const server = serve(options);
src/server/live.ts
+82 -54
View File
@@ -1,40 +1,60 @@
import { tmpdir } from "node:os";
/**
* Server-side live microphone transcription over WebSocket (D-019): the
* browser streams raw float32 16 kHz mono PCM frames, the server runs the
* same pure `LiveTranscriber`/segmenter as the browser engine (src/lib/live.ts)
* and bridges each pass to the faster-whisper sidecar (src/server/transcribe.ts,
* D-013) by writing the segment audio to a temp file. Wired at
* `GET /api/transcribe/live` in src/index.ts.
*
* `createLiveWebSocket` takes the `Sidecar` as an injected dependency —
* mirroring `createTranscribeHandlers` in transcribe.ts — so tests exercise
* this module against a fake sidecar instead of spawning a real Python
* process (see tests/server-live.test.ts).
*/
import { join } from "node:path";
import type { ServerWebSocket } from "bun";
import { LIVE_TUNING, LiveTranscriber, type LiveUpdate } from "../lib/live";
import { sidecar } from "./transcribe";
import { getSessionUser } from "./auth";
import { getFasterWhisperModel } from "../lib/models";
import { sidecar, type Sidecar } from "./transcribe";
import type { SessionUser } from "./auth";
import type { WhisperTask } from "../worker/protocol";
interface LiveSessionData {
user: any;
export interface LiveSessionData {
user: SessionUser;
/** faster-whisper model name (catalog mapping, src/lib/models.ts). */
model: string;
language?: string;
task?: WhisperTask;
task: WhisperTask;
tempDir: string;
}
export const liveWebSocket = {
open(ws: ServerWebSocket<LiveSessionData>) {
// Initialize the transcriber for this connection.
export interface LiveWebSocketDeps {
sidecar: Sidecar;
}
/** Exported for tests (tests/server-live.test.ts), which fake this shape. */
export type LiveSocket = ServerWebSocket<LiveSessionData> & { transcriber?: LiveTranscriber };
export function createLiveWebSocket(deps: LiveWebSocketDeps) {
return {
open(ws: LiveSocket) {
const transcriber = new LiveTranscriber({
...LIVE_TUNING,
transcribe: async (audio: Float32Array) => {
transcribe: async (audio) => {
const audioPath = join(
ws.data.tempDir,
`whisper-live-${crypto.randomUUID()}.f32`
`whisper-live-${crypto.randomUUID()}.f32`,
);
const sink = Bun.file(audioPath).writer();
// audio.buffer is an ArrayBuffer, we need a Uint8Array view
sink.write(new Uint8Array(audio.buffer as ArrayBuffer, audio.byteOffset, audio.byteLength));
await sink.end();
return new Promise((resolve, reject) => {
let text = "";
let chunks: any[] = [];
sidecar.transcribe(
// audio may be a view into a larger buffer — byteOffset/byteLength
// must be preserved, not just the underlying buffer.
await Bun.write(
audioPath,
new Uint8Array(audio.buffer as ArrayBuffer, audio.byteOffset, audio.byteLength),
);
try {
const text: string[] = [];
const chunks: { text: string; timestamp: [number, number | null] }[] = [];
await deps.sidecar.transcribe(
{
model: ws.data.model,
audioPath,
@@ -42,30 +62,25 @@ export const liveWebSocket = {
task: ws.data.task,
},
(segment) => {
text += segment.text;
chunks.push({
text: segment.text,
timestamp: [segment.start, segment.end],
});
text.push(segment.text);
chunks.push({ text: segment.text, timestamp: [segment.start, segment.end] });
},
);
return { text: text.join("").trim(), chunks };
} finally {
await Bun.file(audioPath).unlink().catch(() => {});
}
)
.then(() => resolve({ text: text.trim(), chunks }))
.catch(reject)
.finally(() => Bun.file(audioPath).unlink().catch(() => {}));
});
},
onUpdate: (update: LiveUpdate) => {
ws.send(JSON.stringify({ type: "update", update }));
},
});
// Store it on the websocket
(ws as any).transcriber = transcriber;
ws.transcriber = transcriber;
},
message(ws: ServerWebSocket<LiveSessionData>, message: string | Buffer | Uint8Array) {
message(ws: LiveSocket, message: string | Buffer) {
if (typeof message === "string") {
let parsed: any;
let parsed: { type?: string };
try {
parsed = JSON.parse(message);
} catch {
@@ -75,29 +90,42 @@ export const liveWebSocket = {
return;
}
if (parsed.type === "stop") {
const transcriber = (ws as any).transcriber as LiveTranscriber;
transcriber.stop().then((result) => {
ws.transcriber
?.stop()
.then((result) => {
ws.send(JSON.stringify({ type: "result", result }));
ws.close();
}).catch((err) => {
ws.send(JSON.stringify({ type: "error", error: err instanceof Error ? err.message : String(err) }));
})
.catch((err) => {
ws.send(
JSON.stringify({
type: "error",
error: err instanceof Error ? err.message : String(err),
}),
);
ws.close();
});
}
} else {
// Audio chunk arrived
const transcriber = (ws as any).transcriber as LiveTranscriber;
// Convert raw bytes to Float32Array (16kHz mono)
const buffer = message instanceof Buffer ? message.buffer : message.buffer;
const byteOffset = message instanceof Buffer ? message.byteOffset : (message as Uint8Array).byteOffset;
const byteLength = message.byteLength;
const audio = new Float32Array(buffer, byteOffset, byteLength / 4);
transcriber.push(audio);
return;
}
// Binary frame: a raw float32 mono 16 kHz PCM chunk (lib/mic.ts already
// resampled it on the main thread, same contract as the browser engine).
const audio = new Float32Array(
message.buffer,
message.byteOffset,
message.byteLength / 4,
);
ws.transcriber?.push(audio);
},
close(ws: ServerWebSocket<LiveSessionData>) {
const transcriber = (ws as any).transcriber as LiveTranscriber;
transcriber?.stop().catch(() => {});
close(ws: LiveSocket) {
ws.transcriber?.stop().catch(() => {});
},
};
};
}
// ---------------------------------------------------------------------------
// Default instance (real sidecar), wired in src/index.ts
// ---------------------------------------------------------------------------
export const liveWebSocket = createLiveWebSocket({ sidecar });
src/server/transcribe.ts
+25 -9
View File
@@ -324,14 +324,16 @@ interface TranscribeParams {
task: WhisperTask;
}
/** Parse + validate query params; a string is the 400 error text. */
function parseParams(url: URL): TranscribeParams | string {
const modelId = url.searchParams.get("model");
if (!modelId) return "missing model parameter";
const fasterWhisperModel = getFasterWhisperModel(modelId);
// Catalog-only (D-006/D-013): never forward arbitrary ids to the sidecar.
if (!fasterWhisperModel) return `unknown model: ${modelId}`;
/**
* Parse + validate the `language`/`task` query params shared by both the
* HTTP (`POST /api/transcribe`) and WebSocket (`GET /api/transcribe/live`,
* src/server/live.ts) entry points, so a live session can't forward an
* unvalidated string into the sidecar protocol just because it skips
* `parseParams` below. A string return is the 400 error text.
*/
export function parseLanguageAndTask(
url: URL,
): { language?: string; task: WhisperTask } | string {
const rawLanguage = url.searchParams.get("language");
let language: string | undefined;
if (rawLanguage !== null && rawLanguage !== "" && rawLanguage !== "auto") {
@@ -344,7 +346,21 @@ function parseParams(url: URL): TranscribeParams | string {
return `invalid task: ${rawTask}`;
}
return { modelId, fasterWhisperModel, language, task: rawTask };
return { language, task: rawTask };
}
/** Parse + validate query params; a string is the 400 error text. */
function parseParams(url: URL): TranscribeParams | string {
const modelId = url.searchParams.get("model");
if (!modelId) return "missing model parameter";
const fasterWhisperModel = getFasterWhisperModel(modelId);
// Catalog-only (D-006/D-013): never forward arbitrary ids to the sidecar.
if (!fasterWhisperModel) return `unknown model: ${modelId}`;
const options = parseLanguageAndTask(url);
if (typeof options === "string") return options;
return { modelId, fasterWhisperModel, ...options };
}
/**
tests/server-live.test.ts
+185
View File
@@ -0,0 +1,185 @@
import { afterAll, describe, expect, test } from "bun:test";
import { mkdtemp, readdir, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { createLiveWebSocket, type LiveSessionData, type LiveSocket } from "../src/server/live";
import type { Sidecar, SidecarSegment, SidecarTranscribeRequest } from "../src/server/transcribe";
import type { SessionUser } from "../src/server/auth";
// D-019: the server-live WebSocket bridge, tested against a fake Sidecar
// (createLiveWebSocket takes it as an injected dependency, mirroring
// createTranscribeHandlers in tests/transcribe.test.ts) — no Python sidecar,
// no real model, but real LiveTranscriber/segmenter logic and real temp files.
const USER: SessionUser = { id: 1, username: "tester" };
const RATE = 16_000;
function speech(seconds: number): Float32Array {
const out = new Float32Array(Math.round(seconds * RATE));
for (let i = 0; i < out.length; i++) out[i] = i % 2 === 0 ? 0.1 : -0.1;
return out;
}
function silence(seconds: number): Float32Array {
return new Float32Array(Math.round(seconds * RATE));
}
function toBuffer(audio: Float32Array): Buffer {
return Buffer.from(audio.buffer, audio.byteOffset, audio.byteLength);
}
async function waitFor(
predicate: () => boolean | Promise<boolean>,
timeoutMs = 3000,
): Promise<void> {
const start = performance.now();
while (!(await predicate())) {
if (performance.now() - start > timeoutMs) {
throw new Error("timed out waiting for condition");
}
await new Promise((resolve) => setTimeout(resolve, 5));
}
}
const tempDirs: string[] = [];
async function makeTempDir(): Promise<string> {
const dir = await mkdtemp(join(tmpdir(), "live-ws-test-"));
tempDirs.push(dir);
return dir;
}
afterAll(async () => {
await Promise.all(tempDirs.map((dir) => rm(dir, { recursive: true, force: true })));
});
interface FakeSidecar extends Sidecar {
requests: SidecarTranscribeRequest[];
audioPayloads: Float32Array[];
}
function fakeSidecar(
behavior: { segments?: SidecarSegment[]; error?: string } = {},
): FakeSidecar {
const segments = behavior.segments ?? [{ start: 0, end: 1, text: " hello world" }];
const sidecar: FakeSidecar = {
requests: [],
audioPayloads: [],
async status() {
return { available: true };
},
async transcribe(request, onSegment) {
sidecar.requests.push(request);
sidecar.audioPayloads.push(
new Float32Array(await Bun.file(request.audioPath).arrayBuffer()),
);
if (behavior.error) throw new Error(behavior.error);
for (const segment of segments) onSegment(segment);
return { cancelled: false };
},
};
return sidecar;
}
type FakeSocket = LiveSocket & {
sent: { type: string; [key: string]: unknown }[];
closed: boolean;
};
function fakeSocket(data: LiveSessionData): FakeSocket {
const sent: { type: string; [key: string]: unknown }[] = [];
const ws = {
data,
sent,
closed: false,
send(message: string) {
sent.push(JSON.parse(message));
},
close() {
ws.closed = true;
},
};
return ws as unknown as FakeSocket;
}
function sessionData(overrides: Partial<LiveSessionData> = {}): LiveSessionData {
return {
user: USER,
model: "tiny",
task: "transcribe",
tempDir: "/tmp",
...overrides,
};
}
describe("createLiveWebSocket", () => {
test("a committed segment writes the exact PCM bytes and passes model/language/task through", async () => {
const tempDir = await makeTempDir();
const sidecar = fakeSidecar();
const handler = createLiveWebSocket({ sidecar });
const ws = fakeSocket(sessionData({ model: "base", language: "da", task: "translate", tempDir }));
handler.open(ws);
handler.message(ws, toBuffer(speech(1)));
handler.message(ws, toBuffer(silence(1.5))); // trailing silence commits the segment
await waitFor(() => sidecar.requests.length > 0);
expect(sidecar.requests[0]).toEqual(
expect.objectContaining({ model: "base", language: "da", task: "translate" }),
);
// The bytes written to the temp file are exactly the committed segment's
// samples — not, e.g., a whole shared buffer (see the byteOffset/byteLength fix).
const payload = sidecar.audioPayloads[0]!;
expect(payload.length).toBeGreaterThan(0);
for (const sample of payload) expect(Math.abs(sample)).toBeLessThanOrEqual(0.1 + 1e-6);
await waitFor(() =>
ws.sent.some((m) => m.type === "update" && (m.update as any).committedText === "hello world"),
);
// The temp file is cleaned up once the pass finishes.
await waitFor(async () => (await readdir(tempDir)).length === 0);
});
test("a sidecar error surfaces on the update stream instead of crashing the session", async () => {
const sidecar = fakeSidecar({ error: "model load failed" });
const handler = createLiveWebSocket({ sidecar });
const ws = fakeSocket(sessionData({ tempDir: await makeTempDir() }));
handler.open(ws);
handler.message(ws, toBuffer(speech(1)));
handler.message(ws, toBuffer(silence(1.5)));
await waitFor(() =>
ws.sent.some((m) => m.type === "update" && (m.update as any).error === "model load failed"),
);
});
test('a "stop" text frame drains, sends the result, and closes the socket', async () => {
const sidecar = fakeSidecar({ segments: [{ start: 0, end: 1, text: " done" }] });
const handler = createLiveWebSocket({ sidecar });
const ws = fakeSocket(sessionData({ tempDir: await makeTempDir() }));
handler.open(ws);
handler.message(ws, toBuffer(speech(1)));
handler.message(ws, JSON.stringify({ type: "stop" }));
await waitFor(() => ws.closed);
const result = ws.sent.find((m) => m.type === "result");
expect(result?.result).toEqual({ text: "done", chunks: [{ text: " done", timestamp: [0, 1] }] });
});
test("a malformed text frame is ignored rather than throwing", () => {
const handler = createLiveWebSocket({ sidecar: fakeSidecar() });
const ws = fakeSocket(sessionData({ tempDir: "/tmp" }));
handler.open(ws);
expect(() => handler.message(ws, "not json")).not.toThrow();
expect(ws.closed).toBe(false);
});
test("close() drains the session without throwing even if never stopped", async () => {
const handler = createLiveWebSocket({ sidecar: fakeSidecar() });
const ws = fakeSocket(sessionData({ tempDir: await makeTempDir() }));
handler.open(ws);
handler.message(ws, toBuffer(speech(0.5)));
expect(() => handler.close(ws)).not.toThrow();
});
});
tests/transcribe.test.ts
+41
View File
@@ -5,6 +5,7 @@ import { join } from "node:path";
import {
createSidecar,
createTranscribeHandlers,
parseLanguageAndTask,
type Sidecar,
type SidecarSegment,
type SidecarTranscribeRequest,
@@ -124,6 +125,46 @@ async function readStreamLines(response: Response): Promise<TranscribeStreamLine
// HTTP handlers (fake sidecar)
// ---------------------------------------------------------------------------
// ---------------------------------------------------------------------------
// parseLanguageAndTask — shared with the WS live route (src/server/live.ts,
// src/index.ts) so it can't skip the validation the HTTP route enforces.
// ---------------------------------------------------------------------------
describe("parseLanguageAndTask", () => {
test("defaults to transcribe with no language", () => {
expect(parseLanguageAndTask(new URL("http://localhost/x"))).toEqual({
language: undefined,
task: "transcribe",
});
});
test("accepts a valid language and task", () => {
expect(parseLanguageAndTask(new URL("http://localhost/x?language=da&task=translate"))).toEqual(
{ language: "da", task: "translate" },
);
});
test('"auto"/empty language means no language override', () => {
expect(parseLanguageAndTask(new URL("http://localhost/x?language=auto"))).toEqual({
language: undefined,
task: "transcribe",
});
expect(parseLanguageAndTask(new URL("http://localhost/x?language="))).toEqual({
language: undefined,
task: "transcribe",
});
});
test("rejects an invalid language or task", () => {
expect(parseLanguageAndTask(new URL("http://localhost/x?language=Danish!"))).toBe(
"invalid language: Danish!",
);
expect(parseLanguageAndTask(new URL("http://localhost/x?task=summarize"))).toBe(
"invalid task: summarize",
);
});
});
describe("GET /api/transcribe/status", () => {
test("401 without a session", async () => {
const handlers = createTranscribeHandlers({