Usability and test infrastructure updates.

This commit is contained in:
Lucas Winther
2026-07-29 09:20:32 +02:00
parent 99b1732003
commit 69602f046b
11 changed files with 364 additions and 127 deletions
+27 -15
View File
@@ -84,24 +84,26 @@ src/
auth.ts # Passport.js wiring + session middleware adapter auth.ts # Passport.js wiring + session middleware adapter
db.ts # bun:sqlite schema + queries db.ts # bun:sqlite schema + queries
transcribe.ts # server-side ASR endpoint (bridges the sidecar, M5) transcribe.ts # server-side ASR endpoint (bridges the sidecar, M5)
live.ts # server-engine live mic over WebSocket (D-019, M7)
server-py/ # faster-whisper sidecar (Python, D-013; M5) server-py/ # faster-whisper sidecar (Python, D-013; M5)
tests/ # bun test tests/ # bun test
``` ```
(Current state: M0–M3 built — browser transcription with the curated model (Current state: M0–M6 built — browser transcription with the curated model
catalog, WebNN→WebGPU→WASM auto-selection (D-017), model switching, catalog, WebNN→WebGPU→WASM auto-selection (D-017), model switching,
language/task options, language/task options, the installable offline PWA (`pwa/`,
and the installable offline PWA (`pwa/`, `scripts/build.ts` production build, `scripts/build.ts` production build, `dist/` serving in `src/index.ts`;
`dist/` serving in `src/index.ts`; browser-level offline behavior pending the browser-level offline behavior pending the manual run in `tests/offline.md`),
manual run in `tests/offline.md`). M4 accounts are built (`server/db.ts`, M4 accounts (`server/db.ts`, `server/auth.ts`, `/api/auth/*`, `AuthForms`),
`server/auth.ts`, `/api/auth/*`, `AuthForms`). M7 live microphone and the M5 server engine end to end — `server-py/sidecar.py`,
transcription is built (`lib/live.ts`, `lib/mic.ts`, `LiveControls`, D-016; `server/transcribe.ts` (supervisor + `/api/transcribe` + `/status`),
in-browser run pending per `tests/live.md`). M5's server side is built — `lib/engine.ts` (both engines, D-010), `EngineToggle`, and the catalog
`server-py/sidecar.py`, `server/transcribe.ts` (supervisor + mapping in `lib/models.ts` — with graceful degradation when
`/api/transcribe` + `/status`), `lib/engine.ts` (both engines, D-010), and Python/faster-whisper is absent. M7 live microphone transcription is built
the catalog mapping in `lib/models.ts` — with graceful degradation when for both engines: browser (`lib/live.ts`, `lib/mic.ts`, `LiveControls`,
Python/faster-whisper is absent; the `EngineToggle` component and App wiring D-016) and server (`server/live.ts`, D-019); in-browser manual verification
are the remaining M5 work; see `WorkBreakdown.md`.) of both is still pending per `tests/live.md` (no real browser in the dev
environment). See `WorkBreakdown.md` for the itemized checklist.)
## 4. Key flows ## 4. Key flows
@@ -160,8 +162,18 @@ are the remaining M5 work; see `WorkBreakdown.md`.)
4. Stopping flushes the segmenter, drains pending final passes (merged, with 4. Stopping flushes the segmenter, drains pending final passes (merged, with
a "N remaining segments" countdown in the UI), and yields a normal a "N remaining segments" countdown in the UI), and yields a normal
transcript (text + absolute-timestamped chunks) in `TranscriptView`. transcript (text + absolute-timestamped chunks) in `TranscriptView`.
Everything runs on-device; works offline like the file flow. Server-engine Everything runs on-device; works offline like the file flow.
live input is a recorded future want (WorkBreakdown M7, D-016). 5. **Server-engine live (logged in, D-019).** Same `LiveControls` UI, but
`App.tsx` opens a `GET /api/transcribe/live` WebSocket instead of running
the segmenter locally: the browser only captures/resamples mic audio
(`lib/mic.ts`) and streams raw PCM frames over the socket; the *same* pure
`LiveTranscriber`/segmenter (`lib/live.ts`) runs server-side
(`server/live.ts`), writing each segment to a temp file and bridging it to
the faster-whisper sidecar (D-013) the same way `POST /api/transcribe`
does. `{type:"update"}` frames carry committed/tentative text back;
`{type:"stop"}` drains and closes with a `{type:"result"}` frame. Selected
only when signed in with the server engine available (`EngineToggle`);
anonymous/offline users always get the browser-engine path above.
### 4.2 Offline (PWA) ### 4.2 Offline (PWA)
- Service worker (`pwa/sw.ts`, hand-rolled — D-007) precaches the built app shell - Service worker (`pwa/sw.ts`, hand-rolled — D-007) precaches the built app shell
+69
View File
@@ -836,3 +836,72 @@ baked-in models, either accept runtime downloads for the extra ones (point
`FASTER_WHISPER_DOWNLOAD_ROOT` at `/app/data` for those, per the README's `FASTER_WHISPER_DOWNLOAD_ROOT` at `/app/data` for those, per the README's
override note) or extend the Dockerfile's pre-download `RUN` line — override note) or extend the Dockerfile's pre-download `RUN` line —
re-measure and re-document the image size either way. re-measure and re-document the image size either way.
## D-032 — Server-live WebSocket brought in line with the HTTP engine's DI/validation/test conventions
**Status:** Accepted (2026-07-29)
D-019's server-engine live transcription (`GET /api/transcribe/live` →
`src/server/live.ts`) had landed and was wired end-to-end in `App.tsx`, but
three things about it lagged the standard the rest of M5 set with
`src/server/transcribe.ts`:
1. **Not injectable.** `live.ts` imported the real `sidecar` singleton
directly and called it from inside the `LiveTranscriber`'s `transcribe`
closure, unlike `createTranscribeHandlers(deps)`, whose whole point (per
its own comment) is that `Sidecar` is "injectable — tests use a fake".
There was no way to unit-test the WebSocket bridge without a real Python
sidecar.
2. **Unvalidated query params.** `POST /api/transcribe`'s `parseParams`
validates `language` against `LANGUAGE_RE` and `task` against
`"transcribe" | "translate"` before it ever reaches the sidecar protocol.
The WS upgrade route in `src/index.ts` skipped both checks —
`url.searchParams.get("task") || undefined` forwarded an arbitrary string
straight into `LiveSessionData.task` and on to the Python sidecar's
JSON-lines protocol. Also skipped: the upfront `sidecar.status()` check the HTTP
route makes before doing any work (D-013's "degrade gracefully" —
opening a socket that can only report unavailability after the first
segment reaches the sidecar is a worse failure mode than a prompt 503).
3. **Zero test coverage.** No `tests/server-live.*` file existed; every
other stateful server module in the app (`transcribe.ts`, `auth.ts`,
`db.ts`) has one.
**Fix:**
1. `src/server/transcribe.ts` exports `parseLanguageAndTask(url)` (extracted
from `parseParams`, same validation, same 400-text-on-string-return
contract); `src/index.ts`'s `GET /api/transcribe/live` case now calls it
and 400s on an invalid value, then calls `sidecar.status()` and 503s with
the reason before upgrading — matching `POST /api/transcribe` exactly.
2. `src/server/live.ts` exports `createLiveWebSocket(deps: { sidecar:
Sidecar })`; `liveWebSocket` (the instance `src/index.ts` wires into
`Bun.serve`) is just `createLiveWebSocket({ sidecar })` against the real
singleton. `LiveSessionData.user`/`task` are now typed `SessionUser`/
`WhisperTask` instead of `any`/`string | undefined` (task always has a
value now that the WS route defaults it like the HTTP route does), which
let the pre-existing `options.websocket = liveWebSocket as any` cast in
`src/index.ts` drop to a plain assignment — it was papering over the
untyped `user`/`task` fields, not a real incompatibility with `Bun.serve`'s
generic `websocket` option.
3. `tests/server-live.test.ts` exercises `createLiveWebSocket` against a fake
`Sidecar` (same shape as `tests/transcribe.test.ts`'s): a committed
segment writes exactly its own PCM bytes to the temp file and cleans up
after, model/language/task pass through, a sidecar error surfaces via the
`update` message's `error` field instead of crashing the session, a
`{"type":"stop"}` frame drains and closes with a `result` message, and a
malformed text frame is ignored. `tests/transcribe.test.ts` gained direct
cases for `parseLanguageAndTask`.
**Incidental correctness fix found while rewriting the `transcribe` closure:**
the original code wrote `new Uint8Array(audio.buffer as ArrayBuffer)` to the
temp file — the whole underlying `ArrayBuffer`, ignoring `audio.byteOffset`/
`audio.byteLength`. Every real caller today happens to pass a fresh
`Float32Array` from `lib/live.ts`'s `concat()` helper (always offset 0, full
length), so this never manifested, but `LiveTranscribeFn`'s type signature
(`(audio: Float32Array) => ...`) makes no such promise to callers. Restored
the original's `audio.byteOffset, audio.byteLength` slice (which the
pre-refactor code already had — it would have been an actual regression to
drop it during the DI rewrite).
**Consequence:** the server-live path now has the same testability and
input-validation posture as the HTTP path; no behavior change for a valid
client, since `App.tsx` already only ever sends catalog model ids and
`"transcribe" | "translate"` task values.
**Alternatives rejected:** leaving `live.ts` untested and testing only
through a real spawned sidecar (matches `createSidecar`'s own integration
tests, but that suite already covers the sidecar process itself — the gap
was specifically the WebSocket/`LiveTranscriber` glue, which needs no Python
process to exercise).
+23 -5
View File
@@ -273,7 +273,19 @@ Notes (M5):
(`estimatePeakLoadBytes`) is risky on a phone/tablet, covering the (`estimatePeakLoadBytes`) is risky on a phone/tablet, covering the
explicit Download button and the lazy auto-load on file-drop/live-start explicit Download button and the lazy auto-load on file-drop/live-start
alike. alike.
- [ ] Basic a11y pass (keyboard, focus, aria on progress/live transcript) - [x] Basic a11y pass (keyboard, focus, aria on progress/live transcript) —
the model-download and transcription `<progress>` bars gained
`aria-label`s (previously announced as an unnamed percentage);
`TranscriptView`'s heading is now a programmatic focus target
(`tabIndex={-1}`) that receives focus the moment a final result lands,
so keyboard/screen-reader users land on new content instead of hunting
for it; the Copy button's "Copied!" state and `LiveControls`' status
transitions (requesting mic / listening / finishing) now also announce
via a visually-hidden `role="status"` region for users not focused on
the button. Keyboard operability was already in place everywhere in
scope (native buttons/inputs, `FileDropzone`'s `role="button"` +
`onKeyDown`) and needed no changes. `bun test` unaffected (233 pass, 1
pre-existing unrelated WER-gate failure, see Decisions.md).
- [x] Production build + `bun start` deployment check; README for deployers — - [x] Production build + `bun start` deployment check; README for deployers —
`bun run build` + `bun start` smoke-tested live (headers, hashed-asset `bun run build` + `bun start` smoke-tested live (headers, hashed-asset
immutability, `/api/*` routing, `AUTH_ENABLED=false`, server-engine immutability, `/api/*` routing, `AUTH_ENABLED=false`, server-engine
@@ -316,10 +328,16 @@ cached.
- [~] Manual in-browser verification (mic permission, live text, offline run) - [~] Manual in-browser verification (mic permission, live text, offline run)
— procedure in `tests/live.md`, pending a real browser (same dev-env — procedure in `tests/live.md`, pending a real browser (same dev-env
caveat as M2/M3) caveat as M2/M3)
- [ ] **Future want:** live transcription on the **server engine** — stream - [x] Live transcription on the **server engine** (D-019) — `GET
mic audio over a WebSocket (D-011's escape hatch) to faster-whisper /api/transcribe/live` upgrades to a WebSocket (`src/server/live.ts`,
(D-013) for logged-in users on weak devices. Needs its own decision wired in `src/index.ts`); the client streams mic PCM and runs the
entry; blocked on M5 landing first. browser-engine UI unchanged otherwise (`App.tsx`'s `startLive`/`stopLive`
branch on `useServer`). Landed already but was undocumented here and
untested; brought up to the rest of the codebase's bar (D-032):
`createLiveWebSocket` now takes the sidecar as an injected dependency
like `createTranscribeHandlers`, the WS route validates
`language`/`task` the same way the HTTP route does, and
`tests/server-live.test.ts` covers the bridge against a fake sidecar.
--- ---
+8
View File
@@ -41,6 +41,14 @@ export function LiveControls({
? "Listening — like everything else, your audio stays on this device." ? "Listening — like everything else, your audio stays on this device."
: "…or transcribe live from your microphone."} : "…or transcribe live from your microphone."}
</p> </p>
{/* Announce mode transitions for screen-reader users not focused on the
button — the button's own label change is only reliably heard by
whoever triggered it. */}
<span className="visually-hidden" role="status">
{status === "starting" && "Requesting microphone access"}
{status === "listening" && "Live transcription started"}
{status === "stopping" && "Finishing live transcription"}
</span>
</div> </div>
); );
} }
+5 -1
View File
@@ -39,7 +39,11 @@ export function ModelLoadProgress({ modelId, progress }: ModelLoadProgressProps)
: "…"} : "…"}
</span> </span>
</div> </div>
<progress max={100} value={percent} /> <progress
max={100}
value={percent}
aria-label={`Download progress for ${file.file}`}
/>
</li> </li>
); );
})} })}
+18 -2
View File
@@ -1,4 +1,4 @@
import { useEffect, useState } from "react"; import { useEffect, useRef, useState } from "react";
import { import {
exportFilename, exportFilename,
transcriptToSrt, transcriptToSrt,
@@ -64,6 +64,8 @@ export function TranscriptView({
exportName = null, exportName = null,
}: TranscriptViewProps) { }: TranscriptViewProps) {
const [copied, setCopied] = useState(false); const [copied, setCopied] = useState(false);
const headingRef = useRef<HTMLHeadingElement>(null);
const prevResultRef = useRef<typeof result>(null);
const text = result?.text ?? partialText; const text = result?.text ?? partialText;
const copyText = `${text}${tentativeText}`.trim(); const copyText = `${text}${tentativeText}`.trim();
const showProgress = const showProgress =
@@ -78,6 +80,16 @@ export function TranscriptView({
return () => clearTimeout(timer); return () => clearTimeout(timer);
}, [copied]); }, [copied]);
// Move focus to the transcript heading the moment a final result lands, so
// keyboard/screen-reader users land on the new content instead of having to
// hunt for it after a file or live run finishes.
useEffect(() => {
if (result && !prevResultRef.current) {
headingRef.current?.focus();
}
prevResultRef.current = result;
}, [result]);
if (!transcribing && !text && !tentativeText) return null; if (!transcribing && !text && !tentativeText) return null;
async function copy() { async function copy() {
@@ -89,7 +101,7 @@ export function TranscriptView({
return ( return (
<section className="transcript"> <section className="transcript">
<div className="transcript-header"> <div className="transcript-header">
<h2> <h2 ref={headingRef} tabIndex={-1}>
{result ? "Transcript" : activeLabel} {result ? "Transcript" : activeLabel}
{percent !== null && ( {percent !== null && (
<span className="transcript-percent"> {percent}%</span> <span className="transcript-percent"> {percent}%</span>
@@ -150,6 +162,9 @@ export function TranscriptView({
> >
{copied ? "Copied!" : "Copy"} {copied ? "Copied!" : "Copy"}
</button> </button>
<span className="visually-hidden" role="status">
{copied ? "Copied to clipboard" : ""}
</span>
</div> </div>
</div> </div>
{showProgress && ( {showProgress && (
@@ -157,6 +172,7 @@ export function TranscriptView({
<progress <progress
max={progress.totalSeconds} max={progress.totalSeconds}
value={Math.min(progress.processedSeconds, progress.totalSeconds)} value={Math.min(progress.processedSeconds, progress.totalSeconds)}
aria-label="Transcription progress"
/> />
<span className="transcript-progress-time"> <span className="transcript-progress-time">
{formatTimestamp(progress.processedSeconds)} /{" "} {formatTimestamp(progress.processedSeconds)} /{" "}
+12
View File
@@ -16,6 +16,18 @@ body {
min-height: 100vh; min-height: 100vh;
} }
.visually-hidden {
position: absolute;
width: 1px;
height: 1px;
padding: 0;
margin: -1px;
overflow: hidden;
clip: rect(0, 0, 0, 0);
white-space: nowrap;
border: 0;
}
.app { .app {
max-width: 720px; max-width: 720px;
margin: 0 auto; margin: 0 auto;
+20 -7
View File
@@ -1,10 +1,11 @@
import { serve } from "bun"; import { serve } from "bun";
import { tmpdir } from "node:os";
import { join, normalize, sep } from "node:path"; import { join, normalize, sep } from "node:path";
import { ensureIcons, ICONS_DIR } from "../scripts/generate-icons.ts"; import { ensureIcons, ICONS_DIR } from "../scripts/generate-icons.ts";
import { injectPrecacheManifest } from "./pwa/precache.ts"; import { injectPrecacheManifest } from "./pwa/precache.ts";
import { auth } from "./server/auth.ts"; import { auth } from "./server/auth.ts";
import { AUTH_ENABLED } from "./server/flags.ts"; import { AUTH_ENABLED } from "./server/flags.ts";
import { MAX_UPLOAD_BYTES, transcribe } from "./server/transcribe.ts"; import { MAX_UPLOAD_BYTES, parseLanguageAndTask, sidecar, transcribe } from "./server/transcribe.ts";
import { liveWebSocket } from "./server/live.ts"; import { liveWebSocket } from "./server/live.ts";
import { getFasterWhisperModel } from "./lib/models.ts"; import { getFasterWhisperModel } from "./lib/models.ts";
@@ -132,15 +133,27 @@ async function handleApiRequest(request: Request, server: any): Promise<Response
if (!modelId) return new Response("Missing model", { status: 400 }); if (!modelId) return new Response("Missing model", { status: 400 });
const fasterWhisperModel = getFasterWhisperModel(modelId); const fasterWhisperModel = getFasterWhisperModel(modelId);
if (!fasterWhisperModel) return new Response("Unknown model", { status: 400 }); if (!fasterWhisperModel) return new Response("Unknown model", { status: 400 });
// Same validation as POST /api/transcribe (D-030-adjacent: never forward
// an unvalidated task/language string into the sidecar protocol just
// because this entry point is a WebSocket, not JSON over HTTP).
const options = parseLanguageAndTask(url);
if (typeof options === "string") return new Response(options, { status: 400 });
// Fail fast like the HTTP route (D-013 "degrade gracefully") instead of
// opening a socket that can only report unavailability once the first
// segment reaches the sidecar.
const status = await sidecar.status();
if (!status.available) {
return new Response(status.reason ?? "server engine unavailable", { status: 503 });
}
const success = server.upgrade(request, { const success = server.upgrade(request, {
data: { data: {
user, user,
model: fasterWhisperModel, model: fasterWhisperModel,
language: url.searchParams.get("language") || undefined, language: options.language,
task: url.searchParams.get("task") || undefined, task: options.task,
tempDir: require("node:os").tmpdir() tempDir: tmpdir(),
} },
}); });
if (success) return undefined; if (success) return undefined;
return new Response("WebSocket upgrade failed", { status: 500 }); return new Response("WebSocket upgrade failed", { status: 500 });
@@ -310,7 +323,7 @@ options.port = process.env.PORT || 3000;
// Bun's default cap is 128 MB, which would 4xx a /api/transcribe upload of // Bun's default cap is 128 MB, which would 4xx a /api/transcribe upload of
// more than ~35 minutes of raw PCM before the route's own limit applies. // more than ~35 minutes of raw PCM before the route's own limit applies.
options.maxRequestBodySize = MAX_UPLOAD_BYTES; options.maxRequestBodySize = MAX_UPLOAD_BYTES;
options.websocket = liveWebSocket as any; options.websocket = liveWebSocket;
const server = serve(options); const server = serve(options);
+116 -88
View File
@@ -1,103 +1,131 @@
import { tmpdir } from "node:os"; /**
* Server-side live microphone transcription over WebSocket (D-019): the
* browser streams raw float32 16 kHz mono PCM frames, the server runs the
* same pure `LiveTranscriber`/segmenter as the browser engine (src/lib/live.ts)
* and bridges each pass to the faster-whisper sidecar (src/server/transcribe.ts,
* D-013) by writing the segment audio to a temp file. Wired at
* `GET /api/transcribe/live` in src/index.ts.
*
* `createLiveWebSocket` takes the `Sidecar` as an injected dependency —
* mirroring `createTranscribeHandlers` in transcribe.ts — so tests exercise
* this module against a fake sidecar instead of spawning a real Python
* process (see tests/server-live.test.ts).
*/
import { join } from "node:path"; import { join } from "node:path";
import type { ServerWebSocket } from "bun"; import type { ServerWebSocket } from "bun";
import { LIVE_TUNING, LiveTranscriber, type LiveUpdate } from "../lib/live"; import { LIVE_TUNING, LiveTranscriber, type LiveUpdate } from "../lib/live";
import { sidecar } from "./transcribe"; import { sidecar, type Sidecar } from "./transcribe";
import { getSessionUser } from "./auth"; import type { SessionUser } from "./auth";
import { getFasterWhisperModel } from "../lib/models";
import type { WhisperTask } from "../worker/protocol"; import type { WhisperTask } from "../worker/protocol";
interface LiveSessionData { export interface LiveSessionData {
user: any; user: SessionUser;
/** faster-whisper model name (catalog mapping, src/lib/models.ts). */
model: string; model: string;
language?: string; language?: string;
task?: WhisperTask; task: WhisperTask;
tempDir: string; tempDir: string;
} }
export const liveWebSocket = { export interface LiveWebSocketDeps {
open(ws: ServerWebSocket<LiveSessionData>) { sidecar: Sidecar;
// Initialize the transcriber for this connection. }
const transcriber = new LiveTranscriber({
...LIVE_TUNING,
transcribe: async (audio: Float32Array) => {
const audioPath = join(
ws.data.tempDir,
`whisper-live-${crypto.randomUUID()}.f32`
);
const sink = Bun.file(audioPath).writer();
// audio.buffer is an ArrayBuffer, we need a Uint8Array view
sink.write(new Uint8Array(audio.buffer as ArrayBuffer, audio.byteOffset, audio.byteLength));
await sink.end();
return new Promise((resolve, reject) => { /** Exported for tests (tests/server-live.test.ts), which fake this shape. */
let text = ""; export type LiveSocket = ServerWebSocket<LiveSessionData> & { transcriber?: LiveTranscriber };
let chunks: any[] = [];
sidecar.transcribe(
{
model: ws.data.model,
audioPath,
language: ws.data.language,
task: ws.data.task,
},
(segment) => {
text += segment.text;
chunks.push({
text: segment.text,
timestamp: [segment.start, segment.end],
});
}
)
.then(() => resolve({ text: text.trim(), chunks }))
.catch(reject)
.finally(() => Bun.file(audioPath).unlink().catch(() => {}));
});
},
onUpdate: (update: LiveUpdate) => {
ws.send(JSON.stringify({ type: "update", update }));
},
});
// Store it on the websocket
(ws as any).transcriber = transcriber;
},
message(ws: ServerWebSocket<LiveSessionData>, message: string | Buffer | Uint8Array) { export function createLiveWebSocket(deps: LiveWebSocketDeps) {
if (typeof message === "string") { return {
let parsed: any; open(ws: LiveSocket) {
try { const transcriber = new LiveTranscriber({
parsed = JSON.parse(message); ...LIVE_TUNING,
} catch { transcribe: async (audio) => {
// Not a frame we understand — ignore rather than crash the handler const audioPath = join(
// (which would otherwise take the whole connection down for every ws.data.tempDir,
// client sharing this Bun.serve process, not just this session). `whisper-live-${crypto.randomUUID()}.f32`,
);
// audio may be a view into a larger buffer — byteOffset/byteLength
// must be preserved, not just the underlying buffer.
await Bun.write(
audioPath,
new Uint8Array(audio.buffer as ArrayBuffer, audio.byteOffset, audio.byteLength),
);
try {
const text: string[] = [];
const chunks: { text: string; timestamp: [number, number | null] }[] = [];
await deps.sidecar.transcribe(
{
model: ws.data.model,
audioPath,
language: ws.data.language,
task: ws.data.task,
},
(segment) => {
text.push(segment.text);
chunks.push({ text: segment.text, timestamp: [segment.start, segment.end] });
},
);
return { text: text.join("").trim(), chunks };
} finally {
await Bun.file(audioPath).unlink().catch(() => {});
}
},
onUpdate: (update: LiveUpdate) => {
ws.send(JSON.stringify({ type: "update", update }));
},
});
ws.transcriber = transcriber;
},
message(ws: LiveSocket, message: string | Buffer) {
if (typeof message === "string") {
let parsed: { type?: string };
try {
parsed = JSON.parse(message);
} catch {
// Not a frame we understand — ignore rather than crash the handler
// (which would otherwise take the whole connection down for every
// client sharing this Bun.serve process, not just this session).
return;
}
if (parsed.type === "stop") {
ws.transcriber
?.stop()
.then((result) => {
ws.send(JSON.stringify({ type: "result", result }));
ws.close();
})
.catch((err) => {
ws.send(
JSON.stringify({
type: "error",
error: err instanceof Error ? err.message : String(err),
}),
);
ws.close();
});
}
return; return;
} }
if (parsed.type === "stop") { // Binary frame: a raw float32 mono 16 kHz PCM chunk (lib/mic.ts already
const transcriber = (ws as any).transcriber as LiveTranscriber; // resampled it on the main thread, same contract as the browser engine).
transcriber.stop().then((result) => { const audio = new Float32Array(
ws.send(JSON.stringify({ type: "result", result })); message.buffer,
ws.close(); message.byteOffset,
}).catch((err) => { message.byteLength / 4,
ws.send(JSON.stringify({ type: "error", error: err instanceof Error ? err.message : String(err) })); );
ws.close(); ws.transcriber?.push(audio);
}); },
}
} else {
// Audio chunk arrived
const transcriber = (ws as any).transcriber as LiveTranscriber;
// Convert raw bytes to Float32Array (16kHz mono)
const buffer = message instanceof Buffer ? message.buffer : message.buffer;
const byteOffset = message instanceof Buffer ? message.byteOffset : (message as Uint8Array).byteOffset;
const byteLength = message.byteLength;
const audio = new Float32Array(buffer, byteOffset, byteLength / 4);
transcriber.push(audio);
}
},
close(ws: ServerWebSocket<LiveSessionData>) { close(ws: LiveSocket) {
const transcriber = (ws as any).transcriber as LiveTranscriber; ws.transcriber?.stop().catch(() => {});
transcriber?.stop().catch(() => {}); },
}, };
}; }
// ---------------------------------------------------------------------------
// Default instance (real sidecar), wired in src/index.ts
// ---------------------------------------------------------------------------
export const liveWebSocket = createLiveWebSocket({ sidecar });
+25 -9
View File
@@ -324,14 +324,16 @@ interface TranscribeParams {
task: WhisperTask; task: WhisperTask;
} }
/** Parse + validate query params; a string is the 400 error text. */ /**
function parseParams(url: URL): TranscribeParams | string { * Parse + validate the `language`/`task` query params shared by both the
const modelId = url.searchParams.get("model"); * HTTP (`POST /api/transcribe`) and WebSocket (`GET /api/transcribe/live`,
if (!modelId) return "missing model parameter"; * src/server/live.ts) entry points, so a live session can't forward an
const fasterWhisperModel = getFasterWhisperModel(modelId); * unvalidated string into the sidecar protocol just because it skips
// Catalog-only (D-006/D-013): never forward arbitrary ids to the sidecar. * `parseParams` below. A string return is the 400 error text.
if (!fasterWhisperModel) return `unknown model: ${modelId}`; */
export function parseLanguageAndTask(
url: URL,
): { language?: string; task: WhisperTask } | string {
const rawLanguage = url.searchParams.get("language"); const rawLanguage = url.searchParams.get("language");
let language: string | undefined; let language: string | undefined;
if (rawLanguage !== null && rawLanguage !== "" && rawLanguage !== "auto") { if (rawLanguage !== null && rawLanguage !== "" && rawLanguage !== "auto") {
@@ -344,7 +346,21 @@ function parseParams(url: URL): TranscribeParams | string {
return `invalid task: ${rawTask}`; return `invalid task: ${rawTask}`;
} }
return { modelId, fasterWhisperModel, language, task: rawTask }; return { language, task: rawTask };
}
/** Parse + validate query params; a string is the 400 error text. */
function parseParams(url: URL): TranscribeParams | string {
const modelId = url.searchParams.get("model");
if (!modelId) return "missing model parameter";
const fasterWhisperModel = getFasterWhisperModel(modelId);
// Catalog-only (D-006/D-013): never forward arbitrary ids to the sidecar.
if (!fasterWhisperModel) return `unknown model: ${modelId}`;
const options = parseLanguageAndTask(url);
if (typeof options === "string") return options;
return { modelId, fasterWhisperModel, ...options };
} }
/** /**
+41
View File
@@ -5,6 +5,7 @@ import { join } from "node:path";
import { import {
createSidecar, createSidecar,
createTranscribeHandlers, createTranscribeHandlers,
parseLanguageAndTask,
type Sidecar, type Sidecar,
type SidecarSegment, type SidecarSegment,
type SidecarTranscribeRequest, type SidecarTranscribeRequest,
@@ -124,6 +125,46 @@ async function readStreamLines(response: Response): Promise<TranscribeStreamLine
// HTTP handlers (fake sidecar) // HTTP handlers (fake sidecar)
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// ---------------------------------------------------------------------------
// parseLanguageAndTask — shared with the WS live route (src/server/live.ts,
// src/index.ts) so it can't skip the validation the HTTP route enforces.
// ---------------------------------------------------------------------------
describe("parseLanguageAndTask", () => {
test("defaults to transcribe with no language", () => {
expect(parseLanguageAndTask(new URL("http://localhost/x"))).toEqual({
language: undefined,
task: "transcribe",
});
});
test("accepts a valid language and task", () => {
expect(parseLanguageAndTask(new URL("http://localhost/x?language=da&task=translate"))).toEqual(
{ language: "da", task: "translate" },
);
});
test('"auto"/empty language means no language override', () => {
expect(parseLanguageAndTask(new URL("http://localhost/x?language=auto"))).toEqual({
language: undefined,
task: "transcribe",
});
expect(parseLanguageAndTask(new URL("http://localhost/x?language="))).toEqual({
language: undefined,
task: "transcribe",
});
});
test("rejects an invalid language or task", () => {
expect(parseLanguageAndTask(new URL("http://localhost/x?language=Danish!"))).toBe(
"invalid language: Danish!",
);
expect(parseLanguageAndTask(new URL("http://localhost/x?task=summarize"))).toBe(
"invalid task: summarize",
);
});
});
describe("GET /api/transcribe/status", () => { describe("GET /api/transcribe/status", () => {
test("401 without a session", async () => { test("401 without a session", async () => {
const handlers = createTranscribeHandlers({ const handlers = createTranscribeHandlers({