WIP: uncommitted work at pause (session limit)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Lucas Winther
2026-10-29 23:03:27 +01:00
co-authored by Claude Opus 5.5
parent f7d0525423
commit 826bcd2e49
2 changed files with 127 additions and 3 deletions
+16 -3
View File
@@ -63,7 +63,7 @@ src/
index.html # HTML entry, imports frontend.tsx
frontend.tsx # React root
App.tsx # top-level UI composition
components/ # ModelPicker, TranscriptionOptions, FileDropzone, LiveControls, TranscriptView, EngineToggle, AuthForms
components/ # ModelPicker, TranscriptionOptions, VocabularyHint, FileDropzone, LiveControls, TranscriptView, EngineToggle, AuthForms
lib/
audio.ts # decode + resample to 16 kHz mono Float32Array (Web Audio API)
live.ts # live-mode segment loop: RMS gate, LiveSegmenter, LiveTranscriber (pure, D-016)
@@ -71,9 +71,11 @@ src/
models.ts # model catalog: ids, sizes, dtype defaults per device, languages
modelCache.ts # "downloaded?" per model via Cache Storage (D-005; reused by M3 offline indicator)
engine.ts # engine abstraction: BrowserEngine | ServerEngine
hint.ts # "Names and terms" hint: normalize, localStorage, prompt truncation rule (D-WS2a)
worker/
inference.worker.ts # Transformers.js pipeline, WebNN/WebGPU/WASM selection
protocol.ts # typed postMessage messages (shared with UI)
prompt.ts # decoder prompt (<|startofprev|> hint) via a generate() wrapper (D-WS2a)
pwa/
sw.ts # service worker (app-shell precache, /ort runtime cache)
precache.ts # placeholder-injection helpers shared by build + dev serve
@@ -141,9 +143,16 @@ environment). See `WorkBreakdown.md` for the itemized checklist.)
`AudioContext.decodeAudioData`, downmixes to mono, resamples to 16 kHz,
transfers the `Float32Array` to the worker. For multilingual models the
`transcribe` message also carries `language` (or auto-detect) and `task`
(transcribe | translate).
(transcribe | translate). For every model it may carry `prompt`: the
optional "Names and terms" hint (`VocabularyHint`, stored in localStorage
by `lib/hint.ts`), raw as typed (D-WS2a).
5. Worker runs the pipeline (chunked, with timestamps), streaming partial results
back via `{type:"partial"}` messages; final result as `{type:"result"}`.
A hint becomes `<|startofprev|> hint.` once per pass (`worker/prompt.ts`,
cut to Whisper's 223-token prompt budget at a whole word) and is prepended
to the decoder input of every 30 s window by a wrapper around the model's
`generate()`, which also strips it from the output again (D-WS2a).
Live passes (4.1b) carry the same `prompt`.
### 4.1b Live microphone transcription (anonymous, browser engine — D-016)
1. `LiveControls` starts a session: `getUserMedia` → `AudioContext` at the
@@ -204,7 +213,11 @@ environment). See `WorkBreakdown.md` for the itemized checklist.)
(`server-py/`) that Bun spawns and supervises (D-013; faster-whisper has no
browser/WASM build, so it is server-only — the browser stays on Transformers.js).
`src/server/transcribe.ts` bridges the sidecar's segment stream to the streamed
JSON-lines HTTP response (D-011). The shared catalog maps model ids to
JSON-lines HTTP response (D-011). `language`, `task` and the vocabulary hint
(`prompt`) travel as query params (also on the live WebSocket) and are
validated by `parseLanguageAndTask` (D-032, D-WS2a); the sidecar cuts the
hint with the same rule as the browser and passes it to faster-whisper
(D-WS2a). The shared catalog maps model ids to
`Systran/faster-whisper-*` CTranslate2 conversions. If the sidecar is absent,
the server engine reports unavailable; the browser/anonymous flow is unaffected.
4. Both engines implement the same `Engine` interface in `src/lib/engine.ts`, so the