WIP: uncommitted work at pause (session limit)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
f7d0525423
commit
826bcd2e49
+16
-3
@@ -63,7 +63,7 @@ src/
|
||||
index.html # HTML entry, imports frontend.tsx
|
||||
frontend.tsx # React root
|
||||
App.tsx # top-level UI composition
|
||||
components/ # ModelPicker, TranscriptionOptions, FileDropzone, LiveControls, TranscriptView, EngineToggle, AuthForms
|
||||
components/ # ModelPicker, TranscriptionOptions, VocabularyHint, FileDropzone, LiveControls, TranscriptView, EngineToggle, AuthForms
|
||||
lib/
|
||||
audio.ts # decode + resample to 16 kHz mono Float32Array (Web Audio API)
|
||||
live.ts # live-mode segment loop: RMS gate, LiveSegmenter, LiveTranscriber (pure, D-016)
|
||||
@@ -71,9 +71,11 @@ src/
|
||||
models.ts # model catalog: ids, sizes, dtype defaults per device, languages
|
||||
modelCache.ts # "downloaded?" per model via Cache Storage (D-005; reused by M3 offline indicator)
|
||||
engine.ts # engine abstraction: BrowserEngine | ServerEngine
|
||||
hint.ts # "Names and terms" hint: normalize, localStorage, prompt truncation rule (D-WS2a)
|
||||
worker/
|
||||
inference.worker.ts # Transformers.js pipeline, WebNN/WebGPU/WASM selection
|
||||
protocol.ts # typed postMessage messages (shared with UI)
|
||||
prompt.ts # decoder prompt (<|startofprev|> hint) via a generate() wrapper (D-WS2a)
|
||||
pwa/
|
||||
sw.ts # service worker (app-shell precache, /ort runtime cache)
|
||||
precache.ts # placeholder-injection helpers shared by build + dev serve
|
||||
@@ -141,9 +143,16 @@ environment). See `WorkBreakdown.md` for the itemized checklist.)
|
||||
`AudioContext.decodeAudioData`, downmixes to mono, resamples to 16 kHz,
|
||||
transfers the `Float32Array` to the worker. For multilingual models the
|
||||
`transcribe` message also carries `language` (or auto-detect) and `task`
|
||||
(transcribe | translate).
|
||||
(transcribe | translate). For every model it may carry `prompt`: the
|
||||
optional "Names and terms" hint (`VocabularyHint`, stored in localStorage
|
||||
by `lib/hint.ts`), raw as typed (D-WS2a).
|
||||
5. Worker runs the pipeline (chunked, with timestamps), streaming partial results
|
||||
back via `{type:"partial"}` messages; final result as `{type:"result"}`.
|
||||
A hint becomes `<|startofprev|> hint.` once per pass (`worker/prompt.ts`,
|
||||
cut to Whisper's 223-token prompt budget at a whole word) and is prepended
|
||||
to the decoder input of every 30 s window by a wrapper around the model's
|
||||
`generate()`, which also strips it from the output again (D-WS2a).
|
||||
Live passes (4.1b) carry the same `prompt`.
|
||||
|
||||
### 4.1b Live microphone transcription (anonymous, browser engine — D-016)
|
||||
1. `LiveControls` starts a session: `getUserMedia` → `AudioContext` at the
|
||||
@@ -204,7 +213,11 @@ environment). See `WorkBreakdown.md` for the itemized checklist.)
|
||||
(`server-py/`) that Bun spawns and supervises (D-013; faster-whisper has no
|
||||
browser/WASM build, so it is server-only — the browser stays on Transformers.js).
|
||||
`src/server/transcribe.ts` bridges the sidecar's segment stream to the streamed
|
||||
JSON-lines HTTP response (D-011). The shared catalog maps model ids to
|
||||
JSON-lines HTTP response (D-011). `language`, `task` and the vocabulary hint
|
||||
(`prompt`) travel as query params (also on the live WebSocket) and are
|
||||
validated by `parseLanguageAndTask` (D-032, D-WS2a); the sidecar cuts the
|
||||
hint with the same rule as the browser and passes it to faster-whisper
|
||||
(D-WS2a). The shared catalog maps model ids to
|
||||
`Systran/faster-whisper-*` CTranslate2 conversions. If the sidecar is absent,
|
||||
the server engine reports unavailable; the browser/anonymous flow is unaffected.
|
||||
4. Both engines implement the same `Engine` interface in `src/lib/engine.ts`, so the
|
||||
|
||||
Reference in New Issue
Block a user