Record D-WS7a: Parakeet runtime decision, engine path and CPU measurements vs Large v3 Turbo

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Lucas WintherandClaude Opus 5.5 committed 2026-10-29 22:32:22 +01:00
1 parent 12e0f6678b
commit 9cd92b4920
6 files changed
+257 -5

No files matched your search

+37 -4
View File
@@ -68,11 +68,15 @@ src/
audio.ts # decode + resample to 16 kHz mono Float32Array (Web Audio API)
live.ts # live-mode segment loop: RMS gate, LiveSegmenter, LiveTranscriber (pure, D-016)
mic.ts # microphone capture: AudioWorklet chunker (Blob URL) + fallback (D-016)
models.ts # model catalog: ids, sizes, dtype defaults per device, languages
models.ts # model catalog: ids, family (whisper|parakeet), sizes, dtype defaults per device, languages
parakeet.ts # pure Parakeet logic: ≤30 s window plan, words→chunks, vocab tokenizer, pinned URLs
modelCache.ts # "downloaded?" per model via Cache Storage (D-005; reused by M3 offline indicator)
engine.ts # engine abstraction: BrowserEngine | ServerEngine
worker/
inference.worker.ts # Transformers.js pipeline, WebNN/WebGPU/WASM selection
inference.worker.ts # Transformers.js pipeline, WebNN/WebGPU/WASM selection; dispatches on catalog `family`
parakeetEngine.ts # Parakeet TDT path: parakeet.js ParakeetModel on the shared onnxruntime-web (D-WS7a)
modelFiles.ts # Cache-API weight download for non-Transformers.js paths (same cache, same progress msgs)
ortShim.ts # re-exports onnxruntime-web/webgpu; parakeet.js's own ort import is aliased here
protocol.ts # typed postMessage messages (shared with UI)
pwa/
sw.ts # service worker (app-shell precache, /ort runtime cache)
@@ -217,11 +221,40 @@ environment). See `WorkBreakdown.md` for the itemized checklist.)
- Anonymous users are never blocked from browser inference — auth only unlocks the
server engine.
### 4.1c Parakeet (English-only, desktop-only browser model — D-WS7a)
Catalog entries carry `family?: "whisper" | "parakeet"` (absent ⇒ whisper). For
`family: "parakeet"` the worker skips the Transformers.js pipeline:
1. **Load:** `modelFiles.ts` fetches `vocab.txt`, the decoder_joint and the
encoder from the Hub (revision-pinned URLs, `lib/parakeet.ts`) into the same
`transformers-cache` Cache Storage cache Transformers.js uses, posting the same
`load-progress` messages — so badges, the storage panel, delete and
ModelLoadProgress work unchanged. A file is cached only once complete.
Devices: `parakeetDeviceAttempts` — WASM int8 unless WebGPU is chosen
explicitly (fp16 encoder on WebGPU + int8 decoder on WASM, needs
`shader-f16`); the D-025 restart rule applies.
2. **Runtime:** `parakeetEngine.ts` builds parakeet.js's `ParakeetModel` from our
own sessions on `onnxruntime-web/webgpu` — the module Transformers.js imports,
so there is one runtime and one `/ort/` wasm. parakeet.js's `initOrt` (own ort
copy, CDN wasm path) is never called; `scripts/ortAlias.ts` points its import
at `worker/ortShim.ts` so the bundle carries one ort (`tests/workerBundle.test.ts`).
3. **Transcribe:** audio is planned into ≤ 30 s windows cut at the quietest 100 ms
near each limit (`subarray` views, no copies); each window's word timestamps
become sentence-sized `TranscriptChunk`s with absolute times. File passes post
one `partial` + `transcribe-progress` per window; live passes resolve via
`result` like Whisper's, so `LiveTranscriber`, TranscriptView and export are
untouched. `language`/`task` are ignored (English only).
4. **UI:** the picker lists it under English-only with a one-line note, hides it on
phones/tablets (`modelsForDevice`), and the server-engine list never includes
it (no faster-whisper name). With it selected, a note replaces the language/task
controls.
## 5. Invariants
- **Privacy:** browser-engine audio and transcripts never touch the network.
- **Anonymous-first:** every feature except the server engine works logged out.
- **Offline-first:** app shell + previously-downloaded models must work offline.
- **One browser inference codebase:** Transformers.js in the browser; the server
engine runs faster-whisper (D-013). The model catalog stays shared across both.
- **One browser inference runtime:** onnxruntime-web, driven by Transformers.js for
Whisper and by parakeet.js for Parakeet (D-WS7a) — the same ort instance, never a
second runtime; the server engine runs faster-whisper (D-013). The model catalog
stays shared across both engines (browser-only entries have no faster-whisper name).
- **Worker isolation:** inference never runs on the main thread.