Better tests and more user feedback while doing transcription; WIP.
This commit is contained in:
1 parent
18acead43c
commit
be82ecfe57
19 files changed
+447
-138
No files matched your search
+13
-9
@@ -24,7 +24,7 @@ Logged-in users may optionally route inference to the server.
|
||||
│ ▼ │
|
||||
│ Inference Web Worker │
|
||||
│ ├─ Transformers.js ASR pipeline (Whisper) │
|
||||
│ ├─ Device selection: WebGPU → WASM fallback │
|
||||
│ ├─ Device selection: WebNN → WebGPU → WASM fallback │
|
||||
│ └─ Model weights cached via browser Cache API │
|
||||
│ │
|
||||
│ Service Worker (PWA) │
|
||||
@@ -72,7 +72,7 @@ src/
|
||||
modelCache.ts # "downloaded?" per model via Cache Storage (D-005; reused by M3 offline indicator)
|
||||
engine.ts # engine abstraction: BrowserEngine | ServerEngine
|
||||
worker/
|
||||
inference.worker.ts # Transformers.js pipeline, WebGPU/WASM selection
|
||||
inference.worker.ts # Transformers.js pipeline, WebNN/WebGPU/WASM selection
|
||||
protocol.ts # typed postMessage messages (shared with UI)
|
||||
pwa/
|
||||
sw.ts # service worker (app-shell precache, /ort runtime cache)
|
||||
@@ -89,7 +89,8 @@ tests/ # bun test
|
||||
```
|
||||
|
||||
(Current state: M0–M3 built — browser transcription with the curated model
|
||||
catalog, WebGPU→WASM auto-selection, model switching, language/task options,
|
||||
catalog, WebNN→WebGPU→WASM auto-selection (D-017), model switching,
|
||||
language/task options,
|
||||
and the installable offline PWA (`pwa/`, `scripts/build.ts` production build,
|
||||
`dist/` serving in `src/index.ts`; browser-level offline behavior pending the
|
||||
manual run in `tests/offline.md`). M4 accounts are built (`server/db.ts`,
|
||||
@@ -108,12 +109,15 @@ are the remaining M5 work; see `WorkBreakdown.md`.)
|
||||
1. User picks a model in `ModelPicker` (curated catalog from `lib/models.ts`, with
|
||||
size estimates and a Cache-API-backed "downloaded" badge); UI posts
|
||||
`{type:"load", modelId, device}` where `device` is a preference
|
||||
(`"auto" | "webgpu" | "wasm"`, default `"auto"`). An explicit `dtype` override
|
||||
is expressible in the message but normally omitted.
|
||||
2. Worker resolves the device: `navigator.gpu?.requestAdapter()` → `"webgpu"` if an
|
||||
adapter exists (and pipeline creation succeeds — otherwise it retries on WASM),
|
||||
else `"wasm"` (D-004). It then resolves the catalog's dtype default for that
|
||||
device and creates the Transformers.js `automatic-speech-recognition` pipeline.
|
||||
(`"auto" | "webnn" | "webgpu" | "wasm"`, default `"auto"`). An explicit `dtype`
|
||||
override is expressible in the message but normally omitted.
|
||||
2. Worker resolves the device (D-004, D-017): for `"auto"` it probes
|
||||
`navigator.ml` (WebNN) first, then `navigator.gpu?.requestAdapter()` (WebGPU),
|
||||
and picks in priority order WebNN → WebGPU → WASM; if pipeline creation on the
|
||||
picked device throws, it retries down the same chain (WASM always last). An
|
||||
explicit `"webnn"`/`"webgpu"` preference falls back straight to WASM. It then
|
||||
resolves the catalog's dtype default for the device actually used and creates
|
||||
the Transformers.js `automatic-speech-recognition` pipeline.
|
||||
Download progress events are forwarded to the UI; the terminal
|
||||
`{status:"ready", device}` message reports the backend actually selected, which
|
||||
the UI surfaces. Loading a different model/device disposes the old pipeline
|
||||
|
||||
Reference in new issue
Block a user