whisper_web/tests/offline.md

Manual offline verification (M3)

The service worker, precache injection, and production serving are covered by unit tests and curl smoke tests; actual offline behavior (SW lifecycle, Cache Storage, airplane mode) can only be exercised in a real browser. This is that procedure. Re-run it after any change to src/pwa/*, scripts/build.ts, or the serve paths in src/index.ts.

Prerequisites

• Chrome or Edge (WebGPU + full PWA support; Firefox works for the WASM/offline parts but not WebGPU as of mid-2026).
• Production build served locally:

  bun run build
  bun run start        # NODE_ENV=production bun src/index.ts → http://localhost:3000
  

  Service workers require a secure context; http://localhost qualifies.

1. First visit — install + precache

1. Open http://localhost:3000/ in a fresh profile (or after DevTools → Application → Storage → "Clear site data").
2. DevTools → Application → Service Workers: /sw.js should be activated and running. No reload prompt should appear on this first install (register.ts only reloads after the user accepts the update toast).
3. Application → Cache Storage: expect
   • whisper-shell-<version> — the precached app shell: /, the hashed chunk-*.js / chunk-*.css, /inference.worker.js, /manifest.webmanifest (+ a hashed copy), /icons/icon-192.png, /icons/icon-512.png (+ a hashed 192 copy). The <version> matches the "cache version" printed by bun run build.
   • whisper-runtime-<version> appears only after step 2 below (it holds /ort/* fetched during model load).
4. The ModelPicker status line should read "The app is cached for offline use. Load a model once to make it available offline too."
5. Installability: the browser should offer an install icon in the address bar (manifest + SW + icons). Optional: install and confirm standalone display.

Result: Passed

2. Load a model once (online)

1. Select a model (tiny.en is fastest) and let it load fully.
2. Application → Cache Storage now also contains:
   • transformers-cache — Whisper weights (.onnx, config/tokenizer JSON), written by Transformers.js itself, never touched by our SW (D-005).
   • whisper-runtime-<version> — /ort/ort-wasm-simd-threaded*.{wasm,mjs} (onnxruntime-web loads one variant per backend, e.g. the .jsep build for WebGPU; the worker points wasmPaths at /ort/ so nothing hits the jsdelivr CDN).
3. The model now shows the "available offline" badge in ModelPicker, and the status line switches to "Offline ready — …".

Result: Works with WebGPU, however there was a WASM problem: When trying to run Whisper Tiny with Backend = WASM (CPU):

Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: qdq_actions.cc:137
TransposeDQWeightsForMatMulNBits Missing required scale:
model.decoder.embed_tokens.weight_merged_0_scale for node:
model.decoder.embed_tokens.weight_transposed_DequantizeLinear

Diagnosed & fixed (2026-07-10, D-014): a level-2 graph-optimizer bug in the onnxruntime-web build bundled with Transformers.js 4.2, triggered by the 8-bit decoders' tied embed_tokens weight — not a model or cache problem. The worker now caps WASM sessions at graphOptimizationLevel: "basic" (src/worker/inference.worker.ts), verified against the same ort-web build outside the browser. Re-run this step with Backend = WASM to confirm in a real browser.

Update (2026-10-30, D-WS1c): the Transformers.js 4.3.2 bump brings an onnxruntime-web build without that bug, so the cap is gone and WASM sessions run at the default level again. Re-run this step with Backend = WASM (Whisper Tiny and Tiny.en) to confirm the session still loads in a real browser.

3. Go offline and transcribe

1. DevTools → Network → throttling Offline (or OS airplane mode — stronger, also proves nothing relies on localhost). Optionally also stop bun run start to prove the app survives our server being down.
2. Reload the page. The app shell must render normally (served from whisper-shell-*, cache-first). Also try a deep link like http://localhost:3000/foo — the SW falls back to the cached shell for navigations.
3. Load the previously-downloaded model. It must become ready with no network (weights from transformers-cache, ort runtime from whisper-runtime-*). The DevTools Network tab should show every request answered by the service worker / cache, zero failed fetches escaping to the network.
4. Drop a local audio file and confirm a complete transcription offline.
5. Still offline, confirm a not-downloaded model fails gracefully (error surfaced, app usable) rather than hanging.

Result: Works.

4. Update flow (toast)

1. Back online, with the old tab still open: make any visible change (e.g. edit a string in src/App.tsx), then bun run build and restart bun run start. The cache version in sw.js changes.
2. Reload the tab once so the browser re-checks /sw.js (served Cache-Control: no-cache). When the new SW reaches "installed/waiting", the toast "A new version of Whisper Web is available." appears.
3. "Later" dismisses it; the old version keeps running.
4. "Reload" posts SKIP_WAITING, the new SW activates, and the page reloads itself once. After activation, Cache Storage must contain only the new whisper-shell-* / whisper-runtime-* pair — old versions deleted — while transformers-cache is left intact (model still "available offline").

Result: Works

Recording the result

When this passes in a real browser, flip the [~] offline-verification item in WorkBreakdown.md M3 to [x] and note browser + version here.

• Verified on: browser/version, date