whisper_web/tests/offline.md

Manual offline verification (M3)

The service worker, precache injection, and production serving are covered by unit tests and curl smoke tests; actual offline behavior (SW lifecycle, Cache Storage, airplane mode) can only be exercised in a real browser. This is that procedure. Re-run it after any change to src/pwa/*, scripts/build.ts, or the serve paths in src/index.ts.

Prerequisites

• Chrome or Edge (WebGPU + full PWA support; Firefox works for the WASM/offline parts but not WebGPU as of mid-2026).
• Production build served locally:

  bun run build
  bun run start        # NODE_ENV=production bun src/index.ts → http://localhost:3000
  

  Service workers require a secure context; http://localhost qualifies.

1. First visit — install + precache

1. Open http://localhost:3000/ in a fresh profile (or after DevTools → Application → Storage → "Clear site data").
2. DevTools → Application → Service Workers: /sw.js should be activated and running. No reload prompt should appear on this first install (register.ts only reloads after the user accepts the update toast).
3. Application → Cache Storage: expect
   • whisper-shell-<version> — the precached app shell: /, the hashed chunk-*.js / chunk-*.css, /inference.worker.js, /manifest.webmanifest (+ a hashed copy), /icons/icon-192.png, /icons/icon-512.png (+ a hashed 192 copy). The <version> matches the "cache version" printed by bun run build.
   • whisper-runtime-<version> appears only after step 2 below (it holds /ort/* fetched during model load).
4. The ModelPicker status line should read "The app is cached for offline use. Load a model once to make it available offline too."
5. Installability: the browser should offer an install icon in the address bar (manifest + SW + icons). Optional: install and confirm standalone display.

Result: Passed

2. Load a model once (online)

1. Select a model (tiny.en is fastest) and let it load fully.
2. Application → Cache Storage now also contains:
   • transformers-cache — Whisper weights (.onnx, config/tokenizer JSON), written by Transformers.js itself, never touched by our SW (D-005).
   • whisper-runtime-<version> — /ort/ort-wasm-simd-threaded*.{wasm,mjs} (onnxruntime-web loads one variant per backend, e.g. the .jsep build for WebGPU; the worker points wasmPaths at /ort/ so nothing hits the jsdelivr CDN).
3. The model now shows the "available offline" badge in ModelPicker, and the status line switches to "Offline ready — …".

Result: Works with WebGPU, however there was a WASM problem: When trying to run Whisper Tiny with Backend = WASM (CPU):

Can't create a session. ERROR_CODE: 1, ERROR_MESSAGE: qdq_actions.cc:137
TransposeDQWeightsForMatMulNBits Missing required scale:
model.decoder.embed_tokens.weight_merged_0_scale for node:
model.decoder.embed_tokens.weight_transposed_DequantizeLinear

Diagnosed & fixed (2026-07-10, D-014): a level-2 graph-optimizer bug in the onnxruntime-web build bundled with Transformers.js 4.2, triggered by the 8-bit decoders' tied embed_tokens weight — not a model or cache problem. The worker now caps WASM sessions at graphOptimizationLevel: "basic" (src/worker/inference.worker.ts), verified against the same ort-web build outside the browser. Re-run this step with Backend = WASM to confirm in a real browser.

Update (2026-10-30, D-043): the Transformers.js 4.3.2 bump brings an onnxruntime-web build without that bug, so the cap is gone and WASM sessions run at the default level again. Re-run this step with Backend = WASM (Whisper Tiny and Tiny.en) to confirm the session still loads in a real browser.

3. Go offline and transcribe

1. DevTools → Network → throttling Offline (or OS airplane mode — stronger, also proves nothing relies on localhost). Optionally also stop bun run start to prove the app survives our server being down.
2. Reload the page. The app shell must render normally (served from whisper-shell-*, cache-first). Also try a deep link like http://localhost:3000/foo — the SW falls back to the cached shell for navigations.
3. Load the previously-downloaded model. It must become ready with no network (weights from transformers-cache, ort runtime from whisper-runtime-*). The DevTools Network tab should show every request answered by the service worker / cache, zero failed fetches escaping to the network.
4. Drop a local audio file and confirm a complete transcription offline.
5. Still offline, confirm a not-downloaded model fails gracefully (error surfaced, app usable) rather than hanging.

Result: Works.

4. Update flow (toast)

1. Back online, with the old tab still open: make any visible change (e.g. edit a string in src/App.tsx), then bun run build and restart bun run start. The cache version in sw.js changes.
2. Reload the tab once so the browser re-checks /sw.js (served Cache-Control: no-cache). When the new SW reaches "installed/waiting", the toast "A new version of Whisper Web is available." appears.
3. "Later" dismisses it; the old version keeps running.
4. "Reload" posts SKIP_WAITING, the new SW activates, and the page reloads itself once. After activation, Cache Storage must contain only the new whisper-shell-* / whisper-runtime-* pair — old versions deleted — while transformers-cache is left intact (model still "available offline").

Result: Works

5. Transcript translation offline (D-045)

1. Online: transcribe anything, open Translate transcript…, pick a direction, click Translate. The hint said "Downloads a translation model once (~113 MB)"; the status shows "Downloading the translation model… N%", then "Translating… n of m sentences", then a second card "Translation (English|Danish)" with .txt / .srt / .txt+ts exports named <file>.<en|da>.<ext>.
2. Cache Storage → transformers-cache now holds Xenova/opus-mt-… files; whisper-shell-* holds /translate.worker.js. The hint reads "Translation model downloaded — works offline".
3. Go offline (DevTools → Network → Offline), reload, transcribe with a cached Whisper model, translate again in the same direction: it must work with no network request. The other (never downloaded) direction must fail with "You appear to be offline…" and a Retry button, not hang.

Result: pending (needs a real browser)

6. "Skip silence" (long-form pipeline, D-049/D-050)

1. Online, open Advanced, tick Skip silence in uploaded files, and transcribe a file. DevTools → Network shows one request for onnx-community/silero-vad/…/onnx/model.onnx (2.2 MB); the console logs "speech-detection (VAD) model ready". The transcript arrives with progress moving in jumps over pauses. Repeat on WebGPU (the owner's backend) and note wall-clock against the same file with the option off.
2. Go offline (DevTools "Offline"), reload, transcribe again: no network request at all, same long-form path (console line as above).
3. Clear only the silero-vad entries from transformers-cache (Application → Cache Storage), stay offline, transcribe: still no network request; the console says the VAD model is unavailable and the file is transcribed the usual way.
4. Transcribe a file of pure silence or music with the option on: the transcript reads "No speech was found in this recording."

Result: pending (no real browser in the dev environment)

Recording the result

When this passes in a real browser, flip the [~] offline-verification item in WorkBreakdown.md M3 to [x] and note browser + version here.

• Verified on: browser/version, date