Optimize max peak memory usage and add warnings on mobile for higher models

docs/Decisions.md

@@ -539,3 +539,81 @@ more than it saves. The check is one line, logged, scoped to `"auto"`, and
 targets the one engine with field evidence.
 **Revisit:** on any onnxruntime-web bump — retest explicit WebGPU on Firefox
 (file `tests/live.md` note) and drop the check when it reaches parity.
+
+## D-027 — Mobile OOM crash on Medium+ loads: sequential session loading (patched) + a warn/confirm gate
+**Status:** Accepted (2026-07-28)
+Reported symptom: downloading Whisper Medium (English) on a phone crashed the
+whole browser — on Android, badly enough to briefly kick the OS back to the
+home screen (consistent with the low-memory killer reaping background
+processes, including the launcher, once the tab's memory spike triggered
+system-wide pressure).
+**Cause.** Whisper Medium's smallest available dtype on any device is q4
+(D-022) — still ~460 MB combined encoder+decoder weights. Two independent,
+compounding sources of transient memory at load time:
+1. `@huggingface/transformers` buffers each ONNX weight file fully into one
+   JS-heap `Uint8Array` (`readResponse`/`loadResourceFile`,
+   `src/utils/hub.js`/`hub/utils.js`) before it is cached or handed to
+   onnxruntime — unavoidable either way (with or without a
+   `progress_callback`, and regardless of whether a URL string or a buffer
+   is passed to `InferenceSession.create`): onnxruntime-web's WASM backend
+   has no streaming ONNX parser, so the whole file must exist as one
+   contiguous buffer to construct a session.
+2. `constructSessions` (`src/models/session.js`) loaded the encoder and
+   decoder sessions **concurrently** via `Promise.all`, so both full-size
+   JS buffers were resident at once — needlessly roughly doubling the
+   already-large per-file peak (on top of onnxruntime-web's own
+   WASM-heap `_malloc` + `HEAPU8.set` copy inside `copyFromExternalBuffer`,
+   made once per file at session-creation time, which is itself already
+   serialized across sessions in web environments).
+Net: peak transient memory for Medium approached ~2-3x the ~460 MB combined
+weight size — comfortably inside WASM32's 4 GiB address space, but well
+past the much tighter per-tab memory budgets mobile Safari/Chrome enforce,
+which respond to going over budget with an outright, uncatchable tab kill
+(no JS `error` event fires — `worker.onerror` in `App.tsx` never sees it).
+There was no mobile detection anywhere in the app: Medium downloaded/loaded
+identically on a phone and a workstation.
+**Fix (two parts):**
+1. **Sequential session loading (real fix, no UX cost).** `constructSessions`
+   is patched (`bun patch`, `patches/@huggingface%2Ftransformers@4.2.0.patch`,
+   tracked via `patchedDependencies` in `package.json` so it survives
+   `bun install`) to load sub-model sessions one at a time instead of via
+   `Promise.all`. This lets the previous file's JS buffer become garbage
+   before the next file is fetched, roughly halving the load-time peak for
+   any multi-file model (Whisper's encoder+decoder) — a strict improvement
+   with no downside, so it applies unconditionally, not just on mobile.
+   Investigated and rejected as *not* actionable: skipping the
+   `progress_callback`, `env.customCache`, and passing a URL string instead
+   of a buffer to onnxruntime-web — all three still end up materializing the
+   whole file, per the analysis above. Fixing that fully would require
+   forking onnxruntime-web itself (out of scope).
+2. **Warn/confirm gate on mobile (`src/lib/mobileMemoryGuard.ts`,
+   `estimatePeakLoadBytes` in `src/lib/models.ts`).** Even halved, Medium's
+   peak (~555 MB estimated: `2 × max(encoderBytes, decoderBytes)`, reflecting
+   the now-sequential loading) can still exceed a constrained phone's tab
+   budget. `App.tsx`'s `confirmRiskyMobileLoad` gates all three ways a
+   browser-engine load can start — the explicit Download button, and the
+   lazy auto-load on file-drop / live-start (D-020) — behind
+   `shouldWarnBeforeLoad`: mobile UA detection (Android/iPhone/iPod, plus
+   iPadOS's desktop-masquerading Safari UA via touch-point count, same
+   UA-sniffing rationale as D-026 — no memory-pressure API exists to probe
+   instead) combined with `navigator.deviceMemory` when Chromium reports it,
+   falling back to a flat ~400 MB threshold where it doesn't (all of
+   iOS/Safari). A native `window.confirm()` states the estimated memory need
+   and the crash risk in plain language; declining aborts the load with no
+   state changes, accepting remembers the modelId+device combination for the
+   rest of the session so the same load doesn't ask twice. Desktop/laptop
+   browsers are never gated — no crash evidence exists there.
+**Alternatives rejected:** hard-blocking Medium+ on mobile with no override
+(some phones have enough RAM to be fine; the confirm lets the user judge
+their own device); forking onnxruntime-web for a streaming session-creation
+path (large undertaking for a runtime we don't own upstream).
+**Consequence:** every model load is now faster-to-safe (sequential
+sessions cost a little wall-clock time — files download one after another
+instead of in parallel — in exchange for materially lower peak memory).
+Mobile users choosing Medium/Large now see one extra confirmation step;
+Tiny/Base/Small are unaffected (their estimated peaks stay under the
+warning thresholds).
+**Revisit:** if field reports show the sequential-load patch alone resolves
+mobile crashes in practice, consider loosening the confirm thresholds; if
+onnxruntime-web ever gains a streaming/incremental session-creation API,
+drop the patch and re-measure before touching the thresholds again.

docs/WorkBreakdown.md

@@ -255,6 +255,15 @@ Notes (M5):
       WebGPU EP on Firefox completes but runs far slower than WASM, so
       "auto" skips WebGPU on Firefox (D-026, explicit selection still
       honored — retest on ort bumps).
+- [x] Mobile OOM crash on Medium+ model loads (D-027) — patched
+      `@huggingface/transformers` (`bun patch`) to load the encoder/decoder
+      sessions sequentially instead of concurrently, roughly halving
+      load-time peak memory for every model; added a mobile UA +
+      `deviceMemory` warn/confirm gate (`src/lib/mobileMemoryGuard.ts`)
+      before any browser-engine load whose estimated peak
+      (`estimatePeakLoadBytes`) is risky on a phone/tablet, covering the
+      explicit Download button and the lazy auto-load on file-drop/live-start
+      alike.
 - [ ] Basic a11y pass (keyboard, focus, aria on progress/live transcript)
 - [ ] Production build + `bun start` deployment check; README for deployers