Fixed for decoder dl speed and mobile warnings on low memory devices.

docs/Decisions.md

@@ -617,3 +617,88 @@ warning thresholds).
 mobile crashes in practice, consider loosening the confirm thresholds; if
 onnxruntime-web ever gains a streaming/incremental session-creation API,
 drop the patch and re-measure before touching the thresholds again.
+**Amendment (2026-07-29, D-028):** "a little wall-clock time" undersold the
+cost — see D-028. The sequential patch no longer applies unconditionally;
+`options.sequentialSessionLoad` (set by `App.tsx`, true only when
+`shouldWarnBeforeLoad` is true for the load) now gates it.
+
+## D-028 — Sequential session loading gated to actual mobile-crash risk, not unconditional
+**Status:** Accepted (2026-07-29)
+Reported symptom: `encoder_model_q4.onnx` (Whisper Medium) downloaded far
+slower after D-027 landed than before it — reproduced on desktop too, ruling
+out phone CPU/network as the cause. Measured in Chrome DevTools: ~1.6 MB/s
+sustained on a connection that otherwise performs fine, downloading from
+Hugging Face's Xet-backed CDN (`us.aws.cdn.hf.co/xet-bridge-us/...`).
+**Cause.** D-027's `constructSessions` patch made the encoder and decoder
+downloads always sequential — one HTTP request in flight at a time — to keep
+both weight buffers from being resident together. On a long-RTT path (this
+CDN is US-hosted), a single HTTP/2 stream typically can't fill the available
+bandwidth; two concurrent requests (the pre-D-027 `Promise.all`) filled more
+of it. Serializing the downloads dropped every load back to one flow's
+throughput ceiling — not "a little" wall-clock cost as D-027 claimed, but
+multiple times slower on exactly this kind of connection, and paid by every
+user regardless of device.
+**Fix.** The sequential/parallel choice is now conditional, not baked into
+the patch: `constructSessions` branches on a new `options.sequentialSessionLoad`
+flag (`patches/@huggingface%[email protected]`) — parallel
+`Promise.all` (the original, fast behavior) when falsy/absent, the D-027
+for-loop when true. `App.tsx` computes this per load via `isMobileLoadRisk`
+(extracted from `confirmRiskyMobileLoad`'s existing hint/`estimatePeakLoadBytes`
+logic — the same `shouldWarnBeforeLoad` check the crash-warning dialog uses)
+and threads it through a new `sequentialSessionLoad` field on the worker's
+`LoadMessage` (`protocol.ts`) into the `pipeline()` call
+(`inference.worker.ts`). Crucially this is evaluated independently of the
+confirm dialog's answer — even a load the user clicked "Continue anyway" on
+still loads sequentially, since the memory risk that motivated D-027 doesn't
+go away just because the user accepted it.
+**Consequence:** desktop/laptop loads (and any mobile load under the
+size/`deviceMemory` risk threshold) get back the full concurrent-download
+speed that predates D-027; only loads that would already trigger the mobile
+crash warning pay the sequential-loading wall-clock cost, which is exactly
+where D-027's OOM fix is actually needed.
+**Revisit:** if Hugging Face's Xet CDN behavior changes, or if a future model
+in the catalog is large enough to risk mobile OOM at a size where the
+confirm-dialog threshold doesn't yet trigger, re-check that the two
+thresholds (`shouldWarnBeforeLoad`'s and this gate's) are meant to stay
+identical.
+
+## D-029 — Mobile crash warning was silently disabled on every phone but the lowest-end
+**Status:** Accepted (2026-07-29)
+Reported symptom: the D-027 mobile crash `window.confirm()` never appeared
+for Whisper Medium (or any model), even on devices it's explicitly meant to
+protect.
+**Cause.** `shouldWarnBeforeLoad`'s `deviceMemory`-scaled branch used
+`SAFE_TAB_FRACTION_OF_DEVICE_MEMORY = 0.5`. Checked against the actual
+catalog (`estimatePeakLoadBytes`) for every `navigator.deviceMemory` value
+Chromium can report:
+
+| deviceMemory | "safe" budget (old 0.5) | Medium (556 MB) | Large v3 Turbo (762 MB) |
+|---|---|---|---|
+| 1 | 500 MB | warns | warns |
+| 2 | 1 GB | **silent** | **silent** |
+| 4 | 2 GB | **silent** | **silent** |
+| 8 | 4 GB | **silent** | **silent** |
+
+`deviceMemory` of 2 or more is ordinary today, not just flagship-tier — so
+in practice the warning only ever fired on the oldest/cheapest Android
+phones and on iOS (which lacks the API entirely and always used the flat
+400 MB fallback, D-027). The exact mid/low-range phones the original crash
+report was about — the whole reason D-027 exists — cleared the threshold and
+got no warning at all.
+**Fix:** `SAFE_TAB_FRACTION_OF_DEVICE_MEMORY` lowered to 0.1
+(`src/lib/mobileMemoryGuard.ts`). Re-checking the same table: deviceMemory
+2 and 4 now warn for Medium+ (200 MB / 400 MB budgets, both under 556 MB);
+deviceMemory 6-8 (600-800 MB budgets) still don't, treating genuine
+high-end devices as having real headroom. `tests/mobileMemoryGuard.test.ts`
+gained a regression case pinning deviceMemory 2/4 to `true` for Medium's
+peak — the exact gap this decision closes.
+**Alternatives rejected:** dropping the `deviceMemory` scaling entirely in
+favor of always using the flat 400 MB fallback (simpler, but throws away the
+one signal that lets genuinely high-RAM devices skip an unnecessary
+confirm); a hard absolute cap on top of the fraction (more robust long-term,
+but two knobs to calibrate against no real data beats one — revisit if a
+future model's peak lands awkwardly relative to the 0.1 fraction).
+**Revisit:** this is still a guess, same as D-027's original 0.5 was — there
+is no real crash-telemetry feedback loop. If field reports show 0.1 is still
+too generous (or too aggressive) for some device class, adjust the constant
+rather than the shape of the heuristic.

patches/@huggingface%2Ftransformers@4.2.0.patch

@@ -1,8 +1,11 @@
+diff --git a/node_modules/@huggingface/transformers/.bun-tag-8719566ebeed846f b/.bun-tag-8719566ebeed846f
+new file mode 100644
+index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
 diff --git a/dist/transformers.web.js b/dist/transformers.web.js
-index bb7b32717a42475a5f66411a33d5e5964dfda3e2..799cb402e98d8d0d58bf59629c4c2992407477e0 100644
+index bb7b32717a42475a5f66411a33d5e5964dfda3e2..03de63967e99f8f3c847858621d72577ef037391 100644
 --- a/dist/transformers.web.js
 +++ b/dist/transformers.web.js
-@@ -18558,22 +18558,28 @@ async function getSession(pretrained_model_name_or_path, fileName, options, cach
+@@ -18558,22 +18558,49 @@ async function getSession(pretrained_model_name_or_path, fileName, options, cach
    return { buffer_or_path, session_options, session_config };
  }
  async function constructSessions(pretrained_model_name_or_path, names, options, cache_sessions = void 0) {
@@ -22,14 +25,35 @@ index bb7b32717a42475a5f66411a33d5e5964dfda3e2..799cb402e98d8d0d58bf59629c4c2992
 -      })
 -    )
 -  );
-+  // Patched (au_whisper_web): load sub-model sessions one at a time instead of
++  // Patched (au_whisper_web): each entry briefly holds a full-size decoded
-+  // Promise.all-ing them. Each entry briefly holds a full-size decoded model
++  // model buffer in JS-heap memory (on top of onnxruntime-web's own WASM-heap
-+  // buffer in JS-heap memory (on top of onnxruntime-web's own WASM-heap copy
++  // copy during session creation); loading Whisper's encoder and decoder
-+  // during session creation); loading Whisper's encoder and decoder
 +  // concurrently keeps both buffers alive at once and roughly doubles the
 +  // load-time memory peak, which is enough to OOM-crash mobile browsers on
-+  // larger models (e.g. medium, ~460 MB combined). Sequential loading lets the
++  // larger models (e.g. medium, ~460 MB combined). `options.sequentialSessionLoad`
-+  // previous entry's buffer become garbage before the next one is fetched.
++  // (set by the caller only when that risk is real, D-027/D-029) loads
++  // sub-model sessions one at a time instead of Promise.all-ing them, so the
++  // previous entry's buffer becomes garbage before the next one is fetched —
++  // at the cost of losing the download concurrency a fast connection could
++  // otherwise use, which is the default (and always was, pre-D-027).
++  if (!options?.sequentialSessionLoad) {
++    return Object.fromEntries(
++      await Promise.all(
++        Object.keys(names).map(async (name) => {
++          const cache_config = cache_sessions?.[name] ?? false;
++          const { buffer_or_path, session_options, session_config } = await getSession(
++            pretrained_model_name_or_path,
++            names[name],
++            options,
++            cache_config,
++            name
++          );
++          const session = await createInferenceSession(buffer_or_path, session_options, session_config);
++          return [name, session];
++        })
++      )
++    );
++  }
 +  const entries = [];
 +  for (const name of Object.keys(names)) {
 +    const cache_config = cache_sessions?.[name] ?? false;

src/App.tsx

@@ -213,6 +213,38 @@ export function App() {
     return deviceOverrides.current.get(overrideKey(forModelId, preference)) ?? preference;
   }
 
+  /** Mirrors ModelPicker's size-estimate fallback: until the worker reports an
+   * active device, assume WASM unless the user asked for an accelerator. */
+  function guessDevice(preference: DevicePreference): InferenceDevice {
+    return (
+      activeDevice ??
+      (preference === "webgpu" ? "webgpu" : preference === "webnn" ? "webnn" : "wasm")
+    );
+  }
+
+  /**
+   * Whether loading `forModelId` on `device` risks crashing a mobile tab
+   * (D-027) — independent of whether the user has already been asked. Used
+   * both to decide whether to show the confirm dialog and, regardless of the
+   * user's answer, to decide whether the worker should load sub-model
+   * sessions sequentially (memory-safe but slower) or in parallel (fast,
+   * safe everywhere this isn't true).
+   */
+  function isMobileLoadRisk(forModelId: string, device: InferenceDevice): boolean {
+    const targetModel = getModel(forModelId);
+    if (!targetModel) return false;
+    const peakBytes = estimatePeakLoadBytes(targetModel, resolveDefaultDtype(forModelId, device));
+    const hint: MobileMemoryHint = {
+      userAgent: typeof navigator !== "undefined" ? navigator.userAgent : undefined,
+      maxTouchPoints: typeof navigator !== "undefined" ? navigator.maxTouchPoints : undefined,
+      deviceMemoryGB:
+        typeof navigator !== "undefined"
+          ? (navigator as Navigator & { deviceMemory?: number }).deviceMemory
+          : undefined,
+    };
+    return shouldWarnBeforeLoad(hint, peakBytes);
+  }
+
   /**
    * Confirm before a browser-engine load that risks crashing the tab on
    * mobile (D-027). Returns true if the load should proceed — either it
@@ -223,25 +255,12 @@ export function App() {
   function confirmRiskyMobileLoad(forModelId: string, preference: DevicePreference): boolean {
     const targetModel = getModel(forModelId);
     if (!targetModel) return true;
-    // Mirrors ModelPicker's size-estimate fallback: until the worker reports
+    const device = guessDevice(preference);
-    // an active device, assume WASM unless the user asked for an accelerator.
-    const device: InferenceDevice =
-      activeDevice ??
-      (preference === "webgpu" ? "webgpu" : preference === "webnn" ? "webnn" : "wasm");
     const key = overrideKey(forModelId, device);
     if (mobileWarningAcknowledged.current.has(key)) return true;
+    if (!isMobileLoadRisk(forModelId, device)) return true;
 
     const peakBytes = estimatePeakLoadBytes(targetModel, resolveDefaultDtype(forModelId, device));
-    const hint: MobileMemoryHint = {
-      userAgent: typeof navigator !== "undefined" ? navigator.userAgent : undefined,
-      maxTouchPoints: typeof navigator !== "undefined" ? navigator.maxTouchPoints : undefined,
-      deviceMemoryGB:
-        typeof navigator !== "undefined"
-          ? (navigator as Navigator & { deviceMemory?: number }).deviceMemory
-          : undefined,
-    };
-    if (!shouldWarnBeforeLoad(hint, peakBytes)) return true;
-
     const proceed = window.confirm(
       `${targetModel.displayName} needs roughly ${formatModelSize(peakBytes)} of memory at ` +
         "once to load — more than many phones and tablets allow a browser tab to use " +
@@ -379,6 +398,7 @@ export function App() {
             type: "load",
             modelId: message.modelId,
             device: message.device,
+            sequentialSessionLoad: isMobileLoadRisk(message.modelId, message.device),
           });
           break;
         }
@@ -417,10 +437,16 @@ export function App() {
     setLoadStarted(true);
     setError(null);
     newLoadGate();
+    const device = effectiveDevice(nextModelId, preference);
     postToWorker(worker, {
       type: "load",
       modelId: nextModelId,
-      device: effectiveDevice(nextModelId, preference),
+      device,
+      // D-027/D-029: sub-model sessions load sequentially (memory-safe, but
+      // slower — halves the CDN concurrency a fast connection could otherwise
+      // use) only where the peak-memory crash risk is real; everywhere else
+      // (desktop, low-risk models) the worker loads them in parallel.
+      sequentialSessionLoad: isMobileLoadRisk(nextModelId, guessDevice(device)),
     });
     return worker;
   }
@@ -540,10 +566,12 @@ export function App() {
       if (!loadStarted) {
         startLoad(modelId, devicePreference);
       } else {
+        const device = effectiveDevice(modelId, devicePreference);
         postToWorker(workerRef.current!, {
           type: "load",
           modelId,
-          device: effectiveDevice(modelId, devicePreference),
+          device,
+          sequentialSessionLoad: isMobileLoadRisk(modelId, guessDevice(device)),
         });
       }
     }
@@ -788,10 +816,12 @@ export function App() {
     if (!loadStarted) {
       startLoad(modelId, devicePreference);
     } else {
+      const device = effectiveDevice(modelId, devicePreference);
       postToWorker(worker, {
         type: "load",
         modelId,
-        device: effectiveDevice(modelId, devicePreference),
+        device,
+        sequentialSessionLoad: isMobileLoadRisk(modelId, guessDevice(device)),
       });
     }
 

src/lib/mobileMemoryGuard.ts

@@ -52,11 +52,18 @@ export function isLikelyMobile(
 
 /**
  * Fraction of reported device RAM a single tab can safely peak at before
- * mobile OSes intervene. Conservative on purpose — `deviceMemory` already
+ * mobile OSes intervene.
- * caps at 8 regardless of actual RAM, and the tab competes with the rest of
+ *
- * the OS, the browser process itself, and other open tabs.
+ * D-029: the original 0.5 was not conservative enough — `deviceMemory` is
+ * commonly 2-4 on ordinary (not just flagship) Android phones today, and at
+ * 0.5 that already yields a 1-2 GB "safe" budget, clearing every model in
+ * the catalog including Medium (~556 MB peak) and Large v3 Turbo (~762 MB) —
+ * silently skipping the warning for exactly the phones D-027's crash report
+ * was about. 0.1 keeps mid/low-end phones (deviceMemory ≤ 4) warned for
+ * Medium+ while still exempting genuine high-end devices (deviceMemory ≥ 6,
+ * 600+ MB budget) that have real headroom to spare.
  */
-const SAFE_TAB_FRACTION_OF_DEVICE_MEMORY = 0.5;
+const SAFE_TAB_FRACTION_OF_DEVICE_MEMORY = 0.1;
 
 /**
  * Fallback risk threshold when `deviceMemory` isn't reported (notably all of

src/worker/inference.worker.ts

@@ -13,6 +13,7 @@ import {
   pipeline,
   WhisperTextStreamer,
   type AutomaticSpeechRecognitionPipeline,
+  type PretrainedModelOptions,
   type ProgressInfo,
 } from "@huggingface/transformers";
 import { resolveDefaultDtype } from "../lib/models";
@@ -256,7 +257,7 @@ async function createPipeline(
     configureWasmThreads(device);
     const dtype = message.dtype ?? resolveDefaultDtype(message.modelId, device);
     try {
-      const asr = await pipeline("automatic-speech-recognition", message.modelId, {
+      const pipelineOptions: PretrainedModelOptions & { sequentialSessionLoad?: boolean } = {
         device,
         dtype,
         // WASM sessions cap graph optimization at "basic" (D-014): the level-2
@@ -268,7 +269,12 @@ async function createPipeline(
           session_options: { graphOptimizationLevel: "basic" as const },
         }),
         progress_callback: forwardProgress,
-      });
+        // D-027/D-029: sequential sub-model session loading trades load speed
+        // for a lower peak-memory footprint — only worth it where the UI
+        // determined this load risks crashing a mobile tab.
+        sequentialSessionLoad: message.sequentialSessionLoad ?? false,
+      };
+      const asr = await pipeline("automatic-speech-recognition", message.modelId, pipelineOptions);
       // What ort actually ended up with (it clamps to 1 without isolation) —
       // surfaced to the UI so degraded threading is visible in the field.
       const threads = env.backends.onnx.wasm?.numThreads ?? 1;

src/worker/protocol.ts

@@ -47,6 +47,16 @@ export interface LoadMessage {
    * default for the device it actually selected (src/lib/models.ts).
    */
   dtype?: DtypeSpec;
+  /**
+   * Load the encoder/decoder ONNX sessions one at a time instead of
+   * concurrently (D-027/D-029). Sequential halves the peak JS-heap/WASM
+   * memory a load needs (avoiding mobile tab crashes on larger models) at
+   * the cost of losing the CDN-concurrency benefit a fast connection would
+   * otherwise get from two simultaneous downloads — worth paying only where
+   * the memory-crash risk is real. The UI computes this (`isMobileLoadRisk`
+   * in App.tsx); omitted/false loads in parallel.
+   */
+  sequentialSessionLoad?: boolean;
 }
 
 export interface TranscribeMessage {

tests/mobileMemoryGuard.test.ts

@@ -70,6 +70,20 @@ describe("shouldWarnBeforeLoad", () => {
     ).toBe(false);
   });
 
+  // D-029: deviceMemoryGB 2-4 is the ordinary (not just budget) Android
+  // range today — these must still warn for Medium+, the exact class of
+  // phone the original D-027 crash report was about.
+  test("warns on mobile mid-range devices (deviceMemory 2-4) for Medium+", () => {
+    for (const deviceMemoryGB of [2, 4]) {
+      expect(
+        shouldWarnBeforeLoad(
+          { userAgent: ANDROID_UA, maxTouchPoints: 5, deviceMemoryGB },
+          MEDIUM_PEAK_BYTES,
+        ),
+      ).toBe(true);
+    }
+  });
+
   test("falls back to a flat threshold when deviceMemory is unreported (iOS)", () => {
     expect(
       shouldWarnBeforeLoad(