Focus on speedup and not hanging on large audio files.

docs/Decisions.md

@@ -294,3 +294,34 @@ download is a full pipeline load, so it also warms up inference and surfaces the
 available), `ModelPicker` states explicitly that nothing is downloaded to the
 device; the Download button stays (with a matching tooltip) for on-device /
 offline preparation.
+
+## D-021 — Long-file hardening: no idle timeout on the transcribe stream, heartbeats, single-pass Whisper generate
+**Status:** Accepted (2026-07-15)
+Long uploads (tested: 15-min Opus, `tests/audio_files/en_alien.opus`) hung or died
+on both engines. Three independent causes, all length-dependent:
+1. **Bun's default 10 s `idleTimeout` reset `/api/transcribe` mid-stream.** Segment
+   gaps legitimately exceed 10 s (sidecar spawn + model load/download before the
+   first segment; slow decode windows — 30 s+ gaps observed with large-v3-turbo on
+   CPU). The stream then ended without a `result` line, so the client reported
+   "connection was lost" after minutes of apparent hang. Fix: `src/index.ts` calls
+   `server.timeout(request, 0)` for `POST /api/transcribe` (and 90 s for
+   `GET /api/transcribe/status`, whose first probe waits up to 60 s for the sidecar
+   import), and `src/server/transcribe.ts` enqueues a blank ndjson line every 5 s as
+   a heartbeat — reverse proxies get liveness, dead connections surface as enqueue
+   failures, and `ServerEngine` already skips blank lines.
+2. **Bun's default 128 MB `maxRequestBodySize` rejected uploads > ~35 min** (raw
+   f32 PCM is 3.84 MB/min) before the route's own 512 MB limit ever applied. Fix:
+   `maxRequestBodySize` is set to the shared `MAX_UPLOAD_BYTES` export.
+3. **Browser engine: Transformers.js 4.2's Whisper `generate()` takes a seek-loop
+   path when `return_timestamps` is set without `max_new_tokens`** — 1–3 full decode
+   passes per window, redundant under the pipeline's own 30 s/5 s chunking. Measured
+   on the 15-min clip (tiny.en, onnxruntime-node): 80 generate passes for 44 windows,
+   1.4× slower, WER 44.7 % vs 31.5 % (duplicated boundary text), and the streamer's
+   `on_finalize` fired per pass instead of per window, so `TranscribeProgressTracker`
+   saturated the progress bar at ~60 % of the run and the UI looked frozen for the
+   remainder (on WASM: many minutes). A window whose last boundary timestamp is 0 can
+   even loop forever. Fix: the worker passes `max_new_tokens: 440`, which selects the
+   classic single-pass-per-window path (440 ≈ 448 decoder positions minus prompt).
+**Revisit:** when Transformers.js changes the seek-loop condition (currently
+`return_timestamps && !max_new_tokens`, `src/models/whisper/modeling_whisper.js`),
+retest whether the explicit `max_new_tokens` is still needed.

src/index.ts

@@ -4,7 +4,7 @@ import { ensureIcons, ICONS_DIR } from "../scripts/generate-icons.ts";
 import { injectPrecacheManifest } from "./pwa/precache.ts";
 import { auth } from "./server/auth.ts";
 import { AUTH_ENABLED } from "./server/flags.ts";
-import { transcribe } from "./server/transcribe.ts";
+import { MAX_UPLOAD_BYTES, transcribe } from "./server/transcribe.ts";
 import { liveWebSocket } from "./server/live.ts";
 
 const ROOT = join(import.meta.dir, "..");
@@ -81,8 +81,17 @@ async function handleApiRequest(request: Request, server: any): Promise<Response
     case "GET /api/auth/me":
       return auth.me(request);
     case "GET /api/transcribe/status":
+      // The first probe spawns the sidecar and waits for faster-whisper to
+      // import (READY_TIMEOUT_MS = 60 s) — give it headroom over Bun's 10 s
+      // default idle timeout.
+      server.timeout(request, 90);
       return transcribe.status(request);
     case "POST /api/transcribe":
+      // Long-running ndjson stream (D-021): quiet stretches (model
+      // load/download, slow decode windows) exceed Bun's 10 s idle timeout,
+      // which would reset the connection mid-transcription. Heartbeat lines
+      // (transcribe.ts) cover proxies; disable Bun's timer entirely here.
+      server.timeout(request, 0);
       return transcribe.post(request);
     case "GET /api/transcribe/live": {
       const user = auth.getSessionUser(request);
@@ -261,6 +270,9 @@ async function developmentServeOptions(): Promise<ServeOptions> {
 
 const options = IS_PRODUCTION ? await productionServeOptions() : await developmentServeOptions();
 options.port = process.env.PORT || 3000;
+// Bun's default cap is 128 MB, which would 4xx a /api/transcribe upload of
+// more than ~35 minutes of raw PCM before the route's own limit applies.
+options.maxRequestBodySize = MAX_UPLOAD_BYTES;
 options.websocket = liveWebSocket as any;
 
 const server = serve(options);

src/server/transcribe.ts

@@ -282,6 +282,19 @@ export function createSidecar(command?: readonly string[]): Sidecar {
 /** Default cap ≈ 2¼ hours of raw float32 16 kHz mono (64 kB/s). */
 const DEFAULT_MAX_UPLOAD_BYTES = 512 * 1024 * 1024;
 
+/**
+ * Effective upload cap (env-overridable). Exported so src/index.ts can raise
+ * Bun.serve's `maxRequestBodySize` to match — its 128 MB default would
+ * otherwise reject uploads longer than ~35 minutes of audio before this
+ * module ever sees them.
+ */
+export const MAX_UPLOAD_BYTES =
+  Number(process.env.TRANSCRIBE_MAX_UPLOAD_BYTES) || DEFAULT_MAX_UPLOAD_BYTES;
+
+/** Keep the ndjson stream visibly alive through quiet stretches (D-021). */
+const HEARTBEAT_INTERVAL_MS = 5_000;
+const HEARTBEAT = new TextEncoder().encode("\n");
+
 const LANGUAGE_RE = /^[a-z]{2,3}$/;
 
 export interface TranscribeHandlers {
@@ -410,6 +423,20 @@ export function createTranscribeHandlers(deps: TranscribeDeps): TranscribeHandle
         async start(controller) {
           const enqueue = (line: TranscribeStreamLine) =>
             controller.enqueue(encoder.encode(`${JSON.stringify(line)}\n`));
+          // Segment gaps can exceed any proxy/server idle timeout (model
+          // load/download before the first segment; slow decode windows on
+          // big models — 30 s+ observed on a 15-min file). Heartbeat blank
+          // lines keep the connection visibly alive; ServerEngine's line
+          // reader skips empty lines. A dead client makes the enqueue throw,
+          // which is caught here (request.signal separately cancels the
+          // sidecar pass).
+          const heartbeat = setInterval(() => {
+            try {
+              controller.enqueue(HEARTBEAT);
+            } catch {
+              clearInterval(heartbeat);
+            }
+          }, HEARTBEAT_INTERVAL_MS);
           const chunks: TranscriptChunk[] = [];
           try {
             await deps.sidecar.transcribe(
@@ -445,6 +472,7 @@ export function createTranscribeHandlers(deps: TranscribeDeps): TranscribeHandle
               message: error instanceof Error ? error.message : String(error),
             });
           } finally {
+            clearInterval(heartbeat);
             await Bun.file(audioPath).unlink().catch(() => {});
             try {
               controller.close();
@@ -475,5 +503,5 @@ export const sidecar = createSidecar();
 export const transcribe: TranscribeHandlers = createTranscribeHandlers({
   sidecar,
   getSessionUser: appGetSessionUser,
-  maxUploadBytes: Number(process.env.TRANSCRIBE_MAX_UPLOAD_BYTES) || undefined,
+  maxUploadBytes: MAX_UPLOAD_BYTES,
 });

src/worker/inference.worker.ts

@@ -232,6 +232,18 @@ async function transcribe(message: TranscribeMessage): Promise<void> {
     chunk_length_s: CHUNK_LENGTH_S,
     stride_length_s: STRIDE_LENGTH_S,
     return_timestamps: true,
+    // With return_timestamps and no explicit max_new_tokens, Transformers.js
+    // 4.2's Whisper generate() takes a seek-loop path that re-decodes each
+    // window's tail (1–3 generate passes per window). Under this pipeline's
+    // own 30 s/5 s chunking that is pure overhead: ~1.4× slower on long
+    // files, duplicated boundary text, on_finalize firing once per pass
+    // (breaking TranscribeProgressTracker's one-finalize-per-window
+    // assumption — the progress bar saturated early, then looked frozen),
+    // and a potential infinite loop when a window's last boundary timestamp
+    // is 0. Setting max_new_tokens selects the classic single-pass path;
+    // 440 ≈ Whisper's 448 decoder positions minus prompt headroom, far more
+    // than any real 30 s window produces.
+    max_new_tokens: 440,
   };
   if (message.stream ?? message.requestId === undefined) {
     // File-flow pass (id-less, or routed through BrowserEngine with an

tests/audio_files/en_alien.txt

@@ -0,0 +1,279 @@
+Alright, well, now that I've sufficiently trashed this place, I think we're good to-
+Oh, oh, not done trashing just yet!
+Wait a minute, cause-
+No, that's not gonna blow.
+Guys, I have been addicted to Voices of the Void. This game is so good.
+And I'm proud to announce that after many, many hours, I have finally made contact with Alien Life.
+It might have just been a naked guy in the woods, though, now that I'm looking at a freeze frame here.
+But regardless, I did make contact with it, and it did scare me so very bad.
+This was live-streamed, so if you like the videos, come check out the streams.
+Oh, hey, Maxwell! Forgot about you.
+How you doing? Okay, let's pet him.
+Oh, he loves to be petted.
+And you know, guys, I think I'm gonna leave the music on the whole time today, too.
+Just kidding. I just want Maxwell to be happy.
+Yeah, Maxwell, we love the song.
+Alright, let's turn the music off for now, though, Maxwell.
+You're really killing it with that song.
+Why is he still real-looking?
+He is real.
+Because he's real.
+Alright, let's go get to booting these things up.
+Can you put Maxwell outside to guard against the aliens?
+Well, what if they abduct him?
+I do love, in theory, having him, uh, protect us.
+But I don't think I could ever live it down if he, uh, got taken.
+Alright, come on, come on.
+I'm gonna stick to the path so I don't hit any trees.
+I'm real bad about hitting trees.
+Alright, the path ends here, guys.
+Yeah, we're going all the way to India.
+Sure as shit, this is our stump.
+India.
+You think I could hop up here?
+I'm alright, that didn't hurt my body at all.
+I'm resilient.
+Alright, India is back.
+Next.
+Yankee.
+Do I try it?
+Do I try to jump from the tippy top all the way down to the bitty bottom
+and land on the, on the bike without taking fall damage?
+God.
+Whoa.
+Oh, shoot.
+How much damage did that do to me?
+This bike can't take much more.
+I've already crashed it a million times.
+I think I just ran over a small animal.
+Whoops.
+If an alien offered you a job, would you take it?
+I guess it depends on what it is.
+If it's maybe, uh,
+massaging the babes at the alien babe factory?
+Maybe.
+But if it was something worse, like,
+I don't know,
+vivisection on me?
+Maybe no.
+Unless the alien babes were doing it.
+Then maybe yes.
+Yes.
+Ugh.
+No, no, no, no, no.
+It really sucks because I know I'm going to pass out again
+and then this stupid motorcycle is going to go
+shooting off into the woods.
+I have to try to make it home.
+Ugh.
+We're fine.
+It's you.
+I'm walking.
+I'm just going on foot.
+We're almost there.
+Goddamn stupid.
+Ugh.
+Help, help.
+Alright, well, the motorcycle's trying to beat me home.
+It's not even a motorcycle.
+Alright, let's go calibrate these, uh,
+last little things.
+Wait, no.
+Real quick.
+Let's see
+what this sounds...
+Maxwell.
+Are you alright?
+That really looked like it hurt.
+No, he's fine?
+Guys, he's alright.
+He's totally good.
+Alright, let's see what we've got here.
+Exoplanet.
+This could be huge for us.
+Sounds like my stomach.
+Oh, hey, buddy.
+We are listening to Object 1.
+Someone's saying some shit.
+Oh, God.
+And what a beautiful language it is.
+Save signal.
+This one will be called...
+Alien be speaking.
+Guys, did you hear somebody approach me?
+From behind over here?
+Oh, so you've come crawling back, have you?
+I'm working!
+Give him the hammer.
+Give him the hammer.
+Hit him.
+Is that noise?
+Wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait.
+Something there! Something there!
+And it's moving. This is...
+This way. I'm going for it. We're gonna see what this is.
+Where's my car? Oh, it...
+This could be huge for us. I'm gonna make a quick...
+Whoa!
+Alright, we should be approaching it. Oh, once I get to the top of this hill.
+What? Oh, that's a tree.
+See anything?
+I thought that was, like, some giant dude up there, like Slenderman or something.
+I was like, whoa!
+Attempt number 45. Jumping off the tippy-top of this.
+And landing on my... on my... vehicle.
+Wait.
+I'll get there one day.
+What if a big UFO flew on by?
+Alright, let's start to stop.
+I thought that was a little green man right there.
+That's just a... Whoa!
+This isn't us, by the way.
+Oh, Jesus.
+Here we go.
+This is us.
+Boop!
+Alright, let's go push this button.
+Oh, it's my lucky day.
+What?
+What do I do with this?
+What is this, like...
+England in the 1400s?
+Why do I have coal?
+No one uses coal anymore.
+We're on to better, more cool fuels, like...
+I don't know.
+Oil or something.
+Okay.
+Food.
+Alright, well, now that I've sufficiently trashed this place, I think we're going to...
+Oh, oh, not done trashing just yet.
+Wait a minute, because...
+No, that's not going to blow.
+I'm...
+Oh!
+Oh!
+Alright, now that I've trashed the place, time to get out of here.
+Oh, I'm burning.
+I hope it starts raining before long.
+This really hurts.
+Okay, no, we're fine.
+Huh?
+Maxwell, what did you do to my PC?
+You've been playing that RuneScape game again, haven't you?
+I can hear the virus is coming in.
+Hold on a minute.
+Oh, Maxwell, you've screwed something up here.
+What is all this crap, Maxwell?
+You owe me a new PC.
+Okay, I can't be mad at you.
+What?
+Whoa!
+Something.
+Something big.
+What?
+Oh my god, what do I do?
+Holy snot.
+I'm going to investigate.
+God.
+God.
+This is proof.
+Definitive proof of alien existence.
+I saw their mothership, or maybe their...
+Maybe their daughtership or sonship.
+It looked kind of small.
+What was this thing?
+What was this thing?
+It was right there, wasn't it?
+Already left?
+Did I miss it?
+Oh my god, this is the worst day of my stupid life.
+It's gone.
+Anything on here?
+There's still something that way.
+And it's moving.
+I'm coming for you.
+You alien bastard.
+As soon as I get off this fence.
+Oh good, you're here.
+Thank god.
+Come with me.
+We've got work to do.
+Alright.
+You see anything...
+Anything moves, anything out of the ordinary, you let me know.
+This is history in the making.
+Anything...
+What?
+Guys, there's nothing here.
+Wait.
+What is that?
+Holy mother of god.
+We've got a moving light over there.
+Oh my god, we're going.
+There it is.
+There it is.
+It's coming straight for me.
+I'm gonna hit it.
+Never should have landed here, you alien bastard.
+Where am I?
+Where is it?
+I'm scared.
+Never should have come out here.
+I should have stayed in the safety of my home.
+Huh.
+Oh.
+There's the bike.
+Oh, I must have flown.
+Where's the beast though?
+Did you see it?
+It was huge.
+Nine or ten feet tall.
+Real ugly bastard.
+Wait.
+What is that?
+Oh.
+Ah.
+Ah.
+Ah.
+Ah.
+Ah.
+I was trying to do that so I could get to the bottom of the hill quicker.
+What the hell?
+This thing's having a picnic out here.
+Oh.
+Ah.
+Ah.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+Oh.
+nothing out of the ordinary
+it's just my mind playing tricks on me
+where did it go dude i could have seen this thing
+if i didn't
+oh my fucking lord don't you ever do that to me again
+you took way too long to get here you scared me
+i didn't see him when i was approaching right
+it was definitely something freaky
+we got a blurry image of the creature on discord okay
+oh my gosh this is freaky
+oh it looks mad
+guys i don't think they come in peace look i almost ran his ass over
+holy smokes we could sell that pick

tests/live-real-model.test.ts

@@ -194,6 +194,8 @@ describe("live pipeline accuracy (real model, real recordings)", () => {
             chunk_length_s: 30,
             stride_length_s: 5,
             return_timestamps: true,
+            // Selects the single-pass generate path, like the worker (D-021).
+            max_new_tokens: 440,
             language: clip.language,
             task: "transcribe",
           });