Commit Graph
143 Commits
Author SHA1 Message Date
Lucas WintherandClaude Opus 5.5 a0748b5fa6 Translate transcripts with Opus-MT in a dedicated translation worker
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 22:10:51 +01:00
Lucas WintherandClaude Opus 5.5 12e0f6678b Offer Parakeet in the picker as English-only, hide it on phones, and explain the missing language controls
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 22:08:08 +01:00
Lucas WintherandClaude Opus 5.5 708fae5778 Run Parakeet TDT in the inference worker via parakeet.js on the shared onnxruntime-web
Adds parakeet.js 1.4.4 (MIT) as a devDependency bundled into the worker
only; a build plugin keeps its own onnxruntime-web copy out of the bundle.
Includes an opt-in real-model test (PARAKEET_REAL=1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 22:08:05 +01:00
Lucas WintherandClaude Opus 5.5 f55ea6ecf8 Run browser file transcription through the long-form pipeline when asked and the VAD model is available
Adds an Advanced "Skip silence in uploaded files" option (off by default
pending measurement), the vad/longForm protocol fields, the old-path
fallback when the VAD model is missing, and a plain message for files
with no speech.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 22:02:47 +01:00
Lucas WintherandClaude Opus 5.5 0c3a83c23c Run Silero VAD and Whisper decodes for the long-form pipeline on Transformers.js
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 22:02:47 +01:00
Lucas WintherandClaude Opus 5.5 2a4f5b16a5 Add Danish fine-tune catalog entries under a single pending Hugging Face owner constant
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 22:00:33 +01:00
Lucas WintherandClaude Opus 5.5 f7d0525423 Wait for the PCM payload, not just the request, in the live WebSocket test (race under load)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:58:03 +01:00
Lucas WintherandClaude Opus 5.5 d6c3d1ff70 Add the optional "Names and terms" field and send it with every file and live pass
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:57:30 +01:00
Lucas WintherandClaude Opus 5.5 8236ea3504 End the hint prompt in a full stop: an unterminated list made Whisper drop punctuation and loop
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:55:48 +01:00
Lucas WintherandClaude Opus 5.5 c5a57cd33a Keep Parakeet on WASM unless WebGPU is chosen explicitly, and never offer it on phones
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:51:26 +01:00
Lucas WintherandClaude Opus 5.5 827e9a8dae Add a Cache-API model file downloader that reports load progress and caches only complete files
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:50:40 +01:00
Lucas WintherandClaude Opus 5.5 c9fdab1726 Add Parakeet window planning, word-to-chunk mapping, vocab tokenizer and pinned asset URLs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:49:52 +01:00
Lucas WintherandClaude Opus 5.5 d554162a2c Add a model family field and the Parakeet v3 catalog entry with exact file sizes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:48:39 +01:00
Lucas WintherandClaude Opus 5.5 c3a490548a Plan workstream 7: Parakeet TDT engine path
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:47:11 +01:00
Lucas WintherandClaude Opus 5.5 19318a56f2 Add the long-form transcription pipeline behind transcribeLongForm
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:46:16 +01:00
Lucas WintherandClaude Opus 5.5 1c5952f212 Validate the vocabulary hint server-side and pass it to faster-whisper as initial_prompt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:45:39 +01:00
Lucas WintherandClaude Opus 5.5 ad2eb10a28 Add compression-ratio and log-prob quality checks for decode fallback
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:44:41 +01:00
Lucas WintherandClaude Opus 5.5 221ac594eb Let the transcribe progress tracker follow variable-length windows
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:44:19 +01:00
Lucas WintherandClaude Opus 5.5 753ea03979 Upgrade @huggingface/transformers to 4.3.2 and carry the sequential-session patch
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:44:18 +01:00
Lucas WintherandClaude Opus 5.5 313892b6df Carry the vocabulary hint as prompt from the engines through the worker into every window
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:44:06 +01:00
Lucas WintherandClaude Opus 5.5 99b9cd71cd Type the prompt wrapper's tensor slice for open-ended ranges
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:43:29 +01:00
Lucas WintherandClaude Opus 5.5 87617bef4c Fix a type error in the hint truncation test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:43:19 +01:00
Lucas WintherandClaude Opus 5.5 e0a4b2796f Add pure VAD speech-timestamp and chunking functions for the long-form pipeline
Port of faster-whisper 1.2.1 vad.py (checked identical on a random
20 000-window probability trace for both min-silence settings).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:43:18 +01:00
Lucas WintherandClaude Opus 5.5 e6de097cfd Feed Whisper a decoder prompt through the ASR pipeline and strip it from the output
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:43:11 +01:00
Lucas WintherandClaude Opus 5.5 942799d3cb Add the vocabulary hint's normalization, storage and prompt truncation rule
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:42:25 +01:00
Lucas WintherandClaude Opus 5.5 438af265b4 Add plan 001-3 for the browser long-form pipeline
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:42:07 +01:00
Lucas WintherandClaude Opus 5.5 225634c165 Plan workstream 2: vocabulary hint
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:41:45 +01:00
Lucas WintherandClaude Opus 5.5 82ed34c7f9 Plan transcript translation units and sentence pieces for Opus-MT
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:39:37 +01:00
Lucas WintherandClaude Opus 5.5 02a5eab119 Offer Whisper's Translate to English only on models trained for it (canTranslate)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:37:28 +01:00
Lucas WintherandClaude Opus 5.5 afce794cbf Mark server-live sidecar passes as live mode and cover the stream heartbeat with a test
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:37:01 +01:00
Lucas WintherandClaude Opus 5.5 dc64bb8198 Run server-engine file transcriptions through faster-whisper's batched pipeline with VAD
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:36:21 +01:00
Lucas WintherandClaude Opus 5.5 036635e854 Add workstream 4 plan: canTranslate and Opus-MT transcript translation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:35:49 +01:00
Lucas WintherandClaude Opus 5.5 f8ce3120fe Plan workstream 6: Danish fine-tuned Whisper conversions and catalog entries
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:35:40 +01:00
Lucas WintherandClaude Opus 5.5 bbcc123124 Add plan for server engine batched + VAD inference (spec 001, workstream 5)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:35:05 +01:00
Lucas WintherandClaude Opus 5.5 9607c8558f Add plan for the Transformers.js 4.3.2 upgrade (workstream 1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:34:15 +01:00
Lucas WintherandClaude Opus 5.5 9159e2ab5e Coordination plan: assign the synthetic meeting fixture
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:29:11 +01:00
Lucas WintherandClaude Opus 5.5 2642462e33 Add coordination plan for spec 001
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:28:52 +01:00
Lucas WintherandClaude Opus 5.5 3d9c7bfd34 Approve spec 001
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:27:32 +01:00
Lucas WintherandClaude Opus 5.5 2bef4cb80a Draft spec 001: transcription quality and speed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:27:01 +01:00
Lucas WintherandClaude Opus 5.5 e195883b45 Record the product audience and copy tone in PRODUCT.md
Primary users are non-technical people who want Whisper without docs or
ML setup; technical users must keep full control as if running it on
their own device.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:14:41 +01:00
Lucas WintherandClaude Opus 5.5 4942461f79 Add typecheck + CI checks, light theme with self-hosted Inter, fix real-model test
- Typecheck: typescript 7 devDependency, `bun run typecheck`; CI gains a
  `check` stage (install, typecheck, fast tests) gating image builds (D-038).
- Light theme: every colour is a token; prefers-color-scheme: light
  overrides them. Inter Variable self-hosted from @fontsource-variable/inter,
  kept out of Bun's CSS inlining and precached (scripts/fonts.ts, D-037).
  Backend select no longer overflows on narrow phones.
- tests/live-real-model.test.ts: time budget scales with clip length and
  en_alien gets a measured WER bar; it had failed since July (D-039).
- Delete the temporary isolate_wer2.ts; fix icon "not checked in" comments
  and the README's model-weights location.
- Agent docs: branch naming, typecheck, CI, Bun CSS gotcha, permissions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 20:52:13 +01:00
Lucas WintherandClaude Opus 5.5 0ece4b3a2a Bootstrap agent-ready docs and permissions (D-036)
Root AGENTS.md is now the single source of truth for coding agents;
CLAUDE.md, GEMINI.md and docs/CLAUDE.md are pointers. The product spec
moves from docs/CLAUDE.md to PRODUCT.md, DESIGN.md records the shipped
visual system, and docs/PERMISSIONS.md defines agent permission tiers
rendered to .claude/, .codex/ and .gemini/. Adds docs/specs/ and
docs/plans/, logs issues noticed (not fixed) in WorkBreakdown.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 19:51:30 +01:00
Lucas WintherandClaude Opus 5.5 c1b710cb23 Remove AU branding
Rename the app from "AU Whisper Web" to "Whisper Web" in the README
title and app header, rename the package to whisper_web, and drop the
au_ prefix from the transformers patch comment and the scratch path in
isolate_wer2.ts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 19:15:58 +01:00
Lucas Winther 7ab41f4299 Document D-035: whisper-tiny multilingual vs .en speed gap is a content-dependent hallucination-loop effect, not a config bug
Measured with the ort-web-under-Bun harness: no reliable speed difference on
clean, language-matched audio. On out-of-distribution audio (wrong/undetected
language), the multilingual model's extra forced task token measurably
changes how long its repetition-loop hallucination runs before EOS (434 vs 67
tokens on the da_festuge fixture), which is where the reported slowness comes
from. Both models resolve to identical, merged, KV-cache-enabled decoders at
the same dtype — no catalog/config fix applies here. Documents the mechanism
and flags a future no_repeat_ngram_size/repetition_penalty mitigation as the
actionable follow-up if this recurs.
2026-07-30 08:31:58 +02:00
Lucas Winther 500cf3b36e Don't show the live "recording"/listening indicator until the model is ready
startLive()'s browser-engine branch flipped liveStatus to "listening" (the
red live-dot, "Listening..." hint) the instant the mic was granted, even if
the Whisper pipeline was still loading — the user saw an active-looking
session while nothing was being transcribed yet. transcribeLiveSegment()
already correctly awaits the model-ready gate before posting a segment, so
this was purely a misleading-status bug.

Adds a "loading-model" LiveStatus between "starting" and "listening";
startLive() now waits on the same browserModelReady() gate the transcribe
path uses before claiming "listening", with race-safe cleanup if the load
fails while waiting. Server-engine live is unaffected.

Adds Decisions.md D-034; updates tests/live.md's manual verification steps.
2026-07-30 08:31:23 +02:00
Lucas Winther c5d193600a Show "Loading model from cache" instead of "Downloading" when weights are already cached
Transformers.js's progress_callback always fires status:"download" regardless
of whether the file is actually served from the Cache API — there's no
cacheHit signal for the worker to forward. Since App.tsx already tracks which
models are downloaded (for ModelPicker's badges), it now checks that before
starting a load and threads a loadedFromCache flag into ModelLoadProgress, so
the heading/hint reflect reality for the common already-cached case.

Adds Decisions.md D-033.
2026-07-30 08:30:19 +02:00
Lucas Winther bb77185e3f Merge remote-tracking branch 'origin/main' 2026-07-29 09:20:50 +02:00
Lucas Winther 7040f031a6 Usability and test infrastructure updates. 2026-07-29 09:20:43 +02:00
Lucas Winther 69602f046b Usability and test infrastructure updates. 2026-07-29 09:20:32 +02:00
Lucas Winther 99b1732003 Fix Docker image bloat: exclude local venvs from build context, move transformers.js to devDependencies. 2026-07-29 08:25:00 +02:00