STSheets/docs/superpowers/plans/2026-10-30-synthetic-benchmark.md

Synthetic Benchmark Harness Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Generate original tier-1 songs as Guitar Pro files, render them to audio, run a baseline browser-portable transcription pipeline on the audio, and score the output against the source file.

Architecture: One shared symbolic Song model is written to alphaTex and turned into Guitar Pro / MIDI / audio by alphaTab. The pipeline (Demucs-6s ONNX → Basic Pitch ONNX / heuristic drums → tempo grid → quantise → tab assignment) produces timed notes plus a Song that goes back out through the same alphaTex writer. A scorer compares timed notes to the ground truth extracted from the GP file.

Tech Stack: Bun 1.3 + TypeScript (strict), @coderline/alphatab@1.8.4 (MPL-2.0), onnxruntime-node@1.31.1 (MIT), @spotify/basic-pitch@1.0.1 post-processing only (Apache-2.0), ffmpeg CLI (mp3 encode only), models: Kani95/htdemucs-6s-ort (Apache-2.0, sha256 35cb61d4d35173ff1d16df7393f41f05651ffdfe13d3c10e06abc26708bd6933), Spotify nmp.onnx (Apache-2.0).

Spec: docs/superpowers/specs/2026-10-30-synthetic-benchmark-design.md

Global Constraints

• Bun + TypeScript for everything; follow CLAUDE.md (bun test, Bun.file/Bun.write, Bun.$, bunx). No Python on any path in this plan.
• Models run via onnxruntime-node; code touching ORT must only use APIs that also exist in onnxruntime-web (InferenceSession.create, Tensor, session.run).
• CPU only. Use executionProviders: ["cpu"].
• Pipeline code (src/pipeline/**, src/dsp/**, src/score/**) must not import node:fs/Bun.*; file I/O lives in src/bench/**, src/audio/wav.ts, src/audio/mp3.ts. (Keeps the pipeline browser-portable.)
• Generated material is original; no reference songs are recreated.
• Tier-1 benchmark songs: at most two guitars, standard tuning [64,59,55,50,45,40] (MIDI, highest string first), 4/4.
• Pitch score: onset tolerance ±50 ms, offsets ignored, string ignored; computed on timed notes before quantisation.
• Song pass: merged-guitar pitch F1 ≥ 0.85. Tier pass: median song passes.
• Rhythm alignment allows tempo factor ½×, 1×, 2× and whole-beat bar offsets; factor/offset reported separately.
• Every result is labelled with its oracle flags (oracleStems, oracleTempo, oracleNotes).
• Reports order instruments: guitar, vocals, then bass, drums, keys.
• Generated audio, data/, runs/, models/*.onnx are git-ignored; results/*.md + results/*.json summaries are committed.
• SoundFont: alphaTab's bundled sonivox.sf2 (Apache-2.0).
• Commits end with Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>.

Out of Scope for This Plan

• The optional Python comparison track (spec: "optional") — deferred to a follow-up plan once baseline numbers exist.
• Website UI, YouTube ingestion, later tiers.

Review Focus

1. Durations that aren't a single note value or that cross a barline (e.g. 5 sixteenths, a whole note starting on beat 3) — writer must emit tied notes so the GP round trip preserves every onset and pitch. Test in Task 2.
2. Silent or absent stems (no vocals in t1a; a stem Demucs leaves near-silent) — transcribers return [], scorer reports null metrics (not NaN) and the report shows "n/a". Tests in Tasks 6, 8.
3. Audio length not a multiple of the Demucs segment, or shorter than one segment — separation output length must equal input length exactly. Test in Task 7.
4. Pitches outside the instrument's fret range (octave errors from transcription) — tab assigner drops them, counts them, never throws. Test in Task 11.
5. Synth onset latency — rendered audio must put note energy within 20 ms of ground-truth onset, otherwise every score is silently shifted. Test in Task 4.

---

File Structure

scripts/fetch-models.ts        download + sha256-verify models into models/
src/core/types.ts              Song, Track, ScoreNote, TimedNote, TruthTrack, Role, Grid, Audio
src/core/time.ts               tick/second conversion (TPQ = 960)
src/core/rng.ts                seeded PRNG
src/score/alphatex.ts          songToAlphaTex(song)
src/score/gp.ts                songToScore / songToGp / truthFromGp (alphaTab)
src/gen/harmony.ts             keys, progressions, guitar chord shapes
src/gen/parts.ts               per-role part generators
src/gen/tiers.ts               tier specs t1a..t1d
src/gen/generate.ts            generateSong(tier, seed)
src/audio/wav.ts               read/write WAV, toMono, resample
src/audio/mp3.ts               ffmpeg encode
src/audio/render.ts            per-track stem rendering via alphaSynth
src/audio/fx.ts                drive, chorus, reverb, pan, gain
src/audio/mix.ts               mix stems
src/dsp/fft.ts                 radix-2 FFT, torch-compatible stft/istft
src/pipeline/separate.ts       Demucs 6s ONNX
src/pipeline/transcribe.ts     Basic Pitch ONNX → TimedNote[]
src/pipeline/drums.ts          heuristic kick/snare/hat transcriber
src/pipeline/tempo.ts          tempo + downbeat grid
src/pipeline/quantize.ts       TimedNote[] → ScoreNote[]
src/pipeline/tab.ts            string/fret assignment (Viterbi)
src/pipeline/run.ts            orchestrates pipeline incl. oracle modes
src/eval/notes.ts              note matching / F1 / octave errors
src/eval/rhythm.ts             grid alignment + rhythm score
src/eval/score.ts              scoreSong
src/bench/dataset.ts           build song dirs on disk
src/bench/report.ts            markdown + JSON tier report
src/bench/cli.ts               `bun run bench …` entry
tests/**                       bun tests mirroring src

On-disk layout: data/<tier>/<seed>/{song.json, truth.gp, stems/<trackName>.wav, mix.wav, mix.mp3}; runs/<runId>/<tier>/<seed>/{notes.json, out.gp, score.json}; results/<runId>.{md,json}.

---

Task 1: Scaffolding, core types, time, RNG, model fetch

Files:

• Create: src/core/types.ts, src/core/time.ts, src/core/rng.ts, scripts/fetch-models.ts
• Modify: package.json (scripts), .gitignore, delete index.ts
• Test: tests/core/time.test.ts, tests/core/rng.test.ts

Interfaces:

• Produces (src/core/types.ts):

  export type Role = "guitar" | "bass" | "keys" | "vocals" | "drums";
  export type FxPreset = "none" | "clean" | "clean-chorus" | "light-drive";
  export interface ScoreNote { tick: number; dur: number; pitch: number; string?: number; fret?: number }
  // string numbering: 1 = highest-pitched string (alphaTex convention). Drums: pitch = GM drum number.
  export interface Track { name: string; role: Role; program: number; tuning?: number[]; pan: number; fx: FxPreset; gainDb: number; notes: ScoreNote[] }
  export interface Song { title: string; tier: string; seed: number; bpm: number; timeSig: [number, number]; bars: number; key: string; tracks: Track[] }
  export interface TimedNote { onset: number; offset: number; pitch: number; velocity?: number; string?: number; fret?: number }
  export interface TruthTrack { name: string; role: Role; tuning?: number[]; notes: TimedNote[] }
  export interface Grid { bpm: number; firstDownbeat: number; beatsPerBar: number }
  export interface Audio { sampleRate: number; channels: Float32Array[] }
  export const TPQ = 960;
  export const GUITAR_STANDARD = [64, 59, 55, 50, 45, 40];
  export const BASS_STANDARD = [43, 38, 33, 28];
  

• Produces: ticksToSeconds(tick: number, bpm: number): number, secondsToTicks(s: number, bpm: number): number in src/core/time.ts; createRng(seed: number): Rng with next(): number (0..1), int(lo: number, hi: number): number (inclusive), pick<T>(a: readonly T[]): T, chance(p: number): boolean in src/core/rng.ts (mulberry32).

• Step 1: Write failing tests

  test("ticks/seconds", () => {
    expect(ticksToSeconds(960, 60)).toBeCloseTo(1);
    expect(ticksToSeconds(3840, 120)).toBeCloseTo(2);
    expect(secondsToTicks(0.5, 120)).toBeCloseTo(960);
  });
  test("rng deterministic", () => {
    const a = createRng(42), b = createRng(42), c = createRng(43);
    const sa = [a.next(), a.next(), a.next()];
    expect([b.next(), b.next(), b.next()]).toEqual(sa);
    expect(c.next()).not.toBe(sa[0]);
    const r = createRng(1); for (let i = 0; i < 1000; i++) { const v = r.int(3, 5); expect(v >= 3 && v <= 5).toBe(true); }
  });
  

• Step 2: bun test tests/core → FAIL (modules missing).

• Step 3: Implement types, time, rng. package.json scripts: "test": "bun test", "typecheck": "bunx tsc --noEmit", "fetch-models": "bun scripts/fetch-models.ts", "bench": "bun src/bench/cli.ts". .gitignore add data/, runs/, models/**/*.onnx. scripts/fetch-models.ts downloads https://huggingface.co/Kani95/htdemucs-6s-ort/resolve/main/htdemucs_6s.onnx → models/demucs/htdemucs_6s.onnx and https://github.com/spotify/basic-pitch/raw/main/basic_pitch/saved_models/icassp_2022/nmp.onnx → models/basic-pitch/nmp.onnx, skips existing files whose sha256 matches, fails loudly on mismatch (Demucs hash above; record the nmp hash from the current file on first run as a constant).

• Step 4: bun test tests/core → PASS; bun run typecheck → no errors.

• Step 5: Commit feat: core types, time, rng, model fetch.

Task 2: alphaTex writer and Guitar Pro round trip

Files:

• Create: src/score/alphatex.ts, src/score/gp.ts
• Test: tests/score/roundtrip.test.ts

Interfaces:

• Consumes: Task 1 types.
• Produces: songToAlphaTex(song: Song): string; songToScore(song: Song): alphaTab.model.Score; songToGp(song: Song): Uint8Array (Gp7Exporter); truthFromGp(bytes: Uint8Array): { bpm: number; timeSig: [number, number]; tracks: TruthTrack[] }; songToMidi(song: Song): alphaTab.midi.MidiFile.

Writer rules (the implementer cannot infer these):

• Header \title, \tempo <bpm>, \ts <n> <d>; per track \track "<name>", \instrument <program> (drums: \instrument percussion \articulation defaults, notes written as GM numbers), guitar/bass: \staff {tabs} + \tuning (...) written as note names highest string first, notes as fret.string; keys/vocals: \staff {score}, notes as note names (e.g. C#4).

• Each track is one voice. Notes with equal tick form a chord beat; their beat duration is the shortest dur in the group, clipped to the next onset in the track. Gaps become rests.

• Durations are decomposed greedily into {1, 2, 4, 8, 16, 32} with optional single dots, split at barlines, joined with ties (- tie syntax on the continuation note; verify alphaTex tie syntax against alphaTab 1.8 docs/diagnostics).

• truthFromGp uses absolute tick = masterBar.start + beat.playbackStart, seconds via ticksToSeconds with the score tempo (tier 1 is constant tempo). Tied continuation notes are merged into the original note (one TimedNote). String converted to our numbering: ours = stringCount - model.string + 1. Drum pitch = track.percussionArticulations[n.percussionArticulation].outputMidiNumber. Role is stored as the alphaTex track short name (\track "<name>" "<role>") and recovered from track.shortName.

• Step 1: Write failing tests — build a hand-made Song (bpm 90) with: a guitar track containing a C-major chord at tick 0 (5 notes with string/fret), a single note of dur = 5 * 240 (5 sixteenths), and a whole note starting at beat 3 of bar 1 (crosses barline); a vocals track with a rest gap; a drums track with kick 36 + hat 42 together and a snare 38. Assertions:

  const truth = truthFromGp(songToGp(song));
  // every source note appears exactly once with same pitch and onset (±1 ms), ties merged:
  expect(sortNotes(truth.tracks[0].notes).map(n => [n.pitch, round3(n.onset)])).toEqual(expectedGuitar);
  expect(truth.tracks[0].notes.find(n => n.pitch === tiedPitch)!.offset - onset).toBeCloseTo(4 * 60/90, 3);
  expect(truth.tracks[0].notes.every(n => n.string! >= 1 && n.string! <= 6)).toBe(true);
  expect(truth.tracks.find(t => t.role === "drums")!.notes.map(n => n.pitch).sort()).toEqual([36, 38, 42]);
  expect(truth.tracks.map(t => t.role)).toEqual(["guitar", "vocals", "drums"]);
  

  plus: alphaTex parsing emits no error diagnostics (capture Logger or use the importer's diagnostics API) for this song.

• Step 2: bun test tests/score → FAIL.

• Step 3: Implement writer + gp.ts.

• Step 4: bun test tests/score → PASS.

• Step 5: Commit feat: alphaTex writer and GP round trip.

Task 3: Tier-1 song generator

Files:

• Create: src/gen/harmony.ts, src/gen/parts.ts, src/gen/tiers.ts, src/gen/generate.ts
• Test: tests/gen/generate.test.ts

Interfaces:

• Consumes: types, rng, songToGp/truthFromGp (test only).
• Produces: type TierId = "t1a" | "t1b" | "t1c" | "t1d"; TIERS: Record<TierId, TierSpec>; generateSong(tier: TierId, seed: number): Song.

Tier content (decided):

Tier	Tracks	Notes
t1a	1 clean guitar	solo; 66–96 bpm; mix of strummed open chords and arpeggios; one short single-note melodic phrase allowed
t1b	guitar + bass + drums	bass on chord roots (quarters/halves); drums: kick/snare/closed-hat groove, crash on section starts
t1c	t1b + keys (pad/piano chords) + vocals (program 53 "Voice Oohs", monophonic melody in C4–G5)	keys voiced in C3–C5
t1d	t1c + second guitar	guitars get distinct roles (strum vs arpeggio/lead line), pans −0.6 / +0.6, one clean, one clean-chorus or light-drive

Common: 4/4, 16 bars split into sections (e.g. A A B A, 4 bars each, repeats identical), keys from {C, G, D, A, E, F} major + relative minors, progressions from a fixed list of pop progressions (I–V–vi–IV, vi–IV–I–V, I–vi–IV–V, I–IV–vi–V). Guitar chords come from an open-chord shape dictionary in harmony.ts (each shape = per-string fret or mute, highest string first) so every guitar note carries a valid string/fret; arpeggios draw from the same shape. Guitar programs: 25 (steel) or 27 (clean electric). Bass program 33, keys 0 or 89, drums percussion.

• Step 1: Write failing tests

  for (const tier of ["t1a","t1b","t1c","t1d"] as const) for (const seed of [1,2,3]) {
    const s = generateSong(tier, seed);
    expect(generateSong(tier, seed)).toEqual(s);                       // deterministic
    const guitars = s.tracks.filter(t => t.role === "guitar");
    expect(guitars.length).toBe(tier === "t1d" ? 2 : 1);
    for (const g of guitars) for (const n of g.notes) expect(GUITAR_STANDARD[n.string! - 1]! + n.fret!).toBe(n.pitch);
    expect(s.tracks.map(t => t.role).sort()).toEqual(EXPECTED_ROLES[tier]);
    const end = s.bars * 4 * TPQ;
    for (const t of s.tracks) for (const n of t.notes) expect(n.tick + n.dur).toBeLessThanOrEqual(end);
    expect(truthFromGp(songToGp(s)).tracks.length).toBe(s.tracks.length); // valid GP
  }
  const d = generateSong("t1d", 7); const [g1, g2] = d.tracks.filter(t => t.role === "guitar");
  expect(Math.sign(g1!.pan)).not.toBe(Math.sign(g2!.pan));
  expect(generateSong("t1a", 1)).not.toEqual(generateSong("t1a", 2));
  

• Step 2: bun test tests/gen → FAIL.
• Step 3: Implement.
• Step 4: bun test tests/gen → PASS.
• Step 5: Commit feat: tier-1 song generator.

Task 4: Audio I/O, stem rendering, effects, mixing

Files:

• Create: src/audio/wav.ts, src/audio/mp3.ts, src/audio/render.ts, src/audio/fx.ts, src/audio/mix.ts
• Test: tests/audio/audio.test.ts

Interfaces:

• Consumes: songToMidi, songToScore, types.
• Produces: readWav(path: string): Promise<Audio>; writeWav(path: string, a: Audio): Promise<void> (PCM16); toMono(a: Audio): Float32Array; resample(x: Float32Array, from: number, to: number): Float32Array (windowed-sinc, used for 44100→22050 for Basic Pitch); encodeMp3(wavPath: string, mp3Path: string): Promise<void> (ffmpeg -y -loglevel error -i … -b:a 192k); renderStems(song: Song, sf2: Uint8Array, sampleRate = 44100): Map<string, Audio> (stereo, centred, one per track name, all the same length = song length + 2 s tail); applyFx(a: Audio, preset: FxPreset, sampleRate: number): Audio; pan(a: Audio, p: number): Audio (constant-power); mixStems(stems: { audio: Audio; gainDb: number }[]): Audio (sum, then peak-normalise to −1 dBFS).

Rendering: one AlphaSynth.exportAudio per track with AudioExportOptions.trackVolume muting all other tracks (verify key semantics; fallback: build a single-track Score per track), useSyncPoints = false, render(500) until undefined, de-interleave. Fx are simple and deterministic: light-drive = tanh waveshaper with pre-gain 4 + one-pole low-pass 5 kHz; clean-chorus = 2 modulated delay lines (~15 ms, 0.8 Hz); every pitched track gets a light shared Freeverb-style reverb (wet 0.15). Oracle stems written to disk are post-fx, post-pan — exactly what went into the mix.

• Step 1: Write failing tests
  • WAV round trip: random stereo 1 s → write → read → max abs diff < 1e-4.
  • pan(a, -1): right-channel RMS < 1e-3 × left RMS.
  • Rendering generateSong("t1b", 1): stems map has 3 entries, all equal length ≥ song seconds; each stem RMS > 1e-3.
  • Onset latency: render a song with one guitar note at tick 960 (bpm 60 → 1.0 s): first sample where the 5 ms-RMS envelope exceeds 10% of its max lies in [0.98, 1.02] s.
  • mixStems peak ≈ 10^(−1/20) ± 1e-3.
• Step 2: bun test tests/audio → FAIL.
• Step 3: Implement.
• Step 4: bun test tests/audio → PASS.
• Step 5: Commit feat: stem rendering, fx and mixing.

Task 5: Dataset builder and CLI skeleton

Files:

• Create: src/bench/dataset.ts, src/bench/cli.ts
• Test: tests/bench/dataset.test.ts

Interfaces:

• Produces: buildSong(tier: TierId, seed: number, root = "data"): Promise<string> (returns song dir; writes song.json, truth.gp, stems/<trackName>.wav, mix.wav, mix.mp3; idempotent — skips if mix.mp3 exists and song.json matches); loadSongDir(dir: string): Promise<{ song: Song; truth: ReturnType<typeof truthFromGp>; mix: Audio; stems: Map<string, Audio> }>.

• CLI: bun run bench gen --tiers t1a,t1b,t1c,t1d --seeds 1-5.

• Step 1: Write failing test — buildSong("t1a", 1, tmpDir) creates all five artefacts; loadSongDir returns truth.tracks[0].notes.length === song.tracks[0].notes.length (ties are not generated by the tier-1 generator, so counts match) and mix.channels.length === 2.

• Step 2: FAIL. Step 3: implement. Step 4: PASS; run bun run bench gen --tiers t1a --seeds 1-2 and confirm files exist.

• Step 5: Commit feat: dataset builder.

Task 6: Evaluation (notes, rhythm, song score)

Files:

• Create: src/eval/notes.ts, src/eval/rhythm.ts, src/eval/score.ts
• Test: tests/eval/notes.test.ts, tests/eval/rhythm.test.ts, tests/eval/score.test.ts

Interfaces:

• Produces:

  interface MatchResult { tp: number; fp: number; fn: number; precision: number | null; recall: number | null; f1: number | null; octaveErrors: number; matches: [number, number][] }
  matchNotes(ref: TimedNote[], est: TimedNote[], opts?: { onsetTol?: number /*0.05*/; key?: "pitch" | "pitchString" }): MatchResult
  interface GridAlignment { factor: 0.5 | 1 | 2; beatOffset: number }
  alignGrid(refNotes: ScoreNote[], estNotes: ScoreNote[]): GridAlignment
  rhythmScore(ref: ScoreNote[], est: ScoreNote[], align: GridAlignment): { onBeat: number | null; durationOk: number | null }
  scoreSong(truth: { bpm: number; timeSig: [number, number]; tracks: TruthTrack[] }, out: PipelineOutput, flags: OracleFlags): SongScore
  

  PipelineOutput and OracleFlags are defined here in src/eval/score.ts (re-exported to Task 12):

  interface OracleFlags { oracleStems: boolean; oracleTempo: boolean; oracleNotes: boolean }
  interface OutTrack { name: string; role: Role; notes: TimedNote[]; scoreNotes: ScoreNote[] }
  interface PipelineOutput { grid: Grid; tracks: OutTrack[] }
  interface SongScore { flags: OracleFlags; guitarMerged: MatchResult & { pass: boolean }; partSplit: { f1: number | null; outTracks: number; refTracks: number } | null;
    perRole: Partial<Record<Role, MatchResult>>; drumsPerClass: Record<"kick"|"snare"|"hihat"|"cymbal"|"tom", MatchResult> | null;
    rhythm: { factor: number; beatOffset: number; onBeat: number | null; durationOk: number | null } | null;
    stringAgreement: number | null; spuriousRate: number | null }
  

Rules: matching is maximum bipartite matching (onset within tol AND equal pitch); octave errors = unmatched est notes whose onset is within tol of an unmatched ref note with pitch ±12. F1 etc. are null when both ref and est are empty; precision null when est empty; recall null when ref empty. guitarMerged pools all guitar ref notes and all guitar out notes; pass = f1 >= 0.85. partSplit: when ≥2 guitar out tracks, Hungarian assignment maximising tp between out and ref guitar tracks, F1 over the assignment. stringAgreement = matched guitar notes with equal string / matched notes having a string. spuriousRate = fp / est count (guitar merged). Drum classes: kick {35,36}, snare {37,38,40}, hihat {42,44,46}, cymbal {49,51,52,53,55,57,59}, tom {41,43,45,47,48,50}; pitch compared by class. alignGrid tries factor ∈ {0.5,1,2} and beatOffset ∈ {−3..3} beats, picking the pair maximising exact tick-onset agreement of pitch-matched notes. Convention: estTick = refTick * factor + beatOffset * TPQ. rhythmScore.onBeat = share of pitch-matched notes landing on the identical aligned tick; durationOk = share with identical aligned duration.

• Step 1: Write failing tests

  const ref = [n(1.0,60), n(1.5,62), n(2.0,64)];
  expect(matchNotes(ref, [n(1.03,60), n(1.5,74), n(2.2,64)]).tp).toBe(1);   // 74 = 62+12 → octave error; 2.2 s is outside tol
  expect(matchNotes(ref, [n(1.03,60), n(1.5,74), n(2.2,64)]).octaveErrors).toBe(1);
  expect(matchNotes(ref, ref).f1).toBe(1);
  expect(matchNotes([], []).f1).toBeNull();
  expect(matchNotes(ref, []).recall).toBe(0);
  expect(matchNotes(ref, []).precision).toBeNull();
  // grid: est written at double tempo and shifted one beat
  const r = sn([[0,240,60],[960,960,62]]); const e = sn([[960,480,60],[2880,1920,62]]);
  expect(alignGrid(r, e)).toEqual({ factor: 2, beatOffset: 1 });
  expect(rhythmScore(r, e, { factor: 2, beatOffset: 1 })).toEqual({ onBeat: 1, durationOk: 1 });
  

  plus a scoreSong test with two ref guitar tracks and one merged out track (partSplit null-free path: outTracks: 1), and one with an absent vocals track on both sides → perRole.vocals.f1 === null.
• Step 2: FAIL. Step 3: implement. Step 4: PASS.
• Step 5: Commit feat: evaluation metrics.

Task 7: FFT/STFT and Demucs separation

Files:

• Create: src/dsp/fft.ts, src/pipeline/separate.ts
• Test: tests/dsp/fft.test.ts, tests/pipeline/separate.test.ts

Interfaces:

• Produces: fft(re: Float32Array, im: Float32Array): void (in-place radix-2); stft(x: Float32Array, nFft: number, hop: number): { re: Float32Array[]; im: Float32Array[] } (frames × bins, torch semantics: hann window, center=true, reflect pad, normalized=true); istft(spec, nFft, hop, length): Float32Array; type Stem = "drums" | "bass" | "other" | "vocals" | "guitar" | "piano"; createSeparator(modelBytes: Uint8Array): Promise<(mix: Audio) => Promise<Record<Stem, Audio>>>.

Demucs details (from htdemucs.py, MIT): segment = 343980 samples, overlap 0.25, linear-fade (triangular) weights for overlap-add, whole-mix normalisation (mix − mean(mono)) / std(mono) before and inverse after. Per segment: spectral input x = _spec: nfft 4096, hop 1024, pad hop/2*3 = 1536 each side reflect (right pad extended to le*hop - T, le = ceil(T/hop)), stft, drop last bin (→ 2048), keep frames [2, 2+le) (→ 336), complex-as-channels [re_L, im_L, re_R, im_R] → [1,4,2048,336]. Outputs: output [1,6,4,2048,336] → _ispec (re-add zero bin, pad 2 frames both sides, istft, crop pad … pad+T) and add add_67 [1,6,2,T]. Stem order from manifest: drums, bass, other, vocals, guitar, piano. Verify graph-internal normalisation by checking the result quality test below; if separation is garbage, the per-segment normalisation must be applied outside the graph.

• Step 1: Write failing tests
  • fft of an impulse → all-ones magnitude; istft(stft(x)) ≈ x (max err < 1e-4) for random 50 000-sample x with nFft 4096, hop 1024.
  • Separation on data/t1b/1 (built via buildSong): output length === input length for every stem; SDR(guitar stem vs oracle guitar stem) > 3 dB and > SDR(mix vs oracle guitar stem); bass likewise.
  • A 3-second slice of the mix (shorter than one segment) returns 3 s stems without throwing. (Separation tests are slow; mark with a 300 s timeout.)
• Step 2: FAIL. Step 3: implement. Step 4: PASS (bun test tests/dsp tests/pipeline/separate.test.ts --timeout 300000).
• Step 5: Commit feat: demucs 6-stem separation.

Task 8: Basic Pitch transcription

Files:

• Create: src/pipeline/transcribe.ts
• Test: tests/pipeline/transcribe.test.ts

Interfaces:

• Produces: createTranscriber(modelBytes: Uint8Array): Promise<(audio: Audio, opts?: TranscribeOpts) => Promise<TimedNote[]>> with TranscribeOpts = { onsetThresh?: number /*0.5*/; frameThresh?: number /*0.3*/; minNoteMs?: number /*58*/; minPitch?: number; maxPitch?: number; monophonic?: boolean }.

Approach: downmix + resample to 22050; window exactly like @spotify/basic-pitch inference.js (AUDIO_N_SAMPLES = 43844, overlap 30 frames, half-overlap zero pad, trim 15 frames each side, truncate to original frame count); ORT input serving_default_input_2:0 [n,43844,1]; outputs map StatefulPartitionedCall:{0,1,2} → identify contour/note/onset by last-dim size (264 = contour, 88 = note & onset; disambiguate note vs onset by name order matching inference.js OUTPUT_TO_TENSOR_NAME). Post-process with outputToNotesPoly, noteFramesToTime imported from @spotify/basic-pitch/esm/toMidi.js (must not pull in tfjs — verify; if it does, vendor those functions with the Apache-2.0 header). monophonic: at each time keep only the note with highest amplitude, trimming overlaps. Silent input (RMS < 1e-4) returns [] without running the model.

• Step 1: Write failing tests
  • On the oracle guitar stem of data/t1a/1: matchNotes(truthGuitar, est).f1 > 0.5 (sanity floor; the real number goes in the benchmark).
  • On a vocals-like monophonic synthetic line (from data/t1c/1 oracle vocals stem) with monophonic: true: no two output notes overlap in time.
  • 2 s of zeros → [].
• Step 2: FAIL. Step 3: implement. Step 4: PASS.
• Step 5: Commit feat: basic pitch transcription.

Task 9: Tempo and downbeat grid

Files:

• Create: src/pipeline/tempo.ts
• Test: tests/pipeline/tempo.test.ts

Interfaces:

• Produces: estimateGrid(audio: Audio, opts?: { minBpm?: number /*60*/; maxBpm?: number /*180*/; beatsPerBar?: number /*4*/ }): Grid; gridFromTruth(bpm: number, beatsPerBar: number): Grid (oracle; firstDownbeat = 0).

Approach: spectral-flux onset envelope (hop 512 @ 44.1 kHz, log-magnitude, half-wave rectified), tempo by autocorrelation with a log-Gaussian prior centred at 100 bpm, beat phase by maximising envelope sum on the beat comb, downbeat by the beat-phase (0..3) whose comb has the most low-frequency (< 150 Hz) flux energy. Constant tempo only (tier 1).

• Step 1: Write failing test — for t1b and t1c seeds 1–5 mixes: |est.bpm/true − f|/f < 0.02 for some f ∈ {0.5,1,2} in ≥ 8 of 10 songs; firstDownbeat mod beat period within 50 ms of 0 in ≥ 8 of 10.
• Step 2: FAIL. Step 3: implement. Step 4: PASS.
• Step 5: Commit feat: tempo grid estimation.

Task 10: Heuristic drum transcriber

Files:

• Create: src/pipeline/drums.ts
• Test: tests/pipeline/drums.test.ts

Interfaces:

• Produces: transcribeDrums(audio: Audio): TimedNote[] (pitches 36 kick, 38 snare, 42 hat, 49 cymbal).

Approach: band-split flux (kick < 150 Hz, snare 150 Hz–2 kHz with broadband noise, hats > 7 kHz, cymbal = hat-band onset with long decay > 300 ms), peak-pick each band with adaptive threshold (median + k·MAD), 30 ms min inter-onset per class. Labelled in reports as "heuristic baseline".

• Step 1: Write failing test — on oracle drum stems of t1b seeds 1–3: kick F1 > 0.7, snare F1 > 0.6 (class-wise via scoreSong drum classes or direct matchNotes on mapped pitches); silence → [].
• Step 2: FAIL. Step 3: implement. Step 4: PASS.
• Step 5: Commit feat: heuristic drum transcription.

Task 11: Quantisation and tab assignment

Files:

• Create: src/pipeline/quantize.ts, src/pipeline/tab.ts
• Test: tests/pipeline/quantize.test.ts, tests/pipeline/tab.test.ts

Interfaces:

• Produces: quantize(notes: TimedNote[], grid: Grid, stepsPerBeat = 4): ScoreNote[] (ticks relative to grid.firstDownbeat, notes before it dropped; onsets snapped to nearest step, duration snapped to ≥ 1 step; chord notes within 35 ms collapse to the same tick); assignTab(notes: ScoreNote[], tuning: number[], maxFret = 20): { notes: ScoreNote[]; dropped: number }.

Tab assignment: group notes by tick into chords; candidates = all injective note→string mappings with 0 ≤ fret ≤ maxFret; cost = fret span within chord (penalise > 4) + movement from previous hand position (mean non-zero fret) + 0.1·mean fret (prefer low positions) − small bonus for open strings; Viterbi over chords. Notes with no valid position on any string are dropped and counted; if a chord has more notes than strings, keep the highest-pitched tuning.length notes and count the rest as dropped.

• Step 1: Write failing tests

  const g = { bpm: 120, firstDownbeat: 0.1, beatsPerBar: 4 };
  const q = quantize([t(0.1+0.5+0.02, 0.1+1.0, 60), t(0.1+0.75-0.015, 0.1+1.0, 64)], g);
  expect(q.map(n => [n.tick, n.dur, n.pitch])).toEqual([[960, 960, 60], [1440, 480, 64]]);
  const chordC = [48,52,55,60,64].map(p => sn(0, 960, p));
  const a = assignTab(chordC, GUITAR_STANDARD);
  expect(a.dropped).toBe(0);
  for (const n of a.notes) expect(GUITAR_STANDARD[n.string!-1]! + n.fret!).toBe(n.pitch);
  expect(new Set(a.notes.map(n => n.string)).size).toBe(5);
  expect(assignTab([sn(0, 960, 30)], GUITAR_STANDARD)).toEqual({ notes: [], dropped: 1 }); // below low E
  

• Step 2: FAIL. Step 3: implement. Step 4: PASS.
• Step 5: Commit feat: quantisation and tab assignment.

Task 12: Pipeline runner with oracle modes

Files:

• Create: src/pipeline/run.ts
• Test: tests/pipeline/run.test.ts

Interfaces:

• Consumes: separator (T7), transcriber (T8), estimateGrid/gridFromTruth (T9), transcribeDrums (T10), quantize/assignTab (T11), songToGp (T2), PipelineOutput/OracleFlags (T6).
• Produces: createPipeline(models: { demucs: Uint8Array; basicPitch: Uint8Array }): Promise<(input: PipelineInput, flags: OracleFlags) => Promise<{ output: PipelineOutput; gp: Uint8Array }>> where PipelineInput = { mix: Audio; oracle?: { stems: Map<string, Audio>; truth: { bpm: number; timeSig: [number, number]; tracks: TruthTrack[] } } }.

Behaviour: stems — non-oracle: Demucs; map guitar→one "Guitar" track (role guitar), bass→bass, piano→keys, vocals→vocals (monophonic, pitch 48–84), drums→drums; other is ignored (reported in notes.json as unused). Oracle stems: one output track per truth track, from its own oracle stem. Grid — oracleTempo ? gridFromTruth : estimateGrid(mix). oracleNotes: pitched tracks take their truth notes instead of transcription (tab assignment still runs, measuring fingering only). Pitch ranges for Basic Pitch: guitar 40–88, bass 28–67, keys 21–108. Every pitched track is quantised; guitar/bass get assignTab with standard tunings; output Song (bpm from grid) → songToGp. Pipeline code does no file I/O.

• Step 1: Write failing test — on data/t1a/1 with { oracleStems: true, oracleTempo: true, oracleNotes: false }: output has 1 guitar track with > 0 notes; truthFromGp(gp) re-imports with a guitar track; with oracleNotes: true merged guitar F1 from scoreSong === 1 and stringAgreement is a number. On data/t1c/1 with all flags false: output roles ⊇ {guitar, bass, keys, vocals, drums}.
• Step 2: FAIL. Step 3: implement. Step 4: PASS.
• Step 5: Commit feat: pipeline runner with oracle modes.

Task 13: Benchmark orchestration and reports

Files:

• Create: src/bench/report.ts; Modify: src/bench/cli.ts
• Test: tests/bench/report.test.ts

Interfaces:

• Produces: renderReport(run: BenchRun): string (markdown), BenchRun = { runId: string; date: string; modes: { name: string; flags: OracleFlags }[]; results: { tier: TierId; seed: number; mode: string; score: SongScore; seconds: number }[] }.
• CLI: bun run bench run --tiers t1a,t1b,t1c,t1d --seeds 1-5 --modes full,oracle-stems,oracle-tempo,oracle-notes --run-id <id> builds missing songs, runs each mode, writes runs/<id>/<tier>/<seed>/{notes.json,out.gp,score.json} and results/<id>.{md,json}. Modes: full = all false; oracle-stems = stems only; oracle-tempo = stems + tempo; oracle-notes = all true.

Report layout: header (run id, date, machine, model hashes, SoundFont, vocals caveat); per tier a table of median/min/max per mode in this column order: guitar merged F1 (pass ✓/✗), guitar octave errors, spurious rate, part-split F1 (t1d), vocals F1, bass F1, drums kick/snare/hihat F1, keys F1, rhythm onBeat/durationOk with tempo factor/offset, string agreement; tier pass line (median song passes, full mode); per-song appendix. null renders as n/a.

• Step 1: Write failing test — renderReport on a fixture BenchRun (2 songs, 2 modes, one with vocals null): contains "Guitar" column before "Vocals" before "Bass"; contains "n/a"; tier pass line reads "PASS" when median guitar F1 ≥ 0.85 and "FAIL" otherwise.
• Step 2: FAIL. Step 3: implement. Step 4: PASS.
• Step 5: Commit feat: benchmark runner and report.

Task 14: Baseline benchmark run

• Step 1: bun run fetch-models && bun run typecheck && bun test --timeout 300000 → all pass.
• Step 2: bun run bench run --tiers t1a,t1b,t1c,t1d --seeds 1-5 --modes full,oracle-stems,oracle-tempo,oracle-notes --run-id baseline-2026-10-30 → results/baseline-2026-10-30.md exists with 4 tier sections.
• Step 3: Listen-check is not possible; instead spot-check one out.gp per tier re-imports via truthFromGp and one mix.mp3 is non-silent (ffprobe duration > 30 s).
• Step 4: Add a short README.md (what the harness is, commands, where results live). Commit chore: baseline benchmark results and push to origin main.