fix(align): calibrated fit (precision, lift, beat-shift check, baseline) and aligner guards (A7, A8)

Fit flags on precision (<0.6) and chance-corrected lift (<0.25), null with <8
recording onsets; old match rate kept as audibleRate (info). Symbolic-chroma
beat/bar-shift check per window. Constant-offset baseline (-2..+2 s) replaces
DTW without manual anchors unless DTW precision is better by >=0.02. Stretch
warning counts in the fit summary. DTW: slope-limited steps, open begin/end
options; per-bar outlier rejection (1/4 beat, Theil-Sen-transported +-3 bar
median); no anchor in the final partial bar. bench real: saved/computed
source, baseline + method, corrupt alignment.json skipped. Parabolic onset
interpolation; warp extrapolation documented; spec amended.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Lucas Winther
2026-10-30 19:50:48 +01:00
co-authored by Claude Opus 5.5
parent 3b002059e9
commit 3db87c0317
19 changed files with 1016 additions and 252 deletions
+2 -1
View File
@@ -77,5 +77,6 @@ This writes `results/real-<id>.md` and `.json` (committed) and per-song output u
Caveats:
- The fit summary (matched MIDI onsets, median residual, flagged windows) is a consistency measure between the MIDI and the audio, not ground truth; a high match rate does not prove the alignment is right.
- The fit summary is a consistency measure between the MIDI and the audio, not ground truth: *precision* (share of the recording's detected attacks with a warped MIDI onset within ±70 ms; a 30 s window is flagged below 60 %), *lift* (precision minus its off-grid chance level; flagged below 0.25), a chroma check that flags suspected whole-beat/bar shifts, "MIDI onsets audible as attacks" (information only: a full mix exposes far fewer attacks than the score has notes), and the number of warned stretches (> 5 % and > 15 % tempo change). A high precision does not prove the alignment is right.
- `align auto` also reports a constant-offset baseline (best single offset, −2..+2 s); with no manual anchors it keeps the baseline (one auto anchor) unless the DTW alignment beats its precision by 2 points.
- The page plays Ogg Opus; Safari needs 18.4 or later.
@@ -93,7 +93,7 @@ aligned MIDI.
starting from the raw MIDI; a "reset to raw MIDI" action exists.
- **Fit quality:** median absolute onset residual (ms) between matched recording
onsets and warped MIDI onsets, overall and per 30 s window; shown on the page
and in benchmark output.
and in benchmark output. (Amended 2026-10-30: see "Fit quality — Amended" below.)
- **Playback of the warped MIDI:** the server re-renders the warped MIDI audio
when anchors change (alphaTab renders a 4-minute song in ~1–2 s); the page
requests it on Play. Waveforms are sent as precomputed peaks, not full audio.
@@ -118,6 +118,49 @@ aligned MIDI.
match rate = share of warped MIDI onsets with a recording onset within ±70 ms,
overall and per 30 s window. A window is flagged "don't trust" when its match
rate < 0.5 or median residual > 60 ms; the page highlights flagged windows.
*(Superseded by the amendment below.)*
- **Fit quality — Amended 2026-10-30 after real-song evidence (rulings A7, A8).**
On real full mixes the onset picker finds ~1.4 attacks/s while a scored MIDI
has ~13 unique onsets/s, so the match rate above caps near 40 % even when
perfectly aligned (every window was flagged on every song), and onset metrics
cannot see whole-beat shifts (they alias onto the grid).
- **Flag metric = precision:** share of recording onsets (onset flux, k = 4,
parabolic sub-frame peak times) with a warped scored-role MIDI onset within
±70 ms; per 30 s window by recording time. A window with < 8 recording
onsets is null and never flagged; otherwise flagged when precision < 0.6.
- **Chance-corrected lift:** precision minus the mean precision with the
warped MIDI shifted by +0.11, +0.23 and +0.37 s (off-grid); reported per
window and overall; a window is also flagged when lift < 0.25.
- **Beat/bar-shift check:** per window, mean chroma cosine between the
recording (features chroma, 10 Hz) and a symbolic MIDI chroma (piano roll of
the warped pitched scored-role notes, no render) at the current alignment
vs the MIDI moved by ±1, ±2 beats and ±1 bar along the MIDI's own
tempo/time-signature grid. Flagged "beat shift suspected: ±k beat(s)/bar"
when a shift beats the current alignment by > 0.05 (the margin keeps
repetitive harmony, where shifts tie, from flagging). "+1 beat" means the
MIDI content belongs one beat later in the recording.
- **Information only:** the old match rate, renamed "MIDI onsets audible as
attacks" (JSON `audibleRate`), and the median residual (recording onsets →
nearest warped MIDI onset).
- **Stretch warnings:** the fit summary (CLI, page, real report) includes the
number of warned stretches (> 5 % tempo change) and of stretches with
|tempo change| > 15 %.
- **Constant-offset baseline:** grid search −2..+2 s in 10 ms steps for the
offset maximising overall precision; precision peaks within 0.1 of the best
are told apart by the symbolic chroma cosine (beat aliasing), and within a
peak the offset with the smallest mean residual is taken. Reported with its
precision. Without manual anchors, `autoAlign` returns the baseline as one
auto anchor (`chosen: "baseline"`) unless the DTW alignment's precision
beats it by ≥ 0.02 (`chosen: "dtw"`); with manual anchors there is never a
substitution.
- **Aligner guards:** both DTW passes use slope-limited steps (Sakoe–Chiba
P = 1: (1,1), (1,2), (2,1)), so the path cannot jump; the first segment may
start late in the recording (open begin) and the final segment uses an open
end (the path may stop before the recording ends, or before the MIDI ends,
each unconsumed MIDI frame costing 1), so a length mismatch is not absorbed
as a tempo ramp. Per-bar offsets more than ¼ beat from the median of their
±3 neighbouring bars are rejected (iterated until stable), and no auto
anchor is placed in the MIDI's final partial bar.
- **Editing model:** dragging an anchor only re-fits its two neighbouring
stretches. Adding or moving a manual anchor deletes the auto anchors in the two
affected stretches (those stretches become plain uniform scaling) until the user
+283 -54
View File
@@ -2,12 +2,12 @@
// Pure: no I/O, no Bun/process/fs. The SoundFont bytes are passed in.
import type { Audio, Role } from "../core/types.ts";
import type { Smf } from "./smf.ts";
import { makeWarp, type Anchor, type Warp } from "./warp.ts";
import { makeWarp, stretches, type Anchor, type Warp } from "./warp.ts";
import { renderMidi } from "./render.ts";
import { features, pickOnsets, type Features } from "./features.ts";
import { dtw } from "./dtw.ts";
import { dtw, type DtwOptions } from "./dtw.ts";
import { trackKeys } from "./roles.ts";
import { fitQuality, type FitReport } from "./fit.ts";
import { MATCH_TOL_SEC, fitQuality, meanCosine, precisionOf, uniqueSorted, type FitReport, type ShiftInputs, type StretchCounts } from "./fit.ts";
const FINE_RATE = 50;
const COARSE_FACTOR = 5; // coarse pass at 10 Hz
@@ -16,10 +16,23 @@ const FINE_BANDS_SEC = [0.5, 1, 2]; // fine pass band half-widths tried in turn
const ANCHOR_HALF_WIN_SEC = 1; // path points within ±1 s (warped time) of a bar estimate its offset
const MIN_POINTS = 10; // diagonal path points required to place a bar anchor
const MANUAL_CLEARANCE_SEC = 1; // no auto anchor within 1 s (MIDI time) of a manual anchor
const OUTLIER_BEATS = 0.25; // per-bar offsets further than ¼ beat from the robust median of ±3 neighbours are rejected
const OUTLIER_NEIGHBOURS = 3;
const BASELINE_RANGE_SEC = 2, BASELINE_STEP_SEC = 0.01; // constant-offset baseline grid
const BASELINE_MARGIN = 0.02; // DTW must beat the baseline's precision by this much to be kept (no manual anchors)
const BASELINE_PLATEAU = 0.05; // within a precision peak, offsets this close to its best are refined by residual
const BASELINE_TIE = 0.1; // precision peaks this close to the best are told apart by symbolic chroma (beat aliasing)
const OVER_TEMPO = 0.15; // stretches with |tempo change| above this are counted separately in the fit
const ZERO = 1e-9;
/** Bar start times (MIDI seconds) from the tempo map and time signatures (4/4 when none), up to durationSec. */
export function barTimes(smf: Smf): number[] {
export interface MeterGrid { beats: number[]; bars: number[]; /** end of the last bar */ end: number; /** the MIDI ends before its last bar does */ lastBarPartial: boolean }
/**
* Beat and bar start times (MIDI seconds) from the tempo map and time signatures (4/4 when none; a beat is one
* denominator note), for bars starting before `untilSec` (default: the MIDI duration). A signature change mid-bar
* restarts the bar there.
*/
export function meterGrid(smf: Smf, untilSec = smf.durationSec): MeterGrid {
const tempos = smf.tempoMap.length ? [...smf.tempoMap].sort((a, b) => a.sec - b.sec) : [{ sec: 0, bpm: 120 }];
const sigs = [...smf.timeSigs].sort((a, b) => a.sec - b.sec);
const bpmAt = (t: number) => { let b = tempos[0]!.bpm; for (const p of tempos) if (p.sec <= t + ZERO) b = p.bpm; return b; };
@@ -34,19 +47,78 @@ export function barTimes(smf: Smf): number[] {
}
return t;
};
const out: number[] = [];
const beats: number[] = [], bars: number[] = [];
let t = 0;
while (t < smf.durationSec - ZERO && out.length < 100000) {
out.push(t);
while (t < untilSec - ZERO && bars.length < 100000) {
bars.push(t);
let num = 4, den = 4;
for (const s of sigs) if (s.sec <= t + ZERO) { num = s.num; den = s.den; }
let next = advance(t, (num * 4) / den);
const change = sigs.find((s) => s.sec > t + ZERO && s.sec < next - ZERO); // signature change mid-bar: restart there
if (change) next = change.sec;
if (!(next > t)) break;
t = next;
const change = sigs.find((s) => s.sec > t + ZERO)?.sec ?? Infinity;
let b = t;
for (let k = 0; k < num; k++) {
beats.push(b);
const nb = advance(b, 4 / den);
if (nb >= change - ZERO) { b = change; break; } // signature change mid-bar: restart there
b = nb;
}
if (!(b > t)) break;
t = b;
}
return { beats, bars, end: t, lastBarPartial: bars.length > 0 && t > smf.durationSec + 1e-3 };
}
/** Bar start times (MIDI seconds), up to durationSec. */
export const barTimes = (smf: Smf): number[] => meterGrid(smf).bars;
/** Beat start times (MIDI seconds) of the bars starting before durationSec. */
export const beatTimes = (smf: Smf): number[] => meterGrid(smf).beats;
/** Time `k` grid steps after `t` on a sorted grid (fractional positions; linear extrapolation outside it). */
function gridShift(grid: number[], t: number, k: number): number {
const n = grid.length;
if (n < 2) return t;
const posOf = (x: number): number => {
if (x <= grid[0]!) return (x - grid[0]!) / (grid[1]! - grid[0]!);
if (x >= grid[n - 1]!) return n - 1 + (x - grid[n - 1]!) / (grid[n - 1]! - grid[n - 2]!);
let lo = 0, hi = n - 1;
while (hi - lo > 1) { const m = (lo + hi) >> 1; if (grid[m]! <= x) lo = m; else hi = m; }
return lo + (x - grid[lo]!) / (grid[hi]! - grid[lo]!);
};
const timeOf = (p: number): number => {
if (p <= 0) return grid[0]! + p * (grid[1]! - grid[0]!);
if (p >= n - 1) return grid[n - 1]! + (p - n + 1) * (grid[n - 1]! - grid[n - 2]!);
const i = Math.floor(p);
return grid[i]! + (p - i) * (grid[i + 1]! - grid[i]!);
};
return timeOf(posOf(t) + k);
}
const med = (xs: number[]): number => { const s = [...xs].sort((a, b) => a - b), h = s.length >> 1; return s.length % 2 ? s[h]! : (s[h - 1]! + s[h]!) / 2; };
/**
* Per-bar offsets (recording − warped, s) at MIDI time `midiSec`, with each bar's beat length (s): iteratively drops
* the bar deviating most from the robust median of its ±3 kept neighbours' offsets, while that deviation exceeds ¼
* beat. The neighbours' offsets are transported to the bar along their median pairwise (Theil–Sen) slope, so a smooth
* ramp (a genuine tempo difference) is kept whole instead of being eroded from its ends, where a plain one-sided median
* lags the ramp; beat/bar jumps are still rejected.
*/
export function rejectOffsetOutliers<T extends { midiSec: number; off: number; beatSec: number }>(bars: T[]): T[] {
const kept = [...bars];
for (;;) {
let worst = -1, worstDev = 0;
for (let k = 0; k < kept.length; k++) {
const nb = [...kept.slice(Math.max(0, k - OUTLIER_NEIGHBOURS), k), ...kept.slice(k + 1, k + 1 + OUTLIER_NEIGHBOURS)];
if (!nb.length) continue;
const slopes: number[] = [];
for (let a = 0; a < nb.length; a++) for (let b = a + 1; b < nb.length; b++) slopes.push((nb[b]!.off - nb[a]!.off) / (nb[b]!.midiSec - nb[a]!.midiSec));
const slope = slopes.length ? med(slopes) : 0;
const x = kept[k]!.midiSec;
const pred = med(nb.map((b) => b.off + slope * (x - b.midiSec)));
const dev = Math.abs(kept[k]!.off - pred) / kept[k]!.beatSec;
if (dev > OUTLIER_BEATS && dev > worstDev) { worst = k; worstDev = dev; }
}
if (worst < 0) return kept;
kept.splice(worst, 1);
}
return out;
}
/** Aggregate 50 Hz features by `k` frames (centred): chroma energy summed and re-normalised, onset max, re-z-scored. */
@@ -91,19 +163,38 @@ function costFn(A: Features, B: Features): (i: number, j: number) => number {
};
}
type Path = [number, number][];
/** Align rendered frames [i0, i1] with recording frames [j0, j1] (inclusive, fine rate), coarse then banded fine. */
function alignSegment(fine: [Features, Features], coarse: [Features, Features], i0: number, i1: number, j0: number, j1: number, bandsSec: number[]): Path {
type Path = [number, number][];
type Open = { openBegin: boolean; openEnd: boolean };
/** Slope-limited DTW; falls back to the unconstrained symmetric steps when the slope limit makes the end unreachable. */
function dtwSlope(costAt: (i: number, j: number) => number, n: number, m: number, open: Open, band?: { center: (i: number) => number; width: number }): Path {
const opts: DtwOptions = { steps: "slope", ...open };
try {
return dtw(costAt, n, m, band, opts);
} catch (e) {
if (band || !(e instanceof Error) || !/unreachable/.test(e.message)) throw e;
return dtw(costAt, n, m, undefined, { ...open, steps: "symmetric" });
}
}
/**
* Align rendered frames [i0, i1] with recording frames [j0, j1] (inclusive, fine rate), coarse then banded fine, both
* slope-limited. `open.openBegin`/`openEnd` let the path start after j0 / stop before j1 (or before i1).
*/
function alignSegment(fine: [Features, Features], coarse: [Features, Features], i0: number, i1: number, j0: number, j1: number, bandsSec: number[], open: Open): Path {
const K = COARSE_FACTOR;
const ci0 = Math.round(i0 / K), ci1 = Math.max(ci0, Math.min(coarse[0].chroma.length - 1, Math.round(i1 / K)));
const cj0 = Math.round(j0 / K), cj1 = Math.max(cj0, Math.min(coarse[1].chroma.length - 1, Math.round(j1 / K)));
const cc = costFn(coarse[0], coarse[1]);
const cp = dtw((i, j) => cc(ci0 + i, cj0 + j), ci1 - ci0 + 1, cj1 - cj0 + 1);
// coarse map: mean recording frame per rendered frame, in segment-relative coarse units
const sum = new Float64Array(ci1 - ci0 + 1), cnt = new Float64Array(ci1 - ci0 + 1);
const cp = dtwSlope((i, j) => cc(ci0 + i, cj0 + j), ci1 - ci0 + 1, cj1 - cj0 + 1, open);
// coarse map: mean recording frame per rendered frame, in segment-relative coarse units; rows the path does not
// reach (open end on the last column) continue at slope 1
const nRows = ci1 - ci0 + 1;
const sum = new Float64Array(nRows), cnt = new Float64Array(nRows);
for (const [i, j] of cp) { sum[i]! += j; cnt[i]! += 1; }
const cmap = Array.from(sum, (s, i) => s / cnt[i]!);
const cmap = Array.from(sum, (v, i) => (cnt[i] ? v / cnt[i]! : NaN));
for (let i = 1; i < nRows; i++) if (Number.isNaN(cmap[i]!)) cmap[i] = cmap[i - 1]! + 1;
const centre = (i: number): number => { // fine row (segment-relative) -> fine column (segment-relative)
const x = Math.min(cmap.length - 1, Math.max(0, (i0 + i) / K - ci0));
const a = Math.floor(x), b = Math.min(cmap.length - 1, a + 1);
@@ -111,7 +202,7 @@ function alignSegment(fine: [Features, Features], coarse: [Features, Features],
return (y + cj0) * K - j0;
};
const fc = costFn(fine[0], fine[1]);
const p = bandedWithRetry((i, j) => fc(i0 + i, j0 + j), i1 - i0 + 1, j1 - j0 + 1, centre, bandsSec);
const p = bandedWithRetry((i, j) => fc(i0 + i, j0 + j), i1 - i0 + 1, j1 - j0 + 1, centre, bandsSec, open);
if (p) return p.map(([i, j]) => [i0 + i, j0 + j]);
return densify(cp, ci0, cj0, i0, i1, j0, j1);
}
@@ -119,19 +210,20 @@ function alignSegment(fine: [Features, Features], coarse: [Features, Features],
/**
* Fallback: the coarse path expanded to a contiguous fine path from (i0, j0) to (i1, j1). Each coarse step becomes K
* fine steps in the same direction (so coarse diagonals stay diagonal and still yield anchors), clamped to the
* segment; any remainder at the end is closed with horizontal/vertical steps.
* segment; the lead-in to the coarse path's first cell and any remainder at the end use horizontal/vertical steps
* (which carry no timing).
*/
function densify(cp: Path, ci0: number, cj0: number, i0: number, i1: number, j0: number, j1: number): Path {
const K = COARSE_FACTOR;
const out: Path = [[i0, j0]];
let i = i0, j = j0;
// the coarse grid origin may sit up to K/2 fine frames off (i0, j0); align to it first
const step = (di: number, dj: number) => {
const ni = Math.min(i1, i + di), nj = Math.min(j1, j + dj);
if (ni !== i || nj !== j) { i = ni; j = nj; out.push([i, j]); }
};
while (i < Math.min(i1, ci0 * K)) step(1, 0);
while (j < Math.min(j1, cj0 * K)) step(0, 1);
const [si, sj] = cp[0] ?? [0, 0]; // the coarse path may start off the corner (open begin)
while (i < Math.min(i1, (ci0 + si) * K)) step(1, 0);
while (j < Math.min(j1, (cj0 + sj) * K)) step(0, 1);
for (let k = 1; k < cp.length; k++) {
const di = cp[k]![0] - cp[k - 1]![0], dj = cp[k]![1] - cp[k - 1]![1];
for (let q = 0; q < K; q++) step(di, dj);
@@ -140,13 +232,14 @@ function densify(cp: Path, ci0: number, cj0: number, i0: number, i1: number, j0:
return out;
}
/** Banded fine DTW, widening the band (±0.5 s, ±1 s, ±2 s) when the end cell is unreachable; null if all fail. */
/** Banded fine DTW (slope-limited), widening the band (±0.5 s, ±1 s, ±2 s) when the end is unreachable; null if all fail. */
export function bandedWithRetry(
costAt: (i: number, j: number) => number, n: number, m: number, centre: (i: number) => number, bandsSec = FINE_BANDS_SEC,
open: Open = { openBegin: false, openEnd: false },
): Path | null {
for (const sec of bandsSec) {
try {
return dtw(costAt, n, m, { center: centre, width: Math.round(sec * FINE_RATE) });
return dtw(costAt, n, m, { center: centre, width: Math.round(sec * FINE_RATE) }, { steps: "slope", ...open });
} catch (e) {
if (!(e instanceof Error) || !/unreachable/.test(e.message)) throw e;
}
@@ -189,38 +282,153 @@ function compose(manual: Anchor[], auto: Anchor[]): Anchor[] {
const durationOf = (a: Audio) => (a.channels[0]?.length ?? 0) / a.sampleRate;
/** Fit of `warp` against recording onsets: warped onsets of scored-role notes (roles keyed by `trackKeys`). */
function fitFor(recOnsets: number[], smf: Smf, roles: Record<string, Role | null>, warp: Warp, durSec: number): FitReport {
/** What the fit needs from the recording: onset-flux peaks (s) and 10 Hz chroma. Compute once per song. */
export interface RecordingCues { onsets: number[]; chroma: Float32Array[]; chromaRate: number; durSec: number }
function cuesFrom(fA: Features, cA: Features, durSec: number): RecordingCues {
return { onsets: pickOnsets(fA.onset, FINE_RATE), chroma: cA.chroma, chromaRate: cA.rate, durSec };
}
/** Onset-flux peaks and chroma of the raw recording. Compute once and reuse with `fitFromCues` to re-score edits cheaply. */
export function recordingCues(audio: Audio): RecordingCues {
const fA = features(audio, FINE_RATE);
return cuesFrom(fA, downsample(fA, COARSE_FACTOR), durationOf(audio));
}
/** Scored-role tracks (by `trackKeys`). */
function scoredTracks(smf: Smf, roles: Record<string, Role | null>) {
const keys = trackKeys(smf.tracks);
const midi = smf.tracks.flatMap((t, i) => (roles[keys[i]!] ? t.notes.map((n) => warp.toAudio(n.onset)) : []));
return fitQuality(recOnsets, midi, durSec);
return smf.tracks.filter((_, i) => roles[keys[i]!]);
}
/** Onset-flux peaks of the raw recording (seconds). Compute once and reuse with `fitFromOnsets` to re-score edits cheaply. */
export function recordingOnsets(audio: Audio): number[] {
return pickOnsets(features(audio, FINE_RATE).onset, FINE_RATE);
/** Piano-roll chroma (pitch classes of sounding pitched notes, 1 per note) with MIDI time placed by `place`. */
function symbolicChroma(tracks: Smf["tracks"], place: (m: number) => number, nFrames: number, rate: number): Float32Array[] {
const out = Array.from({ length: nFrames }, () => new Float32Array(12));
for (const t of tracks) {
if (t.isDrums) continue;
for (const n of t.notes) {
const a = Math.max(0, Math.round(place(n.onset) * rate));
const b = Math.min(nFrames, Math.max(a + 1, Math.round(place(n.offset) * rate)));
for (let f = a; f < b; f++) out[f]![n.pitch % 12]! += 1;
}
}
return out;
}
/** `alignmentFit` with precomputed recording onsets; `durSec` = recording duration. */
export function fitFromOnsets(onsets: number[], smf: Smf, roles: Record<string, Role | null>, anchors: Anchor[], durSec: number): FitReport {
return fitFor(onsets, smf, roles, makeWarp(anchors), durSec);
/** Warped MIDI onsets of scored-role notes. */
const scoredOnsets = (tracks: Smf["tracks"], place: (m: number) => number) => tracks.flatMap((t) => t.notes.map((n) => place(n.onset)));
/** Fit of `anchors`: precision/lift/audible rate, the beat/bar-shift chroma check and stretch-warning counts. */
export function fitFromCues(cues: RecordingCues, smf: Smf, roles: Record<string, Role | null>, anchors: Anchor[]): FitReport {
const warp = makeWarp(anchors);
const tracks = scoredTracks(smf, roles);
const nFrames = cues.chroma.length, rate = cues.chromaRate;
const g = meterGrid(smf, smf.durationSec + 30);
const barSteps = (m: number, k: number) => gridShift(g.bars, m, k);
const beatSteps = (m: number, k: number) => gridShift(g.beats, m, k);
// "+k beat(s)": the MIDI content belongs k beats later in the recording than the current alignment puts it
const cands: [string, (m: number) => number][] = [
["+1 beat", (m) => beatSteps(m, 1)], ["−1 beat", (m) => beatSteps(m, -1)],
["+2 beats", (m) => beatSteps(m, 2)], ["−2 beats", (m) => beatSteps(m, -2)],
["+1 bar", (m) => barSteps(m, 1)], ["−1 bar", (m) => barSteps(m, -1)],
];
const shift: ShiftInputs = {
rate, recording: cues.chroma,
current: symbolicChroma(tracks, warp.toAudio, nFrames, rate),
shifts: cands.map(([label, f]) => ({ label, chroma: symbolicChroma(tracks, (m) => warp.toAudio(f(m)), nFrames, rate) })),
};
const st = stretches(anchors, smf.durationSec);
const counts: StretchCounts = { total: st.length, warned: st.filter((s) => s.warn).length, over15: st.filter((s) => Math.abs(1 / s.ratio - 1) > OVER_TEMPO).length };
return fitQuality(cues.onsets, scoredOnsets(tracks, warp.toAudio), cues.durSec, { shift, stretches: counts });
}
/** Fit quality of any alignment (e.g. a hand-made one): onset-flux peaks of the raw recording vs the warped MIDI. */
/** Fit quality of any alignment (e.g. a hand-made one) against the raw recording. */
export function alignmentFit(audio: Audio, smf: Smf, roles: Record<string, Role | null>, anchors: Anchor[]): FitReport {
return fitFromOnsets(recordingOnsets(audio), smf, roles, anchors, durationOf(audio));
return fitFromCues(recordingCues(audio), smf, roles, anchors);
}
/** Best constant offset (recording − MIDI, s) and its precision; `cosine` = mean symbolic-chroma cosine there. */
export interface Baseline { offsetSec: number; precision: number | null; cosine: number | null }
/**
* Constant-offset baseline: grid search of −2..+2 s in 10 ms steps maximising precision. Whole-beat shifts alias onto
* the onset grid (equal precision), so the precision peaks within 0.1 of the best are told apart by the mean chroma
* cosine of the symbolic MIDI vs the recording. Within a peak precision is flat across the ±70 ms tolerance, so the
* offset with the smallest mean residual (among those within 0.05 of the peak's precision) is reported.
*/
export function constantOffsetBaseline(cues: RecordingCues, smf: Smf, roles: Record<string, Role | null>): Baseline {
const tracks = scoredTracks(smf, roles);
const midi = uniqueSorted(scoredOnsets(tracks, (m) => m));
const steps = Math.round(BASELINE_RANGE_SEC / BASELINE_STEP_SEC);
const offs = Array.from({ length: 2 * steps + 1 }, (_, k) => Math.round((k - steps) * BASELINE_STEP_SEC * 100) / 100);
const prec = offs.map((o) => precisionOf(cues.onsets, midi, o));
if (prec.every((p) => p === null)) return { offsetSec: 0, precision: null, cosine: null };
const best = Math.max(...prec.map((p) => p ?? -1));
// peaks: runs of consecutive offsets within the tie margin
const runs: number[][] = [];
for (let k = 0; k < offs.length; k++) {
if ((prec[k] ?? -1) < best - BASELINE_TIE) continue;
if (k > 0 && (prec[k - 1] ?? -1) >= best - BASELINE_TIE) runs[runs.length - 1]!.push(k); else runs.push([k]);
}
// within a run, precision is flat across the ±70 ms tolerance (with spikes at its edges, where neighbouring notes
// start to match): take the offset with the smallest mean residual among those within 0.05 of the run's best
const refine = (run: number[]): number => {
const top = Math.max(...run.map((k) => prec[k]!));
let pick = run[0]!, bestRes = Infinity;
for (const k of run) {
if (prec[k]! < top - BASELINE_PLATEAU) continue;
const r = meanResidual(cues.onsets, midi, offs[k]!);
if (r < bestRes) { bestRes = r; pick = k; }
}
return pick;
};
const cosAt = (o: number) => meanCosine(cues.chroma, symbolicChroma(tracks, (m) => m + o, cues.chroma.length, cues.chromaRate), 0, cues.chroma.length, 0);
let pick = refine(runs[0]!), pickCos = runs.length > 1 ? cosAt(offs[pick]!) : null;
for (const run of runs.slice(1)) {
const k = refine(run), c = cosAt(offs[k]!);
if (c !== null && (pickCos === null || c > pickCos)) { pick = k; pickCos = c; }
}
return { offsetSec: offs[pick]!, precision: prec[pick]!, cosine: pickCos ?? cosAt(offs[pick]!) };
}
/** Mean |recording onset − nearest (MIDI + shift)| over recording onsets matched within ±70 ms. */
function meanResidual(rec: number[], midi: number[], shift: number): number {
let s = 0, n = 0;
for (const t of rec) {
const x = t - shift;
let lo = 0, hi = midi.length;
while (lo < hi) { const m = (lo + hi) >> 1; if (midi[m]! < x) lo = m + 1; else hi = m; }
const d = Math.min(lo < midi.length ? Math.abs(midi[lo]! - x) : Infinity, lo > 0 ? Math.abs(midi[lo - 1]! - x) : Infinity);
if (d <= MATCH_TOL_SEC + 1e-9) { s += d; n++; }
}
return n ? s / n : Infinity;
}
/** A single auto anchor realising a constant offset: at the bar nearest the middle of the MIDI (positive audio time). */
function baselineAnchors(smf: Smf, offsetSec: number): Anchor[] {
const bars = barTimes(smf).filter((b) => b + offsetSec > 0.01);
const mid = smf.durationSec / 2;
const m = bars.length ? bars.reduce((a, b) => (Math.abs(b - mid) < Math.abs(a - mid) ? b : a)) : Math.max(0, -offsetSec) + 1;
return [{ midiSec: m, audioSec: m + offsetSec, kind: "auto" }];
}
/** `dtwAnchors`/`dtwPrecision`: the DTW alignment and its precision, also when the baseline replaced it. */
export interface AutoResult { anchors: Anchor[]; fit: FitReport; baseline: Baseline; chosen: "baseline" | "dtw"; dtwAnchors: Anchor[]; dtwPrecision: number | null }
/**
* Align `smf` to `audio`, honouring `manual` anchors exactly. The MIDI is rendered under the manual warp, then each
* stretch between consecutive manual anchors (plus start/end) is aligned by a 10 Hz DTW refined at 50 Hz in a band
* around it. One auto anchor is placed per bar (skipping bars within 1 s of a manual anchor). `roles` is keyed by
* `trackKeys`; tracks without a role are rendered but excluded from the fit.
* stretch between consecutive manual anchors (plus start/end) is aligned by a slope-limited 10 Hz DTW refined at 50 Hz
* in a band around it; the first stretch may start late in the recording (open begin) and the last one may end early
* on either axis (open end), so extra intro/outro material does not force the path. One auto anchor is placed per bar
* (skipping bars within 1 s of a manual anchor and the final partial bar); per-bar offsets more than ¼ beat from the
* median of their ±3 neighbours are rejected. Without manual anchors the result is replaced by the constant-offset
* baseline (one auto anchor) unless it beats the baseline's precision by ≥ 0.02. `roles` is keyed by `trackKeys`;
* tracks without a role are rendered but excluded from the fit.
*/
export function autoAlign(
audio: Audio, smf: Smf, roles: Record<string, Role | null>, manual: Anchor[], sf2: Uint8Array,
opts: { fineBandsSec?: number[] } = {}, // band half-widths for the fine pass; [] forces the coarse-path fallback
): { anchors: Anchor[]; fit: FitReport } {
): AutoResult {
const man = makeWarp(manual).anchors.map((a) => ({ ...a, kind: "manual" as const }));
const W0 = makeWarp(man);
const audioDur = durationOf(audio);
@@ -229,17 +437,18 @@ export function autoAlign(
const fR = features(rendered, FINE_RATE), fA = features(audio, FINE_RATE);
const cR = downsample(fR, COARSE_FACTOR), cA = downsample(fA, COARSE_FACTOR);
const lastR = fR.chroma.length - 1, lastA = fA.chroma.length - 1;
const midiEnd = Math.max(1, Math.min(lastR, Math.round(W0.toAudio(smf.durationSec) * FINE_RATE))); // last rendered frame with MIDI content
// segment boundaries: manual anchors map warped time == recording time, so both axes split at the same instants
const cuts = man.map((a) => Math.round(a.audioSec * FINE_RATE)).filter((f) => f > 0 && f < Math.min(lastR, lastA));
const cuts = man.map((a) => Math.round(a.audioSec * FINE_RATE)).filter((f) => f > 0 && f < Math.min(midiEnd, lastA));
const bounds = [...new Set(cuts)].sort((a, b) => a - b);
// per segment: warped-time span and diagonal path steps (they carry timing; horizontal/vertical runs are stalls,
// silence or unmatched material and do not)
// per segment: warped-time span and diagonal path steps (they carry timing; horizontal/vertical steps do not)
const segs: { lo: number; hi: number; diag: { w: number; off: number }[] }[] = [];
let prev = 0;
for (const b of [...bounds, -1]) {
const i1 = b < 0 ? lastR : b, j1 = b < 0 ? lastA : b;
const path = alignSegment([fR, fA], [cR, cA], prev, i1, prev, j1, opts.fineBandsSec ?? FINE_BANDS_SEC);
const i1 = b < 0 ? midiEnd : b, j1 = b < 0 ? lastA : b;
const open = { openBegin: prev === 0, openEnd: b < 0 };
const path = alignSegment([fR, fA], [cR, cA], prev, i1, prev, j1, opts.fineBandsSec ?? FINE_BANDS_SEC, open);
const diag: { w: number; off: number }[] = [];
for (let k = 1; k < path.length; k++) {
const [i, j] = path[k]!, [pi, pj] = path[k - 1]!;
@@ -249,8 +458,12 @@ export function autoAlign(
prev = b;
}
const auto: Anchor[] = [];
for (const m of barTimes(smf)) {
const grid = meterGrid(smf);
const lastBar = grid.lastBarPartial ? grid.bars[grid.bars.length - 1] : undefined;
const beatAt = (m: number) => { const k = grid.beats.findIndex((b) => b > m + ZERO); return k > 0 ? grid.beats[k]! - grid.beats[k - 1]! : 0.5; };
const perBar: { midiSec: number; w: number; off: number; beatSec: number }[] = [];
for (const m of grid.bars) {
if (m === lastBar) continue; // never anchor the final partial bar
if (man.some((a) => Math.abs(a.midiSec - m) < MANUAL_CLEARANCE_SEC)) continue;
const w = W0.toAudio(m);
const seg = segs.find((s) => w < s.hi) ?? segs[segs.length - 1]!;
@@ -258,9 +471,25 @@ export function autoAlign(
if (off === null) continue;
const a = w + off;
if ((m > 0 ? a <= 0 : a < 0) || a > audioDur) continue; // no degenerate first stretch
auto.push({ midiSec: m, audioSec: a, kind: "auto" });
perBar.push({ midiSec: m, w, off, beatSec: beatAt(m) * (W0.toAudio(m + 1) - w) });
}
const auto = rejectOffsetOutliers(perBar).map((p): Anchor => ({ midiSec: p.midiSec, audioSec: p.w + p.off, kind: "auto" }));
const anchors = compose(man, auto);
const fit = fitFor(pickOnsets(fA.onset, FINE_RATE), smf, roles, makeWarp(anchors), audioDur);
return { anchors, fit };
const cues = cuesFrom(fA, cA, audioDur);
const fit = fitFromCues(cues, smf, roles, anchors);
const baseline = constantOffsetBaseline(cues, smf, roles);
const beats = fit.precision !== null && baseline.precision !== null && fit.precision >= baseline.precision + BASELINE_MARGIN;
if (man.length || baseline.precision === null || beats) return { anchors, fit, baseline, chosen: "dtw", dtwAnchors: anchors, dtwPrecision: fit.precision };
const b = baselineAnchors(smf, baseline.offsetSec);
return { anchors: b, fit: fitFromCues(cues, smf, roles, b), baseline, chosen: "baseline", dtwAnchors: anchors, dtwPrecision: fit.precision };
}
/** e.g. "DTW (precision 93.5%) · constant-offset baseline +0.06 s: precision 88.0%"; with "chosen: baseline" the
* single-anchor constant offset replaced a DTW alignment that was not better by 2 points. */
export function autoSummary(r: { chosen: "baseline" | "dtw"; baseline: { offsetSec: number; precision: number | null }; dtwPrecision: number | null }): string {
const pc = (x: number | null) => (x === null ? "n/a" : `${(x * 100).toFixed(1)}%`);
const off = `${r.baseline.offsetSec >= 0 ? "+" : "−"}${Math.abs(r.baseline.offsetSec).toFixed(2)} s`;
const base = `constant-offset baseline ${off}: precision ${pc(r.baseline.precision)}`;
return r.chosen === "dtw" ? `chosen: DTW (precision ${pc(r.dtwPrecision)}) · ${base}`
: `chosen: baseline (DTW precision ${pc(r.dtwPrecision)} not better by ≥ 2 points) · ${base}`;
}
+3 -2
View File
@@ -1,7 +1,7 @@
import { parseArgs } from "node:util";
import { addSong, loadSong, readAlignment, trackKeys, writeAlignment } from "../ref/manifest.ts";
import { parseSmf } from "./smf.ts";
import { autoAlign } from "./auto.ts";
import { autoAlign, autoSummary } from "./auto.ts";
import { fitSummary } from "./fit.ts";
import { SF2_PATH, startServer } from "./server.ts";
@@ -51,8 +51,9 @@ async function auto(argv: string[]) {
const sf2 = new Uint8Array(await Bun.file(SF2_PATH).arrayBuffer());
const t0 = performance.now();
const r = autoAlign(s.audio, s.smf, roles, manual, sf2);
await writeAlignment(id, { anchors: r.anchors, savedAt: new Date().toISOString(), fit: r.fit }, root);
await writeAlignment(id, { anchors: r.anchors, savedAt: new Date().toISOString(), fit: r.fit, auto: { chosen: r.chosen, baseline: r.baseline, dtwPrecision: r.dtwPrecision } }, root);
console.log(`${id}: ${r.anchors.length} anchors (${manual.length} manual kept) in ${((performance.now() - t0) / 1000).toFixed(1)} s -> ${root}/${id}/alignment.json`);
console.log(`method: ${autoSummary(r)}`);
console.log(`fit: ${fitSummary(r.fit)}`);
}
+72 -22
View File
@@ -1,18 +1,33 @@
// Pure (banded) dynamic time warping. Only the two latest accumulated-cost rows are kept; backpointers are one byte
// per cell (O(n·band) when banded), so a 3000×3000 unbanded run uses ≈ 9 MB.
// Pure (banded) dynamic time warping. Only the latest accumulated-cost rows are kept; backpointers are one byte per
// cell (O(n·band) when banded), so a 3000×3000 unbanded run uses ≈ 9 MB.
export interface DtwBand { center: (i: number) => number; width: number }
/**
* steps: "symmetric" (default) — steps (1,0), (0,1), (1,1), each paying the cost of the cell entered.
* "slope" — slope-limited (Sakoe–Chiba P = 1, asymmetric weights): (1,1), (1,2) via (i, j−1), (2,1) via
* (i−1, j). The local slope stays within [1/2, 2], so the path cannot make abrupt jumps; each row (query frame)
* carries weight 1, so paths that consume all rows are comparable whatever columns they cover.
* openBegin: the path may start anywhere on row 0 (the target may have extra material before the query).
* openEnd: the path may end anywhere on the last row or the last column (target or query has extra material at the
* end); candidates are compared by mean cost per query row, each query row left unconsumed costing 1 (the
* cost of sound vs silence), so a trivial path into a cheap corner cannot win.
*/
export interface DtwOptions { steps?: "symmetric" | "slope"; openBegin?: boolean; openEnd?: boolean }
const DIAG = 0, UP = 1, LEFT = 2; // from (i-1,j-1), (i-1,j), (i,j-1)
const DIAG = 0, UP = 1, LEFT = 2, S12 = 3, S21 = 4; // from (i-1,j-1), (i-1,j), (i,j-1), (i-1,j-2) via (i,j-1), (i-2,j-1) via (i-1,j)
const START = 5;
const UNCONSUMED_ROW_COST = 1;
/** Minimum-cost monotone path from (0,0) to (n-1,m-1) with steps (1,0), (0,1), (1,1). */
/** Minimum-cost monotone path from (0,0) to (n-1,m-1) (or open ends, see `DtwOptions`). */
export function dtw(
costAt: (i: number, j: number) => number,
n: number,
m: number,
band?: DtwBand,
opts: DtwOptions = {},
): [number, number][] {
if (n <= 0 || m <= 0) return [];
const slope = opts.steps === "slope", openBegin = !!opts.openBegin, openEnd = !!opts.openEnd;
const lo = new Int32Array(n);
const hi = new Int32Array(n);
const off = new Int32Array(n + 1);
@@ -22,8 +37,8 @@ export function dtw(
const c = Math.round(band.center(i));
a = Math.max(0, c - band.width);
b = Math.min(m - 1, c + band.width);
if (i === 0) a = 0;
if (i === n - 1) b = m - 1;
if (i === 0 && !openBegin) a = 0;
if (i === n - 1 && !openEnd) b = m - 1;
if (a > b) { // centre outside the matrix: clamp to the nearest edge
a = b = Math.min(m - 1, Math.max(0, c));
}
@@ -32,36 +47,71 @@ export function dtw(
off[i + 1] = off[i]! + (b - a + 1);
}
const bp = new Uint8Array(off[n]!);
let prev = new Float64Array(m).fill(Infinity);
let cur = new Float64Array(m).fill(Infinity);
// accumulated cost rows (row i in D[i % 3]) and cell-cost rows (row i in C[i % 2]); full width, cleared by range
const D = [0, 1, 2].map(() => new Float64Array(m).fill(Infinity));
const C = [0, 1].map(() => new Float64Array(m));
const inRow = (r: number, j: number) => r >= 0 && j >= lo[r]! && j <= hi[r]!;
const acc = (r: number, j: number) => (inRow(r, j) ? D[r % 3]![j]! : Infinity);
// open-end candidates: mean cost per query row, each row left unconsumed (ending on the last column) costing 1
let endScore = Infinity, endI = n - 1, endJ = m - 1;
const consider = (i: number, v: number, j: number) => {
const score = (v + (n - 1 - i) * UNCONSUMED_ROW_COST) / n;
if (v < Infinity && score < endScore - 1e-12) { endScore = score; endI = i; endJ = j; }
};
for (let i = 0; i < n; i++) {
const a = lo[i]!, b = hi[i]!, base = off[i]! - a;
const pa = i > 0 ? lo[i - 1]! : 0, pb = i > 0 ? hi[i - 1]! : -1;
cur.fill(Infinity, Math.max(0, a - 1), Math.min(m, b + 2));
const cur = D[i % 3]!, cc = C[i % 2]!, pc = C[(i + 1) % 2]!;
if (i >= 3) cur.fill(Infinity, lo[i - 3]!, hi[i - 3]! + 1);
for (let j = a; j <= b; j++) cc[j] = costAt(i, j);
for (let j = a; j <= b; j++) {
let best = Infinity, dir = DIAG;
if (i === 0 && j === 0) best = 0;
else {
if (i > 0 && j > 0 && j - 1 >= pa && j - 1 <= pb) { best = prev[j - 1]!; dir = DIAG; }
if (i > 0 && j >= pa && j <= pb && prev[j]! < best) { best = prev[j]!; dir = UP; }
const d = cc[j]!;
if (i === 0 && (j === 0 || openBegin)) { best = 0; dir = START; }
else if (slope) {
const dg = acc(i - 1, j - 1);
if (dg + d < best) { best = dg + d; dir = DIAG; }
const h = acc(i - 1, j - 2);
if (h < Infinity) {
const v = h + ((j - 1 >= a ? cc[j - 1]! : costAt(i, j - 1)) + d) / 2;
if (v < best) { best = v; dir = S12; }
}
const u = acc(i - 2, j - 1);
if (u < Infinity) {
const v = u + (inRow(i - 1, j) ? pc[j]! : costAt(i - 1, j)) + d;
if (v < best) { best = v; dir = S21; }
}
} else {
const dg = acc(i - 1, j - 1);
if (dg < best) { best = dg; dir = DIAG; }
const up = acc(i - 1, j);
if (up < best) { best = up; dir = UP; }
if (j > a && cur[j - 1]! < best) { best = cur[j - 1]!; dir = LEFT; }
if (best < Infinity) best += d;
}
if (best === Infinity) { cur[j] = Infinity; continue; }
cur[j] = best + costAt(i, j);
cur[j] = dir === START ? d : best;
bp[base + j] = dir;
}
// clear stale values outside this row's range so the next row never reads them
if (i > 0) prev.fill(Infinity, pa, pb + 1);
[prev, cur] = [cur, prev];
if (openEnd && i < n - 1 && b === m - 1) consider(i, cur[m - 1]!, m - 1);
}
if (prev[m - 1]! === Infinity) throw new Error("dtw: end cell unreachable within band");
// choose the end cell (last-column candidates were collected during the sweep)
let ei = n - 1, ej = m - 1;
if (openEnd) {
for (let j = lo[n - 1]!; j <= hi[n - 1]!; j++) consider(n - 1, acc(n - 1, j), j);
if (!(endScore < Infinity)) throw new Error("dtw: end cell unreachable within band");
ei = endI; ej = endJ;
} else if (acc(n - 1, m - 1) === Infinity) throw new Error("dtw: end cell unreachable within band");
const path: [number, number][] = [];
let i = n - 1, j = m - 1;
let i = ei, j = ej;
for (;;) {
path.push([i, j]);
if (i === 0 && j === 0) break;
const d = bp[off[i]! - lo[i]! + j]!;
if (d === DIAG) { i--; j--; } else if (d === UP) i--; else j--;
if (d === START) break;
if (d === DIAG) { i--; j--; }
else if (d === UP) i--;
else if (d === LEFT) j--;
else if (d === S12) { path.push([i, j - 1]); i--; j -= 2; }
else { path.push([i - 1, j]); i -= 2; j--; }
}
return path.reverse();
}
+17 -8
View File
@@ -108,7 +108,11 @@ export function features(audio: Audio, rate: number): Features {
return { rate, onset, chroma };
}
/** Peak-pick an onset-strength curve (adaptive threshold median + k·MAD, 50 ms minimum gap). Times in seconds. */
/**
* Peak-pick an onset-strength curve (adaptive threshold median + k·MAD, 50 ms minimum gap, the stronger peak wins).
* Peak times are refined to sub-frame precision by parabolic interpolation over the peak and its two neighbours.
* Times in seconds.
*/
export function pickOnsets(onset: Float32Array, rate: number, k = 4): number[] {
const n = onset.length;
if (n === 0) return [];
@@ -118,15 +122,20 @@ export function pickOnsets(onset: Float32Array, rate: number, k = 4): number[] {
const mad = dev[n >> 1]! * 1.4826;
const thr = med + k * Math.max(mad, 1e-3);
const minGap = 0.05;
const picked: number[] = [];
const frames: number[] = [];
for (let i = 0; i < n; i++) {
const v = onset[i]!;
if (v <= thr || v < (onset[i - 1] ?? -Infinity) || v <= (onset[i + 1] ?? -Infinity)) continue;
const t = i / rate;
const last = picked.length - 1;
if (last >= 0 && t - picked[last]! < minGap - 1e-9) {
if (v > onset[Math.round(picked[last]! * rate)]!) picked[last] = t;
} else picked.push(t);
const last = frames.length - 1;
if (last >= 0 && (i - frames[last]!) / rate < minGap - 1e-9) {
if (v > onset[frames[last]!]!) frames[last] = i;
} else frames.push(i);
}
return picked;
return frames.map((i) => {
if (i === 0 || i === n - 1) return i / rate;
const a = onset[i - 1]!, b = onset[i]!, c = onset[i + 1]!;
const den = a - 2 * b + c;
const d = den < 0 ? Math.max(-0.5, Math.min(0.5, (0.5 * (a - c)) / den)) : 0;
return (i + d) / rate;
});
}
+157 -38
View File
@@ -1,13 +1,58 @@
// Fit quality of an alignment: how well warped MIDI onsets land on onsets picked from the raw recording.
// Fit quality of an alignment: how well onsets picked from the raw recording land on the warped MIDI onsets.
// Pure: no I/O. Recording onsets must come from the audio itself (onset flux), never from transcription.
//
// Calibrated on real mixes (ruling A7, spec "Fit quality", amended 2026-10-30): the onset picker finds ~1.4 attacks/s
// in a full mix while a scored MIDI has ~13 unique onsets/s, so the share of MIDI onsets that are heard ("audible
// rate", the old match rate) is capped near 40% even when perfectly aligned and carries no information. The flag
// metric is therefore *precision*: the share of recording onsets with a warped MIDI onset within ±70 ms. Because a
// dense score matches almost anything, precision is chance-corrected by the precision at off-grid shifts (lift), and a
// symbolic chroma check catches whole-beat / whole-bar shifts that onset metrics cannot see (they alias onto the grid).
export interface FitWindow { startSec: number; matchRate: number | null; medianResidualMs: number | null; flagged: boolean }
export interface FitReport { matchRate: number | null; medianResidualMs: number | null; windows: FitWindow[] }
/** A beat/bar shift that explains the recording's chroma better than the current alignment. */
export interface BeatShift { label: string; gain: number }
export interface FitWindow {
startSec: number;
/** Recording onsets in this window (by recording time). */
recOnsets: number;
/** Share of this window's recording onsets with a warped MIDI onset within ±70 ms; null with < 8 recording onsets. */
precision: number | null;
/** precision − mean precision at the off-grid shifts (chance-corrected); null when precision is null. */
lift: number | null;
/** Information only (JSON `audibleRate`): share of unique MIDI onsets (in this window by warped time) heard as attacks. */
audibleRate: number | null;
/** Median |recording onset − nearest warped MIDI onset| over matched recording onsets (ms). */
medianResidualMs: number | null;
beatShift: BeatShift | null;
flagged: boolean;
/** Why the window is flagged, e.g. "precision 41%", "lift 0.12", "beat shift suspected: +1 beat". */
reasons: string[];
}
export interface StretchCounts { total: number; warned: number; over15: number }
export interface FitReport {
precision: number | null;
lift: number | null;
/** Information only: share of unique MIDI onsets heard as attacks ("MIDI onsets audible as attacks"). */
audibleRate: number | null;
medianResidualMs: number | null;
windows: FitWindow[];
/** Stretches of the alignment: total, warned (> 5% tempo change) and with |tempo change| > 15%. */
stretches?: StretchCounts;
}
/** Symbolic-chroma inputs for the beat/bar-shift check: frames at `rate` Hz in recording time, 12-d (any scale). */
export interface ShiftInputs { rate: number; recording: Float32Array[]; current: Float32Array[]; shifts: { label: string; chroma: Float32Array[] }[] }
export const MATCH_TOL_SEC = 0.07;
export const WINDOW_SEC = 30;
const FLAG_MIN_RATE = 0.5;
const FLAG_MAX_RESIDUAL_MS = 60;
export const FLAG_MIN_PRECISION = 0.6;
export const FLAG_MIN_LIFT = 0.25;
export const MIN_REC_ONSETS = 8;
/** Off-grid shifts (s) of the warped MIDI used as the chance level of precision. */
export const CHANCE_SHIFTS_SEC = [0.11, 0.23, 0.37];
/** A shifted alignment must beat the current one by this much mean chroma cosine to flag the window. */
export const SHIFT_MARGIN = 0.05;
/** The current symbolic chroma must cover at least this share of the window's sounding frames for the check to run. */
const SHIFT_MIN_COVER = 0.25;
const UNIQUE_EPS = 1e-6;
function median(xs: number[]): number | null {
@@ -16,52 +61,126 @@ function median(xs: number[]): number | null {
return s.length % 2 ? s[m]! : (s[m - 1]! + s[m]!) / 2;
}
/** |residual| in seconds to the nearest recording onset, or null when none lies within ±70 ms. */
function nearest(sorted: number[], t: number): number | null {
/** |distance| from `t` to the nearest value in `sorted` (Infinity when empty). */
function nearestDist(sorted: number[], t: number): number {
let lo = 0, hi = sorted.length;
while (lo < hi) { const m = (lo + hi) >> 1; if (sorted[m]! < t) lo = m + 1; else hi = m; }
let best = Infinity;
if (lo < sorted.length) best = Math.min(best, Math.abs(sorted[lo]! - t));
if (lo > 0) best = Math.min(best, Math.abs(sorted[lo - 1]! - t));
return best <= MATCH_TOL_SEC + 1e-9 ? best : null;
return best;
}
const hit = (d: number) => d <= MATCH_TOL_SEC + 1e-9;
/** Sorted, de-duplicated copy. */
export function uniqueSorted(xs: number[]): number[] {
const out: number[] = [];
for (const t of [...xs].sort((a, b) => a - b)) if (!out.length || t - out[out.length - 1]! > UNIQUE_EPS) out.push(t);
return out;
}
function summarise(residuals: (number | null)[]): { matchRate: number | null; medianResidualMs: number | null } {
if (!residuals.length) return { matchRate: null, medianResidualMs: null };
const hits = residuals.filter((r): r is number => r !== null);
const med = median(hits);
return { matchRate: hits.length / residuals.length, medianResidualMs: med === null ? null : med * 1000 };
/** Share of `rec` (any order) with a value of sorted `midi` + `shift` within ±70 ms; null when `rec` or `midi` is empty. */
export function precisionOf(rec: number[], midi: number[], shift = 0): number | null {
if (!rec.length || !midi.length) return null;
let n = 0;
for (const t of rec) if (hit(nearestDist(midi, t - shift))) n++;
return n / rec.length;
}
function cos(a: Float32Array, b: Float32Array): number {
let d = 0, na = 0, nb = 0;
for (let q = 0; q < 12; q++) { d += a[q]! * b[q]!; na += a[q]! * a[q]!; nb += b[q]! * b[q]!; }
return na > 1e-12 && nb > 1e-12 ? d / Math.sqrt(na * nb) : 0;
}
/** Mean chroma cosine over the sounding recording frames of [f0, f1); null when the candidate covers too little. */
export function meanCosine(rec: Float32Array[], cand: Float32Array[], f0: number, f1: number, minCover = SHIFT_MIN_COVER): number | null {
let sum = 0, sounding = 0, covered = 0;
for (let t = f0; t < Math.min(f1, rec.length); t++) {
const r = rec[t]!;
if (!r.some((v) => v !== 0)) continue;
sounding++;
const c = cand[t];
if (!c || !c.some((v) => v !== 0)) continue;
covered++;
sum += cos(r, c);
}
if (!sounding || covered / sounding < minCover) return null;
return sum / sounding;
}
function shiftCheck(s: ShiftInputs, f0: number, f1: number): BeatShift | null {
const cur = meanCosine(s.recording, s.current, f0, f1);
if (cur === null) return null;
let best: BeatShift | null = null;
for (const c of s.shifts) {
const v = meanCosine(s.recording, c.chroma, f0, f1, 0);
if (v !== null && v - cur > SHIFT_MARGIN && (!best || v - cur > best.gain)) best = { label: c.label, gain: v - cur };
}
return best;
}
const pct = (x: number) => `${Math.round(x * 100)}%`;
/**
* Recording onsets are matched to the nearest warped MIDI onset within ±70 ms (non-exclusive). Overall and per 30 s
* window (by recording onset time): precision, lift (precision − mean precision with the MIDI shifted by +0.11, +0.23,
* +0.37 s), median residual; plus the audible rate (info) and, with `extra.shift`, the beat/bar-shift chroma check.
* A window with < 8 recording onsets is null and unflagged; otherwise it is flagged when precision < 0.6, lift < 0.25,
* or a shifted alignment beats the current one by > 0.05 mean chroma cosine.
*/
export function fitQuality(audioOnsets: number[], midiOnsets: number[], durSec: number, extra: { shift?: ShiftInputs; stretches?: StretchCounts } = {}): FitReport {
const rec = [...audioOnsets].sort((a, b) => a - b);
const midi = uniqueSorted(midiOnsets);
const nWin = Math.max(1, Math.ceil(Math.max(durSec, (rec[rec.length - 1] ?? 0) + 1e-9) / WINDOW_SEC));
const winOf = (t: number) => Math.min(nWin - 1, Math.max(0, Math.floor(t / WINDOW_SEC)));
const recWin: number[][] = Array.from({ length: nWin }, () => []);
for (const t of rec) recWin[winOf(t)]!.push(t);
const midiHit = midi.map((t) => hit(nearestDist(rec, t)));
const stats = (r: number[]) => {
const precision = midi.length ? precisionOf(r, midi) : null;
if (precision === null) return { precision: null, lift: null, medianResidualMs: null };
const chance = CHANCE_SHIFTS_SEC.map((s) => precisionOf(r, midi, s)!);
const dists = r.map((t) => nearestDist(midi, t)).filter(hit);
const med = median(dists);
return { precision, lift: precision - chance.reduce((a, b) => a + b, 0) / chance.length, medianResidualMs: med === null ? null : med * 1000 };
};
const audible = (lo: number, hi: number) => {
let n = 0, h = 0;
midi.forEach((t, k) => { if (t >= lo && t < hi) { n++; if (midiHit[k]) h++; } });
return n ? h / n : null;
};
const windows = recWin.map((r, w): FitWindow => {
const startSec = w * WINDOW_SEC;
const lo = w === 0 ? -Infinity : startSec, hi = w === nWin - 1 ? Infinity : startSec + WINDOW_SEC;
const s = r.length >= MIN_REC_ONSETS ? stats(r) : { precision: null, lift: null, medianResidualMs: null };
const beatShift = extra.shift && s.precision !== null
? shiftCheck(extra.shift, Math.round(startSec * extra.shift.rate), w === nWin - 1 ? Infinity : Math.round((startSec + WINDOW_SEC) * extra.shift.rate))
: null;
const reasons: string[] = [];
if (s.precision !== null && s.precision < FLAG_MIN_PRECISION) reasons.push(`precision ${pct(s.precision)}`);
if (s.lift !== null && s.lift < FLAG_MIN_LIFT) reasons.push(`lift ${s.lift.toFixed(2)}`);
if (beatShift) reasons.push(`beat shift suspected: ${beatShift.label}`);
return { startSec, recOnsets: r.length, ...s, audibleRate: audible(lo, hi), beatShift, flagged: reasons.length > 0, reasons };
});
const all = rec.length && midi.length ? stats(rec) : { precision: null, lift: null, medianResidualMs: null };
return { ...all, audibleRate: audible(-Infinity, Infinity), windows, ...(extra.stretches ? { stretches: extra.stretches } : {}) };
}
/**
* Match each unique MIDI onset (already warped to audio time) to the nearest recording onset within ±70 ms
* (non-exclusive). Reports match rate and median |residual| overall and per 30 s window (by MIDI onset time).
* A window is flagged when its match rate < 0.5 or its median residual > 60 ms; empty windows are null, unflagged.
* One line, e.g. "precision 91.2% (lift +0.70) · MIDI onsets audible as attacks 38.1% · median residual 14.2 ms ·
* 1/8 windows flagged (0:30 beat shift suspected: +1 beat) · stretches: 3/40 warned, 1 > 15%".
*/
export function fitQuality(audioOnsets: number[], midiOnsets: number[], durSec: number): FitReport {
const audio = [...audioOnsets].sort((a, b) => a - b);
const midi: number[] = [];
for (const t of [...midiOnsets].sort((a, b) => a - b)) if (!midi.length || t - midi[midi.length - 1]! > UNIQUE_EPS) midi.push(t);
const res = midi.map((t) => nearest(audio, t));
const lastOnset = midi.length ? midi[midi.length - 1]! : 0;
const nWin = Math.max(1, Math.ceil(Math.max(durSec, lastOnset + 1e-9) / WINDOW_SEC));
const buckets: (number | null)[][] = Array.from({ length: nWin }, () => []);
midi.forEach((t, k) => buckets[Math.min(nWin - 1, Math.max(0, Math.floor(t / WINDOW_SEC)))]!.push(res[k]!));
const windows = buckets.map((b, w): FitWindow => {
const s = summarise(b);
const flagged = s.matchRate !== null && (s.matchRate < FLAG_MIN_RATE || (s.medianResidualMs ?? 0) > FLAG_MAX_RESIDUAL_MS);
return { startSec: w * WINDOW_SEC, ...s, flagged };
});
return { ...summarise(res), windows };
}
/** One-line summary, e.g. "82.3% of MIDI onsets matched · median residual 14.2 ms · 2/9 windows flagged (0:30, 2:00)". */
export function fitSummary(f: FitReport): string {
if (f.matchRate === null) return "no scored MIDI onsets";
if (f.precision === null) return f.audibleRate === null ? "no scored MIDI onsets" : "no recording onsets";
const mmss = (s: number) => `${Math.floor(s / 60)}:${String(Math.floor(s % 60)).padStart(2, "0")}`;
const flagged = f.windows.filter((w) => w.flagged);
const med = f.medianResidualMs === null ? "n/a" : `${f.medianResidualMs.toFixed(1)} ms`;
const list = flagged.length ? ` (${flagged.map((w) => mmss(w.startSec)).join(", ")})` : "";
return `${(f.matchRate * 100).toFixed(1)}% of MIDI onsets matched · median residual ${med} · ${flagged.length}/${f.windows.length} windows flagged${list}`;
const lift = f.lift === null ? "" : ` (lift ${f.lift >= 0 ? "+" : "−"}${Math.abs(f.lift).toFixed(2)})`;
const aud = f.audibleRate === null ? "n/a" : `${(f.audibleRate * 100).toFixed(1)}%`;
const list = flagged.length ? ` (${flagged.map((w) => `${mmss(w.startSec)} ${w.reasons.join(", ")}`).join("; ")})` : "";
const st = f.stretches ? ` · stretches: ${f.stretches.warned}/${f.stretches.total} warned, ${f.stretches.over15} > 15%` : "";
return `precision ${(f.precision * 100).toFixed(1)}%${lift} · MIDI onsets audible as attacks ${aud} · median residual ${med} · ${flagged.length}/${f.windows.length} windows flagged${list}${st}`;
}
+12 -3
View File
@@ -255,14 +255,23 @@ async function runBusy(label: string, f: () => Promise<void>) {
try { await f(); } catch (e) { setStatus((e as Error).message, true); } finally { busy = false; showOverlay(null); updateUi(); }
}
interface AutoInfo { chosen: "baseline" | "dtw"; baseline: { offsetSec: number; precision: number | null }; dtwPrecision: number | null }
const pctOf = (x: number | null) => (x === null ? "n/a" : `${(x * 100).toFixed(1)}%`);
/** "DTW, precision 91.0% vs constant offset +0.06 s 88.0%" or "constant offset +0.06 s (precision 91.0%; DTW 89.5% was not better)". */
function autoNote(a: AutoInfo): string {
const off = `${a.baseline.offsetSec >= 0 ? "+" : "−"}${Math.abs(a.baseline.offsetSec).toFixed(2)} s`;
return a.chosen === "dtw" ? `DTW, precision ${pctOf(a.dtwPrecision)} vs constant offset ${off} ${pctOf(a.baseline.precision)}`
: `constant offset ${off} (precision ${pctOf(a.baseline.precision)}; DTW ${pctOf(a.dtwPrecision)} was not better by 2 points)`;
}
const autoAlign = () => runBusy("Auto-aligning around your anchors… (a few seconds to half a minute)", async () => {
const manual = anchors.filter((a) => a.kind === "manual");
const j = await http<{ anchors: Anchor[]; fit: FitReport }>(songUrl("/auto"), send("POST", { anchors: manual }));
const j = await http<{ anchors: Anchor[]; fit: FitReport; auto: AutoInfo }>(songUrl("/auto"), send("POST", { anchors: manual }));
selected = -1;
commit(j.anchors, anchors, true);
fit = j.fit; fitKey = keyOf(j.anchors);
updateUi();
setStatus(`Auto-aligned: ${j.anchors.length - manual.length} automatic anchors around your ${manual.length}. Not saved yet.`);
setStatus(`Auto-aligned: ${j.anchors.length - manual.length} automatic anchors around your ${manual.length} (${autoNote(j.auto)}). Not saved yet.`);
});
const save = () => runBusy("Saving…", async () => {
@@ -357,7 +366,7 @@ function updateUi() {
fitEl.classList.toggle("stale", stale);
if (fit) {
fitEl.textContent = `Fit${stale ? " (updating…)" : ""}: ${fitSummary(fit)}`;
fitEl.classList.toggle("bad", (fit.matchRate ?? 1) < 0.5 || fit.windows.some((w) => w.flagged));
fitEl.classList.toggle("bad", fit.windows.some((w) => w.flagged));
}
const man = anchors.filter((a) => a.kind === "manual").length;
countsEl.textContent = `· anchors: ${man} yours, ${anchors.length - man} automatic`;
+11 -9
View File
@@ -8,7 +8,7 @@ import type { Role } from "../core/types.ts";
import { audioOgg, loadSong, readAlignment, readManifest, writeAlignment, type AlignmentFile, type LoadedSong } from "../ref/manifest.ts";
import { trackKeys } from "./roles.ts";
import { makeWarp, stretches, type Anchor } from "./warp.ts";
import { autoAlign, fitFromOnsets, recordingOnsets } from "./auto.ts";
import { autoAlign, fitFromCues, recordingCues, type AutoResult, type RecordingCues } from "./auto.ts";
import { renderMidi } from "./render.ts";
import { peaks } from "./peaks.ts";
import { stretchInfo } from "./tempo.ts";
@@ -32,9 +32,9 @@ export interface AlignServer { url: string; origin: string; token: string; port:
interface SongState extends LoadedSong {
roles: Record<string, Role | null>;
onsets: number[];
cues: RecordingCues;
peaks: { bucketSec: number; min: number[]; max: number[] };
auto?: { anchors: Anchor[]; fit: FitReport }; // first automatic pass, reused while no alignment.json exists
auto?: AutoResult; // first automatic pass, reused while no alignment.json exists
}
class HttpError extends Error { constructor(readonly status: number, msg: string) { super(msg); } }
@@ -98,7 +98,7 @@ export async function startServer(opts: ServeOptions = {}): Promise<AlignServer>
const s = await loadSong(id, root);
const keys = trackKeys(s.smf.tracks);
const roles = Object.fromEntries(keys.map((k) => [k, s.entry.roles[k] ?? null]));
const onsets = recordingOnsets(s.audio);
const cues = recordingCues(s.audio);
const peaksFile = Bun.file(join(root, id, "peaks.json"));
const recWav = Bun.file(join(root, id, "recording.wav"));
let pk: SongState["peaks"] | null = null;
@@ -114,7 +114,7 @@ export async function startServer(opts: ServeOptions = {}): Promise<AlignServer>
await Bun.write(peaksFile, JSON.stringify({ version: PEAKS_VERSION, peaks: pk }));
}
log(`loaded ${id} (${s.audioDur.toFixed(1)} s) in ${((performance.now() - t0) / 1000).toFixed(1)} s`);
return { ...s, roles, onsets, peaks: pk };
return { ...s, roles, cues, peaks: pk };
}
function song(id: string): Promise<SongState> {
@@ -127,17 +127,19 @@ export async function startServer(opts: ServeOptions = {}): Promise<AlignServer>
return p;
}
const fitOf = (s: SongState, anchors: Anchor[]) => fitFromOnsets(s.onsets, s.smf, s.roles, anchors, s.audioDur);
const fitOf = (s: SongState, anchors: Anchor[]) => fitFromCues(s.cues, s.smf, s.roles, anchors);
const stretchesOf = (s: SongState, anchors: Anchor[]) => stretchInfo(stretches(anchors, s.smf.durationSec), s.smf.tempoMap);
const state = (s: SongState, anchors: Anchor[], fit = fitOf(s, anchors)) => ({ anchors, stretches: stretchesOf(s, anchors), fit });
function runAuto(s: SongState, manual: Anchor[]) {
const t0 = performance.now();
const r = autoAlign(s.audio, s.smf, s.roles, manual, sf2);
log(`auto-aligned ${s.entry.id}: ${r.anchors.length} anchors (${manual.length} manual) in ${((performance.now() - t0) / 1000).toFixed(1)} s`);
log(`auto-aligned ${s.entry.id}: ${r.anchors.length} anchors (${manual.length} manual, ${r.chosen}) in ${((performance.now() - t0) / 1000).toFixed(1)} s`);
return r;
}
const autoInfo = (r: AutoResult) => ({ chosen: r.chosen, baseline: r.baseline, dtwPrecision: r.dtwPrecision });
/** Saved alignment with validated anchors; an unreadable/invalid file yields `error` instead of failing the request. */
async function savedAlignment(id: string): Promise<{ file: AlignmentFile | null; error?: string }> {
try {
@@ -175,7 +177,7 @@ export async function startServer(opts: ServeOptions = {}): Promise<AlignServer>
id, title: s.entry.title, audioDur: s.audioDur, midiDur: s.smf.durationSec,
tracks: trackKeys(s.smf.tracks).map((name) => ({ name, role: s.roles[name] ?? null })),
recordingPeaks: s.peaks, ...state(s, anchors, fit), midiTempo: s.smf.tempoMap,
source: saved ? "saved" : "auto", savedAt: saved?.savedAt ?? null,
source: saved ? "saved" : "auto", savedAt: saved?.savedAt ?? null, ...(saved ? {} : { auto: autoInfo(s.auto!) }),
...(error ? { warning: `${error}. Showing the automatic alignment instead; saving will overwrite the file.` } : {}),
});
}
@@ -208,7 +210,7 @@ export async function startServer(opts: ServeOptions = {}): Promise<AlignServer>
const manual = parseAnchors(await readJson(req)).filter((a) => a.kind === "manual");
const s = await song(id);
const r = runAuto(s, manual);
return Response.json(state(s, r.anchors, r.fit));
return Response.json({ ...state(s, r.anchors, r.fit), auto: autoInfo(r) });
}
if (route === "POST reset") return Response.json(state(await song(id), []));
if (route === "PUT alignment") {
+8 -1
View File
@@ -23,7 +23,9 @@ function sortedCopy(anchors: Anchor[]): Anchor[] {
const ratioOf = (a: Anchor, b: Anchor) => (b.audioSec - a.audioSec) / (b.midiSec - a.midiSec);
// Slopes used beyond the outermost anchors: the nearest interior stretch, or 1 with fewer than two anchors.
// Slopes used beyond the outermost anchors: the nearest interior stretch, or 1 with fewer than two anchors. So
// before the first anchor the map extrapolates with the *first* stretch's slope (there is no implicit 0 -> 0 anchor:
// a warp whose first anchor is (m0, a0) maps MIDI 0 to a0 − m0·slope₀), and after the last anchor with the last one's.
function edgeRatios(A: Anchor[]): { first: number; last: number } {
if (A.length < 2) return { first: 1, last: 1 };
return { first: ratioOf(A[0]!, A[1]!), last: ratioOf(A[A.length - 2]!, A[A.length - 1]!) };
@@ -43,6 +45,11 @@ function piecewise(X: number[], Y: number[], x: number, first: number, last: num
return Y[lo]! + ((x - X[lo]!) * (Y[hi]! - Y[lo]!)) / (X[hi]! - X[lo]!);
}
/**
* Monotonic piecewise-linear MIDI <-> recording map through the anchors. Outside the anchors it extrapolates linearly:
* before the first anchor with the first stretch's slope, after the last anchor with the last stretch's slope (slope
* 1 with a single anchor, identity with none).
*/
export function makeWarp(anchors: Anchor[]): Warp {
const A = sortedCopy(anchors);
const midi = A.map((a) => a.midiSec), audio = A.map((a) => a.audioSec);
+51 -16
View File
@@ -7,12 +7,20 @@ import { matchNotes, type MatchResult } from "../eval/notes.ts";
import { DRUM_CLASSES, ROLE_ORDER, drumClass, toClassPitch, type DrumClass } from "../eval/score.ts";
import { readManifest, readAlignment, loadSong, trackKeys } from "../ref/manifest.ts";
import { makeWarp, type Anchor } from "../align/warp.ts";
import { autoAlign, alignmentFit } from "../align/auto.ts";
import { autoAlign, autoSummary, constantOffsetBaseline, fitFromCues, recordingCues, type Baseline } from "../align/auto.ts";
import { fitSummary, type FitReport } from "../align/fit.ts";
export interface RealSongResult {
id: string; title: string;
alignment: { source: "manual" | "auto"; manual: number; auto: number; savedAt: string | null };
/**
* source: "saved" = reference/<id>/alignment.json, "computed" = auto-aligned this run (--auto). chosen: how the auto
* alignment was made ("dtw" or the constant-offset "baseline"), null when unknown (saved from the page).
* baseline: the constant-offset baseline computed this run, for comparison.
*/
alignment: {
source: "saved" | "computed"; manual: number; auto: number; savedAt: string | null;
chosen: "baseline" | "dtw" | null; baseline: Baseline | { offsetSec: number; precision: number | null }; dtwPrecision: number | null;
};
fit: FitReport;
perRole: Partial<Record<Role, MatchResult>>;
/** F1 at the lenient +-100 ms tolerance, per role (null when the role has no notes on either side). */
@@ -37,19 +45,37 @@ export async function runReal(opts: { runId: string; auto: boolean; root?: strin
});
const sf2 = opts.auto ? new Uint8Array(await Bun.file(SF2).arrayBuffer()) : null;
const results: RealSongResult[] = [];
const skipped: string[] = [];
const skipped: SkippedSong[] = [];
const skip = (id: string, reason: string) => { skipped.push({ id, reason }); console.log(`${id}: skipped (${reason})`); };
for (const entry of songs) {
const saved = await readAlignment(entry.id, root);
if (!saved && !opts.auto) { skipped.push(entry.id); console.log(`${entry.id}: skipped (no alignment.json; pass --auto to compute one)`); continue; }
let saved: Awaited<ReturnType<typeof readAlignment>>;
try {
saved = await readAlignment(entry.id, root);
if (saved) makeWarp(saved.anchors); // validates the anchors
} catch (e) { skip(entry.id, (e as Error).message); continue; }
if (!saved && !opts.auto) { skip(entry.id, "no alignment.json; pass --auto to compute one"); continue; }
const t0 = performance.now();
const { smf, audio } = await loadSong(entry.id, root);
const keys = trackKeys(smf.tracks);
const roleOf = (i: number): Role | null => entry.roles[keys[i]!] ?? null;
const roles = Object.fromEntries(keys.map((k) => [k, entry.roles[k] ?? null]));
let anchors: Anchor[], savedAt: string | null, source: "manual" | "auto";
if (saved) { anchors = saved.anchors; savedAt = saved.savedAt ?? null; source = "manual"; }
else { anchors = autoAlign(audio, smf, entry.roles, [], sf2!).anchors; savedAt = null; source = "auto"; }
const fit = alignmentFit(audio, smf, entry.roles, anchors);
const cues = recordingCues(audio);
let anchors: Anchor[], alignment: RealSongResult["alignment"];
if (saved) {
anchors = saved.anchors;
alignment = {
source: "saved", manual: 0, auto: 0, savedAt: saved.savedAt ?? null,
chosen: saved.auto?.chosen ?? null, baseline: constantOffsetBaseline(cues, smf, roles), dtwPrecision: saved.auto?.dtwPrecision ?? null,
};
} else {
const r = autoAlign(audio, smf, roles, [], sf2!);
anchors = r.anchors;
alignment = { source: "computed", manual: 0, auto: 0, savedAt: null, chosen: r.chosen, baseline: r.baseline, dtwPrecision: r.dtwPrecision };
}
alignment.manual = anchors.filter((a) => a.kind === "manual").length;
alignment.auto = anchors.filter((a) => a.kind === "auto").length;
const fit = fitFromCues(cues, smf, roles, anchors);
const warp = makeWarp(anchors);
const res = await pipeline({ mix: audio }, { oracleStems: false, oracleTempo: false, oracleNotes: false });
@@ -77,7 +103,7 @@ export async function runReal(opts: { runId: string; auto: boolean; root?: strin
const seconds = (performance.now() - t0) / 1000;
const result: RealSongResult = {
id: entry.id, title: entry.title,
alignment: { source, manual: anchors.filter((a) => a.kind === "manual").length, auto: anchors.filter((a) => a.kind === "auto").length, savedAt },
alignment,
fit, perRole, lenient, drumsPerClass,
excludedTracks: smf.tracks.map((t, i) => (roleOf(i) ? null : keys[i]!)).filter((k): k is string => k !== null),
capo: entry.capo ?? {}, seconds,
@@ -100,16 +126,25 @@ const NA = "n/a";
const fmt = (x: number | null | undefined) => (x == null ? NA : x.toFixed(2));
const row = (cells: string[]) => `| ${cells.join(" | ")} |`;
export function renderRealReport(runId: string, results: RealSongResult[], skipped: string[] = []): string {
export interface SkippedSong { id: string; reason: string }
export function renderRealReport(runId: string, results: RealSongResult[], skipped: SkippedSong[] = []): string {
const o: string[] = [`# STSheets real-song benchmark: ${runId}`, ""];
o.push("> Reference = the team's Guitar Pro-exported MIDI warped onto the recording by the saved alignment; scores are note-onset matches (pitch must equal; drums compared by class). " +
"The fit line reports the *consistency of the alignment* (warped MIDI onsets vs onset-flux peaks of the recording), not an independent validation of it: " +
"a poor alignment lowers every score below. Capo is informational and not applied.", "");
"The fit line reports the *consistency of the alignment*, not an independent validation of it: precision = share of the recording's onset-flux peaks with a warped MIDI onset within ±70 ms " +
"(flag < 60% per 30 s window), lift = precision minus its value with the MIDI shifted off-grid (flag < 0.25), a symbolic-chroma check for whole-beat/bar shifts, " +
"and \"MIDI onsets audible as attacks\" (information only; a full mix exposes far fewer attacks than the score has notes). " +
"A poor alignment lowers every score below. Capo is informational and not applied.", "");
for (const r of results) {
const a = r.alignment;
const src = a.source === "auto" ? `auto (${a.manual} manual / ${a.auto} auto anchors, computed this run)` : `saved (${a.manual} manual / ${a.auto} auto anchors, saved ${a.savedAt ?? "unknown"})`;
const counts = `${a.manual} manual / ${a.auto} auto anchors`;
const src = a.source === "computed" ? `computed this run (${counts})` : `saved ${a.savedAt ?? "(time unknown)"} (${counts})`;
const pc = (x: number | null) => (x === null ? NA : `${(x * 100).toFixed(1)}%`);
const off = `${a.baseline.offsetSec >= 0 ? "+" : "−"}${Math.abs(a.baseline.offsetSec).toFixed(2)} s`;
const method = a.chosen ? autoSummary({ chosen: a.chosen, baseline: a.baseline, dtwPrecision: a.dtwPrecision })
: `method unknown (saved from the page) · constant-offset baseline ${off}: precision ${pc(a.baseline.precision)}`;
o.push(`## ${r.title} (${r.id})`, "");
o.push(`- Alignment: ${src}`, `- Consistency of the alignment: ${fitSummary(r.fit)}`);
o.push(`- Alignment: ${src}`, `- Method: ${method}`, `- Consistency of the alignment: ${fitSummary(r.fit)}`);
const capos = Object.entries(r.capo);
o.push(`- Capo: ${capos.length ? capos.map(([k, v]) => `${k}: ${v}`).join(", ") : "none listed"}`);
o.push(`- Excluded tracks (no role): ${r.excludedTracks.length ? r.excludedTracks.join(", ") : "none"}`, `- Pipeline + scoring: ${r.seconds.toFixed(1)} s`, "");
@@ -128,6 +163,6 @@ export function renderRealReport(runId: string, results: RealSongResult[], skipp
o.push("");
}
}
if (skipped.length) o.push(`Skipped (no alignment.json): ${skipped.join(", ")}`, "");
if (skipped.length) o.push(`Skipped: ${skipped.map((s) => `${s.id} (${s.reason})`).join("; ")}`, "");
return o.join("\n");
}
+3 -1
View File
@@ -45,7 +45,9 @@ export async function addSong(e: Omit<SongEntry, "roles"> & { roles?: SongEntry[
// ---- per-song caches and alignment file (reference/<id>/) ----
export interface AlignmentFile { anchors: Anchor[]; savedAt: string; fit: FitReport | null }
/** `auto`: how `bun run align auto` produced the anchors (absent for alignments saved from the page). */
export interface AlignmentFile { anchors: Anchor[]; savedAt: string; fit: FitReport | null; auto?: AutoInfo }
export interface AutoInfo { chosen: "baseline" | "dtw"; baseline: { offsetSec: number; precision: number | null }; dtwPrecision: number | null }
const inflight = new Map<string, Promise<string>>();
+125 -16
View File
@@ -1,5 +1,5 @@
import { test, expect, beforeAll } from "bun:test";
import { alignmentFit, autoAlign, bandedWithRetry, barTimes, fitFromOnsets, recordingOnsets } from "../../src/align/auto.ts";
import { alignmentFit, autoAlign, bandedWithRetry, barTimes, beatTimes, constantOffsetBaseline, fitFromCues, meterGrid, recordingCues, rejectOffsetOutliers } from "../../src/align/auto.ts";
import { makeWarp, type Anchor, type Warp } from "../../src/align/warp.ts";
import type { RefTrack, Smf } from "../../src/align/smf.ts";
import type { Audio, Role, TruthTrack } from "../../src/core/types.ts";
@@ -49,18 +49,22 @@ function recoveryErrors(smf: Smf, got: Anchor[], trueMidiToAudio: (m: number) =>
.map((m) => Math.abs(w.toAudio(m) - trueMidiToAudio(m)));
}
test("recovers a known piecewise warp: median error < 40 ms, fit match rate > 0.7", () => {
test("recovers a known piecewise warp: median error < 40 ms, fit precision ≥ 0.8, DTW beats the constant baseline", () => {
const D = distortion(song.song.bpm);
const smf = smfFrom(song.truth.tracks, song.song.tracks.map((t) => t.program), song.song.bpm, D.toAudio);
const t0 = performance.now();
const { anchors, fit } = autoAlign(song.mix, smf, rolesOf(song.truth.tracks), [], SF2);
const { anchors, fit, baseline, chosen } = autoAlign(song.mix, smf, rolesOf(song.truth.tracks), [], SF2);
const ms = performance.now() - t0;
const err = recoveryErrors(smf, anchors, D.toMidi, durOf(song.mix));
debug(`synthetic warp: ${anchors.length} anchors, median err ${(median(err) * 1000).toFixed(1)} ms, p90 ${(p90(err) * 1000).toFixed(1)} ms, fit ${fit.matchRate?.toFixed(3)} / ${fit.medianResidualMs?.toFixed(1)} ms, ${ms.toFixed(0)} ms`);
debug(`synthetic warp: ${anchors.length} anchors, median err ${(median(err) * 1000).toFixed(1)} ms, p90 ${(p90(err) * 1000).toFixed(1)} ms, fit ${fit.precision?.toFixed(3)} lift ${fit.lift?.toFixed(3)} / ${fit.medianResidualMs?.toFixed(1)} ms, baseline ${baseline.offsetSec} s ${baseline.precision?.toFixed(3)}, ${ms.toFixed(0)} ms`);
expect(chosen).toBe("dtw");
expect(anchors.every((a) => a.kind === "auto")).toBe(true);
expect(median(err)).toBeLessThan(0.04);
expect(p90(err)).toBeLessThan(0.08);
expect(fit.matchRate!).toBeGreaterThan(0.7);
expect(fit.precision!).toBeGreaterThanOrEqual(0.8);
expect(fit.precision!).toBeGreaterThanOrEqual(baseline.precision! + 0.02);
expect(fit.windows.some((w) => w.flagged)).toBe(false);
expect(fit.stretches!.total).toBe(anchors.length + 1);
expect(fit.windows.length).toBe(2);
expect(anchors.every((a) => a.audioSec > 0 || a.midiSec === 0)).toBe(true);
}, BIG);
@@ -86,7 +90,7 @@ test("a silent intro in the recording does not drag the path", () => {
const { anchors, fit } = autoAlign(mix, smf, rolesOf(song.truth.tracks), [], SF2);
const err = recoveryErrors(smf, anchors, (m) => m + 4, durOf(mix));
expect(median(err)).toBeLessThan(0.04);
expect(fit.matchRate!).toBeGreaterThan(0.7);
expect(fit.precision!).toBeGreaterThanOrEqual(0.8);
}, BIG);
test("5-minute input aligns in < 15 s", () => {
@@ -94,12 +98,18 @@ test("5-minute input aligns in < 15 s", () => {
const mix: Audio = { sampleRate: song.mix.sampleRate, channels: song.mix.channels.map((c) => { const o = new Float32Array(c.length * reps); for (let r = 0; r < reps; r++) o.set(c, r * c.length); return o; }) };
const smf = smfFrom(song.truth.tracks, song.song.tracks.map((t) => t.program), song.song.bpm, (t) => t + 0.3, reps, period);
const t0 = performance.now();
const { anchors, fit } = autoAlign(mix, smf, rolesOf(song.truth.tracks), [], SF2);
const { anchors, fit, baseline, chosen } = autoAlign(mix, smf, rolesOf(song.truth.tracks), [], SF2);
const sec = (performance.now() - t0) / 1000;
debug(`5-min (${durOf(mix).toFixed(0)} s) input: ${sec.toFixed(2)} s, ${anchors.length} anchors, fit ${fit.matchRate?.toFixed(3)}`);
debug(`5-min (${durOf(mix).toFixed(0)} s) input: ${sec.toFixed(2)} s, ${anchors.length} anchors, fit ${fit.precision?.toFixed(3)}, ${chosen}, baseline ${baseline.offsetSec} ${baseline.precision?.toFixed(3)}`);
expect(durOf(mix)).toBeGreaterThan(290);
expect(sec).toBeLessThan(15);
expect(fit.windows.length).toBe(10);
// a constant offset: DTW cannot beat the baseline by 0.02, so the single-anchor baseline is returned
expect(baseline.offsetSec).toBeCloseTo(-0.3, 1);
expect(chosen).toBe("baseline");
expect(anchors).toHaveLength(1);
expect(anchors[0]!.kind).toBe("auto");
expect(anchors[0]!.audioSec - anchors[0]!.midiSec).toBeCloseTo(baseline.offsetSec, 6);
}, BIG);
test("barTimes follows tempo and time-signature changes", () => {
@@ -130,13 +140,13 @@ test("fit exposes a bad alignment instead of hiding it", () => {
const smf = smfFrom(song.truth.tracks, song.song.tracks.map((t) => t.program), song.song.bpm, D.toAudio);
const roles = rolesOf(song.truth.tracks);
const bad = alignmentFit(song.mix, smf, roles, []); // identity: off by 0.4–0.8 s throughout
expect(bad.matchRate!).toBeLessThan(0.5);
expect(bad.precision!).toBeLessThan(0.6);
// the −0.8 s section is exposed; the +0.4 s section is not, as 0.4 s is ≈ 40 ms from two 16ths at 68 bpm (grid alias)
expect(bad.windows[1]!.flagged).toBe(true);
const good = alignmentFit(song.mix, smf, roles, autoAlign(song.mix, smf, roles, [], SF2).anchors);
expect(good.windows.some((w) => w.flagged)).toBe(false);
// unmapped tracks are excluded: with no scored roles there is nothing to measure
expect(alignmentFit(song.mix, smf, Object.fromEntries(Object.keys(roles).map((k) => [k, null])), []).matchRate).toBeNull();
expect(alignmentFit(song.mix, smf, Object.fromEntries(Object.keys(roles).map((k) => [k, null])), []).precision).toBeNull();
}, BIG);
test("a deliberately wrong manual anchor (+0.3 s) is kept exactly and the result stays monotonic", () => {
@@ -158,23 +168,122 @@ test("when every fine band fails, the coarse-path fallback still yields auto anc
const D = distortion(song.song.bpm);
const smf = smfFrom(song.truth.tracks, song.song.tracks.map((t) => t.program), song.song.bpm, D.toAudio);
const manual: Anchor = { midiSec: D.toAudio(30), audioSec: 30, kind: "manual" };
const { anchors, fit } = autoAlign(song.mix, smf, rolesOf(song.truth.tracks), [manual], SF2, { fineBandsSec: [] });
const { anchors, fit, chosen } = autoAlign(song.mix, smf, rolesOf(song.truth.tracks), [manual], SF2, { fineBandsSec: [] });
expect(chosen).toBe("dtw"); // manual anchors present: never substituted by the baseline
const autos = anchors.filter((a) => a.kind === "auto");
// both segments (before and after the manual anchor) get anchors
expect(autos.filter((a) => a.audioSec < 30).length).toBeGreaterThan(3);
expect(autos.filter((a) => a.audioSec > 30).length).toBeGreaterThan(3);
expect(strictlyMonotone(anchors)).toBe(true);
const err = recoveryErrors(smf, anchors, D.toMidi, durOf(song.mix));
debug(`coarse fallback: ${autos.length} autos, median err ${(median(err) * 1000).toFixed(1)} ms, p90 ${(p90(err) * 1000).toFixed(1)} ms, fit ${fit.matchRate?.toFixed(3)}`);
debug(`coarse fallback: ${autos.length} autos, median err ${(median(err) * 1000).toFixed(1)} ms, p90 ${(p90(err) * 1000).toFixed(1)} ms, fit ${fit.precision?.toFixed(3)}`);
expect(median(err)).toBeLessThan(0.06); // 10 Hz resolution: coarser than the fine pass, still hand-correctable
expect(p90(err)).toBeLessThan(0.12);
}, BIG);
test("fitFromOnsets with cached recordingOnsets equals alignmentFit", () => {
test("fitFromCues with cached recordingCues equals alignmentFit", () => {
const smf = smfFrom(song.truth.tracks, song.song.tracks.map((t) => t.program), song.song.bpm, (t) => t);
const roles = rolesOf(song.truth.tracks);
const anchors: Anchor[] = [{ midiSec: 1, audioSec: 1.05, kind: "manual" }, { midiSec: 20, audioSec: 20.1, kind: "manual" }];
const onsets = recordingOnsets(song.mix);
expect(onsets.length).toBeGreaterThan(10);
expect(fitFromOnsets(onsets, smf, roles, anchors, durOf(song.mix))).toEqual(alignmentFit(song.mix, smf, roles, anchors));
const cues = recordingCues(song.mix);
expect(cues.onsets.length).toBeGreaterThan(10);
expect(cues.chroma.length).toBeGreaterThan(cues.durSec * cues.chromaRate - 2);
expect(fitFromCues(cues, smf, roles, anchors)).toEqual(alignmentFit(song.mix, smf, roles, anchors));
});
// ---- ruling A7: calibrated fit, beat-shift check, constant-offset baseline ----
test("fit precision: correct alignment ≥ 0.8, a 0.12 s off-grid shift < 0.5 (and flagged)", () => {
const smf = smfFrom(song.truth.tracks, song.song.tracks.map((t) => t.program), song.song.bpm, (t) => t);
const roles = rolesOf(song.truth.tracks);
const good = alignmentFit(song.mix, smf, roles, []);
const off = alignmentFit(song.mix, smf, roles, [{ midiSec: 10, audioSec: 10.12, kind: "manual" }]);
debug(`precision: correct ${good.precision?.toFixed(3)} (lift ${good.lift?.toFixed(3)}, audible ${good.audibleRate?.toFixed(3)}), +0.12 s ${off.precision?.toFixed(3)} (lift ${off.lift?.toFixed(3)})`);
expect(good.precision!).toBeGreaterThanOrEqual(0.8);
expect(good.lift!).toBeGreaterThanOrEqual(0.25);
expect(good.windows.some((w) => w.flagged)).toBe(false);
expect(off.precision!).toBeLessThan(0.5);
expect(off.windows.filter((w) => w.precision !== null).every((w) => w.flagged)).toBe(true);
});
test("beat-shift check flags an alignment that is one beat late, though onset precision cannot see it", () => {
const smf = smfFrom(song.truth.tracks, song.song.tracks.map((t) => t.program), song.song.bpm, (t) => t);
const roles = rolesOf(song.truth.tracks);
const beat = 60 / song.song.bpm;
const late = alignmentFit(song.mix, smf, roles, [{ midiSec: 10, audioSec: 10 + beat, kind: "manual" }]);
const good = alignmentFit(song.mix, smf, roles, []);
debug(`beat shift: late precision ${late.precision?.toFixed(3)}, windows ${JSON.stringify(late.windows.map((w) => [w.precision?.toFixed(2), w.lift?.toFixed(2), w.beatShift]))}`);
expect(good.windows.some((w) => w.beatShift)).toBe(false);
const shifted = late.windows.filter((w) => w.beatShift);
expect(shifted.length).toBeGreaterThan(0);
expect(shifted.every((w) => w.flagged && w.reasons.some((r) => r.startsWith("beat shift suspected: −")))).toBe(true);
});
test("constantOffsetBaseline grid-searches −2..+2 s for the offset with the best precision", () => {
const smf = smfFrom(song.truth.tracks, song.song.tracks.map((t) => t.program), song.song.bpm, (t) => t + 0.27);
const b = constantOffsetBaseline(recordingCues(song.mix), smf, rolesOf(song.truth.tracks));
expect(b.offsetSec).toBeCloseTo(-0.27, 1);
expect(Math.round(b.offsetSec * 100)).toBe(b.offsetSec * 100); // 10 ms grid
expect(b.precision!).toBeGreaterThanOrEqual(0.8);
});
// ---- ruling A8: aligner guards ----
test("rejectOffsetOutliers drops bars > ¼ beat from the robust median of ±3 neighbours (slope-transported), iterating until stable", () => {
const bars = Array.from({ length: 20 }, (_, k) => ({ midiSec: k * 2, off: 0.1 + k * 0.004, beatSec: 0.5 }));
bars[5]!.off = 0.6; // a beat out
bars[6]!.off = 0.55; // its neighbour, also out
bars[15]!.off = -0.3;
const kept = rejectOffsetOutliers(bars).map((b) => b.midiSec / 2);
expect(kept).toEqual(Array.from({ length: 20 }, (_, k) => k).filter((k) => k !== 5 && k !== 6 && k !== 15));
// a smooth ramp (genuine tempo change) is kept whole
const ramp = Array.from({ length: 20 }, (_, k) => ({ midiSec: k * 2, off: k * 0.05, beatSec: 0.5 }));
expect(rejectOffsetOutliers(ramp)).toHaveLength(20);
// ... also a steep one (10% tempo difference: 0.2 s per 2 s bar), whose ends a plain one-sided median would erode
// bar by bar; neighbours are transported along their median (Theil–Sen) slope
const steep = Array.from({ length: 20 }, (_, k) => ({ midiSec: k * 2, off: k * 0.2, beatSec: 0.5 }));
expect(rejectOffsetOutliers(steep)).toHaveLength(20);
// a beat jump inside the steep ramp is still rejected
steep[10]!.off += 0.5;
expect(rejectOffsetOutliers(steep).map((b) => b.midiSec / 2)).toEqual(Array.from({ length: 20 }, (_, k) => k).filter((k) => k !== 10));
});
test("meterGrid: beats and bars from tempo/time signatures; the final bar is partial when the MIDI ends mid-bar", () => {
const smf: Smf = { durationSec: 9, tracks: [], tempoMap: [{ sec: 0, bpm: 120 }, { sec: 4, bpm: 60 }], timeSigs: [{ sec: 0, num: 4, den: 4 }, { sec: 4, num: 3, den: 4 }] };
const g = meterGrid(smf);
expect(g.bars).toEqual([0, 2, 4, 7]);
expect(g.beats).toEqual([0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4, 5, 6, 7, 8, 9]);
expect(g.lastBarPartial).toBe(true); // bar 7..10 ends after 9
expect(meterGrid({ ...smf, durationSec: 10 }).lastBarPartial).toBe(false);
expect(beatTimes({ durationSec: 3, tracks: [], tempoMap: [{ sec: 0, bpm: 120 }], timeSigs: [{ sec: 0, num: 6, den: 8 }] })).toEqual([0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2, 2.25, 2.5, 2.75]);
});
test("no auto anchor in the final partial bar", () => {
const D = distortion(song.song.bpm);
const smf = smfFrom(song.truth.tracks, song.song.tracks.map((t) => t.program), song.song.bpm, D.toAudio);
const g = meterGrid(smf);
const last = g.bars[g.bars.length - 1]!;
const cut: Smf = { ...smf, durationSec: last + 0.5 * (last - g.bars[g.bars.length - 2]!) }; // ends half-way into the last bar
expect(meterGrid(cut).lastBarPartial).toBe(true);
const { anchors } = autoAlign(song.mix, cut, rolesOf(song.truth.tracks), [], SF2);
expect(anchors.some((a) => a.midiSec >= last - 1e-9)).toBe(false);
}, BIG);
test("12 s of extra outro in the recording: open-end DTW, no ramp at the end", () => {
const D = distortion(song.song.bpm);
const smf = smfFrom(song.truth.tracks, song.song.tracks.map((t) => t.program), song.song.bpm, D.toAudio);
const sr = song.mix.sampleRate, extra = 12 * sr;
// the outro repeats the first 12 s of the song (material that DTW could be tempted to match)
const mix: Audio = { sampleRate: sr, channels: song.mix.channels.map((c) => { const o = new Float32Array(c.length + extra); o.set(c); o.set(c.subarray(0, extra), c.length); return o; }) };
const { anchors, chosen } = autoAlign(mix, smf, rolesOf(song.truth.tracks), [], SF2);
expect(chosen).toBe("dtw");
const autos = anchors.filter((a) => a.kind === "auto");
const tail = autos.slice(-4).map((a) => a.audioSec - D.toMidi(a.midiSec));
debug(`outro: last anchor offsets vs truth ${tail.map((e) => (e * 1000).toFixed(0)).join(", ")} ms`);
expect(tail.every((e) => Math.abs(e) < 0.06)).toBe(true);
const w = makeWarp(anchors);
const endErr = Math.abs(w.toAudio(smf.durationSec) - D.toMidi(smf.durationSec));
expect(endErr).toBeLessThan(0.1);
const err = recoveryErrors(smf, anchors, D.toMidi, durOf(song.mix));
expect(median(err)).toBeLessThan(0.04);
}, BIG);
+66
View File
@@ -43,3 +43,69 @@ test("3000x3000 unbanded dtw finishes in < 5 s", () => {
expect(p[p.length - 1]).toEqual([n - 1, n - 1]);
expect(ms).toBeLessThan(5000);
});
// ---- slope-limited steps and open begin/end (ruling A8) ----
const stepsOf = (p: [number, number][]) => p.slice(1).map(([i, j], k) => [i - p[k]![0], j - p[k]![1]] as const);
/** No two consecutive non-diagonal steps in the same direction: local slope stays within [1/2, 2]. */
function slopeLimited(p: [number, number][]): boolean {
const s = stepsOf(p);
return s.every(([di, dj], k) => {
if (!((di === 1 && dj === 1) || (di === 1 && dj === 0) || (di === 0 && dj === 1))) return false;
const prev = s[k - 1];
return !(k > 0 && di + dj === 1 && prev![0] === di && prev![1] === dj);
});
}
test("slope-limited steps: path never makes an abrupt jump even when the cost invites one", () => {
// the cost is 0 on a path that jumps 20 columns at i = 50; symmetric steps follow the jump, slope-limited ones cannot
const jumpCost = (i: number, j: number) => Math.min(1, Math.abs(j - (i < 50 ? i : i + 20)) * 0.2);
const sym = dtw(jumpCost, 100, 120);
expect(stepsOf(sym).some(([di, dj], k, s) => di === 0 && dj === 1 && s[k + 1]?.[0] === 0)).toBe(true); // a horizontal run
const p = dtw(jumpCost, 100, 120, undefined, { steps: "slope" });
expect(p[0]).toEqual([0, 0]);
expect(p[p.length - 1]).toEqual([99, 119]);
expect(slopeLimited(p)).toBe(true);
});
test("slope-limited steps follow a smooth diagonal-with-kink path exactly like the default (banded too)", () => {
const p = dtw(cost, N, M, undefined, { steps: "slope" });
expect(slopeLimited(p)).toBe(true);
for (const [i, j] of p) if (i < 45 || i > 60) expect(Math.abs(j - truthJ(i))).toBeLessThanOrEqual(1);
expect(dtw(cost, N, M, { center: truthJ, width: 15 }, { steps: "slope" })).toEqual(p);
});
test("slope-limited steps throw 'unreachable' when the aspect ratio exceeds 2", () => {
expect(() => dtw(cost, 10, 30, undefined, { steps: "slope" })).toThrow(/unreachable/);
});
test("open end: the path may stop before the recording ends (subsequence end)", () => {
// the query (rows) matches columns 0..99 diagonally; the last 40 columns are extra material
const c = (i: number, j: number) => (j < 100 ? Math.min(1, Math.abs(i - j) * 0.2) : 0.5);
const closed = dtw(c, 100, 140, undefined, { steps: "slope" });
expect(closed[closed.length - 1]).toEqual([99, 139]);
const open = dtw(c, 100, 140, undefined, { steps: "slope", openEnd: true });
expect(open[open.length - 1]).toEqual([99, 99]);
for (const [i, j] of open) expect(Math.abs(i - j)).toBeLessThanOrEqual(1);
// the query being longer than the target (its extra rows mismatch everything, costing more than an unconsumed row):
// the path may end on the last column
const longer = dtw((i, j) => (i < 60 ? Math.min(1, Math.abs(i - j) * 0.2) : 1.5), 100, 60, undefined, { steps: "slope", openEnd: true });
expect(longer[longer.length - 1]).toEqual([59, 59]);
});
test("open begin: the path may start after the beginning of the target", () => {
const c = (i: number, j: number) => (j >= 30 ? Math.min(1, Math.abs(i + 30 - j) * 0.2) : 0.5);
const p = dtw(c, 100, 130, undefined, { steps: "slope", openBegin: true });
expect(p[0]).toEqual([0, 30]);
expect(p[p.length - 1]).toEqual([99, 129]);
for (const [i, j] of p) expect(Math.abs(i + 30 - j)).toBeLessThanOrEqual(1);
// banded with a band that does not include column 0 at row 0
expect(dtw(c, 100, 130, { center: (i) => i + 30, width: 10 }, { steps: "slope", openBegin: true })).toEqual(p);
});
test("open begin + end: a trivial path in a zero-cost corner does not win (unconsumed rows cost 1 each)", () => {
// the last column is free everywhere (e.g. silence vs silence), the real match is the diagonal of the first 100 columns
const c = (i: number, j: number) => (j === 119 ? 0 : j < 100 ? Math.min(1, Math.abs(i - j) * 0.2) : 0.6);
const p = dtw(c, 100, 120, undefined, { steps: "slope", openBegin: true, openEnd: true });
expect(p.length).toBeGreaterThan(90);
expect(p[p.length - 1]).toEqual([99, 99]);
});
+13
View File
@@ -55,3 +55,16 @@ test("pickOnsets enforces 50 ms min gap and ignores flat input", () => {
expect(pickOnsets(o, 100)).toEqual([0.22, 0.6]);
expect(pickOnsets(new Float32Array(100), 100)).toEqual([]);
});
test("pickOnsets refines peaks to sub-frame times by parabolic interpolation", () => {
const o = new Float32Array(100);
o[30] = 4; o[31] = 8; o[32] = 6; // peak leans right: δ = ½(4 − 6)/(4 − 16 + 6) = 1/6 frame
const got = pickOnsets(o, 50);
expect(got).toHaveLength(1);
expect(got[0]!).toBeCloseTo((31 + 1 / 6) / 50, 9);
// min-gap competition still compares the peak heights, not the interpolated times
const p = new Float32Array(100);
p[40] = 5; p[41] = 3; p[42] = 7; p[43] = 1;
expect(pickOnsets(p, 100)).toHaveLength(1);
expect(pickOnsets(p, 100)[0]!).toBeCloseTo((42 + 0.5 * (3 - 1) / (3 - 14 + 1)) / 100, 9);
});
+96 -62
View File
@@ -1,72 +1,106 @@
import { test, expect } from "bun:test";
import { fitQuality, fitSummary } from "../../src/align/fit.ts";
import { fitQuality, fitSummary, FLAG_MIN_PRECISION, FLAG_MIN_LIFT, MIN_REC_ONSETS, SHIFT_MARGIN, type FitReport, type FitWindow } from "../../src/align/fit.ts";
test("perfect fit: match rate 1, residual 0, one window per 30 s", () => {
const on = [1, 2, 3, 31, 32, 61];
const r = fitQuality(on, on, 75);
expect(r.matchRate).toBe(1);
expect(r.medianResidualMs).toBe(0);
expect(r.windows.map((w) => w.startSec)).toEqual([0, 30, 60]);
expect(r.windows.every((w) => !w.flagged && w.matchRate === 1)).toBe(true);
const grid = (from: number, to: number, step: number) => { const o: number[] = []; for (let t = from; t < to - 1e-9; t += step) o.push(Math.round(t * 1000) / 1000); return o; };
test("constants match ruling A7", () => {
expect(FLAG_MIN_PRECISION).toBe(0.6);
expect(FLAG_MIN_LIFT).toBe(0.25);
expect(MIN_REC_ONSETS).toBe(8);
expect(SHIFT_MARGIN).toBe(0.05);
});
test("match within ±70 ms only; residual is median |audio − midi| of matched onsets", () => {
const midi = [1, 2, 3, 4];
const audio = [1.01, 2.03, 3.069, 4.2]; // last is 200 ms away: unmatched
const r = fitQuality(audio, midi, 10);
expect(r.matchRate).toBe(0.75);
test("precision = share of recording onsets with a MIDI onset within ±70 ms; audibleRate = the reverse (info)", () => {
// recording hears every 2nd MIDI onset (dense score, sparse attacks) plus 2 extra attacks out of the score
const midi = grid(0.5, 20.5, 0.25); // 80 onsets
const rec = [...midi.filter((_, k) => k % 2 === 0), 3.13, 7.63]; // 40 + 2
const r = fitQuality(rec, midi, 20.5);
expect(r.precision).toBeCloseTo(40 / 42, 6);
expect(r.audibleRate).toBeCloseTo(0.5, 6);
expect(r.medianResidualMs).toBe(0);
expect(r.windows).toHaveLength(1);
expect(r.windows[0]!.recOnsets).toBe(42);
// low audibleRate alone does not flag (that was the old, miscalibrated rule)
expect(r.windows[0]!.flagged).toBe(false);
});
test("lift: precision minus mean precision at off-grid shifts (+0.11, +0.23, +0.37 s)", () => {
const midi = grid(1, 29, 1); // one onset per second: shifts of 0.11–0.37 s never land
const r = fitQuality(midi, midi, 30);
expect(r.precision).toBe(1);
expect(r.lift).toBeCloseTo(1, 6);
expect(r.windows[0]!.lift).toBeCloseTo(1, 6);
// a dense MIDI grid (every 0.12 s) matches anything: precision 1 but no lift -> flagged
const dense = grid(0, 30, 0.12);
const rec = grid(0.5, 29.5, 0.5).map((t) => t + 0.031);
const d = fitQuality(rec, dense, 30);
expect(d.precision).toBe(1);
expect(d.lift!).toBeLessThan(FLAG_MIN_LIFT);
expect(d.windows[0]!.flagged).toBe(true);
expect(d.windows[0]!.reasons.join(" ")).toContain("lift");
});
test("window flagged when precision < 0.6; null and unflagged with < 8 recording onsets", () => {
const midi = grid(1, 89, 1);
const rec = [
...grid(1, 11, 1), // window 0: 10 onsets, all on the score
...grid(31.5, 41.5, 1), // window 1: 10 onsets, all half-way between score onsets
61, 62, 63, // window 2: only 3 onsets
];
const r = fitQuality(rec, midi, 90);
expect(r.windows.map((w) => w.startSec)).toEqual([0, 30, 60]);
expect(r.windows[0]!.precision).toBe(1);
expect(r.windows[0]!.flagged).toBe(false);
expect(r.windows[1]!.precision).toBe(0);
expect(r.windows[1]!.flagged).toBe(true);
expect(r.windows[1]!.reasons[0]).toContain("precision");
const w2 = r.windows[2]!;
expect(w2.recOnsets).toBe(3);
expect(w2.precision).toBeNull();
expect(w2.lift).toBeNull();
expect(w2.flagged).toBe(false);
});
test("no MIDI onsets: overall nulls, nothing flagged", () => {
const r = fitQuality([1, 2], [], 10);
expect(r.precision).toBeNull();
expect(r.audibleRate).toBeNull();
expect(r.lift).toBeNull();
expect(r.windows.every((w) => !w.flagged)).toBe(true);
});
test("residual: median |recording − nearest MIDI| over matched recording onsets", () => {
const midi = grid(1, 11, 1);
const r = fitQuality(midi.map((t, k) => t + (k % 2 ? 0.02 : 0.04)), midi, 12);
expect(r.medianResidualMs!).toBeCloseTo(30, 6);
});
test("nearest recording onset is used and may be shared by several MIDI onsets", () => {
const r = fitQuality([1.0], [0.98, 1.0, 1.02], 5);
expect(r.matchRate).toBe(1);
expect(r.medianResidualMs!).toBeCloseTo(20, 6);
test("beat-shift check: flags a window where a shifted symbolic chroma beats the current one by the margin", () => {
const rate = 10, n = 600; // 60 s, two windows
const pc = (k: number) => { const c = new Float32Array(12); c[k % 12] = 1; return c; };
// chord changes every second (one "beat"); recording = sequence s(t); current MIDI = s shifted by one beat in window 1
const seq = (t: number) => pc(Math.floor(t / rate) * 5);
const recording = Array.from({ length: n }, (_, t) => seq(t));
const current = Array.from({ length: n }, (_, t) => (t < 300 ? seq(t) : seq(t - rate)));
const plus1 = Array.from({ length: n }, (_, t) => seq(t - rate)), minus1 = Array.from({ length: n }, (_, t) => (t < 300 ? seq(t - rate) : seq(t)));
const onsets = grid(0.5, 59.5, 0.5);
const r = fitQuality(onsets, onsets, 60, { shift: { rate, recording, current, shifts: [{ label: "+1 beat", chroma: plus1 }, { label: "−1 beat", chroma: minus1 }] } });
expect(r.windows[0]!.beatShift).toBeNull();
expect(r.windows[0]!.flagged).toBe(false);
expect(r.windows[1]!.beatShift?.label).toBe("−1 beat");
expect(r.windows[1]!.beatShift!.gain).toBeGreaterThan(SHIFT_MARGIN);
expect(r.windows[1]!.flagged).toBe(true);
expect(r.windows[1]!.reasons).toContain("beat shift suspected: −1 beat");
});
test("empty window gives nulls and is not flagged", () => {
const r = fitQuality([1, 2, 65], [1, 2, 65], 90);
expect(r.windows).toHaveLength(3);
const w = r.windows[1]!;
expect(w).toEqual({ startSec: 30, matchRate: null, medianResidualMs: null, flagged: false });
});
test("no MIDI onsets at all gives overall nulls", () => {
const r = fitQuality([1, 2], [], 10);
expect(r.matchRate).toBeNull();
expect(r.medianResidualMs).toBeNull();
expect(r.windows).toEqual([{ startSec: 0, matchRate: null, medianResidualMs: null, flagged: false }]);
});
test("window flagged when match rate < 0.5", () => {
const midi = [1, 2, 3, 4, 31, 32];
const audio = [1, 31, 32]; // window 0: 1/4 matched
const r = fitQuality(audio, midi, 60);
expect(r.windows[0]!.matchRate).toBe(0.25);
expect(r.windows[0]!.flagged).toBe(true);
expect(r.windows[1]!.flagged).toBe(false);
expect(r.matchRate).toBe(0.5);
});
test("window flagged when median residual > 60 ms even with full match", () => {
const midi = [1, 2, 3];
const r = fitQuality(midi.map((t) => t + 0.065), midi, 10);
expect(r.matchRate).toBe(1);
expect(r.windows[0]!.medianResidualMs!).toBeCloseTo(65, 6);
expect(r.windows[0]!.flagged).toBe(true);
});
test("unsorted and duplicate MIDI onsets: duplicates count once", () => {
const r = fitQuality([2, 1], [2, 1, 1, 1], 5);
expect(r.matchRate).toBe(1);
const s = fitQuality([1], [1, 1, 1, 3], 5);
expect(s.matchRate).toBe(0.5); // unique onsets {1, 3}
});
test("fitSummary: one line with match rate, median residual and flagged windows", () => {
const w = (startSec: number, flagged: boolean) => ({ startSec, matchRate: 0.9, medianResidualMs: 10, flagged });
expect(fitSummary({ matchRate: 0.8234, medianResidualMs: 14.24, windows: [w(0, false), w(30, true), w(60, false), w(90, true)] }))
.toBe("82.3% of MIDI onsets matched · median residual 14.2 ms · 2/4 windows flagged (0:30, 1:30)");
expect(fitSummary({ matchRate: 1, medianResidualMs: 3, windows: [w(0, false)] })).toBe("100.0% of MIDI onsets matched · median residual 3.0 ms · 0/1 windows flagged");
expect(fitSummary({ matchRate: null, medianResidualMs: null, windows: [] })).toBe("no scored MIDI onsets");
test("fitSummary: precision, lift, audible rate, residual, flagged windows with reasons, stretch warnings", () => {
const w = (startSec: number, reasons: string[]): FitWindow => ({ startSec, recOnsets: 20, precision: 0.9, lift: 0.7, audibleRate: 0.4, medianResidualMs: 10, beatShift: null, flagged: reasons.length > 0, reasons });
const f: FitReport = {
precision: 0.8234, lift: 0.6512, audibleRate: 0.3811, medianResidualMs: 14.24,
windows: [w(0, []), w(30, ["beat shift suspected: +1 beat"]), w(60, []), w(90, ["precision 41%"])],
stretches: { total: 40, warned: 3, over15: 1 },
};
expect(fitSummary(f)).toBe("precision 82.3% (lift +0.65) · MIDI onsets audible as attacks 38.1% · median residual 14.2 ms · 2/4 windows flagged (0:30 beat shift suspected: +1 beat; 1:30 precision 41%) · stretches: 3/40 warned, 1 > 15%");
expect(fitSummary({ ...f, windows: [w(0, [])], stretches: undefined })).toBe("precision 82.3% (lift +0.65) · MIDI onsets audible as attacks 38.1% · median residual 14.2 ms · 0/1 windows flagged");
expect(fitSummary({ precision: null, lift: null, audibleRate: null, medianResidualMs: null, windows: [] })).toBe("no scored MIDI onsets");
});
+14 -7
View File
@@ -127,12 +127,18 @@ describe("API", () => {
expect(Math.max(...first.recordingPeaks.max)).toBeGreaterThan(0.05);
expect(first.source).toBe("auto");
expect(first.savedAt).toBeNull();
expect(first.anchors.length).toBeGreaterThan(2);
// the fixture's MIDI is the recording's own score (identity alignment): the constant-offset baseline wins
expect(first.auto.chosen).toBe("baseline");
expect(first.anchors).toHaveLength(1);
expect(Math.abs(first.auto.baseline.offsetSec)).toBeLessThan(0.05);
expect(first.anchors.every((a: Anchor) => a.kind === "auto")).toBe(true);
expect(first.stretches.length).toBeGreaterThanOrEqual(first.anchors.length); // +1, minus an empty edge stretch
expect(first.stretches[1].midiBpm).toBeGreaterThan(30);
expect(first.stretches[1].bpm).toBeCloseTo(first.stretches[1].midiBpm / first.stretches[1].ratio, 6);
expect(first.fit.matchRate).toBeGreaterThan(0.5);
expect(first.fit.precision).toBeGreaterThan(0.6);
expect(first.fit.stretches.total).toBe(first.stretches.length);
expect(["dtw", "baseline"]).toContain(first.auto.chosen);
expect(typeof first.auto.baseline.offsetSec).toBe("number");
expect(first.midiTempo[0].bpm).toBeGreaterThan(30);
expect(await Bun.file(join(root, "fix", "recording.wav")).exists()).toBe(true);
expect(await Bun.file(join(root, "fix", "peaks.json")).exists()).toBe(true);
@@ -164,7 +170,7 @@ describe("API", () => {
const r = await api("/api/song/fix/fit", json({ anchors: [{ midiSec: 1, audioSec: 1.4, kind: "manual" }] }));
const j = await r.json();
expect(j.fit.windows.length).toBeGreaterThan(0);
expect(j.fit.matchRate).toBeLessThan(first.fit.matchRate);
expect(j.fit.precision).toBeLessThan(first.fit.precision);
expect(j.stretches.length).toBe(2);
});
test("bad anchors are rejected with 400", async () => {
@@ -173,12 +179,13 @@ describe("API", () => {
expect((await api("/api/song/fix/fit", { method: "POST", body: "{not json" })).status).toBe(400);
});
test("auto: keeps manual anchors and re-solves autos around them", async () => {
const manual: Anchor = { ...first.anchors[3], kind: "manual" };
const r = await api("/api/song/fix/auto", json({ anchors: [manual, ...first.anchors.slice(5)] }));
const manual: Anchor = { midiSec: 10, audioSec: 10, kind: "manual" };
const r = await api("/api/song/fix/auto", json({ anchors: [manual, { midiSec: 15, audioSec: 15.5, kind: "auto" }] }));
const j = await r.json();
expect(j.anchors.filter((a: Anchor) => a.kind === "manual")).toEqual([manual]);
expect(j.anchors.filter((a: Anchor) => a.kind === "auto").length).toBeGreaterThan(2);
expect(j.fit.matchRate).toBeGreaterThan(0.5);
expect(j.fit.precision).toBeGreaterThan(0.6);
expect(j.auto.chosen).toBe("dtw"); // manual anchors present: never the baseline
expect(j.stretches.length).toBeGreaterThanOrEqual(j.anchors.length);
}, BIG);
test("reset: raw MIDI (no anchors)", async () => {
@@ -217,6 +224,6 @@ describe("API", () => {
const j = await g.json();
expect(j.source).toBe("auto");
expect(j.warning).toContain("alignment.json");
expect(j.anchors.length).toBeGreaterThan(2);
expect(j.anchors.length).toBeGreaterThanOrEqual(1);
});
});
+7
View File
@@ -230,3 +230,10 @@ describe("deleteAnchor", () => {
expect(() => deleteAnchor([M(1, 1)], 1)).toThrow();
});
});
test("extrapolation before the first anchor uses the first stretch's slope (no implicit 0 -> 0 anchor)", () => {
const w = makeWarp([{ midiSec: 10, audioSec: 12, kind: "auto" }, { midiSec: 20, audioSec: 23, kind: "auto" }]); // slope 1.1
expect(w.toAudio(0)).toBeCloseTo(12 - 10 * 1.1, 9);
expect(w.toMidi(12 - 10 * 1.1)).toBeCloseTo(0, 9);
expect(makeWarp([{ midiSec: 10, audioSec: 12, kind: "auto" }]).toAudio(0)).toBe(2); // single anchor: slope 1
});
+32 -11
View File
@@ -9,31 +9,46 @@ import { writeAlignment, type SongEntry } from "../../src/ref/manifest.ts";
import { createPipeline } from "../../src/pipeline/run.ts";
import { scoreSong } from "../../src/eval/score.ts";
import type { Role } from "../../src/core/types.ts";
import type { FitReport, FitWindow } from "../../src/align/fit.ts";
const hasReference = await Bun.file("reference/songs.json").exists();
const mr = (f1: number | null, tp = 8, fp = 2, fn = 2) => ({ tp, fp, fn, precision: f1, recall: f1, f1, octaveErrors: 0, matches: [] as [number, number][] });
const fit = (rate: number) => ({ matchRate: rate, medianResidualMs: 12.3, windows: [{ startSec: 0, matchRate: rate, medianResidualMs: 12.3, flagged: false }, { startSec: 30, matchRate: 0.2, medianResidualMs: 80, flagged: true }] });
const win = (startSec: number, precision: number, reasons: string[]): FitWindow => ({ startSec, recOnsets: 40, precision, lift: 0.6, audibleRate: 0.4, medianResidualMs: 12.3, beatShift: null, flagged: reasons.length > 0, reasons });
const fit = (p: number): FitReport => ({
precision: p, lift: 0.62, audibleRate: 0.38, medianResidualMs: 12.3,
windows: [win(0, p, []), win(30, 0.5, ["precision 50%"])], stretches: { total: 41, warned: 4, over15: 2 },
});
const base = { offsetSec: 0.06, precision: 0.88 };
const fixture = (): RealSongResult[] => [
{
id: "a", title: "Song A", alignment: { source: "auto", manual: 0, auto: 40, savedAt: null }, fit: fit(0.8),
id: "a", title: "Song A", alignment: { source: "computed", manual: 0, auto: 40, savedAt: null, chosen: "dtw", baseline: base, dtwPrecision: 0.8 }, fit: fit(0.8),
perRole: { bass: mr(0.5), guitar: mr(0.9), keys: mr(null, 0, 0, 0) }, lenient: { bass: 0.6, guitar: 0.95, keys: null },
drumsPerClass: null, excludedTracks: ["Click", "Pad"], capo: { Lead: 2 }, seconds: 3,
},
{
id: "b", title: "Song B", alignment: { source: "manual", manual: 5, auto: 30, savedAt: "2026-10-29T10:00:00Z" }, fit: fit(0.6),
id: "b", title: "Song B", alignment: { source: "saved", manual: 0, auto: 1, savedAt: "2026-10-29T10:00:00Z", chosen: "baseline", baseline: base, dtwPrecision: 0.85 }, fit: fit(0.6),
perRole: { guitar: mr(0.3) }, lenient: { guitar: 0.4 }, drumsPerClass: null, excludedTracks: [], capo: {}, seconds: 4,
},
];
describe("renderRealReport", () => {
const md = renderRealReport("fx", fixture());
test("guitar row precedes the other roles; auto source and fit shown", () => {
const md = renderRealReport("fx", fixture(), [{ id: "c", reason: "reference/c/alignment.json: not valid JSON" }, { id: "d", reason: "no alignment.json" }]);
test("guitar row precedes the other roles; source, method, baseline, fit and stretch warnings shown", () => {
expect(md).toContain("Song A");
expect(md).toContain("computed this run");
expect(md).toContain("2026-10-29T10:00:00Z");
expect(md).toContain("saved 2026-10-29T10:00:00Z");
expect(md).toContain("0 manual / 40 auto");
expect(md).toContain("80.0% of MIDI onsets matched");
expect(md).not.toMatch(/manual \(/);
expect(md).toContain("chosen: DTW (precision 80.0%)");
expect(md).toContain("chosen: baseline (DTW precision 85.0% not better");
expect(md).toContain("constant-offset baseline +0.06 s: precision 88.0%");
expect(md).toContain("precision 80.0% (lift +0.62)");
expect(md).toContain("MIDI onsets audible as attacks 38.0%");
expect(md).toContain("1/2 windows flagged (0:30 precision 50%)");
expect(md).toContain("stretches: 4/41 warned, 2 > 15%");
expect(md).toContain("consistency of the alignment");
expect(md).toContain("c (reference/c/alignment.json: not valid JSON)");
expect(md).toContain("d (no alignment.json)");
const a = md.slice(md.indexOf("Song A"), md.indexOf("Song B"));
expect(a.indexOf("| guitar")).toBeGreaterThan(-1);
expect(a.indexOf("| guitar")).toBeLessThan(a.indexOf("| bass"));
@@ -59,15 +74,18 @@ describe("runReal", () => {
id: "t1a-1", title: "Synthetic t1a/1", audio: join(dir, "mix.wav"), midi: join(ref, "t1a-1", "song.mid"),
roles: Object.fromEntries(truth.tracks.map((t) => [t.name, t.role])), capo: {},
};
await Bun.write(join(ref, "songs.json"), JSON.stringify([entry, { ...entry, id: "noalign", title: "No alignment" }]));
await Bun.write(join(ref, "songs.json"), JSON.stringify([entry, { ...entry, id: "noalign", title: "No alignment" }, { ...entry, id: "corrupt", title: "Corrupt alignment" }]));
await Bun.write(join(ref, "corrupt", "alignment.json"), "{ not json");
const dur = 600;
await writeAlignment("t1a-1", { anchors: [{ midiSec: 0, audioSec: 0, kind: "manual" }, { midiSec: dur, audioSec: dur, kind: "manual" }], savedAt: "2026-10-30T00:00:00Z", fit: null }, ref);
const results = await runReal({ runId: "t", auto: false, root: ref, out: join(root, "runs"), resultsDir: join(root, "results") });
expect(results.map((r) => r.id)).toEqual(["t1a-1"]);
const r = results[0]!;
expect(r.alignment).toEqual({ source: "manual", manual: 2, auto: 0, savedAt: "2026-10-30T00:00:00Z" });
expect(r.fit.matchRate).not.toBeNull();
expect(r.alignment).toEqual({ source: "saved", manual: 2, auto: 0, savedAt: "2026-10-30T00:00:00Z", chosen: null, baseline: expect.any(Object), dtwPrecision: null });
expect(Math.abs(r.alignment.baseline.offsetSec)).toBeLessThan(0.05); // identity alignment
expect(r.fit.precision).not.toBeNull();
expect(r.fit.stretches).toBeDefined();
const pipe = await createPipeline({
demucs: new Uint8Array(await Bun.file("models/demucs/htdemucs_6s.onnx").arrayBuffer()),
@@ -81,7 +99,10 @@ describe("runReal", () => {
for (const f of ["notes.json", "out.gp", "score.json"]) expect(await Bun.file(join(root, "runs", "real-t", "t1a-1", f)).exists()).toBe(true);
const json = await Bun.file(join(root, "results", "real-t.json")).text();
expect(json).not.toContain('"matches"');
expect(await Bun.file(join(root, "results", "real-t.md")).text()).toContain("Synthetic t1a/1");
const md = await Bun.file(join(root, "results", "real-t.md")).text();
expect(md).toContain("Synthetic t1a/1");
expect(md).toMatch(/corrupt \(.*alignment\.json.*not valid JSON/);
expect(md).toContain("noalign (no alignment.json; pass --auto to compute one)");
} finally { await rm(root, { recursive: true, force: true }); }
}, 600_000);