lucasw89/STSheets
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: baseline benchmark results

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
This commit is contained in:
Lucas Winther
2026-10-30 05:43:50 +01:00
co-authored by Claude Sonnet 5.5
parent d9fd5d8c9f
commit 2b7d5ffbeb
4 changed files with 326469 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+6
View File
@@ -37,3 +37,9 @@ data/
runs/
models/**/*.onnx
.superpowers/
# reference audio/midi supplied by the team (never commit)
*.flac
*.mid
*.midi
reference/
README.md
+29 -8
View File
@@ -1,15 +1,36 @@
# stsheets
# STSheets synthetic benchmark
To install dependencies:
A harness that measures guitar-tab transcription quality without any real recordings. It generates original songs in four tiers (t1a to t1d, from solo guitar up to a full band with a second guitar part), writes them as Guitar Pro files, renders audio with a bundled SoundFont, then runs the pipeline (Demucs source separation, Basic Pitch transcription, tempo and beat detection, tab fingering) and scores the result against the known truth. A song passes when merged-guitar pitch F1 is at least 0.85. Oracle modes feed ground truth to later stages so errors can be attributed to a single stage.
```bash
## Prerequisites
- [Bun](https://bun.sh)
- ffmpeg (and ffprobe) on `PATH`
## Commands
```sh
bun install
bun run fetch-models # downloads the ONNX models into models/
bun test # one test is skipped unless the Demucs drum measurement is enabled by env
bun run bench gen --tiers t1a,t1b,t1c,t1d --seeds 1-5
bun run bench run --tiers t1a,t1b,t1c,t1d --seeds 1-5 \
--modes full,oracle-stems,oracle-tempo,oracle-notes --run-id baseline-2026-10-30
```
To run:
## On-disk layout
```bash
bun run index.ts
```
- `data/<tier>/<seed>/` holds `song.json`, `truth.gp`, `mix.wav`, `mix.mp3` and `stems/`. Git-ignored.
- `runs/<run-id>/<tier>/<seed>/<mode>/` holds `notes.json`, `out.gp` and `score.json`. Git-ignored.
- `results/<run-id>.md` and `.json` are the committed reports.
This project was created using `bun init` in bun v1.3.14. [Bun](https://bun.com) is a fast all-in-one JavaScript runtime.
## Results
The baseline is in `results/baseline-2026-10-30.md` (tier summaries plus a per-song appendix) with the raw numbers in the matching `.json`.
## Known caveats
- Vocals are rendered with a GM "Voice Oohs" patch, not real singing, so vocal numbers are indicative only.
- Drums use a heuristic baseline transcriber.
- The SONiVOX guitar is partly routed to Demucs' piano stem, a synthetic-timbre gap that lowers full-mode guitar scores.
- Fingering (string agreement) is informational, not pass/fail.
results/baseline-2026-10-30.json
+326295
View File
File diff suppressed because it is too large Load Diff
results/baseline-2026-10-30.md
+139
View File
@@ -0,0 +1,139 @@
# STSheets benchmark report: baseline-2026-10-30
- Run id: baseline-2026-10-30
- Date: 2026-10-30T04:19:26.998Z
- Machine: AMD Ryzen 9 9950X 16-Core Processor x16, linux x64, bun 1.3.14
- Models (sha256): htdemucs_6s.onnx 35cb61d4d35173ff1d16df7393f41f05651ffdfe13d3c10e06abc26708bd6933; nmp.onnx 2c3c1d144bfa61ad236e92e169c13535c880469a12a047d4e73451f2c059a0ec
- SoundFont: alphaTab bundled sonivox.sf2 (Apache-2.0)
- Modes: full (stems=false, tempo=false, notes=false); oracle-stems (stems=true, tempo=false, notes=false); oracle-tempo (stems=true, tempo=true, notes=false); oracle-notes (stems=true, tempo=true, notes=true)
> Caveats: vocals are rendered with a GM "Voice Oohs" patch, not real singing, so separation/transcription numbers for vocals are indicative only. Drums use a heuristic baseline transcriber. The SONiVOX guitar is partly routed to Demucs' piano stem (known synthetic-timbre gap). Pitch is pass/fail (song passes at merged-guitar pitch F1 >= 0.85); rhythm is secondary; fingering (string agreement) is informational. Cells are median (min–max).
## Tier t1a
| Mode | Songs | Guitar merged F1 | Guitar octave errors | Spurious rate | Part-split F1 (t1d) | Vocals F1 | Bass F1 | Drums kick F1 | Drums snare F1 | Drums hihat F1 | Keys F1 | Rhythm onBeat | Rhythm durationOk | Tempo factor | Beat offset | String agreement |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| full | 5 | 0.43 (0.18–0.86) ✗ | 3.0 (0.0–3.0) | 0.64 (0.22–0.69) | n/a | n/a (424 spurious) | n/a (1260 spurious) | n/a (80 spurious) | n/a (70 spurious) | n/a (200 spurious) | n/a (1592 spurious) | 0.65 (0.33–1.00) | 0.14 (0.03–0.18) | 1.00 (1.00–1.00) | 0 (-3–0) | 0.85 (0.51–1.00) |
| oracle-stems | 5 | 0.87 (0.64–0.93) ✗ | 0.0 (0.0–0.0) | 0.21 (0.13–0.53) | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 0.99 (0.51–1.00) | 0.19 (0.12–0.45) | 1.00 (1.00–1.00) | 0 (-3–0) | 0.85 (0.81–0.95) |
| oracle-tempo | 5 | 0.87 (0.64–0.93) ✗ | 0.0 (0.0–0.0) | 0.21 (0.13–0.53) | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 1.00 (0.99–1.00) | 0.25 (0.19–0.52) | 1.00 (1.00–1.00) | 0 (0–0) | 0.85 (0.81–0.96) |
| oracle-notes | 5 | 1.00 (1.00–1.00) ✓ | 0.0 (0.0–0.0) | 0.00 (0.00–0.00) | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 0 (0–0) | 1.00 (0.98–1.00) |
**Tier t1a: FAIL** (median merged-guitar F1 0.43 in `full` mode, threshold 0.85)
## Tier t1b
| Mode | Songs | Guitar merged F1 | Guitar octave errors | Spurious rate | Part-split F1 (t1d) | Vocals F1 | Bass F1 | Drums kick F1 | Drums snare F1 | Drums hihat F1 | Keys F1 | Rhythm onBeat | Rhythm durationOk | Tempo factor | Beat offset | String agreement |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| full | 5 | 0.51 (0.18–0.73) ✗ | 3.0 (1.0–7.0) | 0.53 (0.28–0.69) | n/a | n/a (371 spurious) | 0.37 (0.22–0.38) | 0.98 (0.95–1.00) | 0.49 (0.16–0.76) | 1.00 (0.99–1.00) | n/a (1774 spurious) | 0.84 (0.52–0.95) | 0.20 (0.11–0.28) | 1.00 (1.00–1.00) | 0 (0–0) | 0.93 (0.57–0.99) |
| oracle-stems | 5 | 0.87 (0.67–0.92) ✗ | 0.0 (0.0–0.0) | 0.20 (0.15–0.49) | n/a | n/a | 0.50 (0.45–0.52) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | n/a | 1.00 (0.99–1.00) | 0.23 (0.19–0.51) | 1.00 (1.00–1.00) | 0 (0–0) | 0.86 (0.71–0.96) |
| oracle-tempo | 5 | 0.87 (0.67–0.92) ✗ | 0.0 (0.0–0.0) | 0.20 (0.15–0.49) | n/a | n/a | 0.50 (0.45–0.52) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | n/a | 1.00 (0.99–1.00) | 0.23 (0.19–0.51) | 1.00 (1.00–1.00) | 0 (0–0) | 0.86 (0.71–0.96) |
| oracle-notes | 5 | 1.00 (1.00–1.00) ✓ | 0.0 (0.0–0.0) | 0.00 (0.00–0.00) | n/a | n/a | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | n/a | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 0 (0–0) | 1.00 (0.98–1.00) |
**Tier t1b: FAIL** (median merged-guitar F1 0.51 in `full` mode, threshold 0.85)
## Tier t1c
| Mode | Songs | Guitar merged F1 | Guitar octave errors | Spurious rate | Part-split F1 (t1d) | Vocals F1 | Bass F1 | Drums kick F1 | Drums snare F1 | Drums hihat F1 | Keys F1 | Rhythm onBeat | Rhythm durationOk | Tempo factor | Beat offset | String agreement |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| full | 5 | 0.39 (0.16–0.57) ✗ | 4.0 (1.0–11.0) | 0.58 (0.35–0.78) | n/a | 0.21 (0.05–0.24) | 0.32 (0.18–0.43) | 1.00 (0.92–1.00) | 0.75 (0.37–0.92) | 0.99 (0.99–1.00) | 0.11 (0.06–0.18) | 0.81 (0.49–0.94) | 0.19 (0.00–0.29) | 1.00 (1.00–1.00) | 0 (-2–0) | 0.94 (0.90–0.97) |
| oracle-stems | 5 | 0.87 (0.67–0.92) ✗ | 0.0 (0.0–0.0) | 0.20 (0.15–0.49) | n/a | 0.86 (0.84–0.96) | 0.47 (0.46–0.52) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 0.18 (0.06–0.39) | 1.00 (0.99–1.00) | 0.23 (0.19–0.51) | 1.00 (1.00–1.00) | 0 (-2–0) | 0.86 (0.70–0.96) |
| oracle-tempo | 5 | 0.87 (0.67–0.92) ✗ | 0.0 (0.0–0.0) | 0.20 (0.15–0.49) | n/a | 0.86 (0.84–0.96) | 0.47 (0.46–0.52) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 0.18 (0.06–0.39) | 1.00 (0.99–1.00) | 0.23 (0.19–0.51) | 1.00 (1.00–1.00) | 0 (0–0) | 0.86 (0.70–0.96) |
| oracle-notes | 5 | 1.00 (1.00–1.00) ✓ | 0.0 (0.0–0.0) | 0.00 (0.00–0.00) | n/a | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 0 (0–0) | 1.00 (0.98–1.00) |
**Tier t1c: FAIL** (median merged-guitar F1 0.39 in `full` mode, threshold 0.85)
## Tier t1d
| Mode | Songs | Guitar merged F1 | Guitar octave errors | Spurious rate | Part-split F1 (t1d) | Vocals F1 | Bass F1 | Drums kick F1 | Drums snare F1 | Drums hihat F1 | Keys F1 | Rhythm onBeat | Rhythm durationOk | Tempo factor | Beat offset | String agreement |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| full | 5 | 0.60 (0.43–0.72) ✗ | 4.0 (0.0–12.0) | 0.33 (0.24–0.49) | 0.50 (0.35–0.71) | 0.29 (0.13–0.52) | 0.37 (0.34–0.45) | 1.00 (0.98–1.00) | 0.41 (0.07–0.58) | 0.99 (0.89–1.00) | 0.10 (0.07–0.15) | 0.80 (0.73–0.93) | 0.19 (0.16–0.25) | 1.00 (1.00–1.00) | 0 (0–0) | 0.93 (0.82–0.97) |
| oracle-stems | 5 | 0.92 (0.78–0.92) ✗ | 0.0 (0.0–14.0) | 0.15 (0.14–0.34) | 0.92 (0.77–0.92) | 0.88 (0.84–0.96) | 0.47 (0.46–0.53) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 0.65 (0.06–0.95) | 1.00 (0.98–1.00) | 0.39 (0.14–0.48) | 1.00 (1.00–1.00) | 0 (0–0) | 0.83 (0.74–0.89) |
| oracle-tempo | 5 | 0.92 (0.78–0.92) ✗ | 0.0 (0.0–14.0) | 0.15 (0.14–0.34) | 0.92 (0.77–0.92) | 0.88 (0.84–0.96) | 0.47 (0.46–0.53) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 0.65 (0.06–0.95) | 1.00 (0.98–1.00) | 0.39 (0.14–0.48) | 1.00 (1.00–1.00) | 0 (0–0) | 0.83 (0.74–0.89) |
| oracle-notes | 5 | 1.00 (1.00–1.00) ✓ | 0.0 (0.0–0.0) | 0.00 (0.00–0.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 1.00 (1.00–1.00) | 0 (0–0) | 1.00 (0.95–1.00) |
**Tier t1d: FAIL** (median merged-guitar F1 0.60 in `full` mode, threshold 0.85)
## Per-song appendix
| Tier | Seed | Mode | Guitar pass | Guitar merged F1 | Guitar octave errors | Spurious rate | Part-split F1 (t1d) | Vocals F1 | Bass F1 | Drums kick F1 | Drums snare F1 | Drums hihat F1 | Keys F1 | Rhythm onBeat | Rhythm durationOk | Tempo factor | Beat offset | String agreement | Seconds |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| t1a | 1 | full | ✗ | 0.79 | 3.0 | 0.23 | n/a | n/a (77 spurious) | n/a (320 spurious) | n/a (38 spurious) | n/a (2 spurious) | n/a (3 spurious) | n/a (349 spurious) | 1.00 | 0.18 | 1.00 | 0 | 0.88 | 18.4 |
| t1a | 1 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.0 |
| t1a | 1 | oracle-stems | ✓ | 0.87 | 0.0 | 0.21 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 0.99 | 0.23 | 1.00 | 0 | 0.85 | 2.2 |
| t1a | 1 | oracle-tempo | ✓ | 0.87 | 0.0 | 0.21 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 0.99 | 0.23 | 1.00 | 0 | 0.85 | 1.9 |
| t1a | 2 | full | ✗ | 0.43 | 0.0 | 0.66 | n/a | n/a (94 spurious) | n/a (125 spurious) | n/a | n/a (16 spurious) | n/a (84 spurious) | n/a (223 spurious) | 0.57 | 0.14 | 1.00 | -1 | 0.82 | 17.2 |
| t1a | 2 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.0 |
| t1a | 2 | oracle-stems | ✗ | 0.64 | 0.0 | 0.53 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 0.56 | 0.18 | 1.00 | -1 | 0.90 | 2.2 |
| t1a | 2 | oracle-tempo | ✗ | 0.64 | 0.0 | 0.53 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 1.00 | 0.25 | 1.00 | 0 | 0.90 | 1.8 |
| t1a | 3 | full | ✓ | 0.86 | 1.0 | 0.22 | n/a | n/a (96 spurious) | n/a (250 spurious) | n/a (42 spurious) | n/a (1 spurious) | n/a (15 spurious) | n/a (258 spurious) | 0.98 | 0.18 | 1.00 | 0 | 0.85 | 14.1 |
| t1a | 3 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 1.00 | 1.00 | 1.00 | 0 | 0.98 | 0.0 |
| t1a | 3 | oracle-stems | ✓ | 0.87 | 0.0 | 0.20 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 1.00 | 0.19 | 1.00 | 0 | 0.85 | 1.7 |
| t1a | 3 | oracle-tempo | ✓ | 0.87 | 0.0 | 0.20 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 1.00 | 0.19 | 1.00 | 0 | 0.85 | 1.5 |
| t1a | 4 | full | ✗ | 0.32 | 3.0 | 0.64 | n/a | n/a (77 spurious) | n/a (225 spurious) | n/a | n/a (19 spurious) | n/a (64 spurious) | n/a (309 spurious) | 0.65 | 0.08 | 1.00 | 0 | 0.51 | 15.7 |
| t1a | 4 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.0 |
| t1a | 4 | oracle-stems | ✓ | 0.88 | 0.0 | 0.21 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 1.00 | 0.45 | 1.00 | 0 | 0.81 | 1.9 |
| t1a | 4 | oracle-tempo | ✓ | 0.88 | 0.0 | 0.21 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 1.00 | 0.45 | 1.00 | 0 | 0.81 | 1.7 |
| t1a | 5 | full | ✗ | 0.18 | 3.0 | 0.69 | n/a | n/a (80 spurious) | n/a (340 spurious) | n/a | n/a (32 spurious) | n/a (34 spurious) | n/a (453 spurious) | 0.33 | 0.03 | 1.00 | -3 | 1.00 | 14.9 |
| t1a | 5 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.0 |
| t1a | 5 | oracle-stems | ✓ | 0.93 | 0.0 | 0.13 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 0.51 | 0.12 | 1.00 | -3 | 0.95 | 1.7 |
| t1a | 5 | oracle-tempo | ✓ | 0.93 | 0.0 | 0.13 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | 1.00 | 0.52 | 1.00 | 0 | 0.96 | 1.5 |
| t1b | 1 | full | ✗ | 0.60 | 3.0 | 0.33 | n/a | n/a (91 spurious) | 0.23 | 0.95 | 0.25 | 1.00 | n/a (411 spurious) | 0.93 | 0.19 | 1.00 | 0 | 0.93 | 18.3 |
| t1b | 1 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | n/a | n/a | 1.00 | 1.00 | 1.00 | 1.00 | n/a | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.2 |
| t1b | 1 | oracle-stems | ✓ | 0.87 | 0.0 | 0.20 | n/a | n/a | 0.50 | 1.00 | 1.00 | 1.00 | n/a | 0.99 | 0.23 | 1.00 | 0 | 0.86 | 4.2 |
| t1b | 1 | oracle-tempo | ✓ | 0.87 | 0.0 | 0.20 | n/a | n/a | 0.50 | 1.00 | 1.00 | 1.00 | n/a | 0.99 | 0.23 | 1.00 | 0 | 0.86 | 3.9 |
| t1b | 2 | full | ✗ | 0.51 | 2.0 | 0.57 | n/a | n/a (77 spurious) | 0.38 | 1.00 | 0.54 | 0.99 | n/a (301 spurious) | 0.84 | 0.24 | 1.00 | 0 | 0.95 | 16.6 |
| t1b | 2 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | n/a | n/a | 1.00 | 1.00 | 1.00 | 1.00 | n/a | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.2 |
| t1b | 2 | oracle-stems | ✗ | 0.67 | 0.0 | 0.49 | n/a | n/a | 0.50 | 1.00 | 1.00 | 1.00 | n/a | 1.00 | 0.22 | 1.00 | 0 | 0.79 | 3.8 |
| t1b | 2 | oracle-tempo | ✗ | 0.67 | 0.0 | 0.49 | n/a | n/a | 0.50 | 1.00 | 1.00 | 1.00 | n/a | 1.00 | 0.22 | 1.00 | 0 | 0.79 | 3.7 |
| t1b | 3 | full | ✗ | 0.73 | 7.0 | 0.28 | n/a | n/a (80 spurious) | 0.22 | 0.96 | 0.16 | 1.00 | n/a (307 spurious) | 0.95 | 0.28 | 1.00 | 0 | 0.90 | 14.2 |
| t1b | 3 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | n/a | n/a | 1.00 | 1.00 | 1.00 | 1.00 | n/a | 1.00 | 1.00 | 1.00 | 0 | 0.98 | 0.2 |
| t1b | 3 | oracle-stems | ✓ | 0.87 | 0.0 | 0.20 | n/a | n/a | 0.45 | 1.00 | 1.00 | 1.00 | n/a | 1.00 | 0.19 | 1.00 | 0 | 0.86 | 3.3 |
| t1b | 3 | oracle-tempo | ✓ | 0.87 | 0.0 | 0.20 | n/a | n/a | 0.45 | 1.00 | 1.00 | 1.00 | n/a | 1.00 | 0.19 | 1.00 | 0 | 0.86 | 3.1 |
| t1b | 4 | full | ✗ | 0.18 | 1.0 | 0.69 | n/a | n/a (60 spurious) | 0.37 | 0.98 | 0.76 | 1.00 | n/a (373 spurious) | 0.52 | 0.20 | 1.00 | 0 | 0.57 | 15.8 |
| t1b | 4 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | n/a | n/a | 1.00 | 1.00 | 1.00 | 1.00 | n/a | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.2 |
| t1b | 4 | oracle-stems | ✓ | 0.87 | 0.0 | 0.22 | n/a | n/a | 0.52 | 1.00 | 1.00 | 1.00 | n/a | 1.00 | 0.43 | 1.00 | 0 | 0.71 | 3.5 |
| t1b | 4 | oracle-tempo | ✓ | 0.87 | 0.0 | 0.22 | n/a | n/a | 0.52 | 1.00 | 1.00 | 1.00 | n/a | 1.00 | 0.43 | 1.00 | 0 | 0.71 | 3.3 |
| t1b | 5 | full | ✗ | 0.31 | 3.0 | 0.53 | n/a | n/a (63 spurious) | 0.38 | 1.00 | 0.49 | 1.00 | n/a (382 spurious) | 0.69 | 0.11 | 1.00 | 0 | 0.99 | 14.0 |
| t1b | 5 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | n/a | n/a | 1.00 | 1.00 | 1.00 | 1.00 | n/a | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.2 |
| t1b | 5 | oracle-stems | ✓ | 0.92 | 0.0 | 0.15 | n/a | n/a | 0.46 | 1.00 | 1.00 | 1.00 | n/a | 1.00 | 0.51 | 1.00 | 0 | 0.96 | 3.4 |
| t1b | 5 | oracle-tempo | ✓ | 0.92 | 0.0 | 0.15 | n/a | n/a | 0.46 | 1.00 | 1.00 | 1.00 | n/a | 1.00 | 0.51 | 1.00 | 0 | 0.96 | 3.2 |
| t1c | 1 | full | ✗ | 0.57 | 3.0 | 0.40 | n/a | 0.21 | 0.18 | 0.92 | 0.37 | 1.00 | 0.11 | 0.91 | 0.19 | 1.00 | -2 | 0.94 | 19.4 |
| t1c | 1 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | n/a | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.2 |
| t1c | 1 | oracle-stems | ✓ | 0.87 | 0.0 | 0.20 | n/a | 0.87 | 0.52 | 1.00 | 1.00 | 1.00 | 0.18 | 0.99 | 0.23 | 1.00 | -2 | 0.88 | 8.2 |
| t1c | 1 | oracle-tempo | ✓ | 0.87 | 0.0 | 0.20 | n/a | 0.87 | 0.52 | 1.00 | 1.00 | 1.00 | 0.18 | 0.99 | 0.23 | 1.00 | 0 | 0.87 | 8.0 |
| t1c | 2 | full | ✗ | 0.39 | 11.0 | 0.71 | n/a | 0.24 | 0.32 | 1.00 | 0.86 | 0.99 | 0.15 | 0.81 | 0.29 | 1.00 | 0 | 0.94 | 16.9 |
| t1c | 2 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | n/a | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.2 |
| t1c | 2 | oracle-stems | ✗ | 0.67 | 0.0 | 0.49 | n/a | 0.86 | 0.47 | 1.00 | 1.00 | 1.00 | 0.39 | 1.00 | 0.22 | 1.00 | 0 | 0.79 | 7.4 |
| t1c | 2 | oracle-tempo | ✗ | 0.67 | 0.0 | 0.49 | n/a | 0.86 | 0.47 | 1.00 | 1.00 | 1.00 | 0.39 | 1.00 | 0.22 | 1.00 | 0 | 0.79 | 7.3 |
| t1c | 3 | full | ✗ | 0.54 | 7.0 | 0.35 | n/a | 0.05 | 0.18 | 0.99 | 0.53 | 1.00 | 0.09 | 0.94 | 0.22 | 1.00 | 0 | 0.90 | 13.8 |
| t1c | 3 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | n/a | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0 | 0.98 | 0.2 |
| t1c | 3 | oracle-stems | ✓ | 0.87 | 0.0 | 0.19 | n/a | 0.96 | 0.46 | 1.00 | 1.00 | 1.00 | 0.09 | 1.00 | 0.19 | 1.00 | 0 | 0.86 | 6.3 |
| t1c | 3 | oracle-tempo | ✓ | 0.87 | 0.0 | 0.19 | n/a | 0.96 | 0.46 | 1.00 | 1.00 | 1.00 | 0.09 | 1.00 | 0.19 | 1.00 | 0 | 0.86 | 6.1 |
| t1c | 4 | full | ✗ | 0.16 | 4.0 | 0.78 | n/a | 0.09 | 0.36 | 1.00 | 0.92 | 0.99 | 0.18 | 0.49 | 0.18 | 1.00 | 0 | 0.97 | 14.9 |
| t1c | 4 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | n/a | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.2 |
| t1c | 4 | oracle-stems | ✓ | 0.87 | 0.0 | 0.23 | n/a | 0.86 | 0.52 | 1.00 | 1.00 | 1.00 | 0.26 | 1.00 | 0.43 | 1.00 | 0 | 0.70 | 6.8 |
| t1c | 4 | oracle-tempo | ✓ | 0.87 | 0.0 | 0.23 | n/a | 0.86 | 0.52 | 1.00 | 1.00 | 1.00 | 0.26 | 1.00 | 0.43 | 1.00 | 0 | 0.70 | 6.5 |
| t1c | 5 | full | ✗ | 0.26 | 1.0 | 0.58 | n/a | 0.24 | 0.43 | 1.00 | 0.75 | 0.99 | 0.06 | 0.76 | 0.00 | 1.00 | 0 | 0.94 | 13.6 |
| t1c | 5 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | n/a | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.2 |
| t1c | 5 | oracle-stems | ✓ | 0.92 | 0.0 | 0.15 | n/a | 0.84 | 0.46 | 1.00 | 1.00 | 1.00 | 0.06 | 1.00 | 0.51 | 1.00 | 0 | 0.96 | 6.4 |
| t1c | 5 | oracle-tempo | ✓ | 0.92 | 0.0 | 0.15 | n/a | 0.84 | 0.46 | 1.00 | 1.00 | 1.00 | 0.06 | 1.00 | 0.51 | 1.00 | 0 | 0.96 | 6.2 |
| t1d | 1 | full | ✗ | 0.72 | 4.0 | 0.24 | 0.71 | 0.38 | 0.45 | 1.00 | 0.41 | 1.00 | 0.07 | 0.93 | 0.25 | 1.00 | 0 | 0.82 | 17.6 |
| t1d | 1 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.2 |
| t1d | 1 | oracle-stems | ✗ | 0.79 | 3.0 | 0.33 | 0.79 | 0.88 | 0.53 | 1.00 | 1.00 | 1.00 | 0.65 | 0.99 | 0.19 | 1.00 | 0 | 0.83 | 10.1 |
| t1d | 1 | oracle-tempo | ✗ | 0.79 | 3.0 | 0.33 | 0.79 | 0.88 | 0.53 | 1.00 | 1.00 | 1.00 | 0.65 | 0.99 | 0.19 | 1.00 | 0 | 0.83 | 9.8 |
| t1d | 2 | full | ✗ | 0.54 | 2.0 | 0.43 | 0.48 | 0.52 | 0.34 | 1.00 | 0.50 | 0.99 | 0.15 | 0.80 | 0.19 | 1.00 | 0 | 0.97 | 16.8 |
| t1d | 2 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.2 |
| t1d | 2 | oracle-stems | ✓ | 0.92 | 0.0 | 0.15 | 0.92 | 0.88 | 0.47 | 1.00 | 1.00 | 1.00 | 0.66 | 1.00 | 0.39 | 1.00 | 0 | 0.87 | 9.2 |
| t1d | 2 | oracle-tempo | ✓ | 0.92 | 0.0 | 0.15 | 0.92 | 0.88 | 0.47 | 1.00 | 1.00 | 1.00 | 0.66 | 1.00 | 0.39 | 1.00 | 0 | 0.87 | 9.0 |
| t1d | 3 | full | ✗ | 0.71 | 12.0 | 0.33 | 0.63 | 0.22 | 0.40 | 0.98 | 0.07 | 0.89 | 0.11 | 0.89 | 0.16 | 1.00 | 0 | 0.85 | 13.7 |
| t1d | 3 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0 | 0.95 | 0.2 |
| t1d | 3 | oracle-stems | ✗ | 0.78 | 14.0 | 0.34 | 0.77 | 0.96 | 0.47 | 1.00 | 1.00 | 1.00 | 0.06 | 0.98 | 0.14 | 1.00 | 0 | 0.83 | 7.7 |
| t1d | 3 | oracle-tempo | ✗ | 0.78 | 14.0 | 0.34 | 0.77 | 0.96 | 0.47 | 1.00 | 1.00 | 1.00 | 0.06 | 0.98 | 0.14 | 1.00 | 0 | 0.83 | 7.4 |
| t1d | 4 | full | ✗ | 0.43 | 4.0 | 0.49 | 0.35 | 0.13 | 0.37 | 0.98 | 0.58 | 0.99 | 0.10 | 0.73 | 0.24 | 1.00 | 0 | 0.93 | 15.2 |
| t1d | 4 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.2 |
| t1d | 4 | oracle-stems | ✓ | 0.92 | 0.0 | 0.14 | 0.92 | 0.85 | 0.52 | 1.00 | 1.00 | 1.00 | 0.95 | 1.00 | 0.43 | 1.00 | 0 | 0.74 | 8.2 |
| t1d | 4 | oracle-tempo | ✓ | 0.92 | 0.0 | 0.14 | 0.92 | 0.85 | 0.52 | 1.00 | 1.00 | 1.00 | 0.95 | 1.00 | 0.43 | 1.00 | 0 | 0.74 | 8.1 |
| t1d | 5 | full | ✗ | 0.60 | 0.0 | 0.31 | 0.50 | 0.29 | 0.34 | 1.00 | 0.11 | 0.96 | 0.10 | 0.80 | 0.19 | 1.00 | 0 | 0.94 | 13.9 |
| t1d | 5 | oracle-notes | ✓ | 1.00 | 0.0 | 0.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0 | 1.00 | 0.2 |
| t1d | 5 | oracle-stems | ✓ | 0.92 | 0.0 | 0.15 | 0.92 | 0.84 | 0.46 | 1.00 | 1.00 | 1.00 | 0.13 | 1.00 | 0.48 | 1.00 | 0 | 0.89 | 7.7 |
| t1d | 5 | oracle-tempo | ✓ | 0.92 | 0.0 | 0.15 | 0.92 | 0.84 | 0.46 | 1.00 | 1.00 | 1.00 | 0.13 | 1.00 | 0.48 | 1.00 | 0 | 0.89 | 7.6 |