9.2 KiB
9.2 KiB
STSheets benchmark report: exp-repeat-2026-10-31
- Run id: exp-repeat-2026-10-31
- Date: 2026-10-30T20:14:56.152Z
- Machine: AMD Ryzen 9 9950X 16-Core Processor x16, linux x64, bun 1.3.14
- Models (sha256): htdemucs_6s.onnx 35cb61d4d35173ff1d16df7393f41f05651ffdfe13d3c10e06abc26708bd6933; nmp.onnx 2c3c1d144bfa61ad236e92e169c13535c880469a12a047d4e73451f2c059a0ec
- SoundFont: alphaTab bundled sonivox.sf2 (Apache-2.0)
- Modes: full (stems=false, tempo=false, notes=false)
Caveats: vocals are rendered with a GM "Voice Oohs" patch, not real singing, so separation/transcription numbers for vocals are indicative only. Drums use a heuristic baseline transcriber. The SONiVOX guitar is partly routed to Demucs' piano stem (known synthetic-timbre gap). Pitch is pass/fail (song passes at merged-guitar pitch F1 >= 0.85); rhythm is secondary; fingering (string agreement) is informational. Cells are median (min–max).
Tier t1a
| Mode | Songs | Guitar merged F1 | Guitar octave errors | Spurious rate | Part-split F1 (t1d) | Vocals F1 | Bass F1 | Drums kick F1 | Drums snare F1 | Drums hihat F1 | Keys F1 | Rhythm onBeat | Rhythm durationOk | Tempo factor | Beat offset | String agreement | GP-file guitar F1 | Bar count match | Time sig match | Techniques |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| full | 5 | 0.89 (0.63–0.94) ✓ 4/5 | 0.0 (0.0–0.0) | 0.15 (0.12–0.53) | n/a | n/a (424 spurious) | n/a (1260 spurious) | n/a (80 spurious) | n/a (70 spurious) | n/a (200 spurious) | n/a (1592 spurious) | 0.98 (0.52–1.00) | 0.92 (0.08–0.95) | 1.00 (1.00–1.00) | 0 (-3–0) | 0.88 (0.76–0.95) | 0.90 (0.69–0.93) | 0 (0–1) | 1 (1–1) | n/a (tier 1 has no technique marks) |
Tier t1a: PASS (median merged-guitar F1 0.89 in full mode, threshold 0.85)
Tier t1b
| Mode | Songs | Guitar merged F1 | Guitar octave errors | Spurious rate | Part-split F1 (t1d) | Vocals F1 | Bass F1 | Drums kick F1 | Drums snare F1 | Drums hihat F1 | Keys F1 | Rhythm onBeat | Rhythm durationOk | Tempo factor | Beat offset | String agreement | GP-file guitar F1 | Bar count match | Time sig match | Techniques |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| full | 5 | 0.84 (0.75–0.95) ✗ 2/5 | 0.0 (0.0–2.0) | 0.18 (0.09–0.39) | n/a | n/a (371 spurious) | 0.37 (0.22–0.38) | 0.98 (0.95–1.00) | 0.49 (0.16–0.76) | 1.00 (0.99–1.00) | n/a (1774 spurious) | 1.00 (0.99–1.00) | 0.89 (0.78–0.96) | 1.00 (1.00–1.00) | 0 (0–0) | 0.95 (0.80–1.00) | 0.84 (0.77–0.96) | 0 (0–0) | 1 (1–1) | n/a (tier 1 has no technique marks) |
Tier t1b: FAIL (median merged-guitar F1 0.84 in full mode, threshold 0.85)
Tier t1c
| Mode | Songs | Guitar merged F1 | Guitar octave errors | Spurious rate | Part-split F1 (t1d) | Vocals F1 | Bass F1 | Drums kick F1 | Drums snare F1 | Drums hihat F1 | Keys F1 | Rhythm onBeat | Rhythm durationOk | Tempo factor | Beat offset | String agreement | GP-file guitar F1 | Bar count match | Time sig match | Techniques |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| full | 5 | 0.75 (0.56–0.80) ✗ 0/5 | 1.0 (0.0–8.0) | 0.34 (0.30–0.59) | n/a | 0.21 (0.05–0.24) | 0.32 (0.18–0.43) | 1.00 (0.92–1.00) | 0.75 (0.37–0.92) | 0.99 (0.99–1.00) | 0.11 (0.06–0.18) | 0.98 (0.96–1.00) | 0.75 (0.60–0.80) | 1.00 (1.00–1.00) | 0 (-2–0) | 0.64 (0.34–0.94) | 0.76 (0.60–0.79) | 0 (0–1) | 1 (1–1) | n/a (tier 1 has no technique marks) |
Tier t1c: FAIL (median merged-guitar F1 0.75 in full mode, threshold 0.85)
Tier t1d
| Mode | Songs | Guitar merged F1 | Guitar octave errors | Spurious rate | Part-split F1 (t1d) | Vocals F1 | Bass F1 | Drums kick F1 | Drums snare F1 | Drums hihat F1 | Keys F1 | Rhythm onBeat | Rhythm durationOk | Tempo factor | Beat offset | String agreement | GP-file guitar F1 | Bar count match | Time sig match | Techniques |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| full | 5 | 0.74 (0.65–0.78) ✗ 0/5 | 4.0 (2.0–13.0) | 0.23 (0.22–0.33) | 0.68 (0.57–0.72) | 0.29 (0.13–0.52) | 0.37 (0.34–0.45) | 1.00 (0.98–1.00) | 0.41 (0.07–0.58) | 0.99 (0.89–1.00) | 0.10 (0.07–0.15) | 0.94 (0.85–0.97) | 0.66 (0.48–0.78) | 1.00 (1.00–1.00) | 0 (0–0) | 0.82 (0.39–0.97) | 0.75 (0.69–0.80) | 0 (0–0) | 1 (1–1) | n/a (tier 1 has no technique marks) |
Tier t1d: FAIL (median merged-guitar F1 0.74 in full mode, threshold 0.85)
Per-song appendix
| Tier | Seed | Mode | Guitar pass | Guitar merged F1 | Guitar octave errors | Spurious rate | Part-split F1 (t1d) | Vocals F1 | Bass F1 | Drums kick F1 | Drums snare F1 | Drums hihat F1 | Keys F1 | Rhythm onBeat | Rhythm durationOk | Tempo factor | Beat offset | String agreement | GP-file guitar F1 | Bar count match | Time sig match | Techniques | Seconds |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| t1a | 1 | full | ✓ | 0.89 | 0.0 | 0.15 | n/a | n/a (77 spurious) | n/a (320 spurious) | n/a (38 spurious) | n/a (2 spurious) | n/a (3 spurious) | n/a (349 spurious) | 1.00 | 0.94 | 1.00 | 0 | 0.88 | 0.87 | 0 | 1 | n/a (tier 1 has no technique marks) | 24.2 |
| t1a | 2 | full | ✗ | 0.63 | 0.0 | 0.53 | n/a | n/a (94 spurious) | n/a (125 spurious) | n/a | n/a (16 spurious) | n/a (84 spurious) | n/a (223 spurious) | 0.56 | 0.24 | 1.00 | -1 | 0.76 | 0.69 | 0 | 1 | n/a (tier 1 has no technique marks) | 23.5 |
| t1a | 3 | full | ✓ | 0.87 | 0.0 | 0.20 | n/a | n/a (96 spurious) | n/a (250 spurious) | n/a (42 spurious) | n/a (1 spurious) | n/a (15 spurious) | n/a (258 spurious) | 0.98 | 0.95 | 1.00 | 0 | 0.85 | 0.90 | 0 | 1 | n/a (tier 1 has no technique marks) | 20.7 |
| t1a | 4 | full | ✓ | 0.92 | 0.0 | 0.15 | n/a | n/a (77 spurious) | n/a (225 spurious) | n/a | n/a (19 spurious) | n/a (64 spurious) | n/a (309 spurious) | 1.00 | 0.92 | 1.00 | 0 | 0.94 | 0.93 | 0 | 1 | n/a (tier 1 has no technique marks) | 21.5 |
| t1a | 5 | full | ✓ | 0.94 | 0.0 | 0.12 | n/a | n/a (80 spurious) | n/a (340 spurious) | n/a | n/a (32 spurious) | n/a (34 spurious) | n/a (453 spurious) | 0.52 | 0.08 | 1.00 | -3 | 0.95 | 0.91 | 1 | 1 | n/a (tier 1 has no technique marks) | 19.5 |
| t1b | 1 | full | ✗ | 0.84 | 0.0 | 0.18 | n/a | n/a (91 spurious) | 0.23 | 0.95 | 0.25 | 1.00 | n/a (411 spurious) | 1.00 | 0.78 | 1.00 | 0 | 0.99 | 0.84 | 0 | 1 | n/a (tier 1 has no technique marks) | 25.3 |
| t1b | 2 | full | ✗ | 0.75 | 2.0 | 0.39 | n/a | n/a (77 spurious) | 0.38 | 1.00 | 0.54 | 0.99 | n/a (301 spurious) | 0.99 | 0.78 | 1.00 | 0 | 0.80 | 0.77 | 0 | 1 | n/a (tier 1 has no technique marks) | 26.3 |
| t1b | 3 | full | ✗ | 0.82 | 0.0 | 0.19 | n/a | n/a (80 spurious) | 0.22 | 0.96 | 0.16 | 1.00 | n/a (307 spurious) | 0.99 | 0.91 | 1.00 | 0 | 0.93 | 0.83 | 0 | 1 | n/a (tier 1 has no technique marks) | 17.2 |
| t1b | 4 | full | ✓ | 0.90 | 1.0 | 0.15 | n/a | n/a (60 spurious) | 0.37 | 0.98 | 0.76 | 1.00 | n/a (373 spurious) | 1.00 | 0.89 | 1.00 | 0 | 0.95 | 0.91 | 0 | 1 | n/a (tier 1 has no technique marks) | 43.1 |
| t1b | 5 | full | ✓ | 0.95 | 0.0 | 0.09 | n/a | n/a (63 spurious) | 0.38 | 1.00 | 0.49 | 1.00 | n/a (382 spurious) | 1.00 | 0.96 | 1.00 | 0 | 1.00 | 0.96 | 0 | 1 | n/a (tier 1 has no technique marks) | 31.0 |
| t1c | 1 | full | ✗ | 0.75 | 0.0 | 0.34 | n/a | 0.21 | 0.18 | 0.92 | 0.37 | 1.00 | 0.11 | 0.98 | 0.63 | 1.00 | -2 | 0.94 | 0.74 | 1 | 1 | n/a (tier 1 has no technique marks) | 41.9 |
| t1c | 2 | full | ✗ | 0.56 | 8.0 | 0.59 | n/a | 0.24 | 0.32 | 1.00 | 0.86 | 0.99 | 0.15 | 0.96 | 0.60 | 1.00 | 0 | 0.63 | 0.60 | 0 | 1 | n/a (tier 1 has no technique marks) | 47.0 |
| t1c | 3 | full | ✗ | 0.75 | 1.0 | 0.35 | n/a | 0.05 | 0.18 | 0.99 | 0.53 | 1.00 | 0.09 | 0.96 | 0.80 | 1.00 | 0 | 0.74 | 0.76 | 0 | 1 | n/a (tier 1 has no technique marks) | 23.0 |
| t1c | 4 | full | ✗ | 0.79 | 1.0 | 0.33 | n/a | 0.09 | 0.36 | 1.00 | 0.92 | 0.99 | 0.18 | 1.00 | 0.76 | 1.00 | 0 | 0.34 | 0.79 | 0 | 1 | n/a (tier 1 has no technique marks) | 16.1 |
| t1c | 5 | full | ✗ | 0.80 | 0.0 | 0.30 | n/a | 0.24 | 0.43 | 1.00 | 0.75 | 0.99 | 0.06 | 0.98 | 0.75 | 1.00 | 0 | 0.64 | 0.78 | 0 | 1 | n/a (tier 1 has no technique marks) | 20.0 |
| t1d | 1 | full | ✗ | 0.74 | 2.0 | 0.22 | 0.71 | 0.38 | 0.45 | 1.00 | 0.41 | 1.00 | 0.07 | 0.94 | 0.78 | 1.00 | 0 | 0.82 | 0.74 | 0 | 1 | n/a (tier 1 has no technique marks) | 25.3 |
| t1d | 2 | full | ✗ | 0.78 | 7.0 | 0.23 | 0.72 | 0.52 | 0.34 | 1.00 | 0.50 | 0.99 | 0.15 | 0.95 | 0.77 | 1.00 | 0 | 0.80 | 0.80 | 0 | 1 | n/a (tier 1 has no technique marks) | 29.2 |
| t1d | 3 | full | ✗ | 0.69 | 13.0 | 0.33 | 0.61 | 0.22 | 0.40 | 0.98 | 0.07 | 0.89 | 0.11 | 0.89 | 0.58 | 1.00 | 0 | 0.87 | 0.75 | 0 | 1 | n/a (tier 1 has no technique marks) | 20.1 |
| t1d | 4 | full | ✗ | 0.77 | 4.0 | 0.22 | 0.68 | 0.13 | 0.37 | 0.98 | 0.58 | 0.99 | 0.10 | 0.97 | 0.66 | 1.00 | 0 | 0.39 | 0.78 | 0 | 1 | n/a (tier 1 has no technique marks) | 21.7 |
| t1d | 5 | full | ✗ | 0.65 | 4.0 | 0.28 | 0.57 | 0.29 | 0.34 | 1.00 | 0.11 | 0.96 | 0.10 | 0.85 | 0.48 | 1.00 | 0 | 0.97 | 0.69 | 0 | 1 | n/a (tier 1 has no technique marks) | 20.1 |