Add research reports and synthetic benchmark design

Co-Authored-By: Claude Opus 5.5 <[email protected]>

docs/superpowers/specs/2026-10-30-synthetic-benchmark-design.md

@@ -0,0 +1,184 @@
+# Synthetic Benchmark Harness — Design
+
+Date: 2026-10-30
+Status: awaiting user approval of this document
+
+## Intent
+
+STSheets will turn songs into sheet music and Guitar Pro tabs. Before we can improve
+transcription accuracy we need a way to measure it with a perfectly known answer.
+The harness generates original songs as Guitar Pro files, renders them to audio,
+runs our transcription pipeline on that audio, and scores the output against the
+file it came from. Because we write the source file, ground truth is exact: no
+alignment problems, no disputes about whether a human tab matches its recording.
+
+It is also how we work without user data: the product keeps no user data, so
+the improvement loop runs on data we generate ourselves.
+
+Background research: `reports/Music transcription tooling survey.md` and
+`reports/Transcription accuracy improvement ideas.md`.
+
+## Goals
+
+1. **Difficulty ladder.** Generated songs are organised into tiers. Each step adds
+   one kind of difficulty so a drop in score can be attributed.
+   - **Tier 1 (build now):** simple, slow, floaty pop. Style references:
+     *Swimming* (Lunar Vacation), *Last Night on Earth* (Green Day),
+     *Afire Love* (Ed Sheeran). Standard tuning, 4/4, slow-to-moderate tempo,
+     clean or lightly driven guitars with reverb/chorus, open chords, arpeggios,
+     simple melodic lines; bass on roots; simple drums; keys/pads; a vocal melody.
+   - **Later tiers (design for, don't build):** simpler rock/metal in the vein of
+     Skillet and Anarbor (distortion, power chords, palm muting, faster tempos,
+     possibly lower tunings), then up to *Moon Walker*, *Patent Pending*,
+     *Burn the Ballroom* (faster, denser, more technique).
+   - Benchmark tiers use **at most two guitars**. More guitars are allowed in the
+     product later but are not part of this ladder.
+2. **Original material only.** Songs are generated in the *style* of the
+   references, never recreations of them.
+3. **Instrument priority** (drives what we tune first and how reports are ordered):
+   1. Guitar
+   2. Vocals
+   3. Bass, drums, keys (equal)
+   All five are present and scored from the start; only the focus differs.
+4. **Scoring priority** (the user's words: wrong notes are "the hardest no"):
+   1. **Pitch** — correct notes at roughly the right time. Primary pass/fail.
+   2. **Rhythm** — after quantisation, notes on the right beat with the right
+      duration so the tab reads correctly. Secondary.
+   3. **Fingering (string/fret) and techniques (slides, bends, hammer-ons…)** —
+      measured and reported, but *not* part of pass/fail; users can fix these.
+5. **Repeatable.** Same seed → same song, same audio, same scores.
+6. **Browser-portable where we own the code.** The pipeline should be structured
+   so it can later run in the browser (WASM/WebGPU).
+
+## Non-goals
+
+- The website UI, upload flow, YouTube ingestion.
+- Building the metal tiers now (but nothing should prevent adding them).
+- Training or fine-tuning models (later phase; this harness is the prerequisite).
+- Matching the reference songs themselves.
+
+## Anti-goals (would be failure even if it "works")
+
+- Scores that look good because the pipeline was handed information it won't have
+  in real use (tempo, stems, track count) without that being explicitly labelled
+  as an oracle run.
+- Fingering/technique differences dragging down the headline score.
+- Anything that stores or depends on user data.
+- Shipping components whose licence blocks commercial use without flagging it.
+
+## Constraints
+
+- **Language/runtime:** Bun + TypeScript for everything we own: generator, Guitar
+  Pro writing, rendering/mixing, pipeline orchestration, scoring, reports.
+- **Models** run via ONNX Runtime (`onnxruntime-node` in the harness; the same
+  model files can later run under `onnxruntime-web`).
+- **Python** only as an optional *comparison* track for research models that
+  exist only in Python (e.g. Noise2Fret). Never on the product path.
+- **Hardware:** this machine, CPU only (16 cores, ~17 GB RAM, no GPU).
+- **Licences:** anything on the product path must allow commercial use (e.g.
+  Demucs MIT, Basic Pitch Apache-2.0, alphaTab MPL-2.0). Research-only things
+  may be used in the comparison track if clearly labelled. SoundFonts used for
+  rendering must have a licence that permits this use.
+- **Repo:** `/home/user/STSheets`, git, remote
+  `https://gitea.lucaswinther.info/lucasw89/STSheets`. Generated audio is
+  not committed (regenerable from seeds); source, specs, and result summaries are.
+
+## How it works (decided parts)
+
+```
+seed + tier spec
+   → song generator (TS)            → ground-truth score model
+   → Guitar Pro file (alphaTab GP7 exporter)
+   → render each track (alphaTab synth + SoundFont) → per-track stems
+   → effects + pan + mix (our TS DSP) → mix.wav / mix.mp3
+   → pipeline under test (decode → separate → transcribe per stem → tempo/beat
+     → quantise → tab assignment → GP file)
+   → scorer (output vs ground truth) → per-song JSON → tier report (markdown)
+```
+
+### Generator
+- Produces an original arrangement from a seed and a tier spec: tempo, key,
+  chord progression, song sections (with repeats), and per-instrument parts.
+- Guitar parts in tier 1: strummed open chords, fingerpicked/arpeggiated chords,
+  simple single-note melodic lines. Two-guitar songs give the guitars distinct
+  roles (e.g. strum + arpeggio) and pan them apart.
+- Writes the song as a valid Guitar Pro file (alphaTab model → GP7 export) with
+  tracks, tunings, string/fret, and technique marks. This file is the ground truth.
+- Tier 1 should itself ramp: start with a single clean guitar, then add the band,
+  then the second guitar. Exact sub-steps are the builder's call.
+
+### Renderer
+- Renders each track separately (so isolated stems exist for oracle runs), then
+  applies light effects (drive, chorus, reverb), panning, and mixes.
+- Vocals have no real singing synthesis available; the vocal melody is rendered
+  with a voice-like SoundFont patch. **Known limitation:** separation models are
+  trained on real voices and may not treat this as "vocals". Report vocal results
+  with that caveat.
+
+### Pipeline under test (baseline)
+- Separation: Demucs (ONNX). Transcription: Basic Pitch (ONNX/TS) for pitched
+  stems. Drums: a licence-clean option if one is runnable, otherwise report drums
+  as "not yet supported" rather than faking it. Tempo/beat: must be estimated by
+  the pipeline itself. Tab assignment: our own playability-based assigner.
+  Output: Guitar Pro file via alphaTab.
+- Pipeline stages are swappable so later experiments (from the ideas report) can
+  replace one stage at a time.
+
+### Oracle modes (for error attribution)
+The harness can feed the pipeline ground-truth information to find which stage
+causes errors. Every result is labelled with which oracles were on:
+- **Oracle stems:** isolated rendered stems instead of separated ones.
+- **Oracle tempo/beats:** true tempo map instead of estimated.
+- **Oracle notes:** ground-truth notes into the tab assigner (tests fingering only).
+
+### Scorer
+- **Track matching:** output tracks are matched to ground-truth tracks by
+  instrument. The pipeline is not told how many guitars there are.
+  *(User's decision)* For pass/fail, all guitar ground-truth notes are pooled and
+  compared with all guitar output notes (merged guitar score). Whether the
+  pipeline split the guitars into the right separate parts is a separate
+  "part-split" metric: when it outputs several guitar tracks they are matched
+  one-to-one by best note agreement (Hungarian), and unmatched tracks count
+  against this metric only. Part-split becomes a pass criterion in a later tier,
+  not in tier 1.
+- *(Coordinator's call)* Separation uses a model with a dedicated guitar stem
+  (e.g. 6-stem Demucs) so keys are not pooled into "guitar"; keys bleeding into
+  the guitar stem is a known risk that the spurious-note rate will expose.
+- **Pitch (primary):** note-level precision/recall/F1 on onset + pitch (onset
+  tolerance ±50 ms, offsets ignored), per track. String choice is ignored here:
+  same pitch on a different string counts as correct. Also report octave-error
+  rate and spurious-note rate separately.
+  *(Coordinator's call)* The primary pitch score is computed on the pipeline's
+  timed notes in seconds, **before** quantisation, so a tempo or grid error
+  cannot fail a song on pitch. Every pipeline run therefore emits a timed-notes
+  file alongside the Guitar Pro output. Pitch F1 of the final GP file (mapped
+  back to seconds via its own tempo map) is reported as a secondary number.
+- **Rhythm (secondary):** on the quantised output, the share of correctly-pitched
+  notes that sit on the correct beat position and have the correct notated
+  duration; bar count / time signature agreement.
+  *(Coordinator's call)* Before scoring rhythm, the scorer aligns the output grid
+  to the true grid, allowing a tempo factor of ½×, 1× or 2× and a whole-beat bar
+  offset (incl. pickup bars). The rhythm score is computed after that alignment;
+  the detected tempo factor and bar offset are reported separately as their own
+  "grid errors".
+- **Informational:** string/fret agreement on matched notes, technique
+  precision/recall, drum per-class F1 (GM drum map), vocal note F1.
+- **Reports:** per-song JSON plus a tier-level markdown summary, guitar first,
+  then vocals, then bass/drums/keys.
+
+### Pass criteria (builder's/coordinator's call, adjustable)
+Initial targets: a tier-1 song "passes" when the guitar pitch F1 ≥ 0.85; a tier
+passes when its median song passes. These numbers are a starting point to be
+revised once baseline results exist.
+
+## Left to the builder
+
+- Repo layout, module boundaries, CLI shape.
+- Which SoundFont(s) (subject to licence) and exact effect implementations.
+- Generator internals (progression/rhythm/melody rules), as long as output fits
+  the tier descriptions.
+- Exact tier-1 sub-steps and how many songs per tier.
+- How the Demucs ONNX model is obtained or converted, and fallback choices if
+  alphaTab rendering/export doesn't work under Bun (report any substitution).
+- Quantisation method and tab-assignment cost model.
+- Test strategy for the harness itself.

reports/Music transcription tooling survey.md

@@ -0,0 +1,207 @@
+# Separate stems, then transcribe each instrument
+
+As of late 2026, nobody has released an open-source system that turns a full song into good multi-instrument sheets and tabs. The components for building one do exist, and they work best as a chain: **separate the mix into stems, then send each stem to a specialist transcriber, then quantize the notes to a beat grid and render the result with alphaTab**. Quality depends heavily on the instrument. **Kick, snare, bass and lead vocals come out reliably. Piano and lead guitar are workable. String/fret tablature is still only about half right end to end**: the best 2026 research system, TART, scores **54% Tab F1** ([arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)). Hi-hats, cymbals, backing vocals and lead-vs-rhythm guitar separation remain weak. The browser is now a realistic place to run the front of the pipeline. WebGPU ships in every major browser ([web.dev](https://web.dev/blog/webgpu-supported-major-browsers)), MIT-licensed HTDemucs ONNX ports separate stems at **about 8–22x real time on a mid-range GPU** ([HF monteslu](https://huggingface.co/monteslu/htdemucs-web-onnx)), and Basic Pitch (Apache-2.0) has an official browser build. The best models, however, are too large for the browser, have unclear or non-commercial licenses, or both. That includes the RoFormer separators, Kyutai's MuScriptor and the strongest drum models. The architecture that follows from this is browser-first, with a server-side quality tier. The key finding for the next phase: no commercial vendor publishes accuracy numbers ([audio2guitar](https://audio2guitar.com/blog/best-ai-guitar-tab-generators); [Songscription](https://www.songscription.ai/blog/best-ai-guitar-tab-generators)), so a reproducible benchmark against Guitar Pro references would be both an engineering tool and a differentiator.
+
+## RoFormer models lead separation, but only Demucs runs in browsers
+
+Source separation decides how well everything after it can do, and the field has moved past Demucs. **BS-RoFormer SW** is the best open-weight multi-stem model. It is a 6-stem BS-RoFormer variant (vocals, bass, drums, guitar, piano, other), modified by lucidrains with weights from viperx. MVSEP made it the default for new users and reports Multisong SDR of **vocals 11.30, bass 14.62, drums 14.11, guitar 9.05, piano 7.83, other 8.71** ([MVSEP news 64](https://mvsep.com/news/64?lang=en)). The original Demucs repo was **archived on 2025-01-01**. Its author keeps a slow fork at adefossez/demucs that takes bug fixes only ([facebookresearch/demucs](https://github.com/facebookresearch/demucs)). Demucs's 6-stem model is especially weak on the instruments this project cares about most. The README admits "the piano source is not working great" ([Demucs README](https://cdn.jsdelivr.net/gh/facebookresearch/demucs@main/README.md)), and MVSEP measures htdemucs_6s at **2.23 SDR for piano** against 7.83 for BS-RoFormer SW, with "other" dropping to 2.63 ([MVSEP algorithms](https://mvsep.com/en/algorithms)).
+
+| Model | Stems | Bass / drums / vocals / guitar / piano SDR (MVSEP) | Code / weights license | Browser-ready? |
+|---|---|---|---|---|
+| htdemucs_ft | 4 | 12.05 / 11.24 / 8.33 / – / – | MIT / MIT | Yes (WebGPU ONNX, 333 MB) |
+| htdemucs_6s | 6 | 11.42 / 10.59 / 8.17 / n/a / 2.23 | MIT / MIT | Yes (WebGPU ONNX, ~84 MB) |
+| SCNet XL | 4 | 13.81 / 13.42 / – / – / – | code open | No port found |
+| BS-RoFormer SW | 6 | 14.62 / 14.11 / 11.30 / 9.05 / 7.83 | community weights, **no license stated** | Unofficial ONNX export only, ~700 MB checkpoint |
+| MVSEP BS-RoFormer 124-band | 2 (vocals) | vocals 12.33 | service-only | No |
+
+Sources: [MVSEP algorithms](https://mvsep.com/en/algorithms); [demucs-rs](https://github.com/nikhilunni/demucs-rs); [HF monteslu](https://huggingface.co/monteslu/htdemucs-web-onnx). MVSEP numbers come from its own Multisong set and cannot be compared directly with MUSDB18 paper figures.
+
+Two gaps matter. First, **the cleanest stems are not the most legally clean**. Demucs is MIT end to end ([HF monteslu](https://huggingface.co/monteslu/htdemucs-web-onnx)). The RoFormer weights (viperx, KimberleyJensen, becruily, gabox, anvuew, jarredou) are community-trained, usually on undisclosed data, and were found with no license attached. The BS-RoFormer SW checkpoint has already moved hosts once, after the jarredou account disappeared ([HF jarredou commit](https://huggingface.co/jarredou/BS-ROFO-SW-Fixed/commit/f99fc5c2e4bc201413cfc1ca5767cf62ddb1a9a4); [openmirlab/bs-roformer-infer](https://github.com/openmirlab/bs-roformer-infer)). Second, **separation stops at "guitar"**. No open-weight model splits lead from rhythm guitar. Moises markets one as "first-ever" without publishing metrics ([Moises blog](https://moises.ai/blog/moises-news/new-guitar-separation-models/)), MVSEP offers only a demo ([MVSEP news](https://de2.mvsep.com/news)), and LALAL.AI puts all guitars in one stem ([LALAL.AI](https://www.lalal.ai/blog/andromeda-supports-guitars-updates-piano/)). Keyboards other than piano have no dedicated stem and fall into "other".
+
+For the server, two MIT-licensed tools wrap these models. **python-audio-separator** is a pip/CLI wrapper. It loads MDX, VR, Demucs and RoFormer checkpoints, ships 11 ensemble algorithms and karaoke presets, and runs on CUDA, Apple MPS/CoreML and CPU. Version 0.47.0 raised RoFormer overlap defaults, which means 4–6x more forward passes ([python-audio-separator](https://github.com/nomadkaraoke/python-audio-separator)). **ZFTurbo MSST** (`pip install msst`) is the training and inference framework behind most community models. It supports 17 architectures and includes DrumSep checkpoints ([MSST](https://github.com/ZFTurbo/Music-Source-Separation-Training)). The UVR5 GUI is still on v5.6 plus beta Roformer patches ([kocpc](https://en.kocpc.com.tw/?p=12798)). Community sentiment has followed the models. UVR was the top answer in r/edmproduction's stem-splitter thread, and by mid-2026 opinion had "hardened toward MVSEP" ([Futureproof Music School](https://futureproofmusicschool.com/blog/best-ai-stem-separation-tools)).
+
+Two kinds of sub-separation are mature enough to use. **Vocal karaoke models** split lead from backing vocals. Open BS-RoFormer karaoke weights reach about **10.1–10.2 lead SDR**, but backing-vocal stems top out around **5.6–7 SDR**, which is usable for detecting harmony but not for clean transcription ([MVSEP algorithms](https://mvsep.com/en/algorithms)). **DrumSep models** split a drum stem into kick (18–22 SDR), snare (13–17) and toms (13–16) well. Hi-hat, ride and crash come out poorly, at 3–9 SDR ([MVSEP algorithms](https://mvsep.com/en/algorithms)). LarsNet is an academic alternative, but MVSEP could not reproduce its metrics, and the LARS plug-in weights are CC BY-NC ([MVSEP LarsNet](https://mvsep.com/algorithms/31?lang=en); [LARS](https://github.com/EdoardoMor/LARS)).
+
+In the browser, **only Demucs is proven**:
+
+- **monteslu's htdemucs ONNX build** is 126 MB in mixed precision. It is split into 21 chained pieces so the UI never freezes, and keeps convolutions in fp32 because full fp16 "badly degrades drums and bass" ([HF monteslu](https://huggingface.co/monteslu/htdemucs-web-onnx)).
+- **6-stem WebGPU exports** of htdemucs_6s exist ([HF kramp](https://huggingface.co/kramp/htdemucs-6s-webgpu-onnx)).
+- **demucs-rs** (Apache-2.0) runs in the browser via WASM+WebGPU, but "significantly slower than the native CLI" ([demucs-rs](https://github.com/nikhilunni/demucs-rs)).
+- **CPU-only WASM is impractical.** freemusicdemixer took about 41 minutes for a 7-minute track single-threaded. Its weights are proprietary, and the repo was archived in April 2025 ([freemusicdemixer repo](https://github.com/sevagh/freemusicdemixer.com)).
+
+A community ONNX export of BS-RoFormer SW exists, with no model card and no published browser speed ([HF elicwhite](https://huggingface.co/elicwhite)). On a server GPU, htdemucs separates a 4-minute song in roughly 15–30 s, against 2–3 minutes on CPU ([MISST](https://github.com/Frikallo/MISST); [LinuxLinks](https://www.linuxlinks.com/machine-learning-linux-demucs-music-source-separation/2/)). No primary-source RoFormer timings were found.
+
+## Per-stem specialists beat all-in-one models for keys, vocals and bass
+
+Multi-instrument transcription made a real jump in 2026, but with a licensing catch. **Kyutai's MuScriptor** (ISMIR 2026) is a decoder-only Transformer trained on 1.45M rendered MIDI files plus about 11,000 hours of real recordings with aligned notes. It outputs notes over 36 instrument subgroups, including Voice, E-Piano, E-Bass, Clean E-Guitar and Distorted E-Guitar ([arXiv 2607.08168](https://arxiv.org/html/2607.08168v2)). On 372 held-out real tracks it scores **48.2 Multi F1 against 21.9 for YourMT3+**, with onset F1 of 60.4 against 32.5 ([arXiv 2607.08168](https://arxiv.org/html/2607.08168v2)). The code is MIT, but **the weights are CC BY-NC 4.0 and gated**. Released sizes are 103M (CPU-practical), 307M (default) and 1.4B. The paper lists slightly different parameter counts. There is no ONNX or browser path, and sheet output needs MuseScore 4 ([GitHub muscriptor](https://github.com/muscriptor/muscriptor)). **YourMT3+** is GPL-3.0, lightly maintained (30 commits, last news November 2024) and labelled "pre-release" ([GitHub YourMT3](https://github.com/mimbres/YourMT3)).
+
+Even the leader stays below 50% on real music, and adding instruments hurts sharply. In the 2025 AMT Challenge, F1 roughly halved going from one to three instruments. YourMT3-MoE fell from 0.759 to 0.392, and the main failures were hallucinated instruments and dense polyphony ([arXiv 2603.27528](https://arxiv.org/html/2603.27528v1)). These models are best treated as a non-commercial quality reference, not as the product engine. Their weakness argues for separating first and then transcribing each stem, although **no controlled study of stems versus mix was found**. Songscription warns that stem bleed creates false notes ([Songscription](https://www.songscription.ai/blog/separate-stems-before-transcribing)).
+
+**Spotify's Basic Pitch** is the default lightweight transcriber:
+
+- **What it is:** Apache-2.0, polyphonic and instrument-agnostic. It outputs pitch bends, ships as CoreML, TFLite and ONNX, and has an official TypeScript/browser port, `@spotify/basic-pitch` ([basic-pitch](https://github.com/spotify/basic-pitch); [basic-pitch-ts](https://github.com/spotify/basic-pitch-ts)).
+- **Its own guidance:** it "works best on one instrument at a time", which suits a stem-based design.
+- **Where it ranks:** it came last in the AMT Challenge (0.063), mainly because it outputs no instrument labels. It was also the fastest entry ([arXiv 2603.27528](https://arxiv.org/html/2603.27528v1)).
+- **A possible successor:** a September 2026 paper reports its 26K-parameter Harmonica-nano beating Basic Pitch's frame F1 by 14.6 points at 1,600x real time ([arXiv 2609.04640](https://arxiv.org/abs/2609.04640)). Code and license are unknown.
+- **Maintenance:** the TS port's mirror was "last synced over a year ago" ([ecosyste.ms](https://awesome.ecosyste.ms/projects/github.com%2Fspotify%2Fbasic-pitch-ts)). An ONNX re-export for ORT-Web exists ([HF daserge](https://huggingface.co/daserge/basic-pitch-onnx)).
+- **User sentiment:** no primary-source user reviews were found.
+
+The best tool for each pitched instrument:
+
+- **Keyboard.** Solo piano is essentially solved on studio recordings. **Transkun** (12.9M params) reaches **93.48% note-with-offset F1** on MAESTRO v3 and is pip-installable ([arXiv 2605.17405](https://arxiv.org/pdf/2605.17405); [Transkun](https://github.com/yujia-yan/transkun)). The ByteDance high-resolution model (Apache-2.0 code, models pedal) was **archived on Dec 8, 2025** ([bytedance/piano_transcription](https://github.com/bytedance/piano_transcription)). Omnizart (MIT) is TensorFlow-era and incompatible with ARM Macs ([Omnizart](https://github.com/Music-and-Culture-Technology-Lab/omnizart)). The real constraint is the piano stem at about 7.8 SDR, not the transcriber. Organ and synth parts land in "other" and need Basic Pitch.
+- **Vocals.** Pitch tracking is nearly solved, but segmenting it into notes is not. **FCPE** matches RMVPE's 96.8% raw pitch accuracy with 10.6M instead of 90.4M parameters, at 5x the speed ([arXiv 2509.15140](https://arxiv.org/html/2509.15140v1)). **PESTO** handles a 3-minute file in about 13 s on a laptop CPU, against about 12 minutes for CREPE ([PESTO](https://github.com/SonyCSLParis/pesto)). For notes, **RMVPE → ROSVOT** is the de facto research chain ([ROSVOT](https://github.com/RickyL-2000/ROSVOT)), with SOME as a lighter option. ROSVOT's checkpoint is trained mainly on Mandarin ([TelkNet](https://telknet.cc/en/tools/music-vocal-midi)). Vibrato, slides and melisma are where notation breaks down. Whisper runs on WebGPU for lyrics ([Remotion](https://remotion.dev/docs/whisper-webgpu)).
+- **Bass.** No maintained dedicated bass transcriber exists. A 2024 project notes "a lack of automatic transcription solutions for the electric bass guitar" ([HKU](https://wp2023.cs.hku.hk/fyp23026/wp-content/uploads/sites/27/final_paper.pdf)). Bass is mostly monophonic and its stem is the second-cleanest at about 14.6 SDR, so running a pitch tracker or Basic Pitch on the stem is the pragmatic route. The low E (about 41 Hz) needs explicit testing.
+
+Licenses for Transkun, ROSVOT, SOME, RMVPE, FCPE and PESTO were not verified and must be checked before shipping.
+
+## Guitar tabs remain roughly half right end to end
+
+Producing tablature means solving two problems: finding the notes, then choosing a string and fret for each. The 2026 state of the art is **TART** (Berkeley, revised September 2026). It is a four-stage pipeline:
+
+1. A high-resolution audio-to-MIDI model.
+2. A 160K-parameter technique classifier.
+3. A 15M-parameter T5 string/fret assigner, "AudioFret", which is conditioned on audio.
+4. A BeatNet-quantized MusicXML/JAMS renderer.
+
+TART reports **81.35% note F1 (onset ±50 ms)**, **71.8% Tab F1 when given perfect MIDI**, and **54.08% Tab F1 end to end**. It attributes a 17.75-point "propagation cost" to note-detection errors ([arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)). The test sets are GuitarSet and EGDB, mostly solo guitar, so dense distorted rock mixes will score lower. **No TART code or weights were found**, nor any for the September 2026 diffusion model Noise2Fret ([arXiv 2608.30854](https://arxiv.org/abs/2608.30854)). The training data is public and has improved:
+
+- **GOAT:** 5.9 h of DI electric guitar, amp-augmented to 29.5 h, CC BY 4.0, with tabs **in Guitar Pro format** ([arXiv 2509.22655](https://arxiv.org/abs/2509.22655)).
+- **GAPS:** 14 h of classical guitar ([arXiv 2408.08653](https://arxiv.org/abs/2408.08653)).
+- **SynthTab:** DadaGP's 26k songs rendered to audio ([arXiv 2309.09085](https://arxiv.org/pdf/2309.09085)).
+
+Distorted guitar is still under-evaluated. No paper benchmarks high-gain full-mix tabs.
+
+For string/fret assignment, learned models beat heuristics only on hard material. The Fretting-Transformer results ([arXiv 2506.14223](https://arxiv.org/html/2506.14223v1)):
+
+| Material | Fretting-Transformer | Guitar Pro | TuxGuitar | A* |
+|---|---|---|---|---|
+| Easy (GuitarToday) | 98.4% | 97.6% | 98.3% | – |
+| Leduc (jazz) | 72.2% | 56.0% | 57.9% | 62.6% |
+| DadaGP | 81.6% | 76.6% | 78.8% | 78.9% |
+
+On the easy GuitarToday material, a trivial "lowest fret" rule nearly ties the model. Its code was not found. **midi2tab_Deconvolutions** has public code but assumes standard 6-string tuning ([arXiv 2510.10619](https://arxiv.org/html/2510.10619v1)). Given these numbers, a playability-cost Viterbi assigner is a sound license-clean first version. Its costs would be fret span, position shifts and open-string preference, and it would be aware of tuning and capo. A T5 assigner fine-tuned on DadaGP/GOAT is the upgrade path.
+
+Techniques and tuning are partly solved:
+
+- **Technique classification** works on curated data: TART reports 95.9% macro F1 across 9 classes, with vibrato the weakest at 89.4% because it is confused with bends ([arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)). No open technique weights or end-to-end technique scores exist.
+- **Tuning and capo are user inputs** in every research system. No detector was found. Notes below E2 strongly hint at drop or down-tuning. Capo is effectively unobservable from audio.
+- **Bends** need a pitch contour rather than a class label. Basic Pitch's pitch-bend output, or an F0 track on the stem, can provide it.
+
+**Multiple guitars are the least solved requirement**, and users notice. Heise found Songsterr AI "fairly accurate" but unable to always tell multiple guitar parts apart ([heise](https://heise.de/-10356152)). No open model separates guitars from each other, so the practical approaches are untested DSP and symbolic tricks:
+
+- Split the mix into left, right, mid and side channels before and after separation. Hard-panned double-tracked rhythm parts dominate the sides, and centered leads dominate the mid.
+- Split voices symbolically after transcription: monophonic high line as lead, chordal material as rhythm.
+- Let the user declare how many guitars there are and where they are panned.
+
+These approaches need benchmarking in the next phase.
+
+For Guitar Pro I/O, **alphaTab** (MPL-2.0) is the clear fit for a website. Its `Gp7Exporter` writes .gp (GP7+) files to a Uint8Array in browser or Node, available since 1.2.0 ([alphaTab exporter](https://alphatab.net/docs/guides/exporter)). An alphaTex text exporter arrived in 1.7, and a GP8 drum-export bug was fixed in December 2025, which shows active maintenance ([alphaTab 1.7](https://alphatab.net/docs/releases/release1_7); [PR #2456](https://github.com/CoderLine/alphaTab/pull/2456)). On the Python side, **PyGuitarPro** reads and writes only GP3–5 under LGPL-3.0 ([PyGuitarPro](https://github.com/Perlence/PyGuitarPro)). ApolloTab claims GP3–GP8 support, which is unverified ([PyPI apollotab](https://pypi.org/project/apollotab/)). No dedicated bass-tab model exists either. Commercial tools such as Tabtify and Songscription offer 4/5/6-string bass tabs with no published accuracy ([Tabtify](https://tabtify.com/convert/audio-to-bass-tab); [Songscription](https://www.songscription.ai/bass-tab-generator)).
+
+## Drums: kick and snare are reliable, cymbals are not
+
+Drum transcription has a usable open stack, but the licenses split it in two. The 2025–26 options:
+
+| Model | Classes | Onset F1 | License | Notes |
+|---|---|---|---|---|
+| **ADT_STR** (Jan 2026) | 26 GM-derived | claims SOTA on ENST/MDB (no numbers retrieved) | **CC BY-SA 4.0**, weights on HF | Trained on synthetic data only |
+| **ADTOF** | 5 | MDB 86.9, IDMT 94.7, E-GMD 45.0 | **CC BY-NC-SA 4.0** | Trained on real music; PyTorch port; low maintenance (8 commits) |
+| **Noise-to-Notes** (diffusion) | 7 | E-GMD 89.7, MDB 87.9; velocity F1 82.8 | code/weights not found | ~50M params plus a 330M MERT encoder |
+| **OaF-Drums** (Magenta) | GM | E-GMD 83.4, MDB 75.2 | E-GMD data CC BY 4.0; code license not checked | Velocity-aware |
+| MuScriptor / YourMT3+ | drums, onset-only | 50.2 / 41.4 | NC / GPL | Too weak for drum notation on their own |
+
+Sources: [ADT_STR](https://github.com/pier-maker92/ADT_STR); [arXiv 2601.09520](https://arxiv.org/abs/2601.09520); [ADTOF](https://github.com/MZehren/ADTOF); [arXiv 2509.21739](https://arxiv.org/html/2509.21739v2); [Magenta](https://magenta.tensorflow.org/oaf-drums); [arXiv 2607.08168](https://arxiv.org/html/2607.08168).
+
+ADTOF collapses to 45 F1 on electronic-kit audio. That domain gap is a reason to evaluate on the project's own songs rather than trust published scores. An ISMIR 2025 diagnostic study confirms the expected weaknesses ([ISMIR 2025 P5-10](https://ismir2025program.ismir.net/poster_130.html)):
+
+- **Toms and cymbals** are the worst classes.
+- **Simultaneous hits** are the main error source on drum-only audio.
+- **Melodic instruments and vocals** are the main limit in full mixes.
+
+This last point argues for transcribing a separated drum stem. There is no evidence that further splitting the stem into kick, snare and so on with DrumSep or LarsNet improves transcription. Velocity, and therefore ghost-note notation, is modeled only by OaF-Drums, hFT-Transformer and N2N.
+
+No drum transcription demo running in the browser was found ([browserTranscribe](https://github.com/jsphweid/browserTranscribe)). User sentiment is thin. An r/DRUM user found Klangio's Drum2Notes poor even on a simple isolated beat, while another called Songsterr AI "very good" ([Reddit r/DRUM](https://nyc1.lr.ggtyler.dev/r/DRUM/comments/1e5qmzg/ai_that_translates_audio_into_sheet_music)).
+
+## Rhythm, notation and ingestion decide whether output looks like music
+
+Raw note events only become sheet music once they sit on a beat grid. **Beat This!** (CPJKU, ISMIR 2024) is the default beat tracker for a commercial site. Its code and weights are **MIT**, it tracks beats and downbeats without a DBN, falls back to CPU, and has an **8 MB small model**, which makes it a candidate for a browser port ([beat_this](https://github.com/CPJKU/beat_this)). The older options each have a blocker:
+
+- **madmom** has had no PyPI release since 2018, breaks on Python ≥3.10 with NumPy ≥1.24, and has **non-commercial CC BY-NC-SA model files** ([PyPI madmom](https://pypi.org/project/madmom/); [madmom](https://github.com/CPJKU/madmom)).
+- **allin1** (MIT) adds verse/chorus labels but depends on madmom and NATTEN ([all-in-one](https://github.com/mir-aidj/all-in-one)).
+- **essentia.js** is the only mature in-browser analysis library, but it is AGPLv3 unless you buy a license ([Essentia licensing](https://essentia.upf.edu/licensing_information.html)).
+
+For chords, BTC's 170-class checkpoint is the practical baseline ([HF btc-chord](https://huggingface.co/puar-playground/btc-chord)). ChordFormer (2025) claims SOTA but its code is unconfirmed ([arXiv 2502.11840](https://arxiv.org/abs/2502.11840v1)). Research models for turning performance MIDI into a score are almost all piano-only. PM2S beats commercial software on MV2H ([ISMIR 2022](https://archives.ismir.net/ismir2022/paper/000047.pdf)). For guitar, bass, vocals and drums, the workable method is to interpolate onsets between detected beats and snap them to allowed subdivisions. This handles tempo drift better than a fixed BPM grid.
+
+On rendering, **alphaTab** is the only library with multi-track standard notation, guitar and bass tab, drum notation, SoundFont playback, audio export, and an External Cursor API that syncs the score to external audio or a YouTube video ([alphaTab 1.6](https://alphatab.net/docs/releases/release1_6); [features](https://demo.alphatab.net/features.html)). It is the natural core of the product. MusicXML can go to OSMD (BSD-3, playback still sponsor-only) or Verovio (LGPL-3.0 WASM, with MEI tablature since 2025) for print-quality standard notation ([OSMD](https://github.com/opensheetmusicdisplay/opensheetmusicdisplay); [Verovio changelog](https://data.safetycli.com/changelogs/verovio)). Editing is not built into these renderers, so a correction UI has to be built on the score model. Users ask for such an editor in every review.
+
+Ingestion splits cleanly into an easy part and a hard one. In the browser, Web Audio's `decodeAudioData` handles mp3, wav, m4a and ogg/opus in modern browsers, with ffmpeg.wasm (about 31 MB) lazy-loaded as a fallback ([ffmpeg.wasm #623](https://github.com/ffmpegwasm/ffmpeg.wasm/issues/623)). One timing caveat: MP3 decoders add **20–40 ms offsets** ([all-in-one](https://github.com/mir-aidj/all-in-one)), which is close to the 50 ms tolerance used in evaluation. **YouTube links are the risky input**:
+
+- **yt-dlp only works server-side.** Since February 2026 it needs an external JS runtime (Deno) ([openSUSE](https://src.opensuse.org/pool/yt-dlp/pulls/4/files)), and it increasingly needs PO tokens that it cannot generate itself ([yt-dlp PO Token Guide](https://github.com/yt-dlp/yt-dlp/wiki/PO-Token-Guide)).
+- **The token workaround is failing.** The bgutil token provider now warns that tokens "no longer bypass the bot check for majority of cases" ([PyPI bgutil](https://pypi.org/project/bgutil-ytdlp-pot-provider)).
+- **There is legal exposure.** The RIAA's 2020 takedown of youtube-dl cited anti-circumvention under DMCA §1201 ([XDA](https://www.xda-developers.com/youtube-dl-riaa-dmca/)).
+
+The lower-risk design: users upload their own file, and a YouTube link only embeds the official player, synced to the score through alphaTab's cursor API.
+
+## Users treat every AI transcriber as a first draft
+
+Commercial tools set the bar this project has to clear, and the feedback is consistent across them:
+
+- **Klangio** (Guitar2Tabs and siblings) takes YouTube links and exports Guitar Pro and MusicXML. It is rated 3.6/5 on the App Store, with reports of crashes on MusicXML export ([App Store](https://apps.apple.com/app/id1589660528)). Its Trustpilot reviews range from "legendary" to "didn't even come close" ([Trustpilot](https://uk.trustpilot.com/review/klang.io)). An independent August 2026 test saw accuracy drop on phone or noisy recordings ([agent-finder](https://agent-finder.co/reviews/klangio)). A competitor says it "tends to get the notes right" but lays them out unplayably and detects bends inconsistently ([Songscription](https://www.songscription.ai/blog/best-ai-guitar-tab-generators)).
+- **Songsterr AI** does guitar, bass and drums in one pass. Bass players are split between "dead on accurate" and "if you are after accuracy you'll not get that" ([BassBuzz](https://forum.bassbuzz.com/t/new-songsterr-ai-is-impressive/70519)).
+- **Complaints that recur** across products: unplayable fingerings, confused multiple guitars, inconsistent technique detection, paywalls on export, and per-song ticket pricing.
+
+These map directly onto features this design can deliver: a playability-aware assigner, explicit multi-guitar handling, an editor, open GP and MusicXML export, and free processing in the browser.
+
+On the open-source side, the usual recipe is yt-dlp → Demucs → Basic Pitch → music21 → engraver. The most complete example is **oh-sheet**, which outputs two-hand piano PDFs with a 25-fixture evaluation harness. Its license is ambiguous: the sidebar says MIT but the README badge says "Proprietary" ([oh-sheet](https://github.com/Oh-Sheet-Team/oh-sheet)). **No open end-to-end guitar-tab or drum pipeline exists**, so this project would be first in that space.
+
+## Recommended architecture: browser draft, server quality pass
+
+The design runs a complete **"draft" pipeline in the browser**, built only from permissive components, and offers an optional **server "quality" pipeline** that swaps in larger models. Both feed the same symbolic score model, rendered and exported by alphaTab. Each browser stage runs in its own Web Worker, terminated after use. That is the reliable way to free memory, because ORT-Web's 4 GB WASM heap does not shrink when a session is disposed ([ORT large models](https://onnxruntime.ai/docs/tutorials/web/large-models.html); [Chromium group](https://groups.google.com/a/chromium.org/g/chromium-extensions/c/-4CNcu5bUGA/m/SxP0RfNaCAAJ)). The browser draft uses WebGPU through ONNX Runtime Web or transformers.js v4 ([transformers.js releases](https://github.com/huggingface/transformers.js/releases)). The app routes to the server when WebGPU is missing: Firefox on Linux or Android, and older Android devices ([web.dev](https://web.dev/blog/webgpu-supported-major-browsers)).
+
+| Stage | Browser draft (permissive) | Server quality tier | License caveat for commercial use |
+|---|---|---|---|
+| Ingest | decodeAudioData; ffmpeg.wasm fallback; YouTube embed for sync only | yt-dlp extraction only after legal review | yt-dlp and YouTube ToS / DMCA §1201 risk; ffmpeg.wasm core is LGPL/GPL depending on build |
+| Separation | htdemucs_6s or htdemucs ONNX on WebGPU (MIT) | BS-RoFormer SW 6-stem via python-audio-separator / MSST; optional SCNet XL ensemble for bass and drums; htdemucs_ft as clean fallback | RoFormer community weights have **no stated license**; get legal sign-off or ship Demucs only |
+| Multi-guitar | L/R/mid/side split before and after separation; user declares number of guitars and panning | same, plus symbolic lead/rhythm voice split | none (in-house DSP) |
+| Vocals | lead stem → FCPE or PESTO F0 → simple note segmentation; Whisper WebGPU for lyrics | karaoke BS-RoFormer (lead vs backing) → RMVPE/FCPE → ROSVOT or SOME | karaoke weights unlicensed; ROSVOT/SOME/FCPE/PESTO licenses unverified |
+| Keys | piano stem → Basic Pitch ONNX (Apache-2.0) | piano stem → Transkun or ByteDance (archived, Apache code); "other" stem → Basic Pitch for synths and organ | verify Transkun license; ByteDance checkpoint license unstated |
+| Bass | bass stem → Basic Pitch or F0 tracker → 4/5-string Viterbi assigner | same, larger models | none beyond the separator's |
+| Guitar | guitar stem → Basic Pitch (with pitch bends) → in-house Viterbi string/fret assigner with tuning and capo inputs | high-res guitar CRNN (Riley/GAPS-style) or a fine-tuned model → T5 assigner trained on DadaGP/GOAT → technique tagger | GOAT CC BY 4.0; DadaGP and GuitarSet licenses unverified; train in-house since no TART code |
+| Drums | not yet viable (no browser ADT port; best small models are NC) | drum stem → **ADT_STR** (CC BY-SA); ADTOF only for internal comparison | ADTOF, LARS, madmom models are **non-commercial**; CC BY-SA requires share-alike on derived weights |
+| Beat / tempo / chords | Beat This! small (8 MB) via ONNX export, untested | Beat This! full; BTC chords | avoid madmom models and essentia.js (AGPL) |
+| Quantize / notation | beat-interpolated grid snapping → alphaTex → alphaTab render, playback, GP7 export | same, plus MusicXML via music21 for OSMD/Verovio views | alphaTab MPL-2.0 is file-level copyleft (share changes to its files) |
+| Reference / benchmark only | – | MuScriptor 307M for a full-band reference transcription | **CC BY-NC**: not for production without a Kyutai license |
+
+The table hides one real trade-off: **the legally cleanest stack is measurably worse at separation**. htdemucs_6s's piano stem is at 2.23 SDR against 7.83, and the Demucs family has no strong guitar stem ([MVSEP algorithms](https://mvsep.com/en/algorithms)). If the RoFormer weights cannot be cleared, there are two remaining paths. One is to train a 6-stem RoFormer in-house with MSST on licensed data. The other is to treat keys and guitar as "best effort" in the draft tier. On a consumer server GPU, the full quality pipeline is estimated at about 1–3 minutes per 4-minute song. That figure is an inference; no RoFormer timings were published.
+
+## Benchmark against Guitar Pro references by stage, not just end to end
+
+The next phase should copy TART's habit of attributing errors to each stage, because a single end-to-end score hides where the 17-point losses happen ([arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)). No public tool compares two GP files ([arXiv 2506.14223](https://arxiv.org/html/2506.14223v1)), so building that comparison is the first deliverable.
+
+**Corpus and ground truth.** Each user-supplied reference .gp file is paired with the *exact* recording it transcribes. Each pair is tagged by genre, gain level (clean, crunch, high-gain), number of guitars, panning, tuning, capo and presence of keys. Three public sets add controlled anchors: GuitarSet/EGDB for guitar notes, GOAT for electric tabs in GP format, and MDB/ENST for drums. One parsing risk needs an early check. alphaTab's documented importers cover GP3–5, GP6 and alphaTex ([alphaTab features](https://demo.alphatab.net/features.html)). Importing GP7/8, the format most user references will be in, is assumed but needs testing in week one. PyGuitarPro is a fallback for GP3–5 only. The parsed reference is expanded into note events (repeats unrolled; tuning and capo applied), each carrying score position, pitch, string, fret, duration and technique flags. Drum tracks are mapped to the classes the model can output: kick, snare, toms, hi-hat, cymbals.
+
+**Alignment.** Reference tabs have bar and beat positions but no audio timestamps. Bar-level anchors come from aligning the beats of the reference's synthesized audio to Beat This! output on the real recording, refined with chroma DTW. Alignment success is reported separately so that alignment failures are not counted as transcription errors. Audio is decoded to WAV once, server-side, which removes the 20–40 ms MP3 decoder offset ([all-in-one](https://github.com/mir-aidj/all-in-one)).
+
+**Metrics.** Generated tracks are matched to reference tracks with the Hungarian algorithm on note F1. This matching is the multi-guitar test. Swapped lead and rhythm parts, or merged parts, show up directly as matching failures. The per-track metrics (via mir_eval where applicable):
+
+- **Notes:** note F1 on onset ±50 ms plus pitch, then with offsets added.
+- **Tabs:** Tab F1 (pitch + onset + string + fret), and Tab Disambiguation Rate (correct string given correct pitch).
+- **Lenient variant:** counts a different but playable position as acceptable, since human tabs legitimately disagree. Learned assigners reach only 72–82% exact agreement on hard references ([arXiv 2506.14223](https://arxiv.org/html/2506.14223v1)).
+- **Techniques:** precision and recall for each technique.
+- **Rhythm:** beat-level onset accuracy in score time, correct bar count, and correct time signature.
+- **Playability:** a fret-span and shift difficulty score.
+- **Drums:** per-class onset F1.
+
+**Ablations and baselines.** Each benchmark song runs through four controlled comparisons:
+
+- **Mix vs stem:** answers the open question of whether separation helps transcription.
+- **Browser draft vs server quality:** same song, same metrics, plus runtime and peak memory per device class.
+- **Oracle notes:** the reference's own notes fed into the assigner, which isolates fingering quality from note detection.
+- **Separation stems:** Demucs vs RoFormer stems, which prices the license trade-off in F1 points.
+
+Basic Pitch + Viterbi is the floor. MuScriptor is a non-commercial reference ceiling. Klangio and Songsterr GP exports are commercial comparisons where their terms allow. A small human study runs alongside: guitarists rate playability and time how long it takes to correct output in the editor. This captures the "useful draft" quality that users actually judge products on.
+
+## Conclusion
+
+The bottleneck has moved. Separation and note detection improved sharply in 2025–26. What now limits a product like this is everything after the notes: deciding which guitar played what, picking playable fingerings, placing notes on a credible rhythmic grid, and keeping the whole stack licensable. That favors a team that builds strong symbolic post-processing and a correction editor on top of commodity models, rather than one waiting for a better end-to-end model. The strongest end-to-end models (MuScriptor, TART) are either non-commercial or unreleased.
+
+The browser-first design is viable for drafts today. It lowers cost and protects privacy, and that matches the main user complaints about paywalls and per-song tickets. Its quality ceiling, however, is set by the separator the project is legally allowed to ship. Two decisions should therefore come early, before heavy engineering: legal clearance of the RoFormer weights (or a plan to train in-house), and a verified way to import GP7/8 for the benchmark. If both resolve well, a published, reproducible accuracy table against real Guitar Pro tabs would be a claim no competitor currently makes.

reports/Transcription accuracy improvement ideas.md

@@ -0,0 +1,191 @@
+# Search, vote and verify beyond stem transcription
+
+The biggest accuracy gains available to us in late 2026 come from a newly released model plus our own symbolic engineering, not from a bigger end-to-end model. The model is **Noise2Fret**, a diffusion audio-to-tab model with public code that lifts electric-guitar Tab F1 on GOAT from about **0.66 to 0.795** ([arXiv 2608.30854](https://arxiv.org/html/2608.30854)). The engineering is three layers wrapped around the transcriber: search over hypotheses (tuning and capo, fingering, diffusion samples), voting across redundancy the song already contains (repeated riffs, double-tracked guitars, the cleaner bass stem), and verification of the result (render-and-compare, model agreement) that marks the bars a user needs to check. Six of the coordinator's sixteen ideas are cheap to test against our Guitar Pro references now: tuning/capo search, repetition voting, pitch-contour technique detection, bass-informed rescoring, multi-guitar pan clustering and user hints. Four are worth staging behind them: render-and-compare confidence, the note-level ensemble ("note-ROVER"), diffusion n-best rescoring and licence-clean procedural data. Three should wait: per-song tone adaptation, iterative "peeling" separation and video fingering. A licence audit changes the plan as much as any model does. Several datasets the earlier survey treated as open or unverified turn out to be **non-commercial or research-only**: GuitarSet, GOAT, DadaGP and SynthTab. The only commercially clean real-audio guitar data we found is Guitar-TECHS plus our own recordings, so a shippable guitar model has to be trained mostly on audio we render ourselves. The literature offers almost no published numbers for ensembling, repetition voting, LM rescoring or test-time adaptation applied to music transcription. Our benchmark against our Guitar Pro references will therefore produce the first evidence on most of these ideas.
+
+This report builds on the baseline survey in `reports/Music transcription tooling survey.md`. That survey set out the pipeline: separate stems (Demucs in the browser, BS-RoFormer on the server), transcribe each stem (Basic Pitch, Transkun, ADT_STR, F0 trackers), assign string and fret with a Viterbi pass, quantize to Beat This! beats, and render with alphaTab. It also found TART's **54% end-to-end Tab F1** as the 2026 research ceiling ([arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)). Nothing below repeats that material except where a correction is needed.
+
+## Four 2025–26 results move the guitar frontier, and one paradigm does not
+
+### Noise2Fret shows that generative tab models model fingering better
+
+The earlier survey recorded that no code was available for Noise2Fret. **That is now wrong**: code and trained models are on GitHub and Zenodo ([GitHub](https://github.com/RiccardoVib/Noise2Fret); [Zenodo](https://zenodo.org/records/23037260)). Noise2Fret diffuses in a continuous embedding of string-fret targets with a 1D conv U-Net, conditioned on the STFT, spectral flux, brightness and raw audio. Five auxiliary losses (pitch-class, circle-of-fifths, positional, string-similarity and hand-span feasibility) build musical and ergonomic priors into training. The results are large:
+
+| Dataset | Noise2Fret pitch F / Tab F | TabCNN Tab F | FretNet Tab F |
+|---|---|---|---|
+| GuitarSet | 0.856 / **0.845** | 0.717 | 0.727 |
+| GOAT (mostly distorted electric) | 0.800 / **0.795** | 0.656 | 0.662 |
+
+The model is also cheap: about 15M parameters and **341M FLOPs per pass, against 3.4B for TabCNN and 17.4B for FretNet** ([arXiv 2608.30854](https://arxiv.org/html/2608.30854)). The likely reason it wins is that one pitch can be fretted in several places, so the fingering distribution has several peaks, and a generative model can represent that where a classifier averages across them. Two caveats limit what this means for us. First, both test sets are close to solo DI guitar. No numbers exist for separated stems from dense mixes. Second, **the code licence is not stated** (the paper text is CC BY-SA 4.0), so Noise2Fret is cleared for benchmarking only until we get an answer.
+
+### Weak supervision is the most mature route to real-mix training data
+
+Three methods turn imperfectly aligned scores into training signal. Riley et al. align Guitar Pro scores to a pretrained model's activations with DTW and then snap each onset to the local activation peak. That reaches **98% onset F1 at 50 ms, against 62% for DTW alone**. About 4 hours of aligned commercial jazz guitar then set a zero-shot GuitarSet record of **87.3 onset F1 against 82.9** ([arXiv 2402.15258](https://arxiv.org/html/2402.15258v1)). CountEM (ISMIR 2025) goes further and drops local alignment entirely, training EM on note-occurrence histograms. A search snippet reports that fine-tuning on GAPS raised GuitarSet accuracy from **66.2% to 84.6%** ([arXiv 2511.14250](https://arxiv.org/pdf/2511.14250)). Histogram supervision tolerates tabs whose repeats or sections do not match the recording, and user-supplied GP files often have exactly that problem. A 2026 cycle-consistency method reaches 64.81 GuitarSet frame F1 from almost no paired data. Without any pairs, however, it can converge on pitch-shifted degenerate solutions ([arXiv 2605.24193](https://arxiv.org/html/2605.24193v1)). Our reference tabs plug directly into the Riley and CountEM recipes, which is the basis for hypothesis H16c below.
+
+### Foundation encoders help multi-instrument models, and small models keep improving
+
+The 2025 AMT Challenge winner, MIROS, put a **MusicFM** self-supervised encoder into YourMT3+ and added parallel per-instrument-group decoders. It scored **0.600 note-onset F1 against 0.393 for MT3** ([arXiv 2603.27528](https://arxiv.org/html/2603.27528v1)). The same paper shows why the separation front-end still matters: MIROS precision fell from **0.907 on solo pieces to 0.464 on three-instrument pieces**. At the other end of the size range, Harmonica-nano (26.3K parameters) claims frame F1 **14.6 points above Basic Pitch** at 1,622× real time ([arXiv 2609.04640](https://arxiv.org/abs/2609.04640)). No weights are released, but it suggests the browser bottleneck is the separator, not the note model. Neither paper isolates what an SSL encoder contributes on guitar or bass. Testing a MusicFM or MERT head on our stems would therefore produce new evidence, and that test should wait until the encoder licences are verified.
+
+### General audio-language models remain unusable for note-level transcription
+
+Every source agrees on this. Gemini models score near ceiling when given MIDI, but "audio reduced accuracy across tasks" ([arXiv 2510.22455](https://arxiv.org/pdf/2510.22455)). GPT-4o (55.9%) and Gemini 2.5 Pro (53.1%) sit barely above chance on performance-audio reasoning ([arXiv 2601.11968](https://arxiv.org/pdf/2601.11968)). On HMU-Bench, model rankings on notation and on audio do not correlate (Spearman −0.04) ([arXiv 2610.15773](https://arxiv.org/html/2610.15773)). The structural reason is frame rate: a Whisper-style encoder emits one frame per ~40 ms ([arXiv 2407.10759](https://arxiv.org/pdf/2407.10759)), which is too coarse for ±50 ms onsets in dense passages. LLMs still have a narrower job. They can propose *edit operations* on the symbolic score, such as changing a time signature or respelling a passage, and a deterministic validator in the style of [arXiv 2510.02746](https://arxiv.org/pdf/2510.02746) then applies and checks each edit. That safeguard is needed because LLM graders miss octave errors ([arXiv 2610.15773](https://arxiv.org/html/2610.15773v1)).
+
+### The notation side has better quantizers and better metrics than the baseline assumed
+
+A beat-conditioned T5 quantizer takes performance MIDI plus beat annotations, which is exactly what Beat This! produces. It reaches **92.1% onset F1 and 90.2% note-value accuracy on Leduc jazz guitar GP files** ([arXiv 2604.22290](https://arxiv.org/html/2604.22290v1)). It assumes a one-to-one note correspondence, so it must run after note cleanup, and no code has been released. For evaluation, a 2026 study with 106 listeners and 3,180 ABX judgements found that **playback metrics track human preference far better than notation edit distance**. CLEWS scored ρ=0.971 and DTW-chroma ρ=0.891. On the notation side, MUSTER's onset error rate reached ρ=0.84, against 0.38–0.50 for OMR-NED. The code is MIT ([arXiv 2608.04511](https://arxiv.org/html/2608.04511); [AMT-Dual-Eval](https://github.com/pingw220/AMT-Dual-Eval)). The study covers piano only, but it matters twice for us. It argues for adding a playback metric to the benchmark, and it supports render-and-compare (H1 below), at least at system level.
+
+## Licence corrections leave almost no commercially clean real guitar data
+
+The new notes overturn several assumptions in the baseline survey. **These corrections change what we can train on:**
+
+| Item | Baseline survey said | Corrected status | Source |
+|---|---|---|---|
+| **GuitarSet** | licence unverified | **CC BY-NC 4.0**. mirdata's "MIT" applies to the loader code only. | [Zenodo 1422265](https://zenodo.org/records/1422265) |
+| **GOAT** | CC BY 4.0 | **Research-only, data by request**. CC BY covers the paper, not the data, and some content is covers of songs the authors do not own. | [arXiv 2509.22655](https://arxiv.org/html/2509.22655v1) |
+| **DadaGP** | licence unverified | **Research-only, by email request**. The repo code is MIT; no data licence is stated. | [GitHub dadaGP](https://github.com/dada-bots/dadaGP); [arXiv 2107.14653](https://arxiv.org/pdf/2107.14653) |
+| **SynthTab** | public training data | **No licence stated.** It is derived from DadaGP and rendered with commercial Ample Sound plugins, so treat it as research-only. | [arXiv 2309.09085](https://arxiv.org/html/2309.09085v4); [synthtab.dev](https://synthtab.dev) |
+| **Noise2Fret** | no code found | **Code and models public**; code licence not stated. | [GitHub](https://github.com/RiccardoVib/Noise2Fret) |
+
+The practical consequence is that nearly every published guitar model, including any fine-tune we would start from, was trained on non-commercial data. NC licensors do sue. **Jamendo sued NVIDIA in June 2026, seeking about $20.3M**, over commercial training on CC BY-NC MTG-Jamendo; these are allegations so far ([Music Business Worldwide](https://www.musicbusinessworldwide.com/nvidia-hit-with-us-copyright-suit-claiming-it-trained-ai-models-on-winamp-subsidiary-jamendos-music-without-permission/)). The table below scores everything we might train on or ship. "Eval only" means internal benchmarking. Whether even that is safe under each licence needs a legal view, which this report does not give.
+
+| Asset | Licence / status | Train production weights? | Ship in product? | Risk | Source |
+|---|---|---|---|---|---|
+| Guitar-TECHS (electric guitar techniques, chords) | CC BY 4.0 | Yes | n/a | Low. Has no power chords. | [arXiv 2501.03720](https://arxiv.org/html/2501.03720v1) |
+| Slakh2100 (145 h rendered multitrack) | CC BY 4.0 | Yes | n/a | Low | [Zenodo 4599666](https://zenodo.org/records/4599666) |
+| E-GMD, Groove (drums) | CC BY 4.0 (mirdata lists Groove as BY-SA) | Yes | n/a | Low; Magenta is the authoritative source | [Magenta E-GMD](https://magenta.tensorflow.org/datasets/e-gmd); [Magenta Groove](https://magenta.tensorflow.org/datasets/groove) |
+| StemGMD | CC BY 4.0 (per HF card) | Yes | n/a | Low–medium | [HF stemgmd](https://huggingface.co/datasets/schismaudio/stemgmd/blob/main/README.md) |
+| GuitarSet | CC BY-NC 4.0 | **No** | n/a | High | [Zenodo](https://zenodo.org/records/1422265) |
+| IDMT-SMT-Guitar v2 | CC BY-NC-ND 4.0 | **No** (not even derivatives) | n/a | High | [Zenodo 7544110](https://zenodo.org/records/7544110) |
+| GOAT | research-only, on request | **No** | n/a | High | [arXiv 2509.22655](https://arxiv.org/html/2509.22655v1) |
+| DadaGP (and any tab LM trained on it: GTR-CTRL, ShredGP, ProgGP, MIDI-to-Tab) | research-only | **No** | **No** | High; weight licences not checked | [GitHub dadaGP](https://github.com/dada-bots/dadaGP) |
+| SynthTab | unlicensed, DadaGP-derived | **No** | n/a | High | [synthtab.dev](https://synthtab.dev) |
+| MAESTRO, MoisesDB, MedleyDB, M4Singer | CC BY-NC(-SA) | **No** | n/a | High | [MAESTRO](https://magenta.tensorflow.org/datasets/maestro); [moises-db](https://github.com/moises-ai/moises-db) |
+| MUSDB18-HQ | educational only | **No** | n/a | High | [Zenodo](https://zenodo.org/record/3338373) |
+| MTG-Jamendo (pseudo-label pool) | CC BY-NC dataset terms; per-track licences vary | Only per-track CC BY/CC0 subsets | n/a | High; active litigation | [UPF](https://repositori.upf.edu/items/e48c8c42-dba1-404d-85b7-5a0d6ef99e6d/full) |
+| GAPS, EGDB, Riley jazz set, Opencpop, MIR-ST500 | no licence found | Not until cleared | n/a | Unknown | [ISMIR 2024](https://ismir2024program.ismir.net/poster_171.html); [arXiv 2402.15258](https://arxiv.org/html/2402.15258v1) |
+| Noise2Fret code and weights | licence not stated; trained on GuitarSet/GOAT (NC/research) | Retrain on clean data | Not as published | High as published | [GitHub](https://github.com/RiccardoVib/Noise2Fret) |
+| MuScriptor weights | CC BY-NC 4.0, gated | **No** | **No** (Kyutai licence needed) | High; use as benchmark ceiling | [GitHub muscriptor](https://github.com/muscriptor/muscriptor) |
+| ADTOF | CC BY-NC-SA 4.0 | **No** | **No** | High | [ADTOF](https://github.com/MZehren/ADTOF) |
+| ADT_STR | CC BY-SA 4.0 | Share-alike probably attaches to derived weights | With SA obligations | Medium | [ADT_STR](https://github.com/pier-maker92/ADT_STR) |
+| MERT / MusicFM / MuQ encoders | not verified | Not until verified | Not until verified | Unknown | [arXiv 2603.27528](https://arxiv.org/html/2603.27528v1) |
+| Demucs, Basic Pitch | MIT / Apache-2.0 | Yes (as teachers or bases) | Yes | Low | [HF monteslu](https://huggingface.co/monteslu/htdemucs-web-onnx); [basic-pitch](https://github.com/spotify/basic-pitch) |
+| pedalboard, DawDreamer (rendering) | GPL-3.0 | Offline data generation is fine in the usual reading | **No** (keep out of the bundle) | Low offline | [PyPI pedalboard](https://pypi.org/project/pedalboard/0.9.2); [DawDreamer](https://github.com/DBraun/DawDreamer) |
+| NAM trainer / community NAM profiles | MIT / per-uploader terms | Trainer yes; profiles only if terms allow or we capture our own | n/a | Medium | [neural-amp-modeler](https://github.com/sdatkinson/neural-amp-modeler) |
+| Commercial sample libraries (Ample Sound etc.) | EULA silent on ML training | Not until the EULA is checked | n/a | Medium | [arXiv 2309.09085](https://arxiv.org/html/2309.09085v4) |
+| AMT-Dual-Eval metrics code | MIT | n/a | Yes | Low | [GitHub](https://github.com/pingw220/AMT-Dual-Eval) |
+| Our reference GP tabs and recordings | ours, but transcriptions of copyrighted songs | Internal use; ownership of the derivative needs legal review | Not as content | Medium | inference |
+| User corrections and uploads | needs explicit ToS consent | Only with consent | n/a | Medium | inference |
+
+The clean pool is therefore Slakh2100, E-GMD, Groove, StemGMD, Guitar-TECHS, our own recordings, and anything we render ourselves from material we hold rights to. Distilling from NC teachers such as MuScriptor is legally untested, so we treat those teachers as benchmark-only. For guitar specifically, this makes the coordinator's procedural-data idea (H3) close to mandatory.
+
+## Sixteen team hypotheses, judged against the evidence
+
+The ideas below are **untested hypotheses from our team, not literature results**. Each is linked to the nearest evidence in the notes. "Verdict" is our judgement of expected value against effort.
+
+| # | Team hypothesis | Closest evidence | Verdict |
+|---|---|---|---|
+| H1 | Render-and-compare per-bar error heatmap; re-decode the worst bars | Drexel resynthesis training; Inverse Drum Machine; playback metrics ρ≈0.89–0.97 at system level | **Phase 2.** Use it to *rank* candidates rather than as an absolute error score. |
+| H2 | Per-song tone-matched test-time fine-tune | No AMT test-time adaptation found; GOAT shows tone diversity saturates | **Defer.** Measure tone sensitivity first. |
+| H3 | Licence-clean procedural tabs → render → train (also for a guitar splitter) | SynthTab sim-to-real gap; GOAT re-amping gains | **Necessary, high effort.** Pair it with separator-in-the-loop. |
+| H4 | Pan clustering, double-track merge, role classifier, symbolic part splitter | ADRess/DUET; mixing conventions; note-level timbre clustering | **Phase 1–2**, high value |
+| H5 | Tuning/capo inference by decoder search, cross-checked with bass | DLfM 2024 tuning classifier; drop-D ambiguity | **Phase 1**, cheap |
+| H6 | Bass and chords constrain guitar/keys | Bass is the cleanest pitched stem (14.62 dB) | **Phase 1** as a soft prior only |
+| H7 | Two-mode guitar: chord+strum for rhythm, notes for lead | Power-chord templates; MERT strum detection | **Phase 2.** Needs its own metric. |
+| H8 | Repetition consolidation as a free ensemble | Chorus pooling; non-local statistics; drum repetition LMs | **Phase 1**, top value |
+| H9 | Note-ROVER across separators × transcribers | No published note-level ROVER; separator ensembles gain tenths of a dB | **Phase 2**, mainly as confidence |
+| H10 | Iterative peeling / score-informed separation | Jointist; Lin et al.; no measured guitar gain | **Defer** |
+| H11 | Diffusion n-best + LM/playability/render rescoring | Noise2Fret; ASR-style rescoring proposed, unpublished | **Phase 2**, after H1 and the Noise2Fret baseline |
+| H12 | Monophonic F0 contour for bends/vibrato/slides | Vibrato confused with bends in TART; contour inference | **Phase 1**, cheap |
+| H13 | Up-front user hints | Every research system takes tuning as input | **Phase 1** |
+| H14 | "Wrong bar → alternatives" UI, correction logging, time-to-correct | Holzapfel & Benetos; ENSEMBLE; ASR confidence markup | **Build alongside**; it is the product metric |
+| H15 | Audio-visual fingering from playthrough videos | Audio-visual *piano* AMT only | **Park.** Speculative and legally exposed. |
+| H16 | Reference GP tabs for (a) eval, (b) synthetic twins, (c) Riley-style fine-tuning | Riley 98% alignment; SynthTab gap | **Phase 0** for (a)+(b); Phase 3 for (c) |
+
+### Search over hypotheses is cheap and attacks errors the transcriber cannot fix
+
+**H5 (tuning/capo search)** is the best ratio of value to effort in the list. A wrong tuning makes every fret wrong even when every pitch is right. The evidence base is thin. A DLfM 2024 network classifies five tunings, including standard (90 songs) and drop D (86), but its accuracy figures were not retrieved ([ACM DL](https://dl.acm.org/doi/fullHtml/10.1145/3660570.3660574)). A 2023 student project found drop D hard to tell apart from E standard ([Rochester report](https://hajim.rochester.edu/ece/sites/zduan/teaching/ece477/projects/2023/DeclanParker_McCormackChew_FinalReport.pdf)). The proposed search needs no training. First estimate the global cents offset to catch Eb tuning. Then use the lowest sustained guitar pitch to narrow the candidate tunings. Finally run the fingering decoder under each candidate and keep the one with the lowest total cost. One-finger power chords on the low three strings are a strong drop-tuning signature. The baseline survey called capo "effectively unobservable", and that remains true acoustically: a capo-2 G shape and an open A shape sound almost identical. Capo inference is therefore a *notation-convention* prediction, and it is only as good as the fingering prior. Our references will tell us how often human tabbers write a capo at all. The bass cross-check is sound (a low D1 on bass supports drop D), as long as it stays a vote and not a veto.
+
+**H13 (user hints)** pairs naturally with H5. The benchmark should measure the *value of a hint* as the gap between Tab F1 with oracle tuning, number of guitars and lead side and Tab F1 with auto-detection. If that gap is small, we skip the questions and keep the upload flow frictionless. If it is large, we ask, pre-filled with the detected value.
+
+**H11 (diffusion n-best plus rescoring)** follows from Noise2Fret: repeated sampling gives multiple candidate tabs per segment at no extra modelling cost. The rescoring terms are the open question. The tab-LM shallow-fusion idea (log P_audio + λ·log P_LM) has **no published evidence for audio tab transcription** ([arXiv 2609.11904](https://arxiv.org/html/2609.11904v1) is the nearest, a single audio-conditioned T5). Historically, music LMs gave "only small gains" in AMT ([arXiv 2010.00059](https://arxiv.org/pdf/2010.00059)). The available LMs are DadaGP-trained, so they are research-only. A shipped rescorer would need an LM trained on clean data, which couples H11 to H3. The safest, licence-free term is pitch-validity constrained decoding, which masks string/fret outputs that would produce the wrong pitch ([arXiv 2607.26440](https://arxiv.org/pdf/2607.26440)). We should adopt it regardless of the H11 outcome.
+
+### Redundancy inside the song is a free ensemble
+
+**H8 (repetition consolidation)** is the hypothesis the literature most supports in spirit and least in numbers. Pooling repeated choruses improved lyric transcription accuracy and consistency ([McVicar, Ellis, Goto](https://staff.aist.go.jp/m.goto/PAPER/ICASSP2014mcvicar.pdf)). Non-local self-similarity fixes meter and bar-line errors in piano transcription ([arXiv 2008.12710](https://arxiv.org/pdf/2008.12710)). Drum transcribers regularised by bar-periodic LMs improved on ENST and RWC ([arXiv 2010.03749](https://arxiv.org/pdf/2010.03749)). Riff-based rock and metal are the best case for voting. The main failure mode is erasing real variation, such as the turnaround in the fourth repeat or a fill. Voting should therefore *propose* rather than overwrite. Where an instance disagrees with the cluster majority on a confident note, keep it and flag it. Consistent fingering across repeats is a second, readability win that Tab F1 will not capture, so we measure it directly.
+
+**H6 (cross-instrument consistency)** rests on a solid fact: at **14.62 dB SDR the bass stem is the cleanest pitched stem** from BS-RoFormer SW ([MVSEP news 64](https://mvsep.com/news/64?lang=en)), and in metal the bass usually doubles the rhythm-guitar root. The risk is genre. Funk, pop and jazz bass lines walk, play inversions and pedal, so a hard constraint would corrupt correct guitar notes. Bass roots and BTC chord labels should enter as a soft emission bonus inside the decoder. Clashes should be flagged rather than deleted, the same rule the notes recommend for out-of-key filtering. The benchmark should report the effect separately per genre tag.
+
+**H9 (note-ROVER)** has no precedent: no 2023–26 note-level multi-model voting for music was found ([EURASIP 2022, instrument-level voting only](https://link.springer.com/article/10.1186/s13636-022-00245-8)). The separator half of the ensemble promises little. MVSEP shows overlap 2→8 moving SDR from 11.75 to 11.77 ([MVSEP 128](https://hk.mvsep.com/algorithms/128)), and the Demucs shift trick adds roughly 0.2 dB at N× compute ([arXiv 1911.13254](https://arxiv.org/pdf/1911.13254)). The transcriber half is more promising, because Basic Pitch, Noise2Fret and a mix-level model fail in different ways. Even so, models running on the same stem share that stem's artefacts, so their errors correlate. We expect H9's main payoff to be **calibrated confidence** (agreement count) rather than raw F1. It is server-tier only, because every extra model multiplies browser compute.
+
+### Verification beats blind acceptance, but absolute spectral distance is a weak judge
+
+**H1 (render-and-compare)** has real support. Resynthesis has trained guitar tab models on 27+ hours of unlabeled YouTube audio ([Drexel thesis](https://researchdiscovery.drexel.edu/view/pdfCoverPage?instCode=01DRXU_INST&filePid=13549920770004721&download=true)). The Inverse Drum Machine learns separation from transcription labels alone ([arXiv 2505.03337](https://arxiv.org/abs/2505.03337)). Playback-similarity metrics rank whole systems in line with human preference ([arXiv 2608.04511](https://arxiv.org/html/2608.04511)). Bar-level reliability, however, is untested. A soundfont through a rough amp model will never match a real distorted stem, and the stem itself contains separation bleed. An absolute per-bar distance therefore mixes timbre mismatch with note errors. Our refinement is to use H1 *comparatively*: render several candidate transcriptions of the same bar and keep the one closest to the stem. The timbre error is then roughly constant across candidates and largely cancels. The distance features should be timbre-robust (log-CQT after spectral whitening, chroma, onset envelopes). The first measurement is whether per-bar distance predicts per-bar error on the references (AUROC), before anything is re-decoded. If it does not, H1 becomes a candidate ranker inside H11 and not a heatmap.
+
+**Confidence calibration** ties H1, H9 and H11 together. Dedicated work on note-level confidence in music transcription barely exists. TCP_α offers post-hoc confidence heads for frozen MIR models, framed as failure prediction under domain shift ([arXiv 2608.20326](https://arxiv.org/html/2608.20326v1)). In audio captioning, confidence-aware reranking improved both quality and calibration ([arXiv 2512.10170](https://arxiv.org/pdf/2512.10170)). We would combine the signals (posterior, model and sample disagreement, render residual, leakage into other stems, rule flags) into one score per note and per bar. That score would be calibrated on the dev split and judged by *how many reference errors land in the flagged bars*, which is what the user experiences.
+
+**H10 (peeling)** is the least practical idea here. Subtracting a resynthesised guitar from a mix needs a close match in timbre and phase, and a soundfont-plus-amp render cannot provide one for distorted guitar. Score-informed masking is the realistic variant. Older joint models report that transcription helps separation ([arXiv 2302.00286](https://arxiv.org/html/2302.00286); [arXiv 2108.03456](https://arxiv.org/abs/2108.03456)), but no 2024–26 guitar result reports both SDR and F1. Community "residual" multi-pass separation has no measured transcription benefit either. We defer H10 until H1's distance proves informative and H4's pan clustering has been tried, because pan clustering is a much cheaper way to reveal a hidden second guitar.
+
+### Multi-guitar handling remains unsolved
+
+**H4** targets the requirement that users notice most. Heise found Songsterr AI unable to always separate guitar parts ([heise](https://heise.de/-10356152)), and no research system assigns notes to multiple guitar parts in full mixes. The building blocks are old but sound. Avendano's panning index and ADRess find instruments by azimuth ([arXiv 1804.08300](https://arxiv.org/pdf/1804.08300); [Barry et al.](https://dafx.de/paper-archive/2004/P_240.PDF)). Mixing convention hard-pans double-tracked rhythm guitars and places the lead near centre ([Producer Society](https://producersociety.com/how-to-pan-guitars/)). Nobody has measured how often commercial mixes follow that convention, and the benchmark should report it from our corpus. Double-track merging has a further advantage: a hard-L/hard-R pair with near-identical notes is two noisy observations of one part, so merging is itself a two-voter ensemble *and* matches how humans tab. Where guitars share a pan position, the 2025 note-level timbre-clustering model ([arXiv 2509.12712](https://arxiv.org/html/2509.12712v3)) and the GuitarDuets joint framework ([arXiv 2507.01172](https://www.arxiv.org/pdf/2507.01172)) are the nearest research. The symbolic part splitter, which merges multi-track tabs and learns to split them, is clever free supervision. On DadaGP, though, it is research-only, so a shippable version must train on H3's procedural multi-guitar arrangements, where pan and role are known by construction. Mono, live and centre-stacked mixes will defeat the pan stage, so the role classifier (monophonic, high register, bends → lead; chordal, palm-muted, repetitive → rhythm) has to work on its own as well.
+
+**H7 (two-mode transcription)** turns a weakness into a readability win. High-gain rhythm guitar uses a handful of shapes, and power-chord templates for distorted guitar already exist ([GitHub ACR](https://github.com/Driemtax/ACR)). Strum and rhythm detection in polyphonic music works with MERT features ([arXiv 2510.05756](https://arxiv.org/html/2510.05756v1)). The catch is measurement. Our GP references are note-level and record the voicing actually played, octave doublings included, so a canonical two-note power chord will *lose* exact Tab F1 even when a guitarist would call it correct. H7 needs a chord-level metric (root + type + onset per beat against the reference reduced to chords) and an edits-to-correct measure, with Tab F1 reported but not used to decide. Guitar-TECHS has no power chords ([Zenodo](https://zenodo.org/records/14963133)), so the chord model's training data has to be rendered (H3).
+
+**H12 (pitch-contour techniques)** is cheap, browser-feasible and well motivated. TART's technique classifier is weakest on vibrato (89.4%) because it confuses vibrato with bends ([arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)), and older solo-guitar bend detection ranged from about 58% to 88% depending on technique ([ISMIR 2015](https://www.ismir2015.uma.es/articles/119_Paper.pdf)). An F0 contour separates the two by shape: a bend is a sustained rise, vibrato a periodic oscillation. Pitch survives distortion better than spectral shape does, although that is our inference, not a measurement. FCPE matches RMVPE's accuracy at 5× the speed ([arXiv 2509.15140](https://arxiv.org/html/2509.15140v1)), and PESTO runs a 3-minute file in about 13 s on a laptop CPU ([PESTO](https://github.com/SonyCSLParis/pesto)). Both licences still need checking. H12 only works on monophonic lead material, so it depends on H4 to isolate the lead.
+
+### Data and adaptation ideas carry the biggest uncertainty
+
+**H3 (licence-clean procedural data)** is forced by the licence table, but the evidence warns against expecting it to be sufficient. TabCNN trained only on SynthTab scored **43.1 Tab F1 on GuitarSet against 80.1 in-domain**, and 13.8 on IDMT ([arXiv 2309.09085](https://arxiv.org/html/2309.09085v4)). GOAT's NAM re-amping, drawing on about 7,000 profiles from more than 1,000 amps, raised note F1 on distorted test audio from **0.714 to 0.800**, but doubling the augmentation added nothing (0.806) ([arXiv 2509.22655](https://arxiv.org/html/2509.22655v1)). Two refinements raise H3's odds. First, mix the rendered guitars with rendered bass, drums and keys, and **run the mix through our actual separator before training**, so the model learns separator artefacts. This is plausibly the biggest gap in our pipeline, though nothing in the notes measures it. Second, keep acoustic realism and musical realism apart. Procedural riffs teach the acoustic model what notes sound like, but an LM trained on random riffs encodes random statistics, so H3 data should not be used to train H11's rescoring prior unless the generator is genuinely idiomatic. GuitarFlow-style flow-matching renderers ([arXiv 2510.21872](https://arxiv.org/pdf/2510.21872)) and string-wise DDSP guitars ([arXiv 2309.07658](https://arxiv.org/pdf/2309.07658)) could narrow the timbre gap later. Capturing our own NAM profiles avoids per-uploader licence terms.
+
+**H2 (tone-matched test-time adaptation)** is the weakest data idea. No AMT-specific test-time adaptation method was found ([arXiv 2609.01679](https://arxiv.org/pdf/2609.01679) warns of confirmation bias in self-training). Unsupervised domain adaptation did not help a piano model under microphone shift ([Fraunhofer](https://publica.fraunhofer.de/handle/publica/486541)). The GOAT AMP-XL saturation suggests that broad tone diversity already captures most of the benefit. Per-song fine-tuning also costs GPU minutes per upload and cannot run in the browser. Before building anything, run the diagnostic H16b makes possible: render each reference tab in a matched tone and in a clean tone, and compare scores. If the gap is small, tone is not the bottleneck and H2 is dead. If it is large, the cheaper versions come first: picking among a few tone-bucket fine-tunes, or adapting only normalisation statistics.
+
+**H16** gives the reference corpus three jobs, and they conflict. Every song used for fine-tuning (c) or for tuning thresholds and λ weights is spent as test data. We therefore propose a **frozen test split that nothing touches**, a dev split for tuning, and k-fold, leave-one-artist-out cross-validation for any fine-tuning, so that songs by the same band do not leak between folds. **Synthetic twins (b)** are the most underrated element. Rendering each reference GP through our renderer, then mixing and separating it, produces audio with exact ground truth for every stage. Comparing a song's twin with its real recording then splits each error into "rendering gap" and "model error", the per-stage attribution that TART argues for ([arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)). For (c), the Riley recipe (LR 1e-5, 10 s excerpts, about 100K steps) worked at about 4 hours of audio ([arXiv 2402.15258](https://arxiv.org/html/2402.15258v1)). Below about 1 hour it is unproven, and SynthTab saw small real-data fine-tunes sometimes *lower* scores ([arXiv 2309.09085](https://arxiv.org/html/2309.09085v4)). Expect H16c to pay off only once user-consented (recording, GP) pairs accumulate through H14.
+
+### The editor is both a product feature and the final metric
+
+**H14** is where all the confidence work reaches the user. The evidence that editing beats transcribing from scratch is modest and dated. Editing saved time for some participants, and mainly improved quality for less-skilled transcribers ([ISMIR 2019](https://archives.ismir.net/ismir2019/paper/000082.pdf); [DSH 2022](https://academic.oup.com/dsh/article/37/3/747/6408955)). Note-level correction is laborious: ENSEMBLE logged **53,411 actions, about 60 per 3-second clip** ([ISMIR 2016](https://archives.ismir.net/ismir2016/paper/000072.pdf)). The design implications are bar-level alternatives (drawn from H11 samples and H9 voters) rather than note-by-note editing, bulk "apply to every repeat" actions built on H8 clusters, and confidence highlighting modelled on ASR correction UIs ([arXiv 2503.15124](https://arxiv.org/pdf/2503.15124)). Logged corrections need explicit consent before they become training data. They are also biased toward flagged regions, so they are a poor source for estimating overall error rates.
+
+**H15 (audio-visual fingering)** is the most speculative idea. The notes contain only an audio-visual *piano* transcription paper ([TASLP 2024](https://dl.acm.org/doi/pdf/10.1109/TASLP.2024.3426303)) and nothing on guitar fretboard tracking. Video makes the YouTube acquisition risk worse: yt-dlp needs PO tokens that are failing, and youtube-dl drew an RIAA takedown under DMCA §1201 ([yt-dlp PO Token Guide](https://github.com/yt-dlp/yt-dlp/wiki/PO-Token-Guide); [XDA](https://www.xda-developers.com/youtube-dl-riaa-dmca/)). Camera angles, occlusion and edits vary from video to video. One legally clean variant is worth noting: the user's *own* webcam, in a later "play along and we fix your tab" mode. We park H15.
+
+## Ranked experiments: cheap symbolic search first, training last
+
+Each experiment below runs on the same benchmark. That benchmark parses the GP references with tuning, capo and repeats unrolled, aligns them to audio with beats plus DTW plus onset snapping, matches tracks with the Hungarian algorithm, and reports alignment failures separately. The baseline is the survey's **Basic Pitch + Viterbi assigner on BS-RoFormer stems**, with the Demucs stem as a second baseline. Success criteria are our proposed decision thresholds, not literature values, and are evaluated on the frozen test split after tuning on dev. Rank reflects expected Tab F1 or time-to-correct gain divided by engineering effort.
+
+| Rank | Experiment (hypothesis) | Effort | What to measure | Primary metric(s) | Baseline | Success criterion (proposed) |
+|---|---|---|---|---|---|---|
+| 1 | **E1 Noise2Fret drop-in** on separated guitar stems and on synthetic twins | Low (code public; licence check gates shipping) | Tab and note accuracy by gain level (clean/crunch/high-gain) | Tab F1, note F1 (±50 ms), TDR, lenient Tab F1 | BP + Viterbi | ≥ +5 Tab F1 points overall and no loss on clean material. Also confirms whether the GOAT gain survives separation. |
+| 2 | **E2 Tuning/capo search + hint value** (H5, H13) | Low | Tuning/capo classification; Tab F1 under detected vs oracle tuning | Tuning accuracy, capo accuracy, ΔTab F1 | Assume E standard, no capo | ≥ 90% tuning accuracy; Tab F1 within 1 point of oracle tuning. If not, ship the hint UI. |
+| 3 | **E3 Repetition consolidation** (H8) for guitar, bass, drums | Low–medium | Accuracy on repeated vs non-repeated bars; fingering consistency across repeats; variations wrongly erased | Note/Tab F1 on repeated bars, per-class drum F1, repeat-consistency %, false-overwrite rate | No consolidation | ≥ +3 F1 on repeated bars, ≤ 0.5 loss on non-repeated, false overwrites < 5% of changed notes |
+| 4 | **E4 F0-contour techniques** (H12) on the lead stem or cluster | Low | Bend, vibrato, slide detection vs GP technique flags | Per-technique P/R/F1; bend/vibrato confusion | Basic Pitch pitch-bend output | Bend F1 ≥ 0.7 and vibrato↔bend confusion halved |
+| 5 | **E5 Pan clustering + double-track merge + role classifier** (H4) | Medium | Whether we produce the right guitar tracks with the right notes | Hungarian track-match rate, per-track note F1, guitar-count accuracy, lead/rhythm role accuracy; % of corpus following the hard-pan convention | One "guitar" track | Correct guitar count in ≥ 75% of multi-guitar songs; matched-track note F1 ≥ single-stem F1 |
+| 6 | **E6 Bass/chord soft prior** (H6) | Low | Power-chord root errors; spurious clashing notes; per-genre effect | Note F1 on rhythm guitar and keys, root-error rate, clash-flag precision | No prior | ≥ 30% fewer root errors on rock/metal, no F1 loss on other genres |
+| 7 | **E7 Confidence stack**: render-and-compare residual (H1) + model agreement (H9) + calibration | Medium | Whether flags find the errors | Bar-level AUROC for "bar contains an error"; % of errors in top-20% flagged bars; ECE after calibration | Transcriber posterior only | AUROC ≥ 0.75; top-20% of bars capture ≥ 50% of errors |
+| 8 | **E8 Note-ROVER accuracy** (H9) | Medium | Whether voting beats the best single system | Note F1, precision at fixed recall, compute ×RT | Best single transcriber | ≥ +2 F1 at ≤ 3× compute, else keep for confidence only |
+| 9 | **E9 Diffusion n-best + rescoring** (H11, pitch-constrained decoding) | Medium (after E1, E7) | Gain from choosing among K samples | Tab F1 vs K; oracle-best-of-K (upper bound) | Single Noise2Fret sample | Rescored ≥ +2 Tab F1 and closing ≥ 30% of the gap to oracle-best-of-K |
+| 10 | **E10 Two-mode rhythm transcription** (H7) | Medium | Readability and correctness of rhythm sections | Chord-level F1 (root+type+onset), strum-direction accuracy, edits-to-correct; Tab F1 reported | Note-level for all sections | Chord F1 ≥ 0.8 and fewer edits per rhythm bar than note mode |
+| 11 | **E11 Sim-to-real attribution with synthetic twins** (H16b) and tone sensitivity (gates H2) | Medium | Gap between twin and real recording per stage; matched vs clean tone | ΔTab F1 (twin − real) per stage; ΔF1 (matched − clean tone) | n/a (diagnostic) | Produces a per-stage error budget. If tone ΔF1 < 3 points, drop H2. |
+| 12 | **E12 Separator-in-the-loop procedural training** (H3) | High | Commercial-clean model vs NC-trained model | Tab F1 on real references; the same on twins | Noise2Fret / BP as published | Clean model within 5 Tab F1 points of the NC-trained one |
+| 13 | **E13 Fine-tune on aligned references / CountEM** (H16c) | Medium–high | Gain from in-domain real audio | Tab F1 on held-out folds; variance across folds | Pre-fine-tune model | ≥ +3 Tab F1 consistently across folds (k-fold, leave-one-artist-out) |
+| 14 | **E14 SSL-encoder head** (MusicFM/MERT) for guitar and bass | High; licence-gated | Encoder vs Basic Pitch features | Note F1, Tab F1 | BP / Noise2Fret | ≥ +5 note F1 on high-gain material |
+| 15 | E15 Score-informed peeling (H10); E16 tone TTA (H2); E17 audio-visual (H15) | High | n/a | n/a | n/a | Start only if E7/E11 justify it |
+
+Two cross-cutting items run with every experiment. Notation quality is scored with MUSTER-style onset and note-value error rates alongside Tab F1. A playback metric (alphaTab render vs stem via DTW-chroma or CLEWS) is also computed; it needs no reference alignment and therefore covers songs where GP alignment fails ([arXiv 2608.04511](https://arxiv.org/html/2608.04511)). Drums get E3's bar-pattern voting, and keys get E6's harmony prior and E8's voting; both have their own per-instrument metrics.
+
+## A four-phase roadmap folds these experiments into the Guitar Pro benchmark
+
+The ordering follows three principles. Each phase's measurement infrastructure is reused by the next. Post-processing experiments run on a frozen transcriber, so their effects can be attributed. Training work starts only after the cheap wins and the error budget show where training would help.
+
+| Phase | Weeks (indicative) | Build / run | Exit gate |
+|---|---|---|---|
+| **0: Harness** | 1–3 | GP7/8 import check (a week-one risk from the survey); reference expansion; beat + DTW + onset-snap alignment (Riley-style); Hungarian track matching; metric suite (Tab/note F1, TDR, lenient Tab F1, MUSTER, playback DTW-chroma); corpus tags (genre, gain, guitar count, pan, tuning, capo); **frozen test / dev / k-fold splits**; synthetic-twin rendering of every reference; baseline runs on Demucs and RoFormer stems, mix vs stem, oracle-notes assigner | Baseline numbers per stage; alignment success rate reported; twins render cleanly |
+| **1: Symbolic wins on a frozen transcriber** | 3–6 | E2 tuning/capo + hint value; E3 repetition; E4 F0 techniques; E6 bass/chord prior; pitch-validity constraint in the assigner; start editor event logging (H14) | Keep every change that clears its criterion; publish the first internal accuracy table |
+| **2: Model swap, multi-guitar, confidence** | 5–10 | E1 Noise2Fret (licence request sent in Phase 0); E5 multi-guitar; E7 confidence stack and calibration; E8 note-ROVER; E9 n-best rescoring; E10 two-mode; E11 sim-to-real and tone-sensitivity diagnostics | Low-confidence-bar UI backed by measured AUROC; decision on whether the server tier runs ensembles; error budget per stage |
+| **3: Clean training** | 9–16 | E12 procedural + separator-in-the-loop data (own NAM captures, Slakh-style backing); retrain a clean Noise2Fret-class model and a lead/rhythm splitter; E13 CountEM/Riley fine-tune once consented pairs exist; E14 encoder probe if licences clear; distil to a browser student (fp16 WebGPU, test int8) | Clean model within 5 points of the NC-trained one; browser student within the planned budget |
+| **4: Speculative** | gated | E15 peeling, E16 tone TTA, E17 audio-visual. Start only if E7/E11 show the specific bottleneck each addresses. | Evidence-led go/no-go |
+
+Two decisions should be made in Phase 0, because they gate later phases. The first is to request licence terms from the Noise2Fret authors and to check the MERT/MusicFM licences. The second is to get a legal view on our own GP references (internal use, and whether they may later train weights) and on collecting user corrections under the ToS. A related pitfall applies to every experiment from Phase 1 on: λ weights, thresholds and calibration are fitted on dev, and the test split is touched once per phase.
+
+## Conclusion
+
+The research picture has inverted since the baseline survey. Then, the question was which transcriber to adopt. Now one strong generative model exists and is available for testing, and the larger unclaimed gains lie in exploiting structure the transcriber ignores. The constraint has moved from model quality to licensing and measurement. The best public guitar data is unusable for production weights, and almost none of the post-processing ideas have published effect sizes. Two consequences follow. First, our Guitar Pro benchmark, with synthetic twins for attribution and a frozen test split, is research infrastructure that the field lacks, not only QA. Second, the confidence stack carries more weight than any single accuracy trick. A system that is 70% right and says *which* 30% to check gives users more than one that is 75% right but silent about where it is wrong.
+
+The coordinator's ideas sort along one line: those that use evidence already in the song are cheap and testable now (tuning and capo search, repeats, double-tracks, bass roots, F0 contours), while those that generate new evidence (adaptation, peeling, video) are expensive and unproven. Render-and-compare sits between the two groups, and its value depends on a reframing: as a ranker among candidates it should work even though a soundfont never sounds like a real amp, whereas as an absolute error detector it probably will not. If Phase 1 shows the symbolic wins we expect, the product differentiator becomes the published accuracy table, together with an editor that learns from its users' corrections and gives the user a short list of alternatives instead of a blank bar.

research_notes/Music transcription tooling survey/browser_and_commercial.md

@@ -0,0 +1,174 @@
+# In-browser music transcription feasibility and the commercial / open-source landscape (as of Oct 2026)
+
+Note on sourcing: about 19 search/fetch calls. Many results were aggregator or competitor-written pages. These are flagged inline. Anything not backed by a fetched source is in Gaps or marked as an inference.
+
+## A1. Browser ML runtimes (ONNX Runtime Web, transformers.js, TF.js, LiteRT.js, WebNN) and WebGPU support
+
+### Takeaway
+WebGPU now ships in all major browsers: Chrome/Edge 113+, Chrome Android 121+, Safari 26 (Sept 2025, including iOS), and Firefox 141 on Windows and 145 on Apple Silicon macOS. Firefox on Linux/Android and Intel Macs is still partial. ONNX Runtime Web with the WebGPU EP, directly or through transformers.js v4, is the practical runtime for audio models. Google's LiteRT.js (July 2026) is a new option. WebNN is not usable in production: it is behind flags and its origin trial was paused.
+
+### Cited Findings
+- WebGPU is supported in all major browsers as of November 2025 — [Wikipedia](https://en.wikipedia.org/wiki/WebGPU); [web.dev](https://web.dev/blog/webgpu-supported-major-browsers)
+- Chrome/Edge have supported WebGPU since 113. Chrome Android has it since 121, on Android 12+ devices with Qualcomm/ARM GPUs — [Wikipedia](https://en.wikipedia.org/wiki/WebGPU)
+- Firefox: WebGPU on Windows from Firefox 141 and on macOS Tahoe 26 ARM64 from Firefox 145. Linux, Android and Intel Macs are "in progress" — [web.dev](https://web.dev/blog/webgpu-supported-major-browsers)
+- Safari 26 (Sept 2025) shipped WebGPU on macOS, iPadOS, iOS and visionOS. Because all iOS browsers use WebKit, this covers every iOS browser. Some blogs cite earlier Safari versions, which conflicts with this — [Wikipedia](https://en.wikipedia.org/wiki/WebGPU)
+- Chromium Compatibility Mode for older Android (GLES 3.1) is described as a 2026 change — [Khronos SIGGRAPH 2026 deck link](https://www.khronos.org/developers/linkto/webgl-and-webgpu-sigg26). This is a secondary summary and was not fetched.
+- transformers.js v4: preview on npm `next` on Feb 9, 2026. The WebGPU backend was rewritten in C++ with the ONNX Runtime team. GA was reportedly March 2026, and 4.2.0 and 4.3.0 have since shipped (4.3.0 includes a "WebGPU upgrade") — [GitHub releases](https://github.com/huggingface/transformers.js/releases); [newreleases 4.3.0](https://newreleases.io/project/github/huggingface/transformers.js/release/4.3.0); [awesomeagents summary](https://awesomeagents.ai/news/transformers-js-v4-webgpu-browser-ml/). The GA date comes from secondary sources.
+- LiteRT.js was announced by Google on July 9, 2026. It uses XNNPACK for CPU and ML Drift for GPU (WebGPU), with WebNN for NPUs listed as upcoming — [Google Developers Blog](https://developers.googleblog.com/en/litertjs-googles-high-performance-web-ai-inference/); [Gigazine](https://gigazine.net/gsc_news/en/20260710-google-litert-js/)
+- WebNN origin trial: planned for M145–147, delayed, then moved to M147–149. A later post says it was disabled again because of release-blocking issues, with a new milestone to be announced — [blink-dev Intent to Experiment](https://groups.google.com/a/chromium.org/g/blink-dev/c/5CWKSChYo98). A Feb 2026 Phoronix report of an origin trial in Chrome 146 is outdated — [Phoronix](https://www.phoronix.com/news/Chrome-146-Beta)
+- The WebNN spec was at W3C Candidate Recommendation Draft stage as of its June 26, 2026 revision — [kingy.ai](https://kingy.ai/news/litert-js-local-browser-ai-backends-benchmarks-limits/). Secondary source.
+
+### Inferences
+- Target ORT-Web with WebGPU as the main path and the WASM EP as the fallback. Plan for a WASM-only path on Firefox/Linux, Firefox Android and older Android devices.
+- TF.js was not researched separately. Basic Pitch's official TS port uses TF.js, so it remains relevant for that model, but there are ONNX re-exports (see A2).
+
+### Gaps
+- No caniuse share figure for WebGPU was fetched.
+- TF.js maintenance status in 2026 was not checked.
+- MediaPipe Audio tasks were not checked.
+
+## A2. Existing in-browser audio ML projects
+
+### Takeaway
+Browser-side source separation is now proven. There are several 2025–26 HTDemucs ports on WebGPU, and the best of them run many times faster than realtime on a mid-range desktop GPU. Basic Pitch runs in the browser through basic-pitch-ts (TF.js) or an ONNX re-export. Whisper-style ASR on WebGPU is mature. No in-browser drum or guitar-tab transcription demo was found.
+
+### Cited Findings
+- **freemusicdemixer.com (sevagh):** free, unlimited, client-side Demucs using WebAssembly. No WebGPU is mentioned in its description — [sevagh GitHub](https://www.Github.com/sevagh). A third-party listing says its free tier is limited to a lower-quality 4-stem model — [similarlabs](https://similarlabs.com/p/freemusicdemixer). This is an aggregator and unverified.
+- **demucs-wasm (npm):** a wrapper reverse-engineered from freemusicdemixer's MIT code — [README](https://cdn.jsdelivr.net/npm/[email protected]/README.md)
+- **demucs-rs:** Apache-2.0 Rust HTDemucs v4 built on Burn. The browser build is WASM plus WebGPU, and there is also a CLI, a macOS VST3/CLAP plugin and an MCP server. Models: htdemucs 84 MB, htdemucs_6s 84 MB (adds guitar and piano stems), htdemucs_ft 333 MB. The web build is "significantly slower" than the native CLI, and no benchmarks are published. ~158 stars — [GitHub](https://github.com/nikhilunni/demucs-rs)
+- **demucs-js (bakkot):** about 3x realtime on WebGPU. Without WebGPU it is "much, much slower" — [GitHub](https://github.com/bakkot/demucs-js)
+- **monteslu/htdemucs-web-onnx (Hugging Face):**
+  - Size and precision: 126 MB mixed precision vs 173 MB fp32. Conv and ConvTranspose layers stay fp32 because full fp16 degraded the drums and bass stems. Per-stem correlation vs fp32 is at least 0.9998. About 2.4x faster than fp32.
+  - Speed: on a Radeon RX 7600, about 22x realtime back-to-back, or about 8x realtime with UI-friendly pacing.
+  - Why the model is split: it is cut into 21 chained ONNX pieces of 10–30 ms each, because a monolithic session blocks the GPU for 200–330 ms and drops frames. Chaining costs about 7% overhead.
+  - Memory: keep boundary tensors on the GPU with `preferredOutputLocation: 'gpu-buffer'` and dispose them after each segment.
+  - [HF model card](https://huggingface.co/monteslu/htdemucs-web-onnx)
+- Other WebGPU Demucs ONNX exports: [htdemucs-ft-webgpu](https://huggingface.co/monteslu/htdemucs-ft-webgpu), [htdemucs-6s-webgpu-onnx](https://huggingface.co/kramp/htdemucs-6s-webgpu-onnx), [timcsy/demucs-web](https://github.com/timcsy/demucs-web). The openDAW project also has an in-browser stem-separation "AI Demuxer" — [openDAW newsletter](https://buttondown.com/opendaw/archive/ai-demuxer-stem-separation/). None of these were fetched.
+- **Basic Pitch:**
+  - The official web demo at basicpitch.spotify.com converts recordings to MIDI on the site — [About page](https://basicpitch.spotify.com/about)
+  - basic-pitch-ts is a TypeScript library that aims for feature parity with the Python version — [ecosyste.ms](https://awesome.ecosyste.ms/projects/github.com%2Fspotify%2Fbasic-pitch-ts). Its mirror was "last synced over a year ago", so activity is uncertain.
+  - An ONNX re-export for ORT-Web exists — [daserge/basic-pitch-onnx](https://huggingface.co/daserge/basic-pitch-onnx)
+- **Whisper and other ASR in the browser:**
+  - @remotion/whisper-webgpu uses transformers.js and WebGPU with timestamps — [Remotion docs](https://remotion.dev/docs/whisper-webgpu)
+  - The Voxtral-WebGPU HF Space — [HF](https://huggingface.co/spaces/webml-community/Voxtral-WebGPU)
+  - Liquid AI's LFM2.5-Audio-1.5B WebGPU demo — [Liquid docs](https://docs.liquid.ai/examples/web/audio-webgpu-demo)
+  - These are useful for lyrics, not notes.
+- **essentia.js (UPF MTG):** WASM build of Essentia for audio features, onsets and rhythm. AGPLv3 license, which matters for closed products. The README warns that the API is under rapid development — [essentia.js](https://essentia.upf.edu/essentiajs); [TISMIR paper](https://transactions.ismir.net/articles/111)
+- **Drum transcription in the browser:** none found. browserTranscribe lists drum transcription only as future work — [GitHub](https://github.com/jsphweid/browserTranscribe)
+
+### Inferences
+- Separation (the heaviest stage) is about 3–22x realtime on desktop WebGPU, so a 4-minute song would separate in roughly 10 s to 80 s on a desktop GPU. This is extrapolated, not measured. Basic Pitch is a small model and should add little on top.
+- Guitar-tab and drum transcription would be new ground in the browser. No reference implementation was found.
+
+### Gaps
+- No phone or integrated-GPU benchmarks for Demucs in the browser were found.
+- No published WASM-only (no WebGPU) timing for a full song was found.
+- Basic Pitch browser runtime numbers were not found.
+
+## A3. Practical limits: model size, memory, runtime, workers, caching
+
+### Takeaway
+The hard ceiling is the 4 GB wasm32 address space. ONNX Runtime Web cannot load models larger than 4 GB, and disposing sessions does not reclaim wasm linear memory. Demucs (84–333 MB) and Basic Pitch fit easily. The real risks are total memory when several models are loaded together, GPU long-task jank, and the cost of the first download.
+
+### Cited Findings
+- ORT-Web cannot run models larger than 4 GB because of 32-bit wasm addressing. wasm64 or direct GPU weight loading are named as possible future routes — [ONNX Runtime large-models doc](https://onnxruntime.ai/docs/tutorials/web/large-models.html); [V8 4GB blog](https://v8.dev/blog/4gb-wasm-memory)
+- WebGPU can upload weights from an ArrayBuffer outside the wasm heap, so they do not count toward the 4 GB cap — [ORT commit via semanticdiff](https://app.semanticdiff.com/gh/microsoft/onnxruntime/commit/d2a1b7a35398d6612a4d71272a6f784a913c855a)
+- ORT splits inputs larger than maxStorageBufferBindingSize across multiple bindings — [ORT commit](https://app.semanticdiff.com/gh/microsoft/onnxruntime/commit/535f1f37c410575ecbbb47de8f902a137e498869)
+- Firefox bug: WebGPU `queue.writeBuffer()` fails when the wasm heap is larger than 2 GB, so Wasm4GB and Wasm64 modes are not supported there — [Mozilla bug 2022805](https://bugzilla.mozilla.org/show_bug.cgi?id=2022805)
+- Loading several sessions fills the same 4 GB space, and wasm linear memory is not reclaimed after dispose — [Chromium extensions group](https://groups.google.com/a/chromium.org/g/chromium-extensions/c/-4CNcu5bUGA/m/SxP0RfNaCAAJ)
+- Device maxBufferSize distribution data is available — [web3dsurvey](https://web3dsurvey.com/webgpu/limits/maxBufferSize/platform/Linux). The numbers were not fetched.
+- Long GPU bursts block rendering, which is why the model was chained into 21 pieces (see A2) — [monteslu HF](https://huggingface.co/monteslu/htdemucs-web-onnx)
+
+### Inferences
+- Run each pipeline stage in a dedicated Web Worker. Terminating the worker, rather than calling session.dispose(), is the dependable way to free wasm memory between stages, given the reclaim issue above.
+- Process audio in segments, as Demucs already does. Use fp16 or mixed precision to keep downloads around 100 MB for separation.
+
+### Gaps
+- No primary source was fetched on caching models with the Cache API or OPFS, or on storage quotas. transformers.js uses the Cache API by default (from memory, not verified here).
+- No source was found on iOS Safari per-tab memory limits.
+
+## A4. Hybrid browser-first / server-fallback patterns
+
+### Takeaway
+No formal write-up was found. Existing projects show the pattern in pieces: demucs-rs uses the same Rust core for the browser (WASM+WebGPU) and native GPU, and recommends the CLI for long tracks. oh-sheet runs fully server-side with a local and a remote engraving backend.
+
+### Cited Findings
+- demucs-rs: one codebase targets the browser, CLI, plugin and MCP server. It recommends the CLI for batch work or long tracks because the web build is slower — [GitHub](https://github.com/nikhilunni/demucs-rs)
+- oh-sheet: a `local` engraving backend with a `remote_http` fallback when the local one fails — [GitHub](https://github.com/Oh-Sheet-Team/oh-sheet)
+
+### Inferences
+- Suggested pattern: detect `navigator.gpu` and the adapter limits at runtime. Use WebGPU in the browser when it is available, and offer server GPU processing when WebGPU is missing (Firefox Linux/Android, old Android), on low-memory phones, or for the larger htdemucs_ft model. Use the same ONNX files on both sides so outputs match.
+
+### Gaps
+- No published reference architecture or cost analysis for browser-first transcription with server fallback was found.
+
+## B1. Commercial products: features, pricing, user sentiment
+
+### Takeaway
+Users consistently describe AI tabs and sheets as useful drafts that need cleanup. Accuracy is good on clean, solo, single-line parts and drops on dense, distorted or multi-guitar mixes. Recurring complaints are fingering and position choices that a guitarist would not use, inconsistent bend and slide detection, confusion between multiple guitars, and paywalls or ticket friction. There is no independent accuracy benchmark across products.
+
+### Cited Findings
+- **Klangio (Guitar2Tabs, Piano2Notes, Drum2Notes, Sing2Notes):**
+  - Pricing: about 20 s free, then ticket packs ($1.99 for 1, $5.99 for 3, $9.99 for 10, $24.99 for 50, per App Store). Pro subscriptions are about $8.99–9.99/month or $89.99–99.99/year per app. The "Klangio Universe" bundle is $24.99/month or $249/year. Prices differ between listings — [Klangio App Store listings via search](https://apps.apple.com/app/id1589420929); [Drum2Notes](https://apps.apple.com/app/id6717586517). The klang.io/pricing page returned 404.
+  - Features: inputs are live recording, file upload and YouTube. There is a DAW plugin and a public API. Exports are Guitar Pro, MusicXML and MIDI. Bends and slides come through in "Rock Mode" — [Songscription comparison](https://www.songscription.ai/blog/best-ai-guitar-tab-generators). This is a competitor source.
+  - Accuracy: "tends to get the notes right" but lays them out unplayably. Bend and slide detection is inconsistent. The ticket model is described as friction — [same Songscription article](https://www.songscription.ai/blog/best-ai-guitar-tab-generators)
+  - App Store: 3.6★ from 25 reviews. One review reports a crash on MusicXML export — [App Store](https://apps.apple.com/app/id1589660528)
+  - Trustpilot: split between praise for accuracy and time saved and "flawed/unusable". One user reports a support problem after buying a yearly plan — [Trustpilot](https://uk.trustpilot.com/review/klang.io)
+  - Musehub user: "even turns accidental background noises into tabs" — [Musehub](https://musehub.com/app/guitar2tabs)
+- **Songsterr AI:**
+  - Generates guitar, bass and drum tabs in one pass. It is backed by a library of over 1M community tabs. The free tier has limits, and Plus is needed for full access and MIDI or Guitar Pro export. There is no MusicXML export — [Songscription](https://www.songscription.ai/blog/best-ai-guitar-tab-generators)
+  - Heise: "fairly accurate" tabs from your own recordings, but it doesn't always separate multiple guitar parts correctly — [heise](https://heise.de/-10356152)
+  - BassBuzz forum: one user called it "impressive", another said "if you are after accuracy you'll not get that" — [BassBuzz](https://forum.bassbuzz.com/t/new-songsterr-ai-is-impressive/70519)
+- **Songscription:** an AI transcription service from audio or YouTube. Exports Guitar Pro, MusicXML, MIDI and PDF, and includes a piano-roll editor. Free 30 s previews, and full exports need a paid plan. The source is its own marketing, which acknowledges self-interest and says the comparison is "not a controlled accuracy benchmark" — [Songscription](https://www.songscription.ai/blog/best-ai-guitar-tab-generators). It claims cleaner piano output than AnthemScore's "older offline model" — [Songscription vs AnthemScore](https://www.songscription.ai/blog/songscription-vs-anthemscore). This is a self-interested claim.
+- **AnthemScore (Lunaverus):**
+  - Offline desktop app with a one-time purchase. Editions are about $31–$107 per the vendor. Directories list $19.97, which conflicts — [Lunaverus compare](https://www.lunaverus.com/compare); [SourceForge](https://sourceforge.net/software/product/AnthemScore/)
+  - Accuracy: a MuseScore forum user claims about 96% on slow solo piano, with no stated method. Users say it works best on simple, solo-piano audio. The vendor says accuracy "varies from song to song" — [MuseScore forum](https://musescore.org/en/node/272456); [Lunaverus FAQ](https://lunaverus.com/faq)
+  - Tip from a TalkBass user: warping or quantizing an isolated bass stem in Ableton first improves results — [TalkBass](https://www.talkbass.com/threads/absolutely-amazing-scoring-software.1262885/)
+- **Moises:**
+  - Stems, chords and practice tools. Free tier is about 5 tracks/month. Premium is about $3.99/month ($2.99/month annual) and Pro is $9.99/month — [StemSplit review, updated Mar 2026](https://stemsplit.io/blog/moises-ai-review). This is a competitor source.
+  - A roundup ranks Moises for quick editable parts from solo or small-ensemble recordings, and Chordify for chord charts synced to the song — [Worldmetrics](https://worldmetrics.org/best/transcribe-music-software/). This is an SEO roundup.
+- **LALAL.AI:** about $15/month (about $0.17/min). Scored 92% "clean" vocals vs 87% for Moises in a competitor's test with unclear methodology — [StemSplit](https://stemsplit.io/blog/moises-ai-review)
+- **ScoreCloud, AnthemScore:** labelled "freemium" by an aggregator, without tier prices — [ToolMage](https://www.toolmage.com/de/compare/anthemscore-vs-scorecloud/). Unverified.
+
+### Inferences
+- Gaps worth targeting, based on the complaints:
+  - Playable fingering and position assignment for tabs.
+  - Handling more than one guitar part.
+  - An editor for correcting output before export.
+  - Open export formats (MusicXML is missing in Songsterr).
+  - Free or local processing with no per-song tickets. Browser-side inference makes this cheap to offer.
+
+### Gaps
+- Not researched within the call budget: Chordify pricing and sentiment, ScoreCloud, Guitar Pro 8 and MuseScore 4 audio import, Ultimate Guitar, RipX DAW, Melodyne, Samplab, Fadr, Riffusion, and Spleeter-based sites.
+- Searches for Fadr and RipX returned only generic comparison pages.
+- No Reddit threads with substantive accuracy discussion were retrieved. The only one found was an r/Guitar thread from July 2024 about UG tab quality, which is not about AI.
+- No independent cross-product accuracy benchmark exists in the sources found.
+
+## B2. Open-source end-to-end pipelines (audio/YouTube → stems → MIDI → sheet/tab)
+
+### Takeaway
+The common open-source recipe is yt-dlp → Demucs → Basic Pitch → music21 or MusicXML → LilyPond, MuseScore or Verovio. The most complete repo found is oh-sheet (YouTube or audio to a two-hand piano PDF). Several HF Spaces chain the same components. None does guitar tabs or drums end to end with good quality.
+
+### Cited Findings
+- **oh-sheet:**
+  - Pipeline: YouTube (yt-dlp), uploaded audio or MIDI input → optional Demucs stems → Basic Pitch ONNX (about 10 s/song on CPU) with beat, tempo and key detection → rule-based two-hand piano arrangement → "humanize" step → music21 MusicXML and LilyPond PDF.
+  - Stack: FastAPI backend; Flutter web/mobile frontend with OpenSheetMusicDisplay and Tone.js.
+  - Evaluation: an offline P/R/F1 harness on 25 synthesized fixtures, with no scores published.
+  - Licensing: the sidebar says MIT but the README badge says "Proprietary". The premium engraver is closed.
+  - Activity: about 381 stars.
+  - [GitHub](https://github.com/Oh-Sheet-Team/oh-sheet)
+- **StemStudio HF Space:** Basic Pitch, music21 and Verovio, with a playback cursor on the score — [HF Space](https://yair-bab3-stemstudio-chords.hf.space/)
+- **Other HF apps:** one chains Demucs and Basic Pitch into multi-track MIDI with a librosa fallback, but was showing a runtime error. Another uses Demucs, Basic Pitch and LilyPond with .ly or MusicXML export — [search summary; e.g. Voice-To-MIDI Space](https://huggingface.co/spaces/mrblackdev/Voice-To-MIDI-VTM/blob/main/app.py); [roo-score static Space](https://kaidjuric-roo-score.static.hf.space/index.html). Not individually verified.
+- **n8n community proposal:** yt-dlp → Demucs → Basic Pitch per stem → MIDI, MusicXML and PDF using the MuseScore CLI. It is a plan, not a shipped project — [n8n forum](https://community.n8n.io/t/looking-to-build-an-ai-powered-music-transcription-workflow-in-n8n-youtube-multi-instrument-sheet-music/299515)
+- **GitHub topic page:** music-transcription projects, including a YouTube link → PDF score using Basic Pitch — [GitHub topic](https://github.com/topics/music-transcription)
+- Basic Pitch is Apache-2.0 — [PyPI](https://pypi.org/project/basic-pitch)
+
+### Inferences
+- The components that can be reused in the browser line up with what server-side projects use: Demucs ONNX (WebGPU), Basic Pitch ONNX/TS, and Verovio (which has a WASM/JS build) or OpenSheetMusicDisplay for rendering MusicXML. In principle, the whole oh-sheet-style piano chain could run client-side.
+- oh-sheet's arrangement and humanize steps and its evaluation harness are the most useful pieces to study. Check the license before reusing any code.
+
+### Gaps
+- No open-source end-to-end guitar-tab pipeline (audio → playable tab with string and fret assignment) was found.
+- No open-source end-to-end drum pipeline was found.
+- Quality of the HF Spaces was not tested.
+- The latest commit date for oh-sheet was not visible; one listing said its last push was April 2026.

research_notes/Music transcription tooling survey/drum_transcription.md

@@ -0,0 +1,224 @@
+# Open-source Automatic Drum Transcription (ADT): state as of Oct 2026
+
+Research date: 2026-10-30. About 18 tool calls were made, so coverage is uneven. Some important primary pages could not be retrieved: the ADTOF MDPI paper returned 403, the Omnizart docs returned 404, and the YourMT3 README has no drum details. Claims that come from background knowledge and were **not verified** in this session are marked [UNVERIFIED].
+
+## 1. Best open models/tools: classes, accuracy, license, maintenance, compute
+
+### Takeaway
+No single open model wins outright.
+- **For drum-only notation that you can use commercially**, the best options are the 2026 synthetic-data model ADT_STR (CC BY-SA 4.0, 26 GM-derived classes, weights on HF) and Magenta OaF-Drums (velocity-aware, trained on E-GMD).
+- **ADTOF** is the best-known model trained on real music, but it has only 5 classes and is non-commercial.
+- **The newest research SOTA** is Noise-to-Notes (diffusion, ~50M params). Its code/weight availability is unconfirmed.
+- **Multi-instrument models** (MuScriptor 2026, YourMT3+) are much weaker on drums: about 41–50 onset F1.
+
+### Cited Findings
+
+**ADTOF (Zehren et al.)**
+- The repository content is CC BY-NC-SA 4.0, which means non-commercial use only. No separate code or weights license is stated. — [GitHub MZehren/ADTOF](https://github.com/MZehren/ADTOF)
+- Its targets are "5 classes". It ships pre-trained models, including "ADTOF2", which is used in the demo video. — [GitHub MZehren/ADTOF](https://github.com/MZehren/ADTOF)
+- The datasets total 359 h of crowdsourced (rhythm-game) data. They are distributed only as mel spectrograms, on request via Zenodo. — [GitHub MZehren/ADTOF](https://github.com/MZehren/ADTOF)
+- Maintenance signals are low:
+  - the repo has only 8 commits;
+  - the latest noted change added a General MIDI instrument mapping (`instrumentsMapping.py`);
+  - it was tested on macOS Ventura with Python 3.10 (TF/Keras + madmom).
+  — [GitHub MZehren/ADTOF](https://github.com/MZehren/ADTOF)
+- An **ADTOF-pytorch** port exists. It needs only PyTorch (no TF/Keras/madmom) and loses about 0.2% F-measure compared with the original. — [GitHub MZehren/ADTOF](https://github.com/MZehren/ADTOF)
+- In the N2N paper's comparison, ADTOF scores:
+  - 86.87 F1 on MDB-Drums and 94.67 F1 on IDMT, both real or real-ish data;
+  - only 44.95 on E-GMD, which shows a domain mismatch with electronic-kit synthesized audio;
+  - no velocity prediction.
+  — [Noise-to-Notes arXiv 2509.21739v2](https://arxiv.org/html/2509.21739v2)
+
+**Noise-to-Notes (N2N), Sept 2025, v2 Mar 2026**
+- N2N treats ADT as conditional diffusion. It is a transformer decoder (EDGE-style, FiLM-conditioned) that takes log-mel plus MERT-330M layer-10 features. — [arXiv 2509.21739v2](https://arxiv.org/html/2509.21739v2)
+- It uses 7 classes on E-GMD: kick, snare, tom, hi-hat, crash, ride, bell. It uses 5 classes on MDB and 3 on IDMT. — [arXiv 2509.21739v2](https://arxiv.org/html/2509.21739v2)
+- Onset F1 results:
+
+  | Model | E-GMD | E-GMD velocity-F1 | MDB | IDMT |
+  |---|---|---|---|---|
+  | N2N, 10-step | 89.68 | 82.80 | 87.86 | 94.90 |
+  | hFT-Transformer | 86.31 | 80.16 | 70.81 | 70.61 |
+  | OaF Drums | 83.40 | 61.70 | 75.19 | 85.72 |
+  | ADTOF | 44.95 | – | 86.87 | 94.67 |
+
+  — [arXiv 2509.21739v2](https://arxiv.org/html/2509.21739v2)
+- Size and speed:
+  - about 50M parameters, plus the 330M MERT feature extractor;
+  - one sampling step takes 0.163 s per 5 s clip on an A100;
+  - performance saturates at about 10 steps.
+  — [arXiv 2509.21739v2](https://arxiv.org/html/2509.21739v2)
+- The paper text I retrieved states no code, weights or license. — [arXiv 2509.21739v2](https://arxiv.org/html/2509.21739v2)
+
+**ADT_STR, "Towards Realistic Synthetic Data for ADT" (Melucci, Merialdo, Akama; arXiv Jan 2026)**
+- The model is trained only on synthetic audio. That audio is rendered from MIDI using one-shot samples mined semi-supervised from unlabeled audio. It is a seq2seq model. — [arXiv 2601.09520](https://arxiv.org/abs/2601.09520)
+- It claims new SOTA on the ENST and MDB test sets, beating fully supervised methods. — [arXiv 2601.09520](https://arxiv.org/abs/2601.09520)
+- The code is CC BY-SA 4.0, so commercial use is allowed with share-alike.
+- It maps the 47 GM percussion notes onto **26 classes**, merging, for example, the crash cymbals together.
+- Three checkpoints are released (tau 0.4/0.6/0.8, safetensors), and tau-0.8 is the best. Weights are on HF as `Pierfrancesco/adt-str`. The repo is at v1.0.0.
+
+— [GitHub pier-maker92/ADT_STR](https://github.com/pier-maker92/ADT_STR)
+
+**MuScriptor (Rouard, Krause, Roebel, Simon-Gabriel, Défossez; ISMIR 2026 camera-ready; arXiv Jul 2026)**
+- It is an open-weight multi-instrument model in 4 sizes: 60M, 100M, 300M and 1.3B (the paper version). — [arXiv 2607.08168](https://arxiv.org/html/2607.08168)
+- Drums are "onset-only". Drums onset F1 is 50.2 for MuScriptor vs 41.4 for YourMT3+. Per dataset, results are mixed: RWC-P 42.1 vs 36.5, RWC-C 23.7 vs 8.7, RWC-R 36.5 vs 51.3. — [arXiv 2607.08168](https://arxiv.org/html/2607.08168)
+- Licensing and release:
+  - the code is MIT, but the weights are **CC BY-NC 4.0 (non-commercial)**;
+  - the released sizes are small 103M, medium 307M (the default) and large 1.4B;
+  - small is described as practical on CPU, while large "really wants a GPU";
+  - it exports a separate drum-kit PDF through MuseScore 4.
+  — [GitHub muscriptor/muscriptor](https://github.com/muscriptor/muscriptor)
+
+**YourMT3 / YourMT3+**
+- The repo license is GPL-3.0, and the weights license is not stated separately. An HF Spaces demo appeared in Aug 2024, and the last dated news item is Nov 2024. — [GitHub mimbres/YourMT3](https://github.com/mimbres/YourMT3)
+
+**MT3 family**
+- The MT3 design does not model velocity. This is a secondary source (a student review). — [baulab review](https://expo.baulab.info/2022-Fall/PraveenKumarSridhar/second_review.html)
+
+**Magenta OaF-Drums / E-GMD**
+- The E-GMD dataset is CC BY 4.0. — [Magenta OaF Drums](https://magenta.tensorflow.org/oaf-drums)
+- In 2025 comparisons, OaF Drums still reaches 83.4 F1 on E-GMD and 75.2 on MDB. — [arXiv 2509.21739v2](https://arxiv.org/html/2509.21739v2)
+
+**Other**
+- A momentum-NMF ADT method (Jul 2025) offers a classical, interpretable alternative. — [arXiv 2507.12596](https://arxiv.org/abs/2507.12596v1)
+- The STAR Drums dataset (Fraunhofer, 2025) was built from full mixes by separating the drum stem and auto-annotating it with an ADT system. — [Fraunhofer publica](https://publica.fraunhofer.de/handle/publica/499927)
+- A LinkedIn post says "Separate-and-Detect", a latent-diffusion drum transcription paper, was accepted at ISMIR 2026. This is unverified and secondary. — [LinkedIn](https://tw.linkedin.com/in/mia-chen-1788511a9)
+
+### Inferences
+- **Practical choices for a drum-tab product:**
+  - (a) ADT_STR, the only one found with a commercially usable license plus GM-level class granularity;
+  - (b) ADTOF/ADTOF-pytorch, which is robust on real recordings but non-commercial and has only 5 classes;
+  - (c) OaF-Drums, if velocity matters (license not checked here, but Magenta code is typically Apache-2.0 [UNVERIFIED]).
+- **ADTOF's 5 classes** are, to my knowledge, kick / snare / toms / hi-hat / cymbals+ride, and do not separate open from closed hi-hat [UNVERIFIED; the paper page was blocked].
+- **Omnizart drum** (MIT code) and the **madmom** drum models (BSD code; madmom model files are, as I recall, CC BY-NC-SA) are older baselines from 2018–2021, mostly kick/snare/hi-hat. Neither appears as a competitive baseline in 2025–26 papers [UNVERIFIED; docs 404].
+- **Multi-instrument models** are not adequate on their own for drum notation, because drum F1 is about 40–50. Use them only if the product already needs full-band transcription.
+
+### Gaps
+- Exact ADTOF class list and MIDI mapping, per-class F1, and the license of model weights vs dataset (the MDPI paper was 403).
+- ADT_STR numeric F1 values (the README gives no numbers in the retrieved text) and the HF weight license.
+- Whether N2N code/weights are released.
+- Omnizart drum class count and maintenance status (docs 404). The Omnizart project appears dormant since about 2022 [UNVERIFIED].
+- No "Drumscribe" or "DrumTranscriber" open-source projects were found in searches. I did not search for them specifically.
+- No MIREX drum transcription results from 2024–2026 were found.
+
+## 2. Per-class performance (hi-hats/cymbals weakest?)
+
+### Takeaway
+Yes. The 2025 ISMIR diagnostic study confirms that toms and cymbals are the weakest classes. In drum-only audio, the main error source is simultaneous hits. In full mixes, it is interference from melodic instruments.
+
+### Cited Findings
+- Weyers, Uhle, Müller, Lang (ISMIR 2025): toms and cymbals are the classes where SOTA models perform worst. — [ISMIR 2025 P5-10](https://ismir2025program.ismir.net/poster_130.html)
+- Transcription of drums within full mixes is limited mainly by melodic instruments and vocals. Performance improves by about 5% with a single kit, strong onsets only, or no simultaneous notes. — [ISMIR 2025 P5-10](https://ismir2025program.ismir.net/poster_130.html)
+- Drum-only transcription becomes nearly error-free once simultaneous onsets are removed. — [ISMIR 2025 P5-10](https://ismir2025program.ismir.net/poster_130.html)
+- A reviewer comment notes that hi-hat and ride patterns alternate weak and strong strokes, which makes the weak strokes hard to detect. The findings are mostly on re-synthesized audio, and MDB, the real test set, is only about 22 min. — [ISMIR 2025 P5-10](https://ismir2025program.ismir.net/poster_130.html)
+- ADT_STR publishes confusion matrices for its 26 classes on ENST and MDB. — [GitHub ADT_STR](https://github.com/pier-maker92/ADT_STR)
+
+### Inferences
+- Expect kick and snare to be reliable. Expect hi-hat open/closed, crash vs ride vs china, and toms (which tom) to need manual correction.
+- Expect ghost notes on the snare and weak hi-hat strokes to be the most commonly missed onsets.
+
+### Gaps
+- No per-class F1 table was retrieved for any current model.
+
+## 3. Does pre-separation (Demucs drum stem, LarsNet, DrumSep) help?
+
+### Takeaway
+There is indirect evidence that removing melodic interference helps, since it is the main limiting factor in full mixes. I found no controlled study showing that LarsNet or DrumSep sub-stem separation improves ADT F1.
+
+### Cited Findings
+- In full mixes, the main limiting factor is melodic and vocal interference. — [ISMIR 2025 P5-10](https://ismir2025program.ismir.net/poster_130.html)
+- The STAR Drums pipeline separates the drum stem from full mixes and then annotates it with ADT. This shows the "separate then transcribe" pattern in use. — [Fraunhofer publica](https://publica.fraunhofer.de/handle/publica/499927)
+- LarsNet uses 5 parallel U-Nets, one per stem: kick, snare, toms, hi-hat, cymbals. It is trained on StemGMD and beats NMF/NMFD baselines. — [arXiv 2312.09663](https://arxiv.org/html/2312.09663v1)
+- The LarsNet paper treats ADT only as a related task and reports no ADT gains. — [arXiv 2312.09663](https://arxiv.org/html/2312.09663v1)
+- The LARS VST plug-in weights are CC BY-NC 4.0. — [GitHub EdoardoMor/LARS](https://github.com/EdoardoMor/LARS)
+- MVSEP says LarsNet separation quality is inferior to its DrumSep model, and that it could not reproduce the authors' metrics. — [MVSEP](https://mvsep.com/algorithms/31?lang=en)
+- In practice, some users isolate the drum track with Moises before analysis. — [Penn State Brandywine](https://www.brandywine.psu.edu/node/28566)
+
+### Inferences
+- Use a full-drum-stem pre-pass (Demucs/HTDemucs) when the ADT model was trained on isolated drums, such as E-GMD-trained OaF or N2N. It is likely to help, given the interference finding.
+- ADTOF was trained on full mixes, so it may gain less.
+- Sub-stem separation adds compute, and its artifacts (bleed) may create false onsets. Treat it as an experiment, not a default.
+
+### Gaps
+- No quantitative ADT-with/without-separation results were found.
+- DrumSep's architecture and license were not confirmed.
+
+## 4. Velocity, ghost notes, fills
+
+### Takeaway
+Velocity is only modeled by OaF-Drums, hFT-Transformer and N2N. ADTOF, DT-Ensemble, MT3 and MuScriptor are onset-only, so ghost notes and accents must be inferred or are lost.
+
+### Cited Findings
+- ADTOF and DT-Ensemble do not predict velocity. Velocity F1 on E-GMD is N2N 82.8, hFT 80.2 and OaF 61.7. — [arXiv 2509.21739v2](https://arxiv.org/html/2509.21739v2)
+- N2N's Annealed Pseudo-Huber loss raised velocity F1 on E-GMD from 66.1 to 76.2. — [arXiv 2509.21739v2](https://arxiv.org/html/2509.21739v2)
+- MuScriptor's drum hits are "onset-only". — [arXiv 2607.08168](https://arxiv.org/html/2607.08168)
+- Weak (ghost-like) strokes and simultaneous hits are a key error source. — [ISMIR 2025 P5-10](https://ismir2025program.ismir.net/poster_130.html)
+
+### Inferences
+- For ghost-note notation, a velocity threshold on OaF or N2N output (for example, velocity below about 40 drawn as a parenthesized notehead) is the only data-driven path.
+- With onset-only models, you could estimate velocity from onset peak amplitude as a heuristic.
+- Fills (dense tom runs) combine the two weakest factors: tom classification and simultaneous hits. Expect degraded accuracy there.
+
+### Gaps
+- No study specifically evaluates ghost-note or fill accuracy.
+
+## 5. Converting ADT output to notation (quantization, GM map, GP/MusicXML)
+
+### Takeaway
+Most open models output GM-pitched MIDI or class onsets. Building notation requires beat tracking, quantization, meter inference and a percussion map. MIDI-to-score drum work exists, but tooling here is thin.
+
+### Cited Findings
+- ADTOF added a General MIDI instrument mapping file. — [GitHub ADTOF](https://github.com/MZehren/ADTOF)
+- ADT_STR classes follow General MIDI percussion, consolidated from 47 to 26. — [GitHub ADT_STR](https://github.com/pier-maker92/ADT_STR)
+- MuScriptor renders a drum-kit PDF through MuseScore 4. — [GitHub muscriptor](https://github.com/muscriptor/muscriptor)
+- An academic work on "MIDI To Score Automated Drum Transcription" (HAL, 2022) addresses the MIDI-to-score step. — [HAL hal-03847232](https://hal.archives-ouvertes.fr/hal-03847232)
+- Commercial tools ask users for time signature and BPM to improve results. — [Songscription blog](https://www.songscription.ai/blog/transcribe-drums-with-ai)
+- Commercial drum-convert services document manual correction of errors. — [La Touche Musicale help](https://latouchemusicale.crunch.help/en/drum-convert/errors-in-the-transcribed-drum-sheet-music-what-to-do)
+
+### Inferences
+- **Suggested pipeline:** ADT onsets plus classes → beat/downbeat tracking (e.g. madmom or Beat This [UNVERIFIED]) → grid quantization (16th or 32nd notes, triplet detection) → GM drum pitches.
+- **MusicXML percussion output:** a `<unpitched>` display-step/octave plus `<instrument>` per GM note. Typical choices are x noteheads for cymbals and hi-hat, and stems up for hands and down for feet.
+- **Guitar Pro drum track output:** GP uses GM percussion numbers directly, for example via alphaTab/alphaTex or PyGuitarPro [UNVERIFIED].
+- **Open vs closed hi-hat:** ADTOF's merged classes would need to map to a single hi-hat, typically 42.
+
+### Gaps
+- No verified sources were retrieved on Guitar Pro or MusicXML percussion-map specifics, or on alphaTab drum import/export.
+
+## 6. Commercial references and user sentiment
+
+### Takeaway
+User evidence is sparse and anecdotal. Songsterr's AI drum transcription gets the most positive mentions. Drum2Notes (Klangio) got negative early feedback. All tools are treated as drafts needing correction.
+
+### Cited Findings
+- An r/DRUM user (Jul 2024) tried the Drum2Notes trial and found it poor even on isolated drums with a simple beat. — [Reddit r/DRUM](https://nyc1.lr.ggtyler.dev/r/DRUM/comments/1e5qmzg/ai_that_translates_audio_into_sheet_music)
+- In the same thread, a user recommended Songsterr's AI as "very good". — [Reddit r/DRUM](https://nyc1.lr.ggtyler.dev/r/DRUM/comments/1e5qmzg/ai_that_translates_audio_into_sheet_music)
+- The Drum2Notes App Store listing warns that it cannot separate multiple instruments. — [App Store](https://apps.apple.com/app/id6717586517)
+- On the BassBuzz forum, users called Songsterr AI "dead on accurate", while others said it was "mostly hit with some miss". These comments concern bass, not necessarily drums. — [BassBuzz forum](https://forum.bassbuzz.com/t/new-songsterr-ai-is-impressive/70519)
+- Moises is primarily a stem separator. No user reviews of its drum-transcription accuracy were found. — [Penn State Brandywine](https://www.brandywine.psu.edu/node/28566)
+
+### Gaps
+- No 2025–2026 user reviews were found. The Drum2Notes complaint is from 2024, so the product may have improved.
+- Nothing was found on "Drumscrib".
+
+## 7. Browser deployment (ONNX/TF.js), model sizes
+
+### Takeaway
+No ADT project found ships ONNX or TF.js builds. The small CRNN models (ADTOF, OaF) are the realistic browser candidates. N2N (50M plus 330M MERT) and MuScriptor (100M–1.4B) are server-side.
+
+### Cited Findings
+- The ADTOF README makes no mention of ONNX or TF.js. — [GitHub ADTOF](https://github.com/MZehren/ADTOF)
+- The PyTorch port could be exported with `torch.onnx`. — [GitHub ADTOF](https://github.com/MZehren/ADTOF)
+- Model sizes:
+  - hFT-Transformer: 5.5M parameters;
+  - N2N: about 50M, plus the MERT-330M feature extractor;
+  - MuScriptor: 103M, 307M and 1.4B (small is CPU-practical).
+  — [arXiv 2509.21739v2](https://arxiv.org/html/2509.21739v2); [GitHub muscriptor](https://github.com/muscriptor/muscriptor)
+- ADT_STR ships safetensors weights. — [GitHub ADT_STR](https://github.com/pier-maker92/ADT_STR)
+
+### Inferences
+- **Browser path:** ADTOF-pytorch → ONNX → onnxruntime-web (WASM/WebGPU), with the mel front-end reimplemented in JS. ADTOF CRNNs are roughly a few MB [UNVERIFIED size].
+- **Browser path for ADT_STR:** feasible only if the seq2seq model is small, which was not verified.
+- **License caution:** ADTOF and MuScriptor weights are non-commercial. Check this before shipping them in a product.
+
+### Gaps
+- Parameter counts for ADTOF, OaF-Drums and ADT_STR were not verified.
+- No existing web demos running ADT client-side were found.

research_notes/Music transcription tooling survey/guitar_bass_tab.md

@@ -0,0 +1,265 @@
+# Guitar and bass audio-to-tablature transcription (state as of Oct 2026)
+
+Scope note: this covers about 17 search and fetch calls. Items marked "[unverified, prior knowledge]" were not checked in this session. They are listed only under Inferences or Gaps, never as cited findings. Several key arXiv items are dated Aug–Sep 2026 (IDs 2608.x and 2609.x), which fits today's date of 2026-10-30.
+
+## Q1. Audio-to-tab models and SOTA (TabCNN, FretNet, Riley/high-res, GAPS, SynthTab, GOAT, TART, diffusion); electric vs. clean
+
+### Takeaway
+The strongest documented 2026 system is TART (Berkeley; arXiv 2510.02597 in Oct 2025, then a revised "modular" version, arXiv 2609.11904, in Sep 2026, submitted to ISMIR 2026). It is a 4-stage pipeline: audio→MIDI, technique classifier, audio-conditioned T5 string/fret assignment, and tab rendering. It reports 81.35% audio-to-MIDI F50 and 71.8% Tab F1 with oracle MIDI. End to end it gets only 54.08% Tab F1. That means fully automatic tabs are still roughly half right at the string+fret+onset level. Electric and distorted guitar is still the weak spot. Newer data (GOAT, Guitar-TECHS, EGDB, real-tone augmentation) targets it, but no paper reports a dedicated high-gain or distorted benchmark.
+
+### Cited Findings
+- **TART (2609.11904v1, submitted 10 Sep 2026)**
+  - Stage 1 is a high-res CRNN based on Kong et al., fine-tuned on pooled guitar data with noise augmentation.
+  - Stage 2 is a CNN-BiLSTM technique classifier with about 160K parameters.
+  - Stage 3, "AudioFret", is a T5-style encoder-decoder with about 15M parameters. It adds per-note audio embeddings to the MIDI tokens and uses beam search 4 with a chord fret-span limit of 5.
+  - Stage 4 uses BeatNet tempo and a 1/16 grid, and outputs **MusicXML + JAMS**. Guitar Pro export is not mentioned.
+  - — [arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)
+- **TART results**
+  - Audio→MIDI F50: 81.35% average, vs. 74.68% for Riley et al. and 69.88% for NoteEM. Per dataset: GuitarSet 87.4, EGDB 79.0, GuitarSet-noisy 82.2, EGDB-noisy 76.8.
+  - String/fret Tab F1 with oracle MIDI: 71.8%, vs. 63.3% for the Fretting-Transformer. TabCNN averages 27.9% on EGDB.
+  - End-to-end Tab F1: 54.08%. The paper attributes a 17.75-point "propagation cost" to Stage-1 errors.
+  - — [arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)
+- **TART training data**
+  - Stage 1: GAPS, Guitar-TECHS, François Leduc, GOAT-DI.
+  - Stage 3: pretrained on SynthTab + DadaGP, then fine-tuned on GAPS, GOAT-DI, Guitar-TECHS.
+  - Evaluation: GuitarSet, EGDB, and noisy variants of both. There is no separate electric-only evaluation.
+  - — [arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)
+- **TART code**: no code repo was found. The arXiv page has a "Code, Data, Media" tab, but no link surfaced. The paper text is CC BY 4.0. — [arXiv 2609.11904](https://arxiv.org/pdf/2609.11904); [arXiv 2510.02597](https://arxiv.org/abs/2510.02597)
+- **Noise2Fret** (Simionato & Bigo, arXiv 2608.30854, 31 Aug 2026, accepted to ISMIR 2026)
+  - A diffusion model that generates tab through a continuous latent of string/fret targets, conditioned on spectral features.
+  - Adds five playability losses: pitch-class distance, positional distance, circle-of-fifths distance, string similarity, hand-span feasibility.
+  - Evaluated on GuitarSet and GOAT. The abstract claims it beats baselines and is more efficient. No numbers or code are on the abstract page.
+  - — [arXiv 2608.30854](https://arxiv.org/abs/2608.30854)
+- **GOAT** (Loth, Sarmento, Sarkar, Guo, Barthet, Sandler; ISMIR 2025)
+  - 5.9 h of unique DI electric guitar recordings, amp-augmented to 29.5 h.
+  - Tabs with string/fret and techniques, supplied in **Guitar Pro format** plus a token encoding.
+  - License CC BY 4.0 (per the arXiv license link).
+  - Reports "competitive" MIDI transcription and only preliminary tab results.
+  - — [arXiv 2509.22655](https://arxiv.org/abs/2509.22655)
+- **High Resolution Guitar Transcription via Domain Adaptation** (Riley et al., arXiv 2402.15258, 2024; older)
+  - Onset-only F50 on GuitarSet: 89.7 when supervised on Leduc + GuitarSet, and 87.3 zero-shot when trained only on Leduc-derived data. The zero-shot figure is more than 4 points above Maman & Bermano.
+  - This is notes only, not string assignment.
+  - The Leduc data is commercial and not free.
+  - — [arXiv 2402.15258](https://arxiv.org/pdf/2402.15258); [GAPS](https://arxiv.org/html/2408.08653v2)
+- **GAPS** (ISMIR 2024): 14 h of classical (nylon) guitar audio aligned to scores. It is called the largest real-guitar dataset, several times the size of GuitarSet. Its benchmark model is claimed SOTA on GuitarSet in both supervised and zero-shot settings. — [ISMIR 2024 poster](https://ismir2024program.ismir.net/poster_171.html); [arXiv 2408.08653](https://arxiv.org/abs/2408.08653)
+- **SynthTab** (Zang, Zhong, Cwitkowitz, Duan; arXiv 2309.09085; older, 2023–24)
+  - Synthesized audio rendered from DadaGP tablature, which covers 26,181 songs.
+  - Pre-training TabCNNx4 on it improves same-dataset accuracy and especially cross-dataset generalization.
+  - — [arXiv 2309.09085](https://arxiv.org/pdf/2309.09085)
+- **Electric robustness**: Leveraging Real Electric Guitar Tones and Effects (arXiv 2405.14679, 2024) augments TabCNN training with real electric tones and effects to improve robustness. Guitar-TECHS (arXiv 2501.03720) is an electric dataset spanning techniques and hardware. — [arXiv 2405.14679](https://arxiv.org/html/2405.14679v3); [arXiv 2501.03720](https://arxiv.org/pdf/2501.03720)
+- **Survey**: a 2025 IEEE Access journal paper, "Automatic Guitar Transcription With Deep Neural Networks" (Chieppa et al., Univ. Coimbra), exists and could serve as background. Not read in detail. — [PDF](https://mir.dei.uc.pt/pdf/Journals/MERGE/Access_2025_Chieppa.pdf)
+
+### Inferences
+- A practical 2026 architecture is (a) source separation → (b) a strong guitar note transcriber (Riley/GAPS-style high-res CRNN) → (c) a learned string/fret assigner conditioned on tuning and capo → (d) technique tagging → (e) beat/tempo quantization → GP export. That is the TART decomposition.
+- End-to-end Tab F1 of about 54% on research benchmarks with clean or lightly noisy, mostly solo guitar means full-mix, distorted rock and metal will be markedly worse. Expect manual correction.
+- [unverified, prior knowledge] Relevant code: TabCNN (Wiggins & Kim, ISMIR 2019); FretNet and the Cwitkowitz repos (github.com/cwitkowitz/guitar-transcription-continuous, guitar-transcription-with-inhibition, amt-tools, all MIT-licensed as I recall); the Riley/GAPS model (github.com/xavriley). These are the realistic open-source starting points, because no TART code was found.
+
+### Gaps
+- No TART or Noise2Fret code or weights were found. Licenses for any released models are unknown.
+- No paper found that benchmarks high-gain or distorted full-mix rock or metal tabs end to end.
+- Noise2Fret numbers are not retrieved. Only the abstract was read.
+- Licenses for GuitarSet, EGDB, and Guitar-TECHS were not verified this session.
+
+## Q2. MIDI/notes → tab (string/fret assignment) algorithms and tools
+
+### Takeaway
+Learned seq2seq assigners now beat both A* and the built-in Guitar Pro and TuxGuitar heuristics on hard material (jazz, DadaGP). On easy fingerstyle, simple heuristics are nearly as good. Open code exists for some newer work (midi2tab_Deconvolutions). The Fretting-Transformer and TART code links were not found.
+
+### Cited Findings
+- **Fretting-Transformer** (Hamberger, Murgul, Schmidt, Heizmann; arXiv 2506.14223, June 2025)
+  - A reduced T5 (d_model 128, 3+3 layers, 4 heads), with post-processing that forces 100% pitch accuracy.
+  - Tab accuracy on GuitarToday / Leduc / DadaGP:
+
+    | Method | GuitarToday | Leduc | DadaGP |
+    |---|---|---|---|
+    | Fretting-Transformer | 98.41% | 72.19% | 81.58% |
+    | A* | 89.39% | 62.60% | 78.92% |
+    | Guitar Pro | 97.58% | 56.03% | 76.58% |
+    | TuxGuitar | 98.30% | 57.94% | 78.83% |
+
+  - — [arXiv 2506.14223](https://arxiv.org/html/2506.14223v1)
+- **Fretting-Transformer conditioning and caveats**
+  - A conditioned variant takes CAPO and TUNING tokens, trained with capo 0–7 and standard, ½-step down, whole-step down, and drop D.
+  - The model has higher (worse) difficulty scores than TuxGuitar on Leduc and DadaGP.
+  - In the conditioned GuitarToday setting, a trivial lowest-fret baseline (98.30%) slightly beats the model (98.19%).
+  - No code link was found. The paper is CC BY 4.0.
+  - — [arXiv 2506.14223](https://arxiv.org/html/2506.14223v1)
+- **MIDI-to-Tab** (Edwards et al., ISMIR 2024): a masked-language-model Transformer for string assignment. — [ISMIR 2024 poster](https://ismir2024program.ismir.net/poster_175.html); [arXiv 2408.05024](https://arxiv.org/html/2408.05024v1)
+- **midi2tab_Deconvolutions** (Kaliakatsos-Papakostas et al., arXiv 2510.10619, Oct 2025): public code at github.com/maximoskp/midi2tab_Deconvolutions. Assumes standard 6-string tuning and 24 frets. — [arXiv 2510.10619](https://arxiv.org/html/2510.10619v1)
+- **AudioFret**: TART's audio-conditioned assigner gets 71.8% vs. 63.3% Tab F1 for the Fretting-Transformer on GuitarSet/EGDB. Using audio timbre per note helps disambiguate strings. — [arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)
+- **Classical approach**: a weighted DAG plus A* over string/fret candidates to find playable sequences. It is the standard baseline. — [arXiv 2510.02597](https://arxiv.org/pdf/2510.02597)
+
+### Inferences
+- For a website, a cost-based DP/Viterbi assigner (fret span, position shifts, open-string preference, tuning/capo aware) is a cheap, license-clean baseline. It can be swapped later for a T5-style assigner trained on DadaGP/GOAT, which the Fretting-Transformer paper shows is about 98% on easy tabs and 72–82% on hard ones.
+- Benchmarking against hand-made GP tabs will penalize "valid but different" fingerings. See the Q8 metrics.
+
+### Gaps
+- I did not check whether a usable open-source "tabify" library exists (MuseScore's tab conversion, TuxGuitar's algorithm as a library). The Fretting-Transformer paper treats TuxGuitar and Guitar Pro only as black-box baselines.
+
+## Q3. Playing-technique detection
+
+### Takeaway
+Technique classification is reasonably accurate at the note level on curated datasets. TART reports 95.9% macro F1 over 9 classes, with vibrato hardest. Datasets with technique labels exist: Guitar-TECHS, GOAT, EG-IPT, AGPT, IDMT-SMT.
+
+### Cited Findings
+- **TART per-class F1** on a unified test set:
+  - Palm mute 97.9, hammer-on/pull-off 97.4, picking 98.0
+  - Harmonics 94.9, slide 94.7, bend 93.8, vibrato 89.4 (often confused with bend)
+  - Macro F1 95.9
+  - Training data: IDMT-SMT-Chords, Guitar-TECHS, AGPT, EG-IPT, Magcil.
+  - These are classifier scores, not end-to-end tab-with-technique scores.
+  - — [arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)
+- **GOAT** tabs annotate playing techniques. — [arXiv 2509.22655](https://arxiv.org/abs/2509.22655)
+
+### Inferences
+- Bends need pitch-contour output (bend amount and shape), not just a class label, to map onto GP bend points. Continuous-pitch models such as FretNet [unverified, prior knowledge] are relevant here.
+
+### Gaps
+- No end-to-end technique F1 on full songs was found. No open technique-detection model weights were found.
+
+## Q4. Tuning detection and capo inference
+
+### Takeaway
+Current research systems take tuning and capo as user input (TART, Fretting-Transformer) instead of inferring them. No automatic tuning or capo detector was found.
+
+### Cited Findings
+- TART assumes standard tuning by default, and the user can specify tuning, capo, and tempo. It was trained on standard, ½-step down, whole-step down, and drop D, with capo 0–7. — [arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)
+- The Fretting-Transformer uses CAPO/TUNING conditioning tokens with the same four tunings. — [arXiv 2506.14223](https://arxiv.org/html/2506.14223v1)
+- midi2tab_Deconvolutions assumes standard 6-string, 24-fret. — [arXiv 2510.10619](https://arxiv.org/html/2510.10619v1)
+
+### Inferences
+- Down-tuning can be partly inferred from the lowest detected pitch: notes below E2 imply drop or down-tuning, or 7-string. Global detuning (for example A=432 or ½-step) can be inferred from a pitch-deviation histogram.
+- Capo is mostly unobservable from audio. A UI toggle plus "minimize fret positions" search is the pragmatic answer.
+- 7-string and 8-string support is absent from the papers found.
+
+### Gaps
+- No published model was found for tuning, capo, or extended-range detection.
+
+## Q5. Distorted electric guitar; training on DadaGP-rendered or synth audio
+
+### Takeaway
+Synthetic rendering of DadaGP (SynthTab) and real-amp augmentation (2405.14679, GOAT amp renders) are the main strategies. Distortion remains under-evaluated.
+
+### Cited Findings
+- SynthTab renders DadaGP's 26k songs to audio. Pre-training improves cross-dataset results. — [arXiv 2309.09085](https://arxiv.org/pdf/2309.09085)
+- GOAT amp-augments DI recordings from 5.9 h to 29.5 h. — [arXiv 2509.22655](https://arxiv.org/abs/2509.22655)
+- Real electric tones and effects augmentation improves TabCNN robustness. — [arXiv 2405.14679](https://arxiv.org/html/2405.14679v3)
+- TART says prior systems struggle with electric pickups and distortion, but it has no electric-only evaluation. — [arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)
+- Vendors and reviewers all say accuracy is best on clear, solo guitar and degrades on dense mixes and phone recordings. — [Songscription](https://www.songscription.ai/guitar-tab-generator); [agent-finder Klangio review, Aug 2026](https://agent-finder.co/reviews/klangio)
+
+### Gaps
+- No numbers were found for high-gain distortion on full mixes.
+
+## Q6. Bass tab transcription
+
+### Takeaway
+No bass-specific research model for string/fret assignment was found in this session. Research is guitar-centric. Commercial tools (Klangio, Songscription, Tabtify, BassConvert) offer bass tabs, but there are no published accuracy numbers.
+
+### Cited Findings
+- The TART, Noise2Fret, and Fretting-Transformer papers are guitar-only. — [arXiv 2609.11904](https://arxiv.org/html/2609.11904v1); [arXiv 2608.30854](https://arxiv.org/abs/2608.30854)
+- **Songscription**: separates bass from the mix, outputs tab plus notation, and offers Guitar Pro export. — [Songscription bass](https://www.songscription.ai/bass-tab-generator)
+- **Tabtify**: supports 4-, 5-, and 6-string bass. It warns that kick drum, low guitar, and synth bass in dense mixes need cleanup. — [Tabtify](https://tabtify.com/convert/audio-to-bass-tab)
+- **BassConvert**: offers a faithful "transcription" mode vs. an easier "arrangement" mode. — [BassConvert](https://latouchemusicale.com/en/apps/bassconvert/)
+- **Klangio**: claims support for acoustic, electric, and bass guitar. — [Klangio Guitar2Tabs](https://klang.io/guitar2tabs/)
+
+### Inferences
+- Bass is largely monophonic, so a bass stem plus a monophonic pitch tracker plus a cost-based 4/5-string assigner may be competitive. String ambiguity on bass is mostly between positions.
+
+### Gaps
+- No bass tab dataset or benchmark was found. DadaGP contains bass tracks [unverified, prior knowledge].
+
+## Q7. Guitar Pro file I/O and formats
+
+### Takeaway
+Options for each job:
+- **Writing modern .gp (GP7/8)**: alphaTab ships a Gp7Exporter (since 1.2.0), returning a Uint8Array, so it works in browser or Node. It is the best fit for a website.
+- **Python**: PyGuitarPro reads and writes only GP3/4/5 and is LGPL-3.0. A newer PyPI package, ApolloTab, claims GP3–GP8/GPX support under LGPL-2.1 (unverified).
+- **Datasets**: GOAT ships Guitar Pro files.
+
+### Cited Findings
+- **alphaTab export**
+  - `alphaTab.exporter.Gp7Exporter` writes Guitar Pro 7+ (.gp) files via `exporter.export(api.score, api.settings)`, which returns a Uint8Array. Available since 1.2.0.
+  - An alphaTex exporter was added in 1.7.0.
+  - — [alphaTab exporter guide](https://alphatab.net/docs/guides/exporter); [Gp7Exporter ref](https://alphatab.net/docs/reference/types/exporter/gp7exporter/)
+- **alphaTab maintenance**: active. A bug where exported drum tracks were silent in GP8 RSE (wrong `<Pitched>` vs. `<Name>` XML element) was fixed in PR #2456, merged Dec 2025. — [alphaTab issue #2467](https://github.com/CoderLine/alphaTab/issues/2467); [PR #2456](https://github.com/CoderLine/alphaTab/pull/2456)
+- **Community use**: Songsterr Downloader uses alphaTab to convert Songsterr tabs into GP7. — [songsterr-downloader](https://github.com/Metaphysics0/songsterr-downloader)
+- **PyGuitarPro**: reads and writes GP3, GP4, GP5. LGPLv3. — [PyPI](https://pypi.python.org/pypi/PyGuitarPro); [GitHub](https://github.com/Perlence/PyGuitarPro)
+- **ApolloTab**: the PyPI listing (via search snippet) claims "full support for GP3/GP4/GP5/GPX/GP (GP7/GP8)" under LGPL 2.1. The page failed to load on fetch, so this is unverified. — [PyPI apollotab](https://pypi.org/project/apollotab/)
+- **dart_gp_tab_reader**: a Dart port mirroring PyGuitarPro, LGPL-3.0. It notes "Guitar Pro" is a registered trademark of Arobas Music. — [pub.dev](https://pub.dev/packages/dart_gp_tab_reader)
+- **Output formats**: TART outputs MusicXML + JAMS, not GP. — [arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)
+
+### Inferences
+- **Suggested website stack**: a Python ML backend emits an internal score (or alphaTex or MusicXML). The frontend renders it with alphaTab and exports .gp via Gp7Exporter. alphaTab can also import the reference GP files (gp3–gp8) for benchmarking. [unverified, prior knowledge] alphaTab is MPL-2.0, and its importers cover gp3–5, gpx, gp7/8, MusicXML, and alphaTex.
+- **DadaGP** [unverified, prior knowledge]: tokenizes GP files into a text token format via PyGuitarPro, so it inherits the GP3–5 limit. Converting GP7/8 references to GP5 loses some detail.
+- **TuxGuitar** [unverified, prior knowledge]: LGPL, Java desktop app, reads and writes GP3–5, and is not ideal as a server library.
+
+### Gaps
+- alphaTab's license and its exporter's coverage of techniques (bends, slides, palm mute) were not verified this session.
+- ApolloTab's maintenance, maturity, and actual GP8 write fidelity are unknown.
+- The GP7/8 .gp format (zip + "Content/score.gpif" XML) has no official public spec. [unverified, prior knowledge]
+
+## Q8. Evaluation metrics, and comparing against reference Guitar Pro tabs
+
+### Takeaway
+The standard metrics are:
+- **Multi-pitch and note F1** (onset ±50 ms, "F50"), via mir_eval.
+- **Tab F1**: a note counts only if pitch, string, fret, and onset all match.
+- **Tab Disambiguation Rate (TDR)**: string correctness given correct pitch.
+- **Tab accuracy and difficulty scores** for MIDI→tab.
+
+### Cited Findings
+- **TART** reports audio-to-MIDI F50, string/fret Tab F1 with oracle MIDI, and end-to-end Tab F1. It separates out the error propagated from Stage 1, which is 17.75 points. TDR and multipitch F1 are not reported in the version fetched. — [arXiv 2609.11904](https://arxiv.org/html/2609.11904v1)
+- **Fretting-Transformer** uses pitch accuracy, tab accuracy against professional tabs, and a modified difficulty (playability) score. — [arXiv 2506.14223](https://arxiv.org/html/2506.14223v1)
+- **Riley et al.** use onset-only F-measure at 50 ms. — [arXiv 2402.15258](https://arxiv.org/pdf/2402.15258)
+- **SynthTab** reports both multi-pitch and tablature estimation metrics. — [arXiv 2309.09085](https://arxiv.org/pdf/2309.09085)
+
+### Inferences
+- **Proposed benchmark protocol against hand-made GP references**:
+  1. Parse both GP files (alphaTab or PyGuitarPro) to note lists of (onset in beats/seconds, pitch, string, fret, duration, techniques), applying tuning and capo.
+  2. Align tempo or beat grids. Use DTW if the reference has no audio sync.
+  3. Report the following, per track (multiple guitars plus bass need track matching via the Hungarian algorithm on note F1):
+     - (a) note F1 on pitch+onset
+     - (b) Tab F1 on pitch+onset+string+fret
+     - (c) TDR = string-correct / pitch-correct
+     - (d) technique precision and recall per class
+     - (e) rhythm and quantization accuracy at beat level
+     - (f) a playability or difficulty score
+  4. Allow octave- or position-equivalent "acceptable alternative fingering" as a secondary lenient metric. Human tabs often differ in position, as seen in the Fretting-Transformer's Leduc and DadaGP scores in the 72–82% range.
+- [unverified, prior knowledge] TDR originates in TabCNN-era work (Wiggins & Kim 2019). amt-tools implements tab precision, recall, F1, and TDR.
+
+### Gaps
+- No standard public tool was found for "GP-vs-GP" tab diffing.
+
+## Q9. Commercial tools and user sentiment
+
+### Takeaway
+Commercial tools give no numeric accuracy claims. User sentiment is mixed: fine on clean solo guitar, often wrong on self-recorded or full-band audio. Reddit threads did not surface in searches, so sentiment comes from Trustpilot, app stores, and reviews (some vendor-affiliated).
+
+### Cited Findings
+- **Klangio Guitar2Tabs ratings**
+  - Trustpilot reviews range from "accuracy ... is legendary" to one-star: "didn't even come close to being correct" for a user's own recording.
+  - Apple App Store: 3.6/5 from 25 ratings.
+  - An aggregator reports 3.4/5 and crashes on MusicXML export.
+  - — [Trustpilot](https://uk.trustpilot.com/review/klang.io); [App Store](https://apps.apple.com/app/id1589660528); [justuseapp](https://justuseapp.com/en/app/1589660528/guitar2tabs-transcribe-riffs/reviews)
+- **Klangio product details**
+  - Exports Guitar Pro, MIDI, MusicXML, PDF. Has a built-in editor.
+  - Free tier is about a 20-second demo.
+  - One review notes the full transcription not matching the demo.
+  - — [audio2guitar blog (vendor-affiliated)](https://audio2guitar.com/blog/best-ai-guitar-tab-generators); [aipure](https://aipure.ai/products/klangio)
+- **Independent Klangio test** (Aug 2026): accuracy dropped noticeably with phone-recorded or noisy audio. — [agent-finder](https://agent-finder.co/reviews/klangio)
+- **Songsterr** advertises "Paste a YouTube link, and AI will generate accurate tabs in a few minutes." No independent quality evaluation was found. — [App Store](https://apps.apple.com/app/id399211291)
+- **Songscription**: guitar and bass tab generator with Guitar Pro export. Says accuracy is best on clearly audible or solo guitar. — [Songscription](https://www.songscription.ai/guitar-tab-generator)
+- **audio2guitar** (vendor blog)
+  - Separates the guitar, then tabs, chords, and lyrics. MIDI and PDF export only.
+  - 3 free songs, then $8.99/month.
+  - — [audio2guitar](https://audio2guitar.com/blog/best-ai-guitar-tab-generators)
+- **Chordify** does chords only, no tabs or fret positions. **Moises** does stems and chords, no tabs. **Ultimate Guitar** and **Songsterr** are libraries of human tabs (about 1.4M on UG; Songsterr has 400k songs and 1.3M tracks). — [audio2guitar](https://audio2guitar.com/blog/best-ai-guitar-tab-generators)
+
+### Inferences
+- A differentiator for the project could be published, reproducible benchmark numbers against GP references. No commercial vendor publishes any.
+
+### Gaps
+- No info was found on Guitar Pro 8.x "audio import" or transcription features. Its existence or quality is unverified.
+- No Reddit or r/guitarpro sentiment threads surfaced.
+- No independent Songsterr AI-tab quality reviews were found.
+- No numbers were found for Ultimate Guitar AI features.

research_notes/Music transcription tooling survey/pitched_transcription.md

@@ -0,0 +1,216 @@
+# Open-source automatic transcription of pitched instruments (audio to MIDI/notes), as of October 2026
+
+Scope: piano/keyboard, bass, vocals, and general multi-instrument transcription. Guitar-tab (string/fret) models and drum transcription are covered by other researchers, so they are only mentioned here in passing.
+Research date: 2026-10-30. Items marked **[unverified]** come from prior background knowledge and were not re-fetched this session. The report writer should treat them as lower confidence.
+
+## Q1. Spotify Basic Pitch (Python and basic-pitch-ts in the browser): accuracy, polyphony, pitch bends, complaints
+
+### Takeaway
+Basic Pitch is the most permissive and easiest-to-deploy option: Apache-2.0, instrument-agnostic, polyphonic, outputs pitch bends, and runs in the browser through basic-pitch-ts / basicpitch.io. It produces no instrument labels and works best on one instrument at a time. In 2025-26 benchmarks it now sits well behind newer models, both on multi-instrument scoring and on instrument-agnostic frame F1.
+
+### Cited Findings
+- **License and platforms.** Apache-2.0 (Spotify AB, 2022). The Python package ships the TF model converted to CoreML, TFLite and ONNX. Defaults are CoreML on macOS, TFLite on Linux and ONNX on Windows. TensorFlow is optional (`basic-pitch[tf]`). Supports Python 3.7–3.11. — [GitHub spotify/basic-pitch](https://github.com/spotify/basic-pitch)
+- **Capabilities.** "Supports polyphonic instruments" with multipitch support, and outputs MIDI "complete with pitch bends". It is instrument-agnostic, and the README says "Basic pitch works best on one instrument at a time". Stereo is down-mixed to mono and audio is resampled to 22050 Hz. — [GitHub spotify/basic-pitch](https://github.com/spotify/basic-pitch)
+- **Community and activity.** About 6.0k stars, 546 forks, 49 open issues and 16 open PRs at fetch time. No recent release was visible on the page. — [GitHub spotify/basic-pitch](https://github.com/spotify/basic-pitch)
+- **Browser version.** basic-pitch-ts is Apache-2.0 and published on npm as `@spotify/basic-pitch`. Usage is browser-style (AudioContext, decodeAudioData), and the web demo is basicpitch.io. The repo has 15 commits, 9 open issues and 4 open PRs. The page does not name the runtime; it is believed to be TF.js **[unverified]**. — [GitHub spotify/basic-pitch-ts](https://github.com/spotify/basic-pitch-ts)
+- **Model size.** The ICASSP 2022 paper describes a very small model (on the order of ~17K parameters, low memory) **[unverified, not stated on the README]**. — [arXiv 2203.09893](https://arxiv.org/abs/2203.09893)
+- **2025 AMT Challenge.** Multi-instrument, instrument-aware Onset F1 was **0.0634**, last of 14 entries. MT3 scored 0.3932 and the winner 0.5998. Basic Pitch was also the fastest entry at 3.91 ms runtime. The low score is mostly because it emits no instrument labels, while the metric requires instrument+pitch+onset to match. — [arXiv 2603.27528 (AMT Challenge 2025 results)](https://arxiv.org/html/2603.27528v1)
+- **Harmonica comparison (Sep 2026, ICASSP 2027 submission).** The Harmonica-nano model (26.3K params) reports frame F1 0.796 on its dev set, "14.6 percentage points higher than Basic Pitch" (which implies about 0.65 for Basic Pitch). It runs at 1,622.5x real time. — [arXiv 2609.04640](https://arxiv.org/abs/2609.04640)
+- **User complaints.** A search for Reddit/forum complaints returned nothing relevant (see Gaps).
+
+### Inferences
+- For a browser-first app that needs unlabeled notes from a single isolated instrument or stem, Basic Pitch is still the default because of its license and footprint. For labeled multi-instrument output it is not competitive.
+- Harmonica could become a stronger drop-in replacement for Basic Pitch, being instrument-agnostic and even smaller, but its code and license are unknown.
+
+### Gaps
+- No primary-source user sentiment was found for Basic Pitch. Commonly reported anecdotes (ghost notes and octave errors on dense mixes, fragmented notes on vibrato or bends) are **[unverified]**.
+- Exact model file size of basic-pitch-ts, and whether a WebGPU backend exists, are not confirmed.
+- Harmonica's code, weights and license are not stated on the arXiv abstract.
+
+## Q2. Multi-instrument models: MT3, YourMT3/YourMT3+, MR-MT3, and 2025-2026 successors (MuScriptor, MIROS, AMT Challenge)
+
+### Takeaway
+As of October 2026, the clear quality leader on real mixed recordings is **Kyutai's MuScriptor** (July 2026, ISMIR 2026). Its code is MIT but its **weights are CC BY-NC 4.0** (non-commercial). On a real-music test set it roughly doubles YourMT3+'s Multi F1 (48.2 vs 21.9). YourMT3+ (GPL-3.0 code) remains the strongest commercially-ambiguous or academic baseline and wins on Slakh-like synthetic data. The 2025 AMT Challenge winner, MIROS (a MusicFM encoder plus YourMT3+-style decoders), only narrowly beat YourMT3+ on synthetic data.
+
+### Cited Findings
+- **MuScriptor: authorship and training.** By Rouard, Krause, Roebel, Simon-Gabriel and Défossez (Kyutai/Mirelo). arXiv 2607.08168 v1 posted 9 Jul 2026, v2 on 3 Aug 2026, ISMIR 2026 camera-ready. Training runs in three stages: synthetic pre-training, then fine-tuning on real audio, then RL post-training. It adds instrument-presence conditioning. — [arXiv 2607.08168](https://arxiv.org/abs/2607.08168)
+- **MuScriptor: architecture.** Decoder-only Transformer. Input is a 5 s mono 16 kHz mel-spectrogram (512 mel bins, 100 Hz frames) with an optional instrument list. Output is MT3-style tokens over 36 instrument subgroups, including Voice, Acoustic Piano, E-Piano, E-Bass, Acoustic Bass, Acoustic Guitar, Clean E-Guitar and Distorted E-Guitar. — [arXiv 2607.08168v2 HTML](https://arxiv.org/html/2607.08168v2)
+- **MuScriptor: data.** About 1.45M MIDI files rendered with more than 250 soundfonts, plus about 170k real recordings (more than 11,000 h) with aligned note annotations. The test set is 372 held-out real tracks. — [arXiv 2607.08168v2](https://arxiv.org/html/2607.08168v2)
+- **MuScriptor: results on its real test set.** F1 in %:
+
+  | Model | Onset | Frame | Offset | Drums | Multi |
+  |---|---|---|---|---|---|
+  | YourMT3+ | 32.52 | 45.54 | 17.79 | 41.4 | 21.9 |
+  | MuScriptor 1.3B | 60.4 | 73.3 | 49.0 | 50.2 | 48.2 |
+
+  — [arXiv 2607.08168v2](https://arxiv.org/html/2607.08168v2)
+- **MuScriptor vs YourMT3+ per dataset (Multi F1).**
+  - MuScriptor ahead: Bach10 34.7 vs 26.4; Dagstuhl ChoirSet 11.5 vs 2.6; PHENICX 25.7 vs 12.2; RWC-Pop 25.6 vs 19.1; RWC-Genre 23.7 vs 17.2; RWC-Jazz 31.8 vs 26.4.
+  - YourMT3+ ahead: RWC-Classical 40.5 vs 36.0; RWC-Royalty-free 23.1 vs 20.3.
+  - The paper gives no per-instrument (bass/guitar/vocal) F1 and no comparison with MT3 or Basic Pitch.
+  — [arXiv 2607.08168v2](https://arxiv.org/html/2607.08168v2)
+- **MuScriptor: release.**
+  - Code is MIT. Weights are CC BY-NC 4.0 (non-commercial) and gated on Hugging Face (accept the license, then log in).
+  - Variants: small 103M (described as "the practical choice on CPU-only machines"), medium 307M (default), and large 1.4B (which "really wants a GPU"). Apple MPS is supported.
+  - Install with `pip install muscriptor` or `uvx muscriptor serve`. The CLI is `muscriptor transcribe`. Sheet output needs MuseScore 4.
+  - Hosted demo at muscriptor.kyutai.org. No ONNX or browser inference is mentioned.
+  — [GitHub muscriptor/muscriptor](https://github.com/muscriptor/muscriptor)
+- **Discrepancy on parameter counts.** The paper lists about 60M/100M/300M/1.3B. The repo lists 103M/307M/1.4B. — [paper](https://arxiv.org/html/2607.08168v2) vs [repo](https://github.com/muscriptor/muscriptor)
+- **2025 AMT Challenge: setup.** Accepted to the NeurIPS 2025 AI for Music workshop and posted on arXiv in March 2026. Eight teams took part. The test set is 76 newly composed pieces (about 20 s each, at most 3 instruments) covering piano, violin, viola, cello, flute, bassoon, trombone and oboe. All audio was synthesized with FluidSynth and the FluidR3 GM soundfont. — [arXiv 2603.27528](https://arxiv.org/html/2603.27528v1)
+- **2025 AMT Challenge: results (Multi Onset F1).**
+
+  | Entry | Multi Onset F1 |
+  |---|---|
+  | MIROS (winner) | 0.5998 |
+  | YourMT3-YPTF-MoE-M | 0.5938 |
+  | YourMT3-YPTF-S | 0.5581 |
+  | MT3 baseline | 0.3932 |
+  | ByteDance Piano | 0.1721 |
+  | ReconVAT | 0.1415 |
+  | Basic Pitch | 0.0634 |
+
+  — [arXiv 2603.27528](https://arxiv.org/html/2603.27528v1)
+- **MIROS (winner).** A MusicFM conformer SSL encoder, a recurrent adapter conditioned on instrument-group embeddings, and parallel T5-style decoders, about 370M params. It scores Slakh2100 multi F 0.83. It is below YourMT3+ on Slakh but slightly better on challenge data, and the authors suggest YourMT3+ may overfit Slakh. — [arXiv 2603.27528](https://arxiv.org/html/2603.27528v1)
+- **Effect of polyphony and common failures.** F1 drops sharply as instrument count grows (1 to 3 instruments): MT3 0.548 to 0.256; MIROS 0.719 to 0.437; YourMT3-MoE 0.759 to 0.392. Common failures are instrument leakage (hallucinated instruments) and confusion in dense polyphony. — [arXiv 2603.27528](https://arxiv.org/html/2603.27528v1)
+- **YourMT3 / YourMT3+.** Repo code is GPL-3.0, and the weights license is not separately stated. YourMT3+ was published at MLSP 2024 and its code is marked "pre-release". A Hugging Face Space demo runs on a free GPU (launched Aug 2024). The newest dated news item is Nov 2024 ("YouTube is blocked again"), there are only 30 commits, and the repo appears lightly maintained. — [GitHub mimbres/YourMT3](https://github.com/mimbres/YourMT3)
+- **YourMT3+ training data and paper.** Training includes a synthetic dataset of bass-guitar techniques across several basses and pickups, and uses cross-dataset stem augmentation. — [arXiv 2407.04822](https://arxiv.org/pdf/2407.04822); [IEEE](https://ieeexplore.ieee.org/document/10734819/)
+- **MT3 (Google, 2021).** Original multi-task multitrack T5 transcription, now the baseline. — [arXiv 2111.03017](https://arxiv.org/pdf/2111.03017). Code and checkpoints are Apache-2.0, built on JAX/T5X **[unverified]**.
+
+### Inferences
+- **Synthetic vs real.** Slakh and other synthetic results greatly overstate real-world quality. YourMT3+ scores around 0.8+ on Slakh-like data but about 0.22 Multi F1 on MuScriptor's real test set. Even MuScriptor tops out below 0.5 Multi F1 on real music. Instrument labeling, not pitch detection, is the hardest part.
+- **Commercial use.** MuScriptor's weights are non-commercial and YourMT3 is GPL-3.0 (copyleft, which matters if the code is linked). The only permissive multi-instrument option found is MT3 (Apache **[unverified]**), and it is the weakest of the three.
+- **Browser.** None of the multi-instrument Transformer models has a browser/ONNX port. The models are 100M–1.4B params, so client-side inference would be heavy. MuScriptor-small at 103M might be feasible with WebGPU, but nobody is documented doing it.
+
+### Gaps
+- MR-MT3 (memory-retaining MT3, which reduces instrument leakage) was not re-checked this session. Its license and metrics are unknown.
+- No MIREX 2025/2026 multi-F0 or note-tracking results were found.
+- No per-instrument F1 for bass, guitar or vocals from MuScriptor or YourMT3+ on real audio.
+- Diffusion-based multi-instrument transcription papers were not found in this session's searches.
+
+## Q3. Piano transcription: Omnizart, Onsets & Frames, ByteDance high-resolution, hFT-Transformer, Transkun, newest SOTA
+
+### Takeaway
+Piano is effectively solved on MAESTRO: about 97% note-onset F1 and about 93% note+offset F1 for the best models (the onset figure is **[unverified]**). **Transkun** (about 12.9M params, 93.48% note-with-offset F1 on MAESTRO v3) is the best-documented practical SOTA and is pip-installable. The ByteDance repo is now archived, and newer 2025-26 papers (D3RM diffusion refinement, optimal-transport training, sparse-attention transformers) make only marginal gains.
+
+### Cited Findings
+- **Head-to-head on MAESTRO v3, note-with-offset F1.** hFT-Transformer (5.5M params) scores 90.53% (P 92.52, R 88.69). Transkun (12.9M) scores 93.48% (P 94.61, R 92.39). A 2026 optimal-transport model scores 90.78%. That paper attributes offset errors to key release and sustain pedal, and it does not model pedal. — [arXiv 2605.17405](https://arxiv.org/pdf/2605.17405); same numbers in [D3RM arXiv 2501.05068](https://arxiv.org/pdf/2501.05068)
+- **Transkun V2 checkpoint.** On MAESTRO V3 it reports note onset+offset F1 0.9349. The pip-shipped checkpoint is trained without pedal extension of notes and with data augmentation. — [GitHub Yujia-Yan/Transkun](https://github.com/yujia-yan/transkun)
+- **Other recent piano papers.** hFT-Transformer (ISMIR 2023) — [ISMIR archive](https://archives.ismir.net/ismir2023/paper/000024.pdf). Non-hierarchical Transformer interval scoring (Transkun v2 paper) — [arXiv 2404.09466](https://arxiv.org/pdf/2404.09466). Efficient sparse-attention piano transcription (Sep 2025) — [arXiv 2509.09318](https://arxiv.org/pdf/2509.09318). D3RM discrete diffusion refinement (Jan 2025) — [arXiv 2501.05068](https://arxiv.org/pdf/2501.05068)
+- **Robustness on out-of-distribution audio.** Augmentation and re-performance training reached 88.4 note-onset F1 on MAPS without using MAPS training data. — [arXiv 2402.01424](https://arxiv.org/pdf/2402.01424)
+- **Benchmark ceiling and evaluation.** SOTA systems exceed 90% frame-level and 80% note-level F1 (with offsets). The paper argues IR metrics are not musically informed. — [arXiv 2406.08454](https://arxiv.org/html/2406.08454v3)
+- **ByteDance piano_transcription (high-resolution, Kong et al.).** Code is Apache 2.0 and the checkpoint license is not stated. The repo was **archived by the owner on Dec 8, 2025 (read-only)**. Inference uses the `piano_transcription_inference` pip package. The README reports test frame_ap 0.9285, onset MAE 0.097 and offset MAE 0.1353, plus pedal transcription. — [GitHub bytedance/piano_transcription](https://github.com/bytedance/piano_transcription)
+- **ByteDance on non-piano music.** As a piano-only model on the multi-instrument challenge it scored 0.1721. — [arXiv 2603.27528](https://arxiv.org/html/2603.27528v1)
+- **Omnizart.** MIT license. Modules: music (pitched instruments), vocal (note-level), vocal-contour (F0), chord, drum and beat. There is no archived notice, but the README says it is incompatible with ARM macOS and the drum model has unresolved training bugs. 24 open issues and 4 open PRs. No recent release was visible, and it is TensorFlow-based. — [GitHub Omnizart](https://github.com/Music-and-Culture-Technology-Lab/omnizart)
+- **Onsets & Frames.** Part of Magenta (Apache-2.0), and the MAESTRO paper baseline. — [arXiv 1810.12247](https://arxiv.org/pdf/1810.12247v3). A TF.js Onsets & Frames port exists in magenta-js (@magenta/music) **[unverified]**, which is a historical browser option.
+
+### Inferences
+- **Practical choice.** For commercially usable piano transcription, Transkun (12.9M) or ByteDance (archived but Apache code) are the practical picks.
+- **Maintenance.** Omnizart appears effectively unmaintained (TF-era, ARM-Mac incompatible), though not formally archived.
+- **Browser.** Piano models are small (5.5–13M params), so ONNX-web/WebGPU ports are feasible in principle. No maintained browser port was found except the old Magenta.js Onsets & Frames.
+
+### Gaps
+- Transkun's license was not captured this session (believed MIT **[unverified]**).
+- hFT-Transformer's license is likewise uncaptured (Sony, believed MIT **[unverified]**).
+- No Papers-with-Code-style leaderboard for 2026 was found. Papers with Code itself was shut down in 2025 **[unverified]**.
+- No quantified user sentiment was gathered.
+
+## Q4. Vocal melody transcription (F0 and notes) and lyrics alignment
+
+### Takeaway
+The de facto open pipeline for sung notes is **RMVPE (F0) followed by ROSVOT (notes)**, with SOME as a lighter alternative. ROSVOT's checkpoint is mainly trained on Mandarin. For F0 alone:
+- **FCPE** is the fast, accurate choice.
+- **PESTO** is a tiny, CPU-friendly self-supervised choice.
+- **CREPE/torchcrepe** is accurate but slow.
+
+MuScriptor also emits a "Voice" track, but vocal-specific metrics are not reported.
+
+### Cited Findings
+- **ROSVOT (ACL 2024).** Official PyTorch code provides checkpoints for ROSVOT, RMVPE and a word-boundary model. It handles noisy and accompanied singing, and the paper reports SOTA on clean and noisy input. — [GitHub ROSVOT](https://github.com/RickyL-2000/ROSVOT); [ACL Anthology](https://aclanthology.org/2024.acl-long.526/); [arXiv 2405.09940](https://arxiv.org/html/2405.09940)
+- **RMVPE-to-ROSVOT pipeline in use.** It was used again in 2025 dataset work (MRSAudio). — [arXiv 2510.10396](https://arxiv.org/html/2510.10396v3)
+- **TelkNet tool.** It offers SOME (described as fast and lightweight), GAME, ROSVOT and VOCANO side by side, and notes that ROSVOT's released checkpoint is mainly trained on Mandarin singing. — [TelkNet vocal-to-MIDI](https://telknet.cc/en/tools/music-vocal-midi)
+- **FCPE (arXiv Sep 2025).** 96.79% RPA on MIR-1K, comparable to RMVPE. It is 10.64M params vs 90.42M for RMVPE and 1.06 vs 4.91 GFLOPs. RTF is 0.0062 on an RTX 4090, about 5.3x faster than RMVPE and 77x faster than CREPE. — [arXiv 2509.15140](https://arxiv.org/html/2509.15140v1)
+- **PESTO (Sony CSL).** Self-supervised. It processes a 2m51s file in about 13 s on a laptop CPU vs about 12 min for CREPE. Accuracy is close to CREPE with about 800x fewer params, and it outperforms other SSL baselines on MIR-1K and MDB-stem-synth. — [GitHub SonyCSLParis/pesto](https://github.com/SonyCSLParis/pesto); [PyPI](https://pypi.org/project/pesto-pitch/0.1.0/)
+- **Lyrics.** One 2025-26 pipeline (StepAudio 3 Music) separates vocals with Demucs before lyric recognition, saying this "reduces interference from instrumentation and improves the reliability" of lyric predictions. — [arXiv 2609.16034](https://arxiv.org/pdf/2609.16034)
+
+### Inferences
+- **Browser.** PESTO (tiny) and FCPE (about 10M) are realistic for browser F0 via ONNX. ROSVOT/SOME note segmentation could follow on stems.
+- **Vocal transcription is a two-stage problem.** Pitch is near-solved. Note segmentation and quantization (vibrato, slides, melisma) are the weak link.
+
+### Gaps
+- Licenses of ROSVOT, SOME, RMVPE, FCPE and PESTO were not captured this session. PESTO is believed LGPL-3.0, SOME MIT, and RMVPE/FCPE MIT/Apache in RVC/DDSP-SVC ecosystems **[all unverified]**.
+- No head-to-head note-level COnPOff F1 comparison of ROSVOT vs SOME vs VOCANO vs MusicYOLO was found.
+- Whisper-based lyrics alignment tools were not researched in depth this session. Candidates: WhisperX forced alignment, stable-ts, and lyrics-aligner models **[unverified]**.
+
+## Q5. Bass transcription and general-model performance on guitar/bass
+
+### Takeaway
+No established, maintained, dedicated open bass-to-MIDI model was found. In practice people use (a) Basic Pitch or pitch trackers on a Demucs bass stem (bass is mostly monophonic, which suits this), or (b) the bass track from YourMT3+/MuScriptor, both of which label E-Bass and Acoustic Bass. No published per-instrument bass F1 on real music was found.
+
+### Cited Findings
+- **No dedicated electric bass solution.** A 2024 HKU student project notes "a lack of automatic transcription solutions for the electric bass guitar" and builds a monophonic model on synthetic data. — [HKU final paper](https://wp2023.cs.hku.hk/fyp23026/wp-content/uploads/sites/27/final_paper.pdf)
+- **Double bass.** The FiloBass dataset is in Riley's 2025 QMUL PhD thesis on jazz ensemble transcription. — [Riley PhD thesis](https://webspace.eecs.qmul.ac.uk/s.e.dixon/phd/XavierRiley-PhD-Thesis.pdf)
+- **YourMT3+ bass training.** It trains on a synthetic bass-technique dataset recorded on multiple basses and pickups. — [arXiv 2407.04822](https://arxiv.org/pdf/2407.04822)
+- **MuScriptor labels.** Its instrument vocabulary includes E-Bass, Acoustic Bass, Acoustic Guitar, Clean E-Guitar and Distorted E-Guitar, and its demo shows electric guitar and bass with tablature. Per-instrument scores are not given. — [arXiv 2607.08168v2](https://arxiv.org/html/2607.08168v2); [GitHub muscriptor](https://github.com/muscriptor/muscriptor)
+- **Guitar data.** The GOAT guitar dataset (2025) is relevant to guitar generalization on real recordings. Tab models are out of scope here. — [arXiv 2509.22655](https://arxiv.org/pdf/2509.22655)
+
+### Inferences
+- A Demucs bass stem plus a monophonic F0/note tracker (PESTO/FCPE plus note segmentation, or Basic Pitch) is likely the most robust browser-friendly bass approach. The low register (E1 at about 41 Hz) stresses frequency resolution, which is worth testing explicitly.
+
+### Gaps
+- No quantitative bass or guitar results were found for Basic Pitch, YourMT3+ or MuScriptor on real stems.
+- No user reports on bass transcription quality were found.
+
+## Q6. Does transcribing separated stems beat the mixture? And browser feasibility and model sizes
+
+### Takeaway
+No paper was found in this session that directly reports transcription F1 on separated stems vs mixtures. The evidence is qualitative (lyrics pipelines, vendor guidance) plus the structural argument that single-instrument models like Basic Pitch "work best on one instrument at a time". On browser feasibility, only Basic Pitch has an official browser build. Piano models (5–13M) and F0 models (PESTO, FCPE) are small enough to port. Multi-instrument Transformers (100M–1.4B) are not practical client-side today.
+
+### Cited Findings
+- **Basic Pitch on single instruments.** "Basic pitch works best on one instrument at a time." — [GitHub spotify/basic-pitch](https://github.com/spotify/basic-pitch)
+- **Separating vocals before lyric recognition.** Demucs separation before lyrics recognition "reduces interference ... improves the reliability". This is a qualitative claim. — [arXiv 2609.16034](https://arxiv.org/pdf/2609.16034)
+- **Vendor guidance.** Songscription says stems help mainly when parts mask each other in dense mixes, and that a full mix is otherwise fine. It warns about bleed between separated stems. This is a vendor opinion. — [Songscription blog](https://www.songscription.ai/blog/separate-stems-before-transcribing)
+- **Demucs.** Separates drums, bass, vocals and other. — [GitHub facebookresearch/demucs](https://github.com/facebookresearch/demucs)
+- **Polyphony hurts multi-instrument models.** Multi-instrument F1 roughly halves going from 1 to 3 instruments, which indirectly supports separation as preprocessing. — [arXiv 2603.27528](https://arxiv.org/html/2603.27528v1)
+- **YourMT3+ and stems in training.** YourMT3+ uses "cross-dataset stem augmentation" in training, i.e. stem mixing, not inference-time separation. — [arXiv 2407.04822](https://arxiv.org/pdf/2407.04822)
+- **Model sizes.**
+  - Harmonica-nano: 26.3K — [arXiv 2609.04640](https://arxiv.org/abs/2609.04640)
+  - hFT-Transformer: 5.5M; Transkun: 12.9M — [arXiv 2605.17405](https://arxiv.org/pdf/2605.17405)
+  - FCPE: 10.64M; RMVPE: 90.42M — [arXiv 2509.15140](https://arxiv.org/html/2509.15140v1)
+  - MIROS: about 370M — [arXiv 2603.27528](https://arxiv.org/html/2603.27528v1)
+  - MuScriptor: 103M / 307M / 1.4B — [GitHub muscriptor](https://github.com/muscriptor/muscriptor)
+
+### Inferences
+- **Recommended pipeline.** For mixed recordings, use separation (Demucs or a successor) followed by a per-stem specialist:
+  - Piano stem: Transkun
+  - Vocal stem: RMVPE/FCPE plus ROSVOT/SOME
+  - Bass stem: a monophonic tracker or Basic Pitch
+
+  This is likely the best permissively-licensed option. The quantitative benefit is unproven, and separation artifacts and bleed can create false notes.
+
+### Gaps
+- No controlled stems-vs-mix F1 study was found. The 2021 MT3 paper and 2022-23 Demucs+transcription studies may contain such ablations **[unverified]**.
+- ONNX/WebGPU ports of Transkun, PESTO and FCPE were not verified.
+
+## Q7. Quantizing performance MIDI into notation (score-like MIDI / MusicXML)
+
+### Takeaway
+There are three open research lines:
+- **PM2S** (Liu et al., ISMIR 2022): CRNN plus neural beat tracking. It has a toolbox and beats commercial software on MV2H.
+- **End-to-end Transformer PM-to-score** (Beyer and Dai, 2024): RoFormer seq2seq to MusicXML tokens. This is commonly referred to as "MIDI2ScoreTransformer" **[naming unverified]**.
+- **A 2026 T5-based beat-aware rhythm quantizer** (arXiv 2604.22290).
+
+MuScriptor sidesteps the problem for end users by exporting sheet music via MuseScore 4.
+
+### Cited Findings
+- **PM2S (ISMIR 2022).** CRNN plus beat tracking for rhythm quantization. It is significantly better than commercial software on MV2H. Toolbox: github.com/cheriell/PM2S. — [ISMIR 2022 paper](https://archives.ismir.net/ismir2022/paper/000047.pdf)
+- **Beyer and Dai (2024).** End-to-end piano performance-MIDI-to-score with Transformers. Frames it as seq2seq producing MusicXML tokens, uses unpaired scores, and uses ASAP data (222 scores, 1068 performances). — [arXiv 2410.00210](https://www.alphaxiv.org/abs/2410.00210)
+- **Transformer rhythm quantization with beat annotations (Apr 2026).** T5-based, with beat-based pre-quantization and a MIDI tokenizer. It cites the 2024 RoFormer PM-to-score approach. — [arXiv 2604.22290](https://arxiv.org/html/2604.22290v1)
+- **ScoreTranscriber (KAIST MACLab).** Two stages: rhythm quantization plus hand assignment, then a Transformer that produces MusicXML. It is a research prototype. — [GitHub NZX23/ScoreTranscriber](https://github.com/NZX23/ScoreTranscriber)
+- **MuScriptor sheet output.** Sheet-music output requires MuseScore 4+. — [GitHub muscriptor](https://github.com/muscriptor/muscriptor)
+
+### Inferences
+- **Piano vs everything else.** These tools are almost entirely piano-trained (ASAP). For vocals, bass or multi-instrument output, beat tracking (e.g. Beat This!, madmom; **[unverified]**) plus grid quantization in music21/MuseScore remains the practical route.
+
+### Gaps
+- Licenses and maintenance status of PM2S, the Beyer and Dai code, and ScoreTranscriber were not captured.
+- No browser-capable PM-to-score model was found.

research_notes/Music transcription tooling survey/rhythm_notation_ingestion.md

@@ -0,0 +1,201 @@
+# Supporting tooling for an audio-to-sheet-music website: rhythm, harmony, notation, ingestion (as of Oct 2026)
+
+Scope note: about 17 search/fetch calls. Several sub-questions (Chordino, autochord, music21 quantization, PyGuitarPro, MuseScore CLI, abcjs, VexFlow, flat.io) could not be verified within budget and are listed under Gaps. Where a Gap gives a background-knowledge hint, it is labelled UNVERIFIED and must not be cited as fact.
+
+## Beat / downbeat / tempo / meter tracking
+
+### Takeaway
+madmom is effectively unmaintained on PyPI: the last release was in 2018, and the model files are non-commercial. Beat This! (CPJKU, ISMIR 2024, MIT code and weights, CPU-capable, 8 MB small model) is the best default for a commercial site. allin1 (MIT) adds song structure but pulls in madmom and NATTEN. essentia.js is the only mature in-browser option, but it is AGPLv3.
+
+### Cited Findings
+- **madmom status:**
+  - The latest PyPI release is 0.16.1 (Nov 14, 2018). Its classifiers list only Python 2.7 and 3.5–3.7, and it ships only an sdist with no wheels. — [PyPI madmom](https://pypi.org/project/madmom/)
+  - Code is BSD. Model and data files are CC BY-NC-SA 4.0, and commercial use requires contacting Gerhard Widmer (JKU). — [GitHub CPJKU/madmom](https://github.com/CPJKU/madmom)
+  - The repo shows 60 open issues and 22 open PRs, and the README still lists Python 2.7. — [GitHub CPJKU/madmom](https://github.com/CPJKU/madmom)
+- **madmom on modern Python:**
+  - BeatNet's docs warn that madmom may throw errors on Python ≥3.10 with NumPy ≥1.24. The fixes are a `collections` → `collections.abc` import patch and removing `np.int`/`np.float`. — [BeatNet docs](https://docsearch.algolia.com/mcp/docs/repo/mjhydri/beatnet)
+  - A 2024 student project hit the same failure on Python 3.12. — [Rochester ECE477 project](https://hajim.rochester.edu/ece/sites/zduan/teaching/ece477/projects/2024/JiajunWu_SebastianXu_ProjectPresentation.pdf)
+  - The maintainer historically advised installing from GitHub source instead of PyPI. — [madmom issue mirror](https://git.tdem.in/CPJKU/madmom/issues/478)
+- **Beat This!:**
+  - A transformer beat and downbeat tracker, "Accurate Beat Tracking Without DBN Postprocessing" (Foscarin, Schlüter, Widmer, ISMIR 2024). — [Zenodo](https://zenodo.org/records/13922116); [beat_this_annotations](https://github.com/CPJKU/beat_this_annotations)
+  - Code and weights are MIT. Some training data has restrictive licenses, and users are told to assess whether that affects them. — [GitHub CPJKU/beat_this](https://github.com/CPJKU/beat_this)
+  - Install is `pip install beat-this` with PyTorch ≥2.0. Main models are about 78 MB and "small" models about 8.1 MB. It falls back to CPU automatically, and madmom is needed only for the optional DBN. — [GitHub CPJKU/beat_this](https://github.com/CPJKU/beat_this)
+  - Third-party leaderboards report downbeat F1 of 96.6 (HJDB), 90.7 (Harmonix) and 88.8 (Beatles). These are aggregator figures that I have not checked against the paper. — [hyper.ai HJDB](https://hyper.ai/ja/sota/tasks/downbeat-tracking/benchmark/downbeat-tracking-on-hjdb)
+  - Community ports exist: a Rust crate (MIT) and a GGUF conversion. — [docs.rs beat-this](https://docs.rs/crate/beat-this/0.1.0); [HF cstr/beat-this-GGUF](https://huggingface.co/cstr/beat-this-GGUF)
+  - A hosted Replicate version exists. — [Replicate xavriley/beat_this](https://replicate.com/xavriley/beat_this/readme)
+- **allin1 (All-In-One music structure analyzer):**
+  - MIT licensed. It outputs BPM, beats, downbeats, beat positions (1, 2, 3, 4…), and segment boundaries labelled start/end/intro/outro/break/bridge/inst/solo/verse/chorus. — [GitHub mir-aidj/all-in-one](https://github.com/mir-aidj/all-in-one)
+  - Dependencies are PyTorch, NATTEN (built from source on Windows), madmom from GitHub, and Demucs. — [GitHub mir-aidj/all-in-one](https://github.com/mir-aidj/all-in-one)
+  - The README warns that MP3 decoders introduce 20–40 ms offsets and recommends converting to WAV first. — [GitHub mir-aidj/all-in-one](https://github.com/mir-aidj/all-in-one)
+  - Repo stats: 871 stars, 16 open issues, 38 commits, no tagged releases. — [GitHub mir-aidj/all-in-one](https://github.com/mir-aidj/all-in-one)
+- **BeatNet:**
+  - BeatNet has online (causal) and offline modes, and the offline mode uses a DBN. It depends on madmom, so it inherits madmom's Python 3.12 problems. — [BeatNet docs](https://docsearch.algolia.com/mcp/docs/repo/mjhydri/beatnet)
+- **essentia / essentia.js:**
+  - Both are AGPLv3, with a commercial dual license available from UPF. — [Essentia licensing](https://essentia.upf.edu/licensing_information.html); [essentia.js README](https://unpkg.com/[email protected]/README.md)
+  - Beat tracking and BPM estimation run in the browser and Node. The README says it is "under rapid development", so APIs may change. — [essentia.upf.edu](https://essentia.upf.edu/); [essentia.js README](https://unpkg.com/[email protected]/README.md)
+  - One production report gives a WASM binary of about 2 MB, a 1–3 s cold init, and KeyExtractor run in Web Workers. — [dev.to TagMyBeat](https://dev.to/tagmybeat/i-built-a-free-music-tool-i4l)
+
+### Inferences
+- For a commercial public site, avoid madmom's pretrained models (NC license) and the essentia/essentia.js AGPL unless you buy a commercial license. That leaves Beat This! (MIT) as the safest server-side choice.
+- allin1's structure labels are attractive, but it depends on madmom, so it inherits both the packaging fragility and the madmom model licence question. Check which madmom components it actually uses before relying on it commercially.
+- Browser beat tracking is feasible:
+  - essentia.js works directly in the browser, subject to the AGPL.
+  - A small Beat This! model (8 MB) could plausibly run via ONNX Runtime Web after a PyTorch → ONNX export. No official web build was found, so this is an untested inference.
+- Rubato and odd meters: Beat This! drops the DBN, which in older trackers imposes tempo and meter priors. That plausibly helps with tempo changes. No source in this session quantified accuracy on rubato or odd meters.
+
+### Gaps
+- No source found on Beat This! or allin1 accuracy specifically on tempo changes, rubato or odd meters (5/4, 7/8).
+- Could not confirm whether madmom ever published a newer release (0.17) via GitHub. PyPI shows 0.16.1 only.
+- No 2025–26 beat-tracking models beyond Beat This! surfaced in searches. No reliable source either way.
+- librosa was not researched this session. UNVERIFIED background knowledge: it is ISC-licensed and offers a basic DP beat tracker, with no downbeats.
+
+## Key detection and chord recognition
+
+### Takeaway
+The open options are dated but usable. ChordFormer (Feb 2025, conformer, large vocabulary) is the newest academic SOTA claim, but code availability is unconfirmed. BTC (ISMIR 2019) has a usable 170-chord checkpoint. Essentia's key extraction is mature but AGPL.
+
+### Cited Findings
+- **ChordFormer (arXiv 2502.11840, Feb 2025):** a conformer-based large-vocabulary chord recognition model. It reports +2% frame-wise accuracy and +6% class-wise accuracy over prior SOTA. — [arXiv](https://arxiv.org/abs/2502.11840v1); [HF papers](https://huggingface.co/papers/2502.11840)
+- **BTC (bi-directional transformer, ISMIR 2019):**
+  - The large-vocabulary checkpoint covers 170 chord classes, plus a 25-class maj/min model. — [HF puar-playground/btc-chord](https://huggingface.co/puar-playground/btc-chord)
+  - A repo mirror is available. — [GitHub BTC-ISMIR19 mirror](https://github.com/qf6101/BTC-ISMIR19)
+- **CREMA:** a chord model based on McFee & Bello's ISMIR 2017 structured-prediction model. — [CREMA docs](https://crema.readthedocs.io/en/stable/models.html)
+- **Extended chord qualities:** 2025 work evaluates a convolutional-transformer ACR model on extended chord qualities. — [SBCM 2025](https://sol.sbc.org.br/index.php/sbcm/article/view/43948)
+- **LLM refinement:** a 2025 paper uses a 301-class large-vocabulary chord model followed by LLM refinement. — [arXiv 2509.18700](https://www.arxiv.org/pdf/2509.18700)
+- **Essentia key detection:** KeyExtractor works in the browser via essentia.js under AGPLv3. — [dev.to TagMyBeat](https://dev.to/tagmybeat/i-built-a-free-music-tool-i4l); [Essentia licensing](https://essentia.upf.edu/licensing_information.html)
+
+### Inferences
+- For lead-sheet chord symbols, BTC-large (170 classes, Harte-style labels) is the practical open baseline. Its output needs:
+  - a Harte → lead-sheet symbol mapping (e.g., `C:min7` → `Cm7`)
+  - snapping of chord-change times to the beat grid from the beat tracker
+- Chord accuracy on real pop/rock is typically well below key accuracy. Plan a user chord-editing UI.
+
+### Gaps
+- Could not verify ChordFormer code or weights release or licence.
+- BTC's original repo licence was not verified.
+- Not researched: Chordino/NNLS-Chroma (Vamp plugin; UNVERIFIED: GPL), autochord, madmom chord models (CC BY-NC-SA, per the madmom model licence above).
+- No 2026 chord models surfaced.
+
+## Quantization of onsets to beat grid / time signatures / tuplets
+
+### Takeaway
+No dedicated research results were obtained this session. This needs a follow-up.
+
+### Cited Findings
+- No sources were retrieved. The one relevant input found is that beat positions (1–4) from allin1 and downbeats from Beat This! provide the bar grid needed for quantization. — [GitHub all-in-one](https://github.com/mir-aidj/all-in-one); [GitHub beat_this](https://github.com/CPJKU/beat_this)
+
+### Inferences
+- A practical approach is to use detected beat and downbeat times as a variable-tempo grid, and map each note onset to fractional beat position by interpolating between neighbouring beats. Then snap to the nearest allowed subdivision (16th, 8th triplet) with a cost penalty. This handles rubato better than fixed-BPM quantization.
+
+### Gaps
+- music21 `quantize()` was not verified. UNVERIFIED background: music21 is BSD-3 and has `Stream.quantize(quarterLengthDivisors=(4,3))`, which supports triplets but assumes a fixed grid.
+- MuseScore's MIDI import quantization, partitura and pretty_midi were not verified this session.
+
+## Notation formats and toolchains (MusicXML, MEI, MIDI, Guitar Pro, alphaTex)
+
+### Takeaway
+alphaTab can export Guitar Pro 7 (.gp) since 1.2 and alphaTex since 1.7. That makes it a viable single in-browser model for import and export of GP files. MusicXML and MEI interchange is best handled by Verovio (LGPL) or OSMD (BSD-3).
+
+### Cited Findings
+- **alphaTab Guitar Pro export:**
+  - The Gp7Exporter has existed since 1.2.0-alpha.75. — [alphaTab exporter guide](https://next.alphatab.net/docs/guides/exporter); [release 1.2](https://alphatab.net/docs/releases/release1_2)
+  - A Dec 2025 fix addressed exported GP files that had no RSE sound in Guitar Pro (drums in GP8). — [alphaTab PR #2456](https://github.com/CoderLine/alphaTab/pull/2456); [issue #2467](https://github.com/CoderLine/alphaTab/issues/2467)
+- **alphaTab 1.7** (about six months after 1.6) added:
+  - an alphaTex exporter (whole data model → alphaTex)
+  - an AST-based alphaTex parser, a language server, Monaco integration and a VS Code extension prototype
+  - MIDI bank support for GP, MusicXML and alphaTex
+  - improved SMuFL compliance and custom SMuFL fonts
+  - It also fixed tenor/alto clef mapping. — [alphaTab 1.7 release notes](https://alphatab.net/docs/releases/release1_7)
+- **alphaTab 1.6** added:
+  - an External Cursor API to sync the score with external audio or video
+  - GP8 backing-track support
+  - an audio export API that returns synthesized audio — [alphaTab 1.6 release notes](https://alphatab.net/docs/releases/release1_6)
+- **alphaTab inputs and licence:** it loads GP3–5, GP6 and alphaTex, renders standard notation, guitar tab and drum tabs, and is MPL-2.0. — [alphaTab features demo](https://demo.alphatab.net/features.html); [alternativeto](https://alternativeto.net/software/alphatab)
+- **Verovio input formats:** MEI natively, plus MusicXML, ABC, PAE and Humdrum, rendered to SVG. — [gittrend Verovio](https://gittrend.io/repo/rism-digital/verovio)
+
+### Inferences
+- The suggested internal pipeline is:
+  1. Build the internal model from transcription output.
+  2. Emit MusicXML (via music21 or partitura server-side) for standard notation and interchange.
+  3. Emit alphaTex or GP for tab, using alphaTab in the browser.
+- alphaTex is plain text, which makes it a convenient target to generate server-side and load into alphaTab.
+
+### Gaps
+- PyGuitarPro (GP3–5 read/write, LGPL), MuseScore 4 headless CLI conversion (`mscore -o`, GPL), LilyPond (GPL) and partitura (Apache-2.0) were not verified this session. The licences listed here are UNVERIFIED background knowledge.
+- Whether alphaTab can export MusicXML is not confirmed. No source found.
+
+## Browser rendering / editing / playback
+
+### Takeaway
+- **alphaTab (MPL-2.0)** is the only option with first-class multi-track tab, drums and standard notation plus built-in SoundFont playback and external-audio cursor sync. It fits a guitar/bass/drums-centric product best.
+- **OSMD (BSD-3)** handles MusicXML well and can render tab from MusicXML, but its playback is sponsor-only early access.
+- **Verovio (LGPL-3.0, WASM)** is the most rigorous engraver and has had MEI tablature features since 2024–25.
+
+### Cited Findings
+- **OSMD:**
+  - BSD-3-Clause; renders MusicXML via VexFlow. — [GitHub OSMD](https://github.com/opensheetmusicdisplay/opensheetmusicdisplay)
+  - "Can display tablature (guitar tabs) from MusicXML", including bends and glissandi. — [GitHub OSMD](https://github.com/opensheetmusicdisplay/opensheetmusicdisplay)
+  - Audio playback is "Soon" and offered as an early-access build for GitHub sponsors. — [GitHub OSMD](https://github.com/opensheetmusicdisplay/opensheetmusicdisplay)
+- **Verovio:**
+  - The npm WASM build is LGPL-3.0-or-later. Older distro packages list LGPL-3.0-only. — [Verovio changelog (safetycli)](https://data.safetycli.com/changelogs/verovio); [Alpine pkg](https://pkgs.alpinelinux.org/package/v3.20/community/x86_64/verovio)
+  - Tablature: customization arrived in 4.5.0 (Dec 2024) and MEI 5.1 tablature features in 5.2.0 (Apr 2025). A 5.7.0-dev version was dated Sep 2025. — [Verovio changelog (safetycli)](https://data.safetycli.com/changelogs/verovio); [ecosyste.ms](https://science.ecosyste.ms/projects/37342)
+- **Sentiment and platform notes:** community comments describe Verovio as well supported and frequently updated. One developer noted that it does not use native iOS rendering. — [W3C public-music-notation, Apr 2024](https://lists.w3.org/Archives/Public/public-music-notation/2024Apr/0007.html)
+- **alphaTab playback and sync:** built-in SoundFont synth, an audio export API (1.6), an External Cursor API to sync with the original audio or YouTube video (1.6), and MIDI bank selection (1.7). — [alphaTab 1.6](https://alphatab.net/docs/releases/release1_6); [alphaTab 1.7](https://alphatab.net/docs/releases/release1_7)
+- **Obsidian plugin:** an Obsidian plugin uses alphaTab for GP preview, an example of ecosystem adoption. — [Obsidian gtp-preview](https://community.obsidian.md/plugins/gtp-preview)
+
+### Inferences
+- Recommended stack:
+  - alphaTab as the primary viewer and player for a multi-track band score (guitar tab + standard, bass tab, drums, vocals melody, keys grand staff).
+  - Optionally, Verovio or OSMD for a "print-quality standard notation" view of MusicXML.
+- MPL-2.0 (alphaTab) is file-level copyleft: you can use it in a proprietary site, but modifications to alphaTab files themselves must be shared.
+- LGPL-3.0 (Verovio) loaded as a separate WASM module is generally fine for a proprietary web app.
+- The External Cursor API directly enables a "score follows the original YouTube/audio" feature.
+
+### Gaps
+- Not verified this session: VexFlow (MIT, low-level), abcjs (MIT, ABC with synth playback), flat.io embed (commercial API with paid tiers).
+- No in-browser editing comparison found. UNVERIFIED: alphaTab and OSMD are primarily renderers, so editing has to be built on top of the data model.
+- No Reddit or HN sentiment threads were retrieved.
+
+## YouTube ingestion and audio decoding
+
+### Takeaway
+yt-dlp works only server-side. As of 2026 it needs an external JS runtime (Deno) and often a PO-token provider plugin, and it breaks periodically when YouTube changes. A public site that downloads YouTube audio violates YouTube's ToS and carries DMCA §1201 anti-circumvention risk. Safer patterns are user-uploaded files plus an embedded YouTube player synced to the score. In the browser, decodeAudioData covers common formats, and ffmpeg.wasm (~31 MB core, SharedArrayBuffer for the multithreaded build) is the fallback.
+
+### Cited Findings
+- **yt-dlp runtime requirements:**
+  - From early Feb 2026, yt-dlp requires an external JS runtime such as Deno for YouTube downloads to work properly. A later release dropped Deno versions below 2.3.0. — [openSUSE yt-dlp update 2026.02.21](https://src.opensuse.org/pool/yt-dlp/pulls/4/files); [SUSE PackageHub 2026.01.31](https://packagehub.suse.com/packages/yt-dlp/2026_01_31-bp160_1_1)
+  - The Deno signature solver has failed against new player scripts ("found 0 sig function possibilities"). — [yt-dlp issue #15897](https://github.com/yt-dlp/yt-dlp/issues/15897)
+- **yt-dlp PO Token Guide (last edited July 12, 2026):**
+  - yt-dlp cannot generate PO tokens itself. Missing tokens can cause HTTP 403s and account or IP blocks.
+  - The recommended setup is a PO Token Provider plugin with the `mweb` client.
+  - Tokens are bound to the video ID.
+  - YouTube is "rolling out changes to enforce PO Tokens".
+  - Some clients (`tv`, `web_embedded`, `android_vr`) don't need tokens but have limitations: DRM without cookies, embeddable-only, or no made-for-kids videos. — [yt-dlp PO Token Guide](https://github.com/yt-dlp/yt-dlp/wiki/PO-Token-Guide)
+- **bgutil PO token provider:** its PyPI page warns that PO tokens "no longer bypass the bot check for majority of cases". — [PyPI bgutil-ytdlp-pot-provider](https://pypi.org/project/bgutil-ytdlp-pot-provider); [releases](https://github.com/Brainicism/bgutil-ytdlp-pot-provider/releases)
+- **Bot detection in practice:**
+  - Users still report "sign in to confirm you're not a bot" in mid-2026 (yt-dlp 2026.07.04). — [Lemmy thread](https://peachpie.theatl.social/c/[email protected]/p/364782/what-are-some-ways-to-torify-debian-13-6-or-yt-dlp-2026-07-04-when-yt-believes)
+  - Developers describe an ongoing cat-and-mouse cycle. — [dev.to](https://dev.to/talha_ramzan_3878156fea8c/building-a-video-downloader-that-doesnt-break-every-time-youtube-sneezes-lpa)
+- **Legal history:**
+  - The RIAA's 2020 DMCA notice against youtube-dl alleged §1201 circumvention of YouTube's "rolling cipher" and cited a German court decision. — [XDA](https://www.xda-developers.com/youtube-dl-riaa-dmca/); [Law Street Media](https://lawstreetmedia.com/?p=6252)
+  - GitHub reinstated the repo. — [The Next Web](https://thenextweb.com/news/github-took-down-youtube-dl-so-devs-made-more-copies)
+  - No court has ruled on the underlying theory in the sources found.
+- **ffmpeg.wasm:**
+  - The wrapper is MIT, but the core inherits FFmpeg's LGPL or GPL terms depending on the build. — [jsdelivr ffmpeg README](https://cdn.jsdelivr.net/npm/[email protected]/README.md)
+  - Core is about 31–32 MB (reducible with custom builds), with a 2 GB file limit. — [ffmpeg.wasm issue #623](https://github.com/ffmpegwasm/ffmpeg.wasm/issues/623); [dev.to converter](https://dev.to/felix-tech/building-a-privacy-first-video-to-audio-converter-with-ffmpegwasm-1kd0)
+  - Some builds require SharedArrayBuffer. — [Transloadit, May 2025](https://transloadit.com/devtips/extract-thumbnails-from-videos-in-browsers-with-ffmpeg-wasm/)
+
+### Inferences
+- Browser-side YouTube download is not feasible: YouTube media URLs lack CORS headers and need signature, n-challenge and PO-token solving. Any YouTube ingestion must be server-side.
+- Datacenter IPs are heavily bot-flagged, so expect high operational cost (residential proxies, cookies, ban risk) and frequent breakage.
+- For a public website, the lower-risk design is:
+  - "Upload your own audio file" as the primary path.
+  - A YouTube URL used only to embed the official IFrame player for sync, not for extraction.
+  - If extraction is offered anyway, put it behind ToS, DMCA-agent and rate-limit measures, after legal review.
+- For in-browser decoding, the Web Audio `decodeAudioData` path is enough for mp3, wav, aac/m4a and ogg/opus in modern browsers (Safari's ogg support historically lagged). Lazy-load ffmpeg.wasm only for exotic containers, because of its ~31 MB size.
+- SharedArrayBuffer requires COOP/COEP headers, which can conflict with the YouTube iframe embed. This is an inference to test.
+
+### Gaps
+- No official YouTube ToS text was fetched this session. UNVERIFIED: the ToS prohibits downloading except where YouTube provides a download button or link.
+- yt-dlp breakage frequency in 2025–26 was not quantified, for example as a count of emergency releases.
+- Alternatives (Invidious/Piped instances, the Cobalt API) were not researched. They face the same bot-detection pressure and legal posture.
+- Not verified with an MDN fetch: browser codec support for decodeAudioData, specifically Safari ogg/opus support as of 2026.

research_notes/Music transcription tooling survey/stem_separation.md

@@ -0,0 +1,362 @@
+# Open-source music source (stem) separation for downstream transcription (state as of Oct 2026)
+
+Research date: 2026-10-30. About 20 tool calls. MVSEP SDR figures are MVSEP's own "Multisong" or validation-set numbers. They are not MUSDB18-HQ test numbers, so they can't be compared directly with paper numbers. Old information is flagged.
+
+## 1. Demucs / HTDemucs: status after the Meta archive, forks, and quality of htdemucs_6s guitar/piano
+
+### Takeaway
+facebookresearch/demucs was archived on 2025-01-01. The original author (Défossez, now at Kyutai) keeps a low-activity fork at adefossez/demucs that only takes important bug fixes. Demucs is now well behind the RoFormer family. htdemucs_6s guitar and piano stems are weak: the piano stem is close to unusable (MVSEP piano SDR 2.23). Demucs remains the most widely ported model, with ONNX, C++, WASM, Rust and WebGPU ports.
+
+### Cited Findings
+- The facebookresearch/demucs repo was archived 2025-01-01 after the author left Meta. It is "not actively maintained anymore and only important bug fixes will be processed on the new repo" (adefossez/demucs). — [facebookresearch/demucs](https://github.com/facebookresearch/demucs); [DeepWiki](https://deepwiki.com/facebookresearch/demucs)
+- The author reportedly calls adefossez/demucs "the officially maintained Demucs now that I have left Meta to join Kyutai" and says to "expect slow replies and no new feature for now". A low-quality 2026 blog claims the fork shipped v4.1.0 (unverified). — [aireiter blog (secondary)](https://aireiter.com/blog/demucs-stem-separation-models-flags-cost)
+- Another fork exists: Ryan5453/demucs, which is also packaged on Replicate. — [Ryan5453/demucs](https://github.com/Ryan5453/demucs); [Replicate](https://replicate.com/ryan5453/demucs)
+- The official README says htdemucs_6s adds piano and guitar, and that "the piano source is not working great at the moment". The early test found guitar quality OK but "a lot of bleeding and artifacts" on piano. — [Demucs README (jsdelivr mirror)](https://cdn.jsdelivr.net/gh/facebookresearch/demucs@main/README.md)
+- MVSEP Multisong SDR (bass / drums / other / vocals / instrumental):
+  - htdemucs_ft: 12.05 / 11.24 / 5.74 / 8.33 / 14.63
+  - htdemucs: 11.74 / 10.90 / 5.57 / 8.18 / 14.49
+  - htdemucs_6s: 11.42 / 10.59 / 2.63 / 8.17 / 14.48
+  - On MVSEP's piano validation set, htdemucs_6s piano scores 2.23 SDR, against 7.83 for BS-RoFormer SW. MVSEP describes the 6s guitar and piano quality as "still so-so".
+  — [MVSEP algorithms](https://mvsep.com/en/algorithms); [MVSEP algorithm 17](https://www.mvsep.com/algorithms/17)
+- ZFTurbo's pretrained list gives htdemucs_6s Multisong SDR of bass 11.22, drums 10.22 and vocals 8.05. Guitar and piano SDR are not reported. — [MSST pretrained_models.md](https://github.com/ZFTurbo/Music-Source-Separation-Training/blob/main/docs/pretrained_models.md)
+- Model sizes per demucs-rs: htdemucs 84 MB, htdemucs_6s 84 MB, htdemucs_ft 333 MB. htdemucs_ft is a bag of four fine-tuned models, so it is about 4x slower. — [demucs-rs](https://github.com/nikhilunni/demucs-rs); [search summary of StemSplit/aireiter](https://stemsplit.io/blog/demucs-local-setup-guide)
+- Demucs code and weights are MIT. Browser ONNX exports state "MIT, matching the upstream demucs project". — [monteslu/htdemucs-web-onnx](https://huggingface.co/monteslu/htdemucs-web-onnx)
+
+### Inferences
+- For transcription, Demucs makes sense only as a fallback or where portability matters (browser, C++). It should not be the main separator.
+- A plausible split is htdemucs_ft for bass and drums (reasonable at about 11-12 SDR) and BS-RoFormer SW for guitar and piano.
+- Not verified this session (from memory): the HTDemucs paper (arXiv 2211.08553) reports about 9.0 dB average SDR on the MUSDB18-HQ test set, with about 9.2 dB for the fine-tuned / sparse variants.
+
+### Gaps
+- I did not verify adefossez/demucs commit activity for 2026 or confirm the v4.1.0 release on its releases page.
+- I found no published guitar SDR for htdemucs_6s from either MVSEP or ZFTurbo.
+
+## 2. RoFormer family, SCNet, MDX23C and newer models: which give guitar, piano and bass stems, and how good they are
+
+### Takeaway
+The best open-weight multi-stem model as of 2025-26 is BS-RoFormer SW, a 6-stem model (vocals, bass, drums, guitar, piano, other) by viperx/lucidrains, distributed by the community. Its MVSEP SDR:
+
+| Stem | SDR |
+|---|---|
+| Vocals | 11.30 |
+| Bass | 14.62 |
+| Drums | 14.11 |
+| Guitar | 9.05 |
+| Piano | 7.83 |
+| Other | 8.71 |
+
+Guitar and piano stay 3-7 dB behind bass, drums and vocals. For 2-stem vocal separation the leaders are MVSEP's BS-RoFormer 124-band models (12.33 vocal SDR, version 2026.07), but these appear to be MVSEP-hosted only.
+
+### Cited Findings
+- BS-RoFormer (ByteDance SAMI, Lu et al. 2023) won the SDX23 Music Separation track "by a large margin". It reaches 9.80 dB average SDR trained on MUSDB18HQ only, and a reported 11.99 dB with 500 extra songs. — [arXiv 2309.02612](https://arxiv.org/abs/2309.02612v2); [ar5iv](https://ar5iv.labs.arxiv.org/html/2309.02612)
+- Mel-Band RoFormer (arXiv 2310.01809) outperforms BS-RoFormer on vocals, drums and other: vocals 11.21 vs 10.78 dB. It is worse on bass: 9.64 vs 11.43 dB. — [arXiv 2310.01809](https://arxiv.org/html/2310.01809v1)
+- SCNet (ICASSP 2024, Tsinghua and Kunlun) reaches 9.0 dB average SDR on MUSDB18-HQ without extra data, and SCNet-large reaches 9.69. Its CPU inference time is about 48% of HT Demucs. Official code is at starrytong/SCNet. — [arXiv 2401.13276](https://arxiv.org/abs/2401.13276v1); [hyper.ai listing](https://hyper.ai/en/papers/2401.13276)
+- BS-RoFormer SW on MVSEP:
+  - It is a "modified version of initial BS Roformer… Modifications were made by lucidrains… 2nd version of weights (with better quality) was prepared by viperx". It became MVSEP's default model for new users.
+  - Multisong SDR: vocals 11.30, instrumental 17.50, bass 14.62, drums 14.11, guitar 9.05, piano 7.83, other 8.71.
+  - MVSEP's standalone guitar and piano models moved to these numbers: guitar 7.51 → 9.05, piano 6.20 → 7.83.
+  — [MVSEP news 64](https://mvsep.com/news/64?lang=en); [MVSEP algorithms](https://mvsep.com/en/algorithms)
+- BS-RoFormer SW weights are distributable:
+  - jarredou's HF repo BS-ROFO-SW-Fixed (Aug 16 2025) was "edited to allow its use with UVR and MSST".
+  - The jarredou account was reportedly deleted. enerjazzer/BS-ROFO-SW-Fixed is cited as the current host, with a checkpoint of about 700 MB.
+  - Community GGUF and ONNX conversions exist: studio409/BSRoformer-SW-6Stem-GGUF, and elicwhite's ONNX export.
+  — [HF jarredou commit](https://huggingface.co/jarredou/BS-ROFO-SW-Fixed/commit/f99fc5c2e4bc201413cfc1ca5767cf62ddb1a9a4); [studio409 GGUF](https://huggingface.co/studio409/BSRoformer-SW-6Stem-GGUF); [openmirlab/bs-roformer-infer](https://github.com/openmirlab/bs-roformer-infer)
+- MVSEP piano validation SDR:
+
+  | Model | Piano SDR |
+  |---|---|
+  | Demucs4HT 6s | 2.23 |
+  | MDX23C (2023.08) | 4.79 |
+  | MDX23C (2024.09) | 5.59 |
+  | MelRoformer (viperx) | 5.67 |
+  | SCNet Large | 5.89 |
+  | SCNet + Mel ensemble | 6.19 |
+  | BS-RoFormer SW | 7.83 |
+
+  — [MVSEP algorithms](https://mvsep.com/en/algorithms)
+- MVSEP bass SDR:
+
+  | Model | Bass SDR |
+  |---|---|
+  | BS-RoFormer | 12.49 |
+  | HTDemucs4 | 12.52 |
+  | SCNet XL | 13.81 |
+  | BS + HTDemucs + SCNet XL ensemble | 14.07 |
+  | BS-RoFormer SW | 14.62 |
+  | SCNet XL + BS-RoFormer SW | 14.87 |
+
+  — [MVSEP algorithms](https://mvsep.com/en/algorithms)
+- MVSEP drums SDR:
+
+  | Model | Drums SDR |
+  |---|---|
+  | HTDemucs4 | 12.04 |
+  | MelBand | 12.76 |
+  | SCNet Large | 13.01 |
+  | SCNet XL | 13.42 |
+  | BS-RoFormer SW | 14.11 |
+  | MelBand + SCNet XL + SW ensemble | 14.35 |
+
+  — [MVSEP algorithms](https://mvsep.com/en/algorithms)
+- MVSEP's 4/5-stem "Ensemble All-In" (v2025.06.30) scores: bass 14.85, drums 14.33, other 9.00, vocals 11.93, instrumental 18.23. This is an MVSEP service, not open weights. — [MVSEP algorithms](https://mvsep.com/en/algorithms)
+- Top 2-stem vocal models on MVSEP Multisong:
+
+  | Model | Vocals / instrumental SDR |
+  |---|---|
+  | BS-RoFormer 124 bands (ver. 2026.07) | 12.33 / 18.64 |
+  | BS PolarFormer 124 bands | 12.02 / 18.33 |
+  | MelBand "becruily deux" | 11.35 / 17.66 |
+  | SCNet XL IHF | 11.11 / 17.41 |
+
+  — [MVSEP algorithms](https://mvsep.com/en/algorithms)
+- MVSEP's guitar model has been through several architectures:
+  - The original (Sep 2023) used MDX23C.
+  - Later versions use MDX23C, Mel-RoFormer and BS-RoFormer, covering acoustic and electric guitar.
+  - In 2025 MVSEP added a separate acoustic guitar model and a guitar leaderboard (electric and acoustic).
+  — [MVSEP news 28](https://www.mvsep.com/news/28?lang=en); [MVSEP news](https://de2.mvsep.com/news); [MVSEP algorithms](https://de.mvsep.com/algorithms)
+- MVSEP also lists keys, organ and Rhodes models, but publishes no metrics for them. — [MVSEP algorithms](https://mvsep.com/en/algorithms)
+- A 2026 roundup (secondary) says the Roformer family sits "at the top of community quality leaderboards as of mid-2026". — [Futureproof Music School](https://futureproofmusicschool.com/blog/best-ai-stem-separation-tools)
+
+### Inferences
+- **Recommended open pipeline for transcription:**
+  1. Run BS-RoFormer SW for 6 stems.
+  2. Optionally replace the vocal stem with a top open vocal model (e.g. KimberleyJensen Mel-RoFormer, 10.98 in MSST).
+  3. Optionally ensemble bass and drums with SCNet XL.
+- **Guitar stem quality:** at about 9 SDR it is good enough for pitch transcription of a dominant guitar. Expect bleed from keys and synths into the guitar and piano stems.
+- **Piano stem quality:** at about 7.8 SDR, expect more errors, especially on piano that is quiet in the mix or doubled with synth pads.
+- **Keys and synth fall into "other":** BS-RoFormer SW has no synth or organ stem, so keys other than piano land in "other".
+
+### Gaps
+- No independent (non-MVSEP) benchmark of BS-RoFormer SW guitar or piano on a public test set was found.
+- The license of the viperx/SW weights is unclear. HF repos found did not show a license in search results; treat it as unlicensed or redistribution-tolerated community weights. This is a commercial-use risk.
+- No confirmed open-weight 7+ stem music model (e.g. with separate synth, strings or brass) was found. MVSEP has separate single-instrument models (strings, wind, etc.); I could not verify which have downloadable weights.
+
+## 3. UVR5, python-audio-separator, ZFTurbo MSST and MVSEP: availability and licenses
+
+### Takeaway
+- All three open tools are MIT for code, but weights are mixed and often unlicensed community releases.
+- **ZFTurbo MSST:** the training and inference framework, with the broadest architecture support.
+- **python-audio-separator:** the most practical pip/CLI wrapper for UVR-style models plus Roformers, with ensembles. Actively maintained.
+- **UVR5 GUI:** still on v5.6 plus beta Roformer patches; no new official release found for 2025-26.
+- **MVSEP:** a web service hosting the best models, some of them private.
+
+### Cited Findings
+- **ZFTurbo/Music-Source-Separation-Training:**
+  - License: MIT. Installable via `pip install msst`. 634 commits, 72 open issues.
+  - Architectures: mdx23c, htdemucs, segm_models, torchseg, bs_roformer, mel_band_roformer, swin_upernet, bandit, bandit_v2, scnet, apollo, bs_mamba2, conformer, dttnet, bs_conformer, scnet_tran, scnet_masked.
+  - It links an ONNX/TensorRT conversion repo.
+  — [MSST GitHub](https://github.com/ZFTurbo/Music-Source-Separation-Training)
+- MSST pretrained list, selected entries:
+  - Vocals (Multisong SDR): BS PolarFormer 11.00; MelBand RoFormer (KimberleyJensen) 10.98; BS-RoFormer (viperx) 10.87; MDX23C 10.17.
+  - Bass: HTDemucs4 FT bass 11.96.
+  - DrumSep: models are included (see section 5).
+  - No license is stated for the weights. Newest visible tags are v1.0.22 (DTTNet), v1.0.20 (PolarFormer) and v1.0.19 (BS Mamba2).
+  — [MSST pretrained_models.md](https://github.com/ZFTurbo/Music-Source-Separation-Training/blob/main/docs/pretrained_models.md)
+- **nomadkaraoke/python-audio-separator:**
+  - License: MIT.
+  - Architectures: MDX (.onnx), VR (.pth), Demucs (.yaml) and MDXC/RoFormer (.ckpt).
+  - Hardware: CUDA 11.8/12.2; Apple MPS and CoreML; CPU; experimental DirectML (RoFormer falls back to CPU, Demucs unsupported).
+  - Ensembles: 11 algorithms and 9 presets, including karaoke.
+  - In v0.47.0 the RoFormer overlap defaults changed, giving 4-6x more forward passes and slower runs; lowering `overlap` trades seam quality for speed.
+  - The README asks users to credit UVR when using UVR-trained models.
+  — [python-audio-separator](https://github.com/nomadkaraoke/python-audio-separator)
+- **UVR5 GUI:** the latest version cited is v5.6. Roformer support came via the community beta patch "UVR_Patch_4_14_24_18_7_BETA_full_Roformer". One guide says macOS Apple Silicon users may need to run from source for bs_roformer. No 2025-26 official release was found. — [kocpc guide (secondary)](https://en.kocpc.com.tw/?p=12798); [Bilibili guide](https://www.bilibili.com/read/cv27499700/)
+- **Community sentiment (secondary 2026 roundup):** the top-voted answer in r/edmproduction's "Favorite Stem Splitter?" was UVR, and by mid-2026 production-subreddit sentiment had "hardened toward MVSEP" for vocal/instrumental splitting. — [Futureproof Music School](https://futureproofmusicschool.com/blog/best-ai-stem-separation-tools); [KVR forum thread](https://www.kvraudio.com/forum/viewtopic.php?p=8906542)
+- **MVSEP:** a web service with a free tier. Its algorithm list includes several models with no public weights (e.g. "MVSep Team" BS-RoFormer karaoke, the 124-band vocal models, and the "MVSep Lead/Rhythm Guitar (Demo)"). — [MVSEP algorithms](https://mvsep.com/en/algorithms); [MVSEP news](https://de2.mvsep.com/news)
+
+### Inferences
+- For an app pipeline, python-audio-separator (MIT, pip) can load BS-RoFormer SW and Mel-RoFormer checkpoints. MSST is the more flexible path if you need SCNet or DrumSep.
+- **Weight-license caution:** most RoFormer weights (viperx, KimberleyJensen, becruily, gabox, anvuew, jarredou) are community-trained, often on undisclosed or copyrighted data, and usually carry no explicit license. Commercial redistribution is legally unclear. Demucs (MIT) is the cleanest option license-wise.
+
+### Gaps
+- I could not access UVR Discord or GitHub issues directly.
+- I could not verify python-audio-separator's latest release number or date; the page showed no releases list.
+
+## 4. Multi-guitar separation (lead vs rhythm, L/R panned, acoustic vs electric)
+
+### Takeaway
+No open-weight model separating lead from rhythm guitar was found. The lead/rhythm options are commercial:
+- Moises "Lead & Rhythm Guitar Separation" (Apr 2024, vendor claims only).
+- MVSEP's "Lead/Rhythm Guitar (Demo)".
+
+Acoustic-vs-electric splitting exists from Moises, AudioShake and MVSEP. A practical open approach is to separate a single guitar stem, then split it using stereo position (mid/side or pan) or a further acoustic/electric pass.
+
+### Cited Findings
+- **Moises (2024-04-24):** "Lead & Rhythm Guitar Separation" and "Acoustic & Electric Guitar Separation". Each yields 2 guitar tracks plus vocals, bass, drums and other. It claims "the first-ever model that accurately separates lead and rhythm guitars", with no metrics or limitations given. Premium/Pro only. — [Moises blog](https://moises.ai/blog/moises-news/new-guitar-separation-models/)
+- **MVSEP:** lists "MVSep Lead/Rhythm Guitar (Demo)" and an "Electric Guitar" demo with no details. It also has an Acoustic Guitar model (2025). — [MVSEP news](https://de2.mvsep.com/news); [MVSEP news 64](https://mvsep.com/news/64)
+- **LALAL.AI (Andromeda):** its "Guitars" stem captures all guitars, acoustic and electric, in one file, with no lead/rhythm split. — [LALAL.AI blog](https://www.lalal.ai/blog/andromeda-supports-guitars-updates-piano/); [Magnetic Mag 2026-09](https://magneticmag.com/2026/09/lalal-ai-fixes-a-guitar-stem-problem/)
+- **AudioShake:** released electric guitar and acoustic guitar stems (commercial). — [AudioShake](https://www.audioshake.ai/post/audioshake-releases-five-new-music-instrument-stems)
+
+### Inferences
+- **Practical open workarounds (not benchmarked):**
+  1. Separate a guitar stem with BS-RoFormer SW.
+  2. Split it into L, R, mid (L+R) and side (L−R). Double-tracked rhythm guitars panned hard L/R usually dominate the sides, while a centered lead dominates the mid. Hard-panned parts can also be transcribed per channel.
+  3. Do the pan/mid-side split before separation as well (feed L-only and R-only to the separator) to recover panned parts.
+  4. For transcription, run polyphonic transcription on the guitar stem, then separate voices symbolically: monophonic high line = lead, chordal = rhythm.
+- These are standard DSP practices inferred from how mixes work. No source in this session benchmarks them.
+
+### Gaps
+- No academic paper or open model on lead/rhythm guitar separation was found in this session.
+- No independent evaluation of Moises's lead/rhythm model was found.
+
+## 5. Drum sub-stem separation (kick, snare, hats, toms, cymbals)
+
+### Takeaway
+Open drum-sub-stem models are mature:
+- **Best quality:** MVSEP/MSST DrumSep models on Mel-RoFormer, SCNet XL and MDX23C. Kick reaches about 18-22 SDR, snare 13-17 and toms 13-16. Hi-hat, ride and crash are poor at about 3-9.
+- **LarsNet:** an academic 5-stem model with a VST/AU plug-in (LARS). It is faster than real time but lower quality.
+
+All of these expect a drum stem as input, so separate drums first.
+
+### Cited Findings
+- **LarsNet** (Politecnico di Milano, Pattern Recognition Letters 2024):
+  - Trained on the StemGMD dataset.
+  - Outputs 5 stems: kick, snare, toms, hi-hat and cymbals.
+  - Uses a bank of dedicated U-Nets and runs faster than real time.
+  - The LARS plug-in (VST3/AU) wraps it.
+  — [arXiv 2312.09663 (CC BY 4.0)](https://arxiv.org/html/2312.09663v1); [LARS plug-in](https://github.com/EdoardoMor/LARS); [MVSEP LarsNet](https://mvsep.com/algorithms/31?lang=en)
+- **MVSEP DrumSep** comes in 4, 5 and 6-stem variants; the 6-stem variant has kick, snare, toms, hh, ride and crash. — [MVSEP DrumSep](https://de2.mvsep.com/algorithms/29)
+- **MVSEP DrumSep, Multisong SDR:**
+
+  | Model | Kick | Snare | Toms | Cymbals / HH / Ride / Crash |
+  |---|---|---|---|---|
+  | MelBand Roformer 4-stem | 22.22 | 17.09 | 15.86 | cymbals 11.87 |
+  | SCNet XL 4-stem | 20.50 | 14.69 | 15.92 | cymbals 10.08 |
+  | MDX23C 6-stem (aufr33/jarredou) | 18.32 | 13.60 | 13.25 | hh 6.71, ride 5.38, crash 7.56 |
+  | HDemucs (inagoy) 4-stem | 14.13 | 8.42 | 5.67 | cymbals 5.63 |
+
+  — [MVSEP algorithms](https://mvsep.com/en/algorithms)
+- **MVSEP DrumSep, 150-track validation set:**
+
+  | Model | Kick | Snare | Toms | Cymbals / HH / other |
+  |---|---|---|---|---|
+  | MelBand 4-stem | 18.67 | 13.55 | 13.60 | cymbals 8.76 |
+  | BS-RoFormer (jarredou) 5-stem | 18.16 | 12.80 | 13.51 | 4.66 / 6.65 |
+  | HDemucs (inagoy) | 10.52 | 6.05 | 4.68 | 5.03 |
+
+  The page scrape had some column ambiguity for the 5- and 6-stem rows. — [MVSEP algorithms](https://mvsep.com/en/algorithms)
+- **MSST DrumSep:**
+  - mdx23c (jarredou) 5-stem: kick 16.66, snare 11.53, toms 12.33, hh 4.04, cymbals 6.36.
+  - mdx23c (aufr33 & jarredou) 6-stem: kick 14.54, snare 9.79, toms 10.63, hh 3.19.
+  - htdemucs (inagoy): included.
+  — [MSST pretrained_models.md](https://github.com/ZFTurbo/Music-Source-Separation-Training/blob/main/docs/pretrained_models.md)
+- **Other:**
+  - Timbrica drum-part separator (kick/snare/toms, ONNX fp16) on HF. — [HF Faridzar/timbrica](https://huggingface.co/Faridzar/timbrica-drum-separator)
+  - Inference-only `mdxnet-infer` PyPI package for MDX23C drum separation. — [PyPI mdxnet-infer](https://pypi.org/project/mdxnet-infer/)
+  - "Inverse Drum Machine" research (Torres, Peeters, Richard 2025), trained without isolated stems. — [arXiv 2505.03337](https://arxiv.org/pdf/2505.03337v2)
+
+### Inferences
+- For drum transcription, kick, snare and toms stems are clean enough to onset-detect reliably.
+- Hi-hat, ride and crash separation is weak, around 3-9 SDR. A 4-stem model with a combined "cymbals" output (about 9-12 SDR) plus a classifier may beat 6-stem splitting for hat/ride/crash. Dedicated drum-transcription models such as ADT, which operate on the full drum stem, may beat separate-then-detect.
+
+### Gaps
+- No LarsNet SDR numbers comparable to MVSEP's were captured this session.
+- Licenses for the DrumSep weights (jarredou/aufr33/inagoy) were not confirmed.
+
+## 6. Lead vs backing vocal separation
+
+### Takeaway
+"Karaoke" models separate lead from backing+instrumental. The best open-ish weights are Mel/BS-RoFormer karaoke models by becruily, gabox, aufr33/viperx, frazer and anvuew, at about 9.5-10.2 lead SDR. MVSEP's in-house BS-RoFormer (124-band) is best at 10.77 lead and 7.03 back. Backing-vocal stems remain much weaker (about 4.5-7 SDR).
+
+### Cited Findings
+- **MVSEP karaoke SDR:**
+
+  | Model | Lead SDR | Back SDR | Other |
+  |---|---|---|---|
+  | UVR HP-KAROKEE-MSB2-3BAND | 6.42 | — | — |
+  | MDX-B Karaoke | 7.42 | — | — |
+  | MDX-B Karaoke, applied to vocals | 8.28 | 4.46 | — |
+  | MelBand (aufr33/viperx) | 9.45 | — | — |
+  | MelBand (becruily) | 9.61 | — | — |
+  | MelBand (gabox) | 9.67 | — | — |
+  | Fused gabox + aufr33/viperx | 9.85 | — | — |
+  | SCNet XL IHF (becruily) | 9.53 | — | — |
+  | BS-RoFormer (frazer & becruily) | 10.10 | — | — |
+  | BS-RoFormer (anvuew) | 10.22 | — | — |
+  | BS-RoFormer (MVSep Team) | 10.41 | 6.61 | — |
+  | BS-RoFormer 124 bands (MVSep Team) | 10.77 | 7.03 | back+instrum 16.10 |
+  | UVR-BVE-4B_SN | — | 0.87 | — |
+
+  — [MVSEP algorithms](https://mvsep.com/en/algorithms)
+- Extracting vocals first and then applying karaoke changes results. With fused MelBand the lead drops to 9.62 but a back stem of 5.63 becomes available. — [MVSEP algorithms](https://mvsep.com/en/algorithms)
+- python-audio-separator ships karaoke ensemble presets. — [python-audio-separator](https://github.com/nomadkaraoke/python-audio-separator)
+
+### Inferences
+- **Recommended open chain:** best vocal model → karaoke Mel/BS-RoFormer on the vocal stem → lead and backing. The backing stem is usable for harmony detection but not clean transcription.
+
+### Gaps
+- I did not check whether the MVSep Team karaoke weights are downloadable; they appear to be service-only.
+
+## 7. In-browser separation (ONNX/WebGPU/WASM)
+
+### Takeaway
+Only Demucs (htdemucs, htdemucs_6s, htdemucs_ft) has mature browser ports:
+- **WebGPU ONNX ports:** about 22x real time on a mid-range GPU.
+- **demucs-rs:** WASM+WebGPU.
+- **sevagh's freemusicdemixer:** CPU WASM via demucs.cpp. Very slow (tens of minutes per song single-threaded), and its repo was archived Apr 2025.
+
+RoFormer-class models have community ONNX exports (including BS-RoFormer SW 6-stem), but no mature browser app with published speed was found.
+
+### Cited Findings
+- **monteslu/htdemucs-web-onnx:**
+  - htdemucs 4-stem ONNX for onnxruntime-web WebGPU, split into 21 chained pieces to keep the UI responsive.
+  - About 126 MB in mixed precision (173 MB fp32). Mixed precision is about 2.4x faster than fp32.
+  - About 22x real time back-to-back, or about 8x with UI-friendly pacing, on a Radeon RX 7600.
+  - Full fp16 badly degrades drums and bass, so convolutions are kept in fp32.
+  - License: MIT.
+  — [HF monteslu/htdemucs-web-onnx](https://huggingface.co/monteslu/htdemucs-web-onnx)
+- monteslu also has an htdemucs_ft WebGPU build. — [HF monteslu/htdemucs-ft-webgpu](https://huggingface.co/monteslu/htdemucs-ft-webgpu)
+- There are htdemucs_6s WebGPU/ONNX exports with guitar and piano. — [HF kramp/htdemucs-6s-webgpu-onnx](https://huggingface.co/kramp/htdemucs-6s-webgpu-onnx); [HF adowu/htdemucs-6s-onnx](https://huggingface.co/adowu/htdemucs-6s-onnx)
+- **nikhilunni/demucs-rs:**
+  - Rust port supporting htdemucs, htdemucs_6s and htdemucs_ft.
+  - Browser build is WASM + WebGPU and is "significantly slower than the native CLI".
+  - License: Apache-2.0.
+  — [demucs-rs](https://github.com/nikhilunni/demucs-rs); [Korben article](https://korben.info/en/demucs-rs-separate-tracks-stems-browser.html)
+- **sevagh/freemusicdemixer.com:**
+  - Demucs v4 transliterated to C++/Eigen (demucs.cpp), compiled to WASM with Emscripten.
+  - Models: htdemucs (about 81 MB fp16) and htdemucs_6s (about 53 MB).
+  - Speed: a 7-min track takes about 41 min single-run or about 9 min with 8 workers. Another track took about 20 min single-run or about 5 min with 8 workers.
+  - WASM/JS is MIT, but "AI model weights and other website assets are proprietary".
+  - Repo archived Apr 26 2025 as an older snapshot; the live site may differ.
+  — [freemusicdemixer.com repo](https://github.com/sevagh/freemusicdemixer.com); [sevagh GitHub](https://www.Github.com/sevagh)
+- **Other ports:**
+  - demucs-wasm npm, derived from freemusicdemixer code. — [demucs-wasm README](https://cdn.jsdelivr.net/npm/[email protected]/README.md)
+  - timcsy/demucs-web and bakkot/demucs-js. One port notes "if your browser does not support WebGPU it will be much, much slower". — [demucs-web](https://github.com/timcsy/demucs-web); [demucs-js](https://github.com/bakkot/demucs-js)
+- **RoFormer ONNX exports:**
+  - elicwhite/bs-roformer-sw-6stem-onnx (no model card). — [HF elicwhite](https://huggingface.co/elicwhite)
+  - silverdaw 4-stem BS-RoFormer ONNX. — [HF silverdaw](https://huggingface.co/silverdaw/bs-roformer-rhythm-onnx)
+  - schism-audio mini-BS-RoFormer in MLX and CoreML (Apple). — [HF schism-audio](https://huggingface.co/schism-audio/mini-bs-roformer-v2-coreml)
+
+### Inferences
+- **Feasible in browser today:** htdemucs or htdemucs_6s on WebGPU (about 10-30 s for a 4-min song on a decent GPU, based on the 22x figure).
+- **Fallback:** CPU-WASM fallback is impractical (minutes to tens of minutes).
+- **BS-RoFormer SW in browser:** its about 700 MB checkpoint and attention-heavy compute make it a heavy browser download. It is feasible only with WebGPU and quantization, and it is unproven.
+
+### Gaps
+- No published browser speed or quality numbers for RoFormer ONNX models.
+- I did not verify whether the live freemusicdemixer.com has moved to WebGPU or added models (it reportedly added MIDI features; not confirmed this session).
+
+## 8. Typical runtime, CPU vs GPU, for a 4-minute song
+
+### Takeaway
+htdemucs takes about 15-30 s on a modern NVIDIA GPU and about 1.5-8 min on CPU. htdemucs_ft is about 4x slower. SCNet runs at about half of HTDemucs CPU time. RoFormer models are heavier, and higher overlap settings multiply their runtime.
+
+### Cited Findings
+- **MISST measurement:** htdemucs on a 4:09 track took 2m22s on CPU and 28s on GPU. — [MISST GitHub](https://github.com/Frikallo/MISST)
+- **LinuxLinks measurement:** a 6:24 FLAC took 15.6 s on an RTX 3060 Ti and 187.8 s on CPU (about 12x). — [LinuxLinks](https://www.linuxlinks.com/machine-learning-linux-demucs-music-source-separation/2/)
+- **Paper on a Demucs-based tool:** about 5 min on CPU vs under 10 s on GPU (RTX 3060: 7-10 s per track). — [SBrT paper](https://biblioteca.sbrt.org.br/articlefile/4978.pdf)
+- **Vendor estimates (inconsistent):**
+  - htdemucs on an RTX 3080: 20-45 s; on CPU: 90 s to about 8 min.
+  - htdemucs_ft is about 4x slower.
+  — [StemSplit blog (vendor)](https://stemsplit.io/blog/demucs-local-setup-guide)
+- **SCNet:** CPU inference time is about 48% of HT Demucs. — [arXiv 2401.13276](https://arxiv.org/abs/2401.13276v1)
+- **RoFormer in python-audio-separator:** overlap defaults in v0.47.0 cause 4-6x more forward passes, and `torch.compile` speeds up warm runs. — [python-audio-separator](https://github.com/nomadkaraoke/python-audio-separator)
+
+### Inferences
+- **Full pipeline estimate on a single consumer GPU:** BS-RoFormer SW, plus karaoke, plus DrumSep, likely totals about 1-3 minutes per 4-min song.
+- **CPU-only estimate:** RoFormer-class models are likely to take 10+ minutes per pass.
+- Both are inferences; I found no direct RoFormer timing source.
+
+### Gaps
+- No primary-source RoFormer, BS-RoFormer SW or Mel-RoFormer CPU/GPU timing benchmarks were found.
+- No Hacker News threads were captured this session.

research_notes/Transcription accuracy improvement ideas/data_and_training.md

@@ -0,0 +1,154 @@
+# Building / fine-tuning our own transcription and separation models: data, training recipes, compute, compression (as of Oct 2026)
+
+Scope note: complements `reports/Music transcription tooling survey.md` (not repeated here). That survey listed GuitarSet and DadaGP licenses as "unverified" — both are resolved below (GuitarSet is **NC**; DadaGP is **research-only, by request**).
+
+## Q1. Datasets and licenses — which permit commercial use?
+
+### Takeaway
+Almost every *real-audio* AMT/separation dataset with good annotations is non-commercial (GuitarSet, IDMT-SMT-Guitar, MAESTRO, MoisesDB, MedleyDB, MUSDB18-HQ, M4Singer, MTG-Jamendo). The commercially usable core is small and mostly synthetic or drums: **Slakh2100 (CC BY 4.0), E-GMD and Groove (CC BY 4.0), Guitar-TECHS (CC BY 4.0)**, plus DadaGP *code* (MIT) but not its data. A commercial product will have to lean on synthetic rendering + its own recordings.
+
+### Cited Findings
+
+**Guitar**
+- **GuitarSet**: Zenodo record licensed **CC BY-NC 4.0**; 7.5 GB audio+annotations zip, 37 MB annotation-only zip — [Zenodo 1422265](https://zenodo.org/records/1422265). (mirdata's "MIT" refers to loader code only — [mirdata](https://mirdata.readthedocs.io/en/0.3.8/_modules/mirdata/datasets/guitarset.html).)
+- **IDMT-SMT-Guitar v2**: **CC BY-NC-ND 4.0** (no derivatives!); 1.3 GB; 4 subsets: ~4,700 technique note events (3 guitars), 400 note events, 5 short pieces, 64 chord/rhythm pieces at two tempi on three guitars — [Zenodo 7544110](https://zenodo.org/records/7544110).
+- **GOAT**: 5.9 h unique DI electric guitar, re-amped ×5 to 29.5 h; 172 audio files; tabs as .gp/.gp5 + DadaGP tokens + GP-rendered wav + aligned MIDI; guitars: Stratocaster ~235 min, Strandberg ~80 min, Jazzmaster ~26 min, Les Paul ~13 min. **The paper's CC BY 4.0 notice covers the paper, not the data**: data is "available by request ... for research purposes only", and some content is covers whose copyright the authors don't own — [arXiv 2509.22655](https://arxiv.org/html/2509.22655v1). (Correction to prior survey, which listed GOAT as CC BY 4.0.)
+- **SynthTab**: ~13,113 h, 20,715 tracks, 23 timbres (7 acoustic, 16 electric), rendered from DadaGP with commercial Ample Sound plugins via DawDreamer; no license stated in paper ("All code and data is made available at www.synthtab.dev") — [arXiv 2309.09085](https://arxiv.org/html/2309.09085v4). The project site lists ~6,700 h / 15,211 tracks and also no license — [synthtab.dev](https://synthtab.dev). (Hour counts disagree between paper and site.) Because it derives from DadaGP (research-only) and commercial sample libraries, treat as research-only.
+- **DadaGP**: 26,181 Guitar Pro songs, 739 genres, sourced from user-submitted transcriptions of commercial music — [arXiv 2107.14653](https://arxiv.org/pdf/2107.14653). Access is by emailing the authors "to request access to the dataset for research purposes"; repo code is **MIT**, no dataset license stated — [GitHub dadaGP](https://github.com/dada-bots/dadaGP). A search-summary claimed the paper says CC BY 4.0, but that appears to be the paper's own license, not the data — [arXiv 2107.14653](https://arxiv.org/pdf/2107.14653).
+- **Guitar-TECHS** (Jan 2025, electric guitar techniques, excerpts, chords, scales across diverse hardware): paper states dataset and code released under **CC BY 4.0** — [arXiv 2501.03720](https://arxiv.org/html/2501.03720v1). Commercially usable.
+- **GAPS** (classical guitar, ISMIR 2024): paper is CC BY 4.0; no dataset-level license found; described as freely available audio–score pairs — [ISMIR 2024 poster](https://ismir2024program.ismir.net/poster_171.html); [arXiv 2408.08653](https://arxiv.org/html/2408.08653v2).
+- **Riley et al. jazz guitar** (79 GP scores + ~4 h commercial audio from YouTube): no dataset license; aligned MIDI on request — [arXiv 2402.15258](https://arxiv.org/html/2402.15258v1).
+- **EGDB**: no license found in search — [paperswithcode EGDB](https://astro.paperswithcode.com/dataset/egdb).
+
+**Piano / drums / vocals**
+- **MAESTRO**: **CC BY-NC-SA 4.0** — [Magenta MAESTRO](https://magenta.tensorflow.org/datasets/maestro).
+- **E-GMD**: **CC BY 4.0** — [Magenta E-GMD](https://magenta.tensorflow.org/datasets/e-gmd); [arXiv 2004.00188](https://arxiv.org/pdf/2004.00188).
+- **Groove MIDI (GMD)**: CC BY 4.0 per Magenta — [Magenta Groove](https://magenta.tensorflow.org/datasets/groove); mirdata's table shows CC BY-SA — [mirdata quick ref](https://mirdata.readthedocs.io/en/0.3.8/source/quick_reference.html) (conflict; Magenta is authoritative).
+- **STAR Drums** (TISMIR 2025): built by separating real recordings, annotating drums with an ADT model, resynthesising drums with sample-based instruments, and remixing with the real non-drum stem; uses music whose licenses allow distribution; article is CC BY 4.0 but dataset-file license not confirmed — [TISMIR article 244](https://transactions.ismir.net/articles/244); [AudioLabs ePrint](https://www.audiolabs-erlangen.com/content/05_fau/professor/00_mueller/03_publications/2025_WeberUML_DrumSTAR_TISMIR_ePrint.pdf).
+- **StemGMD**: listed CC BY 4.0 on Hugging Face — [HF stemgmd](https://huggingface.co/datasets/schismaudio/stemgmd/blob/main/README.md).
+- **M4Singer**: **CC BY-NC-SA 4.0**, 29.77 h Mandarin a cappella, 20 singers, 700 songs — [NeurIPS 2022](https://papers.neurips.cc/paper_files/paper/2022/file/2de60892dd329683ec21877a4e7c3091-Paper-Datasets_and_Benchmarks.pdf).
+- **Opencpop**: ~5.25 h, one singer, manual alignment + scores (license not found) — [M4Singer paper table](https://papers.neurips.cc/paper_files/paper/2022/file/2de60892dd329683ec21877a4e7c3091-Paper-Datasets_and_Benchmarks.pdf).
+- **MIR-ST500**: 160k+ notes from 500 pop songs, labelled by non-experts (license not found) — [ICASSP 2021](https://rc.signalprocessingsociety.org/conferences/icassp-2021/spsicassp21vid0572).
+
+**Multi-track / separation**
+- **Slakh2100**: **CC BY 4.0**, 145 h of mixtures, ~105 GB download / ~500 GB as wav — [Zenodo 4599666](https://zenodo.org/records/4599666). Commercially usable (synthetic, includes guitar/bass/piano/drum stems from MIDI).
+- **MoisesDB**: **CC BY-NC-SA 4.0** per repo — [GitHub moises-db](https://github.com/moises-ai/moises-db); 240 tracks, 45 artists, 12 genres; guitars are the next most common stem after vocals/bass/drums — [arXiv 2307.15913](https://arxiv.org/abs/2307.15913v1). (Search snippets also mention "NC-RCL"; either way non-commercial.)
+- **MedleyDB**: CC BY-NC-SA 4.0 (non-commercial research), 122 multitracks — [auditory.org 2014](https://auditory.org/postings/2014/614.html); [Zenodo](https://zenodo.org/record/1649325).
+- **MUSDB18-HQ**: "educational purposes only ... should not be used for any commercial purpose"; 46 tracks from MedleyDB (BY-NC-SA) — [Zenodo MUSDB18-HQ](https://zenodo.org/record/3338373).
+
+**Unlabelled audio for pseudo-labelling**
+- **MTG-Jamendo**: CC BY-NC 4.0 dataset terms, research/academic use only, per-track CC licenses set by artists — [UPF repository](https://repositori.upf.edu/items/e48c8c42-dba1-404d-85b7-5a0d6ef99e6d/full). **Jamendo sued NVIDIA (June 2026, N.D. Cal.) alleging commercial training of Fugatto/Audio Flamingo on MTG-Jamendo breached CC BY-NC; ~$20.3M sought; a Ghent court on 11 June 2026 rejected NVIDIA Belgium's jurisdictional objection** — [Music Business Worldwide](https://www.musicbusinessworldwide.com/nvidia-hit-with-us-copyright-suit-claiming-it-trained-ai-models-on-winamp-subsidiary-jamendos-music-without-permission/); [chatgptiseatingtheworld](https://chatgptiseatingtheworld.com/2026/06/23/nvidia-hit-with-8th-copyright-suit-latest-filed-by-s-a-jamendo-over-mtg-jamendo-dataset-of-55000-full-audio-tracks/); refiled Oct 2026 — [aimusicpreneur](https://www.aimusicpreneur.com/ai-music-news/jamendo-refiles-nvidia-ai-training-lawsuit-october-2026/). (Allegations only.)
+
+### Inferences
+- Commercial-clean training pool for us: Slakh2100 + E-GMD + Groove + StemGMD + Guitar-TECHS + our own recordings + anything we render ourselves from MIDI/GP that we have rights to. Real-audio guitar transcription data that is commercially clean is essentially just Guitar-TECHS + our own.
+- NC datasets can still be used for **evaluation/benchmarking** internally in many readings, but training production weights on them is the risk; the Jamendo–NVIDIA case shows NC licensors do litigate.
+- GuitarSet being NC matters: almost every published guitar model (incl. fine-tunes) is trained on it.
+
+### Gaps
+- Licenses not found: EGDB, GAPS dataset files, Opencpop, MIR-ST500, MIR-1K, MAPS, URMP, ADTOF datasets, STAR Drums files, FMA (dataset), Babyslakh, any 2026 bass-specific dataset. Each must be checked on its download page.
+- No dedicated commercially licensed bass or vocal-note dataset found.
+
+## Q2. Synthetic data from Guitar Pro / MIDI: renderers, amp sims, mixing, sim-to-real gap
+
+### Takeaway
+The proven recipe is: GP → per-string MIDI (keyswitches for techniques) → sampled guitar plugin via DawDreamer → NAM amp + cab IR + random reverb via pedalboard. SynthTab-only pretraining transfers poorly zero-shot (GuitarSet Tab F1 43.1 vs 80.1 in-domain) but improves after fine-tuning on real data; GOAT shows NAM re-amping helps electric-guitar generalisation, while doubling the augmentation doesn't help further. Tooling licenses: DawDreamer and pedalboard are GPL-3.0 (fine for offline data generation, not for shipping), NAM is MIT.
+
+### Cited Findings
+- SynthTab pipeline: DadaGP → JAMS → MIDI; each **string rendered separately** "to ensure proper string usage", bends/vibrato via MIDI CC, hammer-ons/pull-offs/slides/palm-mute/harmonics via **keyswitches**; mixes are averages of string signals; Ample Sound plugins (AGL, AGT, AGM, AGSJ, ASH, ALP, ATC, AVC, APF, ASC, AE) hosted in **DawDreamer**; rendering took ~1 week on a 24-core M2 Ultra Mac Studio — [arXiv 2309.09085](https://arxiv.org/html/2309.09085v4).
+- SynthTab sim-to-real: TabCNNx4 trained only on SynthTab scores Tab F1 64.2 on SynthTab val, **43.1 GuitarSet, 13.8 IDMT, 57.0 EGDB**; in-domain TabCNN trained on the real sets reaches 80.1 / 63.1 / 70.5. Pretrain-then-fine-tune "unanimously outperform[s]" single-dataset training, but fine-tuning sometimes *lowered* scores vs the pretrained model; authors blame domain mismatch and propose better humanisation, varied virtual rooms, post-processing and augmentation — [arXiv 2309.09085](https://arxiv.org/html/2309.09085v4).
+- Cross-real-dataset generalisation is also poor: TabCNN trained on IDMT scores 15.3 and 28.8 on other sets — [arXiv 2309.09085](https://arxiv.org/html/2309.09085v4).
+- GOAT amp augmentation: DI re-amped with **Neural Amp Modeler using ~7,000 public profiles from >1,000 amps** (one random per item), cab IRs from NeuralDSP Archetype Nolly with randomised mic/height/distance, room reverb on 25% of items at −10 to 0 dB send, rendered with **pedalboard**, ×5 passes; code public — [arXiv 2509.22655](https://arxiv.org/html/2509.22655v1).
+- GOAT results (note F1): DI-trained 0.820 DI / **0.714 AMP** / 0.860 GuitarSet; AMP-trained 0.843 / **0.800** / 0.843; AMP-XL (double re-amp) 0.818 / 0.806 / 0.835 — i.e. amp augmentation +8.6 pts on distorted test, more is not better. Whisper-style tab token generation "did not work well" (token acc ~0.68, WER ~0.51–0.55) — [arXiv 2509.22655](https://arxiv.org/html/2509.22655v1).
+- STAR Drums shows a hybrid approach: real non-drum stems + resynthesised drums from ADT-derived annotations — [TISMIR 244](https://transactions.ismir.net/articles/244).
+- Licenses: **pedalboard GPL-3.0** (JUCE-derived) — [PyPI pedalboard](https://pypi.org/project/pedalboard/0.9.2); **DawDreamer GPL-3.0** ("to make it easier to comply with all of the dependent projects") — per search result summary of [DawDreamer README](https://github.com/DBraun/DawDreamer) (not fetched directly); **NAM trainer MIT** — [GitHub neural-amp-modeler](https://github.com/sdatkinson/neural-amp-modeler).
+- Slakh2100 is itself a MIDI-render dataset (145 h) under CC BY 4.0 — [Zenodo 4599666](https://zenodo.org/records/4599666).
+
+### Inferences
+- GPL tools used *offline* to generate training audio do not, in the usual reading, impose GPL on the resulting model or audio; they must not be shipped in the browser bundle. Commercial sample libraries (Ample Sound etc.) need their EULA checked for "ML training" use — SynthTab does not address this.
+- NAM profile licensing varies per uploader (tone3000/ToneHunt); profile-level terms were not verified. Capturing our own NAM profiles from owned amps/plugins is the clean route.
+- For multiple guitars: render 2–4 guitar parts from multi-track GP files with random pan, distinct amp/IR per part, and mix with Slakh-style bass/drums/keys — no paper found that evaluates this specifically for lead/rhythm separation.
+- Since DadaGP is research-only, a commercial alternative source of tabs is needed: our own GP files, public-domain/CC tabs, or procedurally generated tabs (random playable riffs/chord progressions).
+
+### Gaps
+- No published quantification of sim-to-real gap for *distorted multi-guitar mixes*, nor of physical-model (vs sample) renderers.
+- FluidSynth / sfizz licenses and quality for guitar were not researched in this pass.
+- Ample Sound / other plugin EULAs regarding dataset generation not checked.
+
+## Q3. Fine-tuning on our own GP tabs + real recordings: alignment, few-shot, overfitting
+
+### Takeaway
+Score-to-audio alignment via DTW against a pretrained transcriber's activations, then per-note onset snapping, gives near-perfect labels (98% onset F1 @50 ms on resynthesised audio vs 62% for DTW alone), and ~4 h of aligned commercial guitar audio was enough to beat the prior zero-shot GuitarSet SOTA. Exact few-shot curves for AMT were not found.
+
+### Cited Findings
+- Riley et al. (high-res guitar via domain adaptation): 79 GP scores + ~4 h commercial jazz guitar audio (33 guitarists); coarse alignment = DTW between score and activations of a pretrained piano model; fine alignment snaps each onset to the local activation maximum. Fine alignment yields **98% onset F1 @50 ms (95% @25 ms) vs 62% / 27% for DTW alone** on resynthesised audio — [arXiv 2402.15258](https://arxiv.org/html/2402.15258v1).
+- Same paper: GuitarSet **zero-shot onset F1 87.3** vs 82.9 previous best; with GuitarSet added 89.7 (Lu et al. 91.1 higher); own jazz test 84.7 vs 80.0. Fine-tuning recipe: piano-pretrained model, 10 s excerpts, batch 4, LR 1e-5 (searched 1e-3–1e-5), 100K steps (~10 epochs), random transposition — [arXiv 2402.15258](https://arxiv.org/html/2402.15258v1).
+- GOAT provides aligned MIDI ("fine-aligned MIDI") with GP tabs, a template for packaging our own data — [arXiv 2509.22655](https://arxiv.org/html/2509.22655v1).
+- A 2022 Fraunhofer study found a piano multi-pitch model "surprisingly robust" to microphone-induced domain shift and that four unsupervised domain-adaptation methods did not notably improve it — [Fraunhofer publica](https://publica.fraunhofer.de/handle/publica/486541) (via search summary).
+- SynthTab notes fine-tuning on small real sets sometimes *reduced* performance vs the synthetic-pretrained model — [arXiv 2309.09085](https://arxiv.org/html/2309.09085v4).
+
+### Inferences
+- Our hand-made GP tabs map directly onto the Riley/GOAT recipe: GP → MIDI (keep string/fret) → DTW against Basic Pitch or our model's activations → onset snap → train. The GP tab itself also supplies string/fret labels for a tab-assignment head.
+- With a small set of songs (tens), use them primarily as a **held-out benchmark**, and fine-tune only with heavy regularisation (low LR ~1e-5, few steps, freeze backbone or LoRA/adapters, mix in synthetic data each batch) and leave-one-song-out evaluation to detect overfitting.
+- ~4 h of aligned real audio is a demonstrated useful scale for guitar; fewer than ~1 h is unproven.
+
+### Gaps
+- No AMT paper found quantifying LoRA/adapter few-shot curves (e.g., 5/10/30 min of target data). synctoolbox and CTC-based alignment for AMT not researched in this pass.
+
+## Q4. Training recipes and compute costs
+
+### Takeaway
+Top-tier separation is expensive: the SDX23-winning BS-RoFormer took **16× A100-80GB for 4 weeks (~10,750 GPU-h, ≈$14–17k at $1.3–1.6/h)** on ~500 in-house songs + MUSDB. Fine-tuning a transcriber is cheap by comparison (Riley's guitar fine-tune: batch 4, 100K steps — a single-GPU job). Exact YourMT3/MT3 training budgets were not found.
+
+### Cited Findings
+- BS-RoFormer: "the training process lasted for 4 weeks using 16 Nvidia A100-80GB GPUs", total batch 128 (8/GPU), 8 s segments, **93.4M params** (72.2M for LL=6), trained on 500 in-house songs + MUSDB18HQ; ablation LL=6 models trained on MUSDB18HQ only using 16× V100-32GB; LR ×0.9 every 40k steps — [arXiv 2309.02612](https://arxiv.org/html/2309.02612). Small version reached 9.80 dB avg SDR on MUSDB18HQ without extra data — [arXiv abs](https://arxiv.org/abs/2309.02612).
+- Cloud A100 prices: Lambda 1× A100 40GB $1.29/GPU-h; Lyceum A100 80GB $1.59/h — [Lambda](https://lambda.ai/welcome/ai-cloud); [Lyceum](https://lyceum.technology/products/training/dedicated/index.html).
+- Riley et al. guitar fine-tune: batch 4, 100K steps, LR 1e-5; compute not reported — [arXiv 2402.15258](https://arxiv.org/html/2402.15258v1).
+- SynthTab rendering compute: ~1 week on one M2 Ultra (CPU) for ~13k h audio — [arXiv 2309.09085](https://arxiv.org/html/2309.09085v4).
+- MSST (ZFTurbo) is the community training framework (17 architectures) — see prior survey; MuScriptor trained on 1.45M rendered MIDI + ~11k h real aligned audio (prior survey) — indicates the scale that beats YourMT3+.
+
+### Inferences
+- Rough budget tiers (inference from the above): (a) fine-tune a Basic Pitch / HPPNet / CRNN-scale guitar or bass model on synthetic + ~few h real: tens of single-GPU hours (<$100); (b) train a stem separator from scratch at BS-RoFormer scale: ~10k A100-h; (c) fine-tune an existing *permissively licensed* separator (e.g., Demucs MIT) on a new guitar or lead/rhythm stem: likely 1–2 orders of magnitude less than (b), feasible on rented single GPUs over days.
+- Free Colab/Kaggle GPUs (session-limited) are viable only for tier (a) experiments.
+
+### Gaps
+- No published GPU-hours for YourMT3+/MT3, Basic Pitch, MSST community models (viperx etc.), or Demucs fine-tunes were found; Kaggle/Colab quotas not verified for 2026.
+
+## Q5. Efficient browser models: distillation, quantization, tiny models
+
+### Takeaway
+Tiny transcribers can match or beat Basic Pitch (Harmonica-nano: 26.3K params, frame F1 0.796, +14.6 pts over Basic Pitch, 1,622× real time) but code/weights are not released. No AMT-specific int8 accuracy-loss numbers were found; speech/audio analogues report ~1–2 point drops. Distilling from NC teachers (MuScriptor, ADTOF, MoisesDB-trained models) inherits legal risk.
+
+### Cited Findings
+- **Harmonica** (arXiv 2609.04640, submitted ICASSP 2027): "multi-depth harmonic convolution"; nano = **26.3K params, frame F1 0.796, 14.6 pts above Basic Pitch, 1,622.5× real time**; x-large claimed SOTA instrument-agnostic; no code/weights link, training data and causality not stated — [arXiv 2609.04640](https://arxiv.org/abs/2609.04640).
+- Int8 analogues: distilled-then-8-bit acoustic event detection RNN lost ~2 pts accuracy — [arXiv 2001.10876](https://arxiv.org/pdf/2001.10876); OpenVINO int8 Distil-Whisper 2.1–6.1× faster with 1–2% accuracy drop, reducible via calibration data choice — [OpenVINO blog](https://blog.openvino.ai/blog-posts/optimizing-whisper-and-distil-whisper-for-speech-recognition-with-openvino-and-nncf); outlier activations in the last encoder layer hurt int8 for seq2seq speech models — [arXiv 2406.11022](https://arxiv.org/pdf/2406.11022).
+- TinyMusician (MusicGen-Small): distillation barely changed quality; mixed-precision quantization had a larger effect — [arXiv 2509.00914](https://arxiv.org/pdf/2509.00914).
+- Distil-Whisper distinguishes sequence-level (train on teacher pseudo-labels) vs word-level (KL to teacher distribution) distillation — [arXiv 2311.00430](https://arxiv.org/pdf/2311.00430).
+- MuScriptor weights CC BY-NC 4.0 (prior survey); MoisesDB CC BY-NC-SA — [GitHub moises-db](https://github.com/moises-ai/moises-db).
+
+### Inferences
+- Plan: train harmonic-conv / Basic-Pitch-size students per stem; export ONNX fp16 for WebGPU (safest), try int8 weight-only for WASM fallback and measure on our benchmark; expect 1–2 pts loss but verify, since onset precision can be more quantization-sensitive than classification.
+- Distillation from NC teachers: the CC BY-NC terms restrict *use* of the weights; using NC model outputs to train a commercial student is legally untested and risky — treat NC teachers as benchmark-only unless licensed (e.g., Kyutai commercial licence). Permissive teachers: Demucs (MIT), Basic Pitch (Apache-2.0), ADT_STR (CC BY-SA — share-alike likely attaches to derived weights).
+- Harmonica's numbers suggest model size is not the browser bottleneck for note transcription; separation (Demucs-scale) is.
+
+### Gaps
+- No published AMT int8/pruning accuracy figures; no AMT distillation study (e.g., MT3 → small CNN) with numbers found; PESTO and streaming/causal AMT not researched in this pass.
+
+## Q6. Pseudo-labelling at scale
+
+### Takeaway
+Pseudo-labelling works in MIR (chord recognition with >1,000 h teacher-labelled audio; singing transcription with noisy-student self-training) and at very large scale in speech (Distil-Whisper), but the unlabelled corpus must itself be commercially licensed — MTG-Jamendo is NC and now the subject of litigation.
+
+### Cited Findings
+- Chord recognition 2026: BTC teacher labels **>1,000 h** unlabelled audio; student trained on pseudo-labels, then on ground truth with selective KD from teacher as regulariser to limit forgetting; students BTC and 2E1D — [arXiv 2602.19778](https://arxiv.org/html/2602.19778v3).
+- Singing transcription (Kum et al., ICASSP 2022): frame-level pitch pseudo-labels → note level via pitch/rhythm quantization → teacher-student self-training; noisy-student helped — [arXiv 2203.13422](https://arxiv.org/abs/2203.13422v2).
+- Pseudo-labelling previously applied in MIR to piano transcription and tagging — [arXiv 2602.19778](https://arxiv.org/html/2602.19778v3).
+- Distil-Whisper: "Robust Knowledge Distillation via Large-Scale Pseudo Labelling" — [arXiv 2311.00430](https://arxiv.org/pdf/2311.00430).
+- MTG-Jamendo NC + Jamendo v. NVIDIA — [UPF](https://repositori.upf.edu/items/e48c8c42-dba1-404d-85b7-5a0d6ef99e6d/full); [MBW](https://www.musicbusinessworldwide.com/nvidia-hit-with-us-copyright-suit-claiming-it-trained-ai-models-on-winamp-subsidiary-jamendos-music-without-permission/).
+- MuScriptor's 11k h of real aligned audio (prior survey) is an example of large-scale real-audio labelling.
+
+### Inferences
+- Viable commercial pseudo-label pool: tracks individually licensed CC BY / CC0 (filter per-track licenses from Jamendo/FMA/Free Music Archive by artist license, not dataset license), user-uploaded songs only with explicit ToS consent, and licensed catalogues.
+- Use confidence filtering and the two-stage (pseudo → ground truth + KD regulariser) recipe from the chord paper; teacher should be a permissive or self-trained large model.
+
+### Gaps
+- No pseudo-labelling study specific to guitar/bass/drum transcription found; FMA dataset license and per-track CC-BY share not quantified.

research_notes/Transcription accuracy improvement ideas/guitar_tab_frontier.md

@@ -0,0 +1,250 @@
+# Frontier and unconventional methods for guitar/bass tablature from full-band recordings (as of Oct 2026)
+
+Scope note: this builds on `/home/user/STSheets/reports/Music transcription tooling survey.md`, which already covers TART (54.08% end-to-end Tab F1, 71.8% from perfect MIDI), Fretting-Transformer, midi2tab, GOAT, GAPS, SynthTab, MuScriptor, and the lack of lead/rhythm separation. That material is not repeated here except where needed. Research budget was limited (about 17 tool calls), so several items are flagged as unverified or gaps. Some arXiv abstract pages gave methods but no numbers; those are marked.
+
+## 1. Symbolic priors for tabs (tab LMs, rescoring, repetition priors)
+
+### Takeaway
+There is a mature family of DadaGP-trained tab language models (GTR-CTRL, ShredGP, ProgGP, all Transformer-XL), but none has been used to rescore audio transcriptions. The 2024–26 work that does use symbolic priors applies them only to string/fret assignment (MIDI-to-Tab MLM, Fretting-Transformer, the 2026 constrained-decoding paper). Using a tab LM as a rescoring prior, as speech recognition does, is an open idea with no published evidence.
+
+### Cited Findings
+- DadaGP: about 26,181 Guitar Pro songs, available both as GP files and as a token format designed for GPT-2/Transformer-XL style models. — [DADABOTS science](https://dadabots.com/science/); [GTR-CTRL arXiv 2302.05393](https://arxiv.org/pdf/2302.05393)
+- GTR-CTRL (2023): a Transformer-XL tab generator conditioned on instrumentation (inst-CTRL) and genre (genre-CTRL). It also introduces GPBERT, a genre classifier over GP token sequences. — [arXiv 2302.05393](https://arxiv.org/pdf/2302.05393)
+- ShredGP (2023): Transformer tab generation conditioned on the style of individual guitarists. — [arXiv 2307.05324](https://arxiv.org/pdf/2307.05324)
+- ProgGP (2023): the DadaGP-pretrained Transformer-XL fine-tuned on a 173-song progressive-metal GP corpus. — [arXiv 2307.05328](https://arxiv.org/pdf/2307.05328)
+- SCORE-SET (2025): a new dataset of Guitar Pro files for phrase generation and sequence learning. It includes technique markings such as bends and palm muting. — [arXiv 2507.18723](https://arxiv.org/html/2507.18723v1)
+- MIDI-to-Tab (ISMIR 2024; Edwards, Riley, Sarmento, Dixon): an encoder-decoder Transformer trained with masked language modeling. It is pretrained on DadaGP (>25k tabs) and fine-tuned on professionally transcribed performances. It "significantly outperforms competing algorithms", and a guitarist user study rated playability. The abstract gives no numbers. The paper is CC BY-NC-SA 4.0 and no code link was found. — [arXiv 2408.05024](https://arxiv.org/abs/2408.05024)
+- "Explicit Note-Event Tokenization and Pitch-Validity Constrained Decoding for MIDI-to-Tablature Transcription" (Hsu, Wang, Hong, Chen; July 2026): adds note-event tokens alongside TAB tokens. During generation it masks string/fret candidates that would produce the wrong pitch, which guarantees pitch fidelity, and it preserves the input timing. Trained on DadaGP and the small Francois Leduc set, it improves on Fretting-Transformer, with larger gains on Leduc. The abstract gives no numbers. Code "will be released" at github.com/MusicGuitarTab/GuitarTab. — [arXiv 2607.26440](https://arxiv.org/pdf/2607.26440)
+- A search for papers that use a tab LM to rescore audio-transcription hypotheses (n-best or lattice rescoring) found none. — search result summary, see [TART 2609.11904](https://arxiv.org/html/2609.11904v1), whose T5 stage conditions on audio plus symbolic input but is a single model, not a rescorer.
+- Older relevant work: "Automatic composition of guitar tabs by Transformers and groove modeling" (ISMIR 2020) is the origin of the tab-token idea. — [ISMIR 2020](https://archives.ismir.net/ismir2020/paper/000349.pdf)
+
+### Inferences
+- **LM rescoring recipe (untested, plausible).** The transcriber emits frame-level pitch posteriors. Generate k-best note/tab hypotheses per bar, for example by varying the onset threshold or the alternative string assignments. Score each hypothesis with log P_audio + λ·log P_tabLM, using a DadaGP-trained GPT on GP tokens. Tune λ on the GP-reference benchmark. This mirrors shallow fusion and n-best rescoring in ASR. The LM should most help with (a) deleting implausible spurious notes (bleed from other stems), (b) filling dropped notes in idiomatic patterns (power-chord fifths, octave doubles), and (c) choosing among string positions.
+- **Pitch-validity constrained decoding** (2607.26440) is the right pattern for any generative tab model: mask outputs inconsistent with the detected pitch set. It is cheap and removes a whole error class.
+- **Repetition prior.** Rock and metal guitar parts repeat by riff. No paper quantifying a repetition prior for tab transcription was found. A cheap DIY method: build a bar-level self-similarity matrix on chroma from the guitar stem, cluster repeated bars, and use majority voting or consensus over repetitions to fix per-instance errors. This also forces consistent fingering across repeats, which human tabbers do.
+- **License risk.** DadaGP derives from community GP tabs. GTR-CTRL/ProgGP weights and DadaGP's license status were not verified in this pass.
+
+### Gaps
+- Licenses and availability of GTR-CTRL/ShredGP/ProgGP weights were not checked.
+- No numeric evidence was found for LM-based correction of audio tab transcription. This would be novel work.
+- MIDI-to-Tab's numbers and the constrained-decoding paper's numbers were not retrieved (abstract pages only).
+
+## 2. Playability / fingering models: which matches human tabbers best
+
+### Takeaway
+Learned seq2seq assigners pretrained on DadaGP (MIDI-to-Tab MLM, Fretting-Transformer, TART's audio-conditioned T5, the 2026 constrained-decoding model) are the current best at matching human tabs. Classic DP/Viterbi with ergonomic costs remains the license-clean baseline. No head-to-head on high-gain rock tabs was found.
+
+### Cited Findings
+- MIDI-to-Tab frames fingering as masked-LM inference rather than the constraint-based DP of prior work. Its premise is that DP methods minimize hand movement, which does not always match what humans write. — [arXiv 2408.05024](https://arxiv.org/abs/2408.05024)
+- TART stage 3 assigns string/fret with a T5 encoder-decoder conditioned on both the pitch sequence and the original audio. Its symbolic-only T5 variant averages 67.0 across benchmarks and beats prior systems on four zero-shot benchmarks. — [arXiv 2609.11904](https://arxiv.org/html/2609.11904v1) (via search summary)
+- Classic method: audio tab transcription with multipitch analysis plus playability constraints, using dynamic programming across frames for temporal continuity. — [ResearchGate, Barbancho et al.](https://www.researchgate.net/publication/261282224_Audio-based_guitar_tablature_transcription_using_multipitch_analysis_and_playability_constraints)
+- Earlier, as noted in the prior survey: Fretting-Transformer vs Guitar Pro vs TuxGuitar vs A*, and the 72–82% exact agreement on hard material. — [arXiv 2506.14223](https://arxiv.org/html/2506.14223v1)
+
+### Inferences
+- **Hybrid assigner.** Run a Viterbi with ergonomic costs (fret span, shift distance, open-string bonus, string-skip penalty). Add a **chord-shape dictionary bonus**: power chord root on E/A/D with a +2-fret fifth, CAGED open chords, barre shapes. Add a **riff-consistency term**: the same pitch sequence in repeated bars gets the same fingering. This approximates human-tabber conventions cheaply. Later, replace the emission/transition scores with a learned model's logits.
+- **Style conditioning.** Genre tags (from GTR-CTRL-like conditioning or a metadata lookup) can switch cost presets. Metal favors low-position power chords and the open low string in drop tunings. Folk/pop favors open chords. Solos favor box positions.
+- **Audio conditioning** (TART-style) is the principled way to inject string timbre (see section 3) into fingering decisions.
+- **Lenient Tab F1** (counting any playable alternative as correct) should be reported alongside exact Tab F1, because human references disagree.
+
+### Gaps
+- No study was found that directly measures which assigner best matches human tabbers on distorted rock/metal GP references.
+- No published chord-shape-dictionary assigner with numbers was found in 2023–26.
+
+## 3. String identity from audio (timbre/inharmonicity)
+
+### Takeaway
+String classification works on clean, isolated, monophonic guitar: about 87–90% in the best cases, and about 37–72% on older and harder setups. No evidence was found that it survives heavy distortion or source-separated full mixes. Use it as a soft prior inside the fingering decoder, not as a hard decision.
+
+### Cited Findings
+- Fretiq (Garg, July/Aug 2026, CC BY 4.0 paper):
+  - Monophonic electric-guitar string classification from a 26-dimensional engineered feature vector (band energies, spectral statistics, 13 MFCCs), with identical Python and TypeScript feature extraction for browser inference.
+  - Accuracy: 97.25% on shuffled frame validation (which the author says overestimates), **86.53% on held-out recording sessions**, and **87.8% on independent free-play**.
+  - MFCCs are the main driver; removing them drops shuffled accuracy to 92.09%.
+  - "Comparison Training", recording same-pitch pairs on adjacent strings, adds about 25.8 points on matched evaluation.
+  - Single guitar and single player; distortion not addressed; no code link found.
+  — [arXiv 2607.18303](https://arxiv.org/pdf/2607.18303)
+- Abeßer (CMMR 2012): automatic string detection for bass and electric guitar with inharmonicity-based features. Reported F-measures look low, roughly 0.37–0.46 (snippet garbled), though it beat MFCC baselines. — [CMMR 2012 PDF](https://cmmr2012.eecs.qmul.ac.uk/sites/cmmr2012.eecs.qmul.ac.uk/files/pdf/papers/cmmr2012_submission_70.pdf)
+- String-Inverse Frequency estimation: SVMs on spectral-envelope features reach about 0.90 F-measure for six-string electric guitar, and earlier SIF features reach up to about 0.72 F1 (per search snippet). — [ResearchGate](https://www.researchgate.net/publication/328602822_Automatic_Guitar_String_Detection_by_String-Inverse_Frequency_Estimation)
+- Barbancho et al., inharmonicity-based method for automatic tablature generation (classic, pre-2023). — [ResearchGate](https://www.researchgate.net/publication/260691838_Inharmonicity-Based_Method_for_the_Automatic_Generation_of_Guitar_Tablature)
+- Physics caveat: inharmonicity grows with string core diameter, so an unwound G3 can be **more** inharmonic than a wound D3. Inharmonicity is not monotonic across strings, which complicates simple rules. — [TUM paper](https://www.ce.cit.tum.de/fileadmin/w00cgn/mmk/Lehre/Musikalische_Akustik/PDF/1048.pdf)
+- Distorted guitar: one 2024 study analyzes inharmonicity, loudness and centroid on Moises-separated distorted guitar. It is descriptive and work-in-progress, not a classifier. Clipping adds many extra harmonics. — [ABRAPEM 2024](https://abrapem.org/wp-content/uploads/2024/09/328.-Distortion-the-sound-of-distorted-electric-guitar-between-perception-spectral-descriptors-and-artificial.pdf)
+
+### Inferences
+- Under distortion, the nonlinearity generates intermodulation and harmonic products that swamp the partial-frequency deviations that inharmonicity estimators rely on. String classification on high-gain stems is therefore likely to degrade substantially. This is an inference with no direct measurement found.
+- Clean and crunch parts (and bass, which is usually less distorted) are where string cues are most likely to help. Bass string ID from inharmonicity was studied by Abeßer.
+- Practical use: feed a per-note string-probability vector as an emission term into the Viterbi/T5 assigner, with a weight learned on the benchmark. Fretiq-style features are cheap and browser-feasible.
+
+### Gaps
+- No paper was found evaluating string classification on distorted guitar or separated stems.
+- Kehling et al. (DAFx 2014) and the IDMT-SMT-Guitar string-ID results were not verified in this pass.
+
+## 4. Multi-guitar handling (spatial, timbre, role, symbolic part assignment)
+
+### Takeaway
+No research system assigns transcribed notes to multiple guitar parts in full mixes. The usable building blocks are classic panning-based separation (Avendano panning index, ADRess/DUET), mixing conventions (rhythm guitars double-tracked and hard-panned, lead near center), and symbolic voice separation. MVSEP reportedly has a lead/rhythm split model, but no metrics were found for it.
+
+### Cited Findings
+- Panning-index separation: Avendano's panning index, then Barry et al.'s ADRess (azimuth discrimination and resynthesis), derived from DUET. ADRess uses gain scaling and phase cancellation to find frequency-dependent nulls in the azimuth domain. It works best on studio mixes where instruments occupy distinct pan positions. — [Overview of lead/accompaniment separation, arXiv 1804.08300](https://arxiv.org/pdf/1804.08300); [Barry et al. DAFx 2004](https://dafx.de/paper-archive/2004/P_240.PDF)
+- DUET's W-disjoint orthogonality assumption does not hold well for music. — [US8027478B2](https://patents.google.com/patent/US8027478)
+- ADRess is implemented in Csound as `pvsdemix`, which takes an azimuth target. This is a ready reference implementation. — [Csound manual](https://csound.com/manual/pvsdemix.html)
+- Practitioner conventions: double-tracked rhythm guitars are hard-panned, and the solo "takes the place of the lead vocal" at or near center. Some engineers place guitars at 10 and 2 o'clock, or the lead around 30% on each side. — [Producer Society](https://producersociety.com/how-to-pan-guitars/); [KVR forum](https://www.kvraudio.com/forum/viewtopic.php?t=546325)
+- A user on the Steinberg forum says the only way to separate lead from other guitars is via panning. Another mentions that MVSEP has a lead/rhythm guitar model. — [Steinberg forum](https://forums.steinberg.net/t/sl12-pro-for-splitting-lead-rhythm-guitar-advice-and-sourcing-video-tutorials/1015919)
+
+### Inferences
+- **Pipeline idea:**
+  1. Run the guitar-stem separator on L and R channels separately, as well as on mid and side. The guitar stem output keeps stereo.
+  2. Compute a per-time-frequency-bin panning index on the guitar stem (ADRess-style) and cluster azimuths per section.
+  3. Transcribe each azimuth cluster separately.
+  4. Merge clusters whose note content is near-identical. A double-tracked rhythm part appears hard L and hard R with the same notes, so it collapses to one GP track, which matches human tabbing.
+  5. Assign the center cluster to lead or other.
+- **Role classification heuristics:** monophonic, high-register, bend/vibrato-heavy and sparse-repetition material suggests lead. Polyphonic, chordal, palm-muted, riff-repetitive and low-register material suggests rhythm. These features are computable from the transcription itself.
+- **Symbolic voice separation / part assignment** (e.g. a GPBERT-like classifier over GP tokens, or a model trained on DadaGP multi-track songs to split a merged guitar note stream into tracks) is plausible because DadaGP contains multi-guitar songs with explicit tracks. That makes it free supervised data: merge tracks, then learn to split them. No published model doing this was found.
+- **Timbre clustering:** amp-tone embeddings per note segment, from any audio embedding model, could separate clean versus distorted guitars even when co-panned. This is untested.
+
+### Gaps
+- No academic evaluation of lead/rhythm guitar separation or of symbolic guitar part assignment was found.
+- MVSEP and Moises lead/rhythm models have no published metrics (consistent with the prior survey).
+- No study was found measuring how often commercial rock mixes follow the double-track hard-pan convention.
+
+## 5. Tuning and capo detection
+
+### Takeaway
+There is a small amount of evidence that tuning classification from audio is feasible (DLfM 2024 deep-learning classifier; a 2023 Rochester student project). Capo detection is explicitly unaddressed ("future work"). Drop-D versus standard is the hard case.
+
+### Cited Findings
+- "Acoustic Classification of Guitar Tunings with Deep Learning" (DLfM 2024, ACM/Cardiff):
+  - Neural networks classify tunings from recordings.
+  - It releases annotated authentic and synthetic guitar datasets.
+  - Five tuning classes, including standard (90 songs), drop D (86) and open D.
+  - Capo and open-string detection is listed as future work, using spectrograms or CQTs.
+  - Exact accuracy figures were not retrieved (the PDF returned 403).
+  — [ACM DL](https://dl.acm.org/doi/fullHtml/10.1145/3660570.3660574); [Cardiff ORCA PDF](https://orca.cardiff.ac.uk/id/eprint/170217/1/dlfm2024-4%20(2).pdf)
+- Rochester ECE477 project (Chew and Parker, 2023): classifies seven tunings from audio. It notes that Drop D is hard to distinguish from E standard, and it was tested only on monophonic audio. — [Report PDF](https://hajim.rochester.edu/ece/sites/zduan/teaching/ece477/projects/2023/DeclanParker_McCormackChew_FinalReport.pdf)
+
+### Inferences
+- **Practical heuristics** (DIY, unvalidated):
+  1. Estimate the global reference deviation from A440 using the pitch-histogram cents offset over the whole song. This catches down-tuned-by-half-step (Eb standard), a common case in rock and metal.
+  2. Take the lowest sustained pitch in the guitar stem. Below E2, this means a drop or down-tuning: D2 gives drop D or D standard, C#2/C2 gives drop C# or drop C, and so on.
+  3. Distinguish drop D from D standard with a playability likelihood: transcribe the notes, then run the fingering decoder under each candidate tuning and pick the tuning that minimizes total ergonomic cost or maximizes tab-LM likelihood. One-finger power chords on the lowest three strings are a strong drop-tuning signature.
+  4. Infer a capo the same way. Open-string resonance pitches (sympathetic ringing, open-string drones) plus a playability minimum under fret offsets give the capo position. Strummed open-chord voicings transposed up are the giveaway.
+  5. Use bass lowest-note evidence to cross-check: for example, a 5-string B0, or a drop-D bass with D1.
+- Let the user override tuning and capo in the UI. That is cheap and removes a large error source.
+
+### Gaps
+- Accuracy numbers for the DLfM 2024 paper and the Rochester project were not retrieved.
+- No paper on capo detection from audio was found.
+- No paper on bass tuning detection was found.
+
+## 6. Retrieval and alignment of existing community tabs
+
+### Takeaway
+The closest research is DECIBEL (TISMIR), which aligns crowd-sourced tabs and chord sheets to audio with "Jump Alignment" and uses them to improve chord estimation. No research or product was found that does note-level "tab retrieval + alignment" for GP tabs. Songsterr's internal methods are unknown. Legally this is the riskiest idea.
+
+### Cited Findings
+- DECIBEL / "Improving Audio Chord Estimation by Alignment and Integration of Crowd-Sourced Symbolic Music" (TISMIR):
+  - A parser extracts untimed chord sequences from guitar tabs and chord sheets.
+  - "Jump Alignment" aligns them to audio while allowing jumps between tab lines, which handles repeats and missing sections.
+  - Selecting the best tab per song reaches 75.0% WCSR.
+  - The paper notes that tabs carry no timing information.
+  — [TISMIR](https://transactions.ismir.net/articles/10.5334/tismir.81); [arXiv 2002.09748](https://arxiv.org/html/2002.09748v1)
+- Audio-to-score alignment with DTW on chroma is standard. Recent work refines precision with neural CQT-to-piano-roll alignment. — [arXiv 2605.20014 "Precise and Simple Audio-to-Score Alignment"](https://arxiv.org/pdf/2605.20014); [arXiv 2506.22237](https://arxiv.org/pdf/2506.22237)
+
+### Inferences
+- **Technical recipe:**
+  1. Identify the song (fingerprint or metadata).
+  2. Fetch candidate tab(s).
+  3. Synthesize them, or derive chroma and onset features from the score.
+  4. Align with a jump-aware DTW (DECIBEL-style, for repeats and structural differences) anchored on beat-tracker downbeats.
+  5. Use the aligned tab as a strong prior: rescoring, fingering, part assignment, tuning. The audio transcription corrects the tab's errors, and the tab corrects the audio errors.
+  6. Detect version mismatch (live versus studio recording, different key) via alignment cost.
+- The same machinery is needed anyway for the benchmark (aligning GP references to audio).
+- **Legal (inference, not legal advice):**
+  - Community tabs are derivative works of copyrighted compositions. Songsterr and Ultimate Guitar license publishers' content, and their ToS typically forbid scraping.
+  - Using DadaGP-style data to train a prior is a different (training-data) risk than serving retrieved tabs.
+  - A safer variant uses only user-supplied tabs ("upload your GP file and we align and fix it"), or retrieval only for internal evaluation.
+  - Get legal review before any scraping.
+
+### Gaps
+- No research or product doing note-level GP tab retrieval plus alignment was found.
+- Songsterr AI's internals (whether it leverages its tab catalogue) are unknown.
+- Songsterr's and Ultimate Guitar's ToS were not fetched in this pass.
+
+## 7. Technique detection on (distorted) electric guitar
+
+### Takeaway
+Datasets now exist: Guitar-TECHS 2025, GOAT 2025 with bends and palm mute in GP format, the Magcil set cited by TART, and classic IDMT-SMT-Guitar. TART reports high per-note classification on curated data (95.9% macro F1 per the prior survey). Evidence for robust technique detection in distorted full-mix stems is still missing.
+
+### Cited Findings
+- TART (earlier version, arXiv 2510.02597) targets slides, bends and percussive hits with a per-note technique classifier stage. It cites Magcil: 549 electric guitar recordings with technique labels including bends and palm mutes. — [arXiv 2510.02597](https://arxiv.org/html/2510.02597)
+- Guitar-TECHS (2025):
+  - Electric guitar recorded on diverse hardware (guitars, amps, mics, environments).
+  - Six single-note techniques: alternate picking, palm mute, vibrato, harmonics, pinch harmonics, bends.
+  - Also chords (maj, min, aug, dim, maj7, dom7, min7, m7b5; no power chords listed), scales and excerpts.
+  - Hosted on Zenodo.
+  — [arXiv 2501.03720](https://arxiv.org/html/2501.03720v1); [Zenodo](https://zenodo.org/records/14963133)
+- GOAT pairs audio with GP tabs including bend and palm-mute annotations, and names technique identification as a use case. — [arXiv 2509.22655](https://arxiv.org/html/2509.22655v1)
+- Classic: ISMIR 2015 electric-guitar technique detection in solo tracks, including bends, with accuracies from about 58% to 88% depending on technique (per search snippet). — [ISMIR 2015](https://www.ismir2015.uma.es/articles/119_Paper.pdf)
+
+### Inferences
+- **Bends and vibrato** are best detected from the F0 contour (Basic Pitch pitch-bend output, or a monophonic F0 tracker on the lead cluster). Contour features survive distortion better than spectral-shape features, because pitch is preserved even when partials are smeared. This is an inference.
+- **Palm muting** under high gain correlates with short decay and reduced high-frequency energy. Train on GOAT's amp-augmented audio, which already has GP-format palm-mute labels.
+- **Pinch harmonics** produce a sudden high-pitch onset with a fundamental 2–4 octaves above the fretted note. Treat them as "pitch far above the playable fret plus a lead context" and assign them via the fingering decoder.
+
+### Gaps
+- No technique-detection results were found on separated full-mix distorted stems.
+- No open technique-detection weights were confirmed.
+
+## 8. Bass-specific
+
+### Takeaway
+No dedicated research bass-tab or slap/pop detection model was found for 2023–26. Notation conventions (T/S for slap, P for pop; slaps on low strings, pops on high) can drive heuristic string assignment and technique marking.
+
+### Cited Findings
+- Slap is notated as T (thumb) or S, and pop as P. P can conflict with pull-off notation. — [BassBuzz](https://www.bassbuzz.com/lessons/how-to-read-bass-tabs); [Soundslice](https://www.soundslice.com/help/en/creating/tablature/130/bass-popping-and-slapping/)
+- Slaps are typically on the lower E/A strings and pops on the higher D/G strings. — [Wikipedia: Slapping](https://en.wikipedia.org/wiki/Slapping_(music))
+- Guitar Pro 8 supports slap/pop marks and a "Slap Bass" playing style. — [Guitar Pro blog](https://www.guitar-pro.com/blog/p/53111-tutorial-how-to-create-a-slap-bass-tab-in-guitar-pro-8)
+- Abeßer (CMMR 2012) covers string detection for bass as well as guitar. — [CMMR 2012](https://cmmr2012.eecs.qmul.ac.uk/sites/cmmr2012.eecs.qmul.ac.uk/files/pdf/papers/cmmr2012_submission_70.pdf)
+
+### Inferences
+- **Bass fingering:** a Viterbi with bass-specific costs (one-finger-per-fret span of about 4 frets at low positions, strong preference for staying in position, open-string use in rock). 4-, 5- or 6-string choice comes from the lowest pitch: below E1 means a 5-string or drop tuning.
+- **Slap/pop detection:** percussive high-frequency transient onsets on bass notes. Pitch and register can assign the thumb (low string) or the pop (high string). Training data could come from DadaGP bass tracks with slap/pop marks rendered via SynthTab-style synthesis. This is untested.
+
+### Gaps
+- No 2023–26 bass-tab or slap/pop detection paper was found, so a targeted search is still needed.
+- IDMT-SMT-Bass and related classic datasets were not re-verified.
+
+## 9. Heavily distorted guitar, power chords, chord-level transcription
+
+### Takeaway
+Distortion is the hardest condition: GOAT, two-thirds distorted, is called the hardest dataset in a 2026 benchmark. Mitigations are amp/effect augmentation in training (2024), de-distortion preprocessing (2022–24), and treating rhythm parts at the chord level (root plus power chord or quality) rather than the note level, then expanding to a canonical voicing via a shape dictionary.
+
+### Cited Findings
+- Harmonica (Sept 2026, submitted to ICASSP 2027):
+  - Instrument-agnostic transcription with multi-depth harmonic convolution.
+  - Nano model: 26.3K parameters, frame F1 0.796 on its development set, 1,622.5x real time, beating Basic Pitch by 14.6 points.
+  - Per the search summary, it describes GOAT as the hardest dataset in its benchmark, with about two-thirds of its recordings distorted.
+  - The abstract page did not show distorted-versus-clean numbers or a code link.
+  — [arXiv 2609.04640](https://arxiv.org/pdf/2609.04640)
+- "Leveraging Real Electric Guitar Tones and Effects to Improve Robustness in Guitar Tablature Transcription Modeling" (Pedroza, Abreu, Corey, Roman, 2024):
+  - Adds synthetic training data made from real guitar tones played through various effects.
+  - Evaluates on a new professional solo-guitar set with many tones.
+  - It reports improved robustness, but no numbers appear on the abstract page.
+  — [arXiv 2405.14679](https://arxiv.org/pdf/2405.14679)
+- Distortion removal and recovery as preprocessing: "Distortion Audio Effects: Learning How to Recover the Clean Signal" (2022) and "Distortion Recovery: A Two-Stage Method for Guitar Effect Removal" (2024). — [arXiv 2202.01664](https://arxiv.org/pdf/2202.01664); [arXiv 2407.16639](https://arxiv.org/pdf/2407.16639)
+- EGDB-related: "Towards automatic transcription of polyphonic electric guitar music" (2022) introduced the EGDB dataset of DI guitar re-amped through multiple amp tones (classic, still relevant). — [arXiv 2202.09907](https://arxiv.org/pdf/2202.09907)
+- ACR (GitHub, bachelor's thesis): chord recognition aimed at heavily distorted electric guitar, with templates for major, minor and **power chord** for all 12 roots. — [GitHub Driemtax/ACR](https://github.com/Driemtax/ACR)
+- Guitar-TECHS chord set has no power chords. — [Zenodo](https://zenodo.org/records/14963133)
+
+### Inferences
+- **Chord-level fallback for rhythm parts:**
+  1. Detect frame-level "root + type" with types {5 (power), maj, min, …} on the rhythm-guitar cluster, together with onsets and palm-mute flags.
+  2. Render each chord as its canonical shape, such as a 2- or 3-note power chord on the string set whose root matches the lowest detected pitch.
+  3. Do this under the detected tuning.
+  This exploits the fact that high-gain rhythm guitar is dominated by a few shapes. It also makes errors "musically plausible", which human-judged playability favors.
+- **Augmentation** is the cheapest win for distorted guitar: GOAT DI plus amp-sim and IR chains, EGDB-style re-amping, and the real-tone approach of 2405.14679.
+- **Bass as an anchor:** in metal, the bass usually doubles the rhythm-guitar root, and the bass stem is cleaner (about 14.6 SDR per the prior survey). The bass transcription can constrain power-chord roots in the guitar part.
+
+### Gaps
+- No benchmark of high-gain full-mix tab transcription was found (consistent with the prior survey).
+- Harmonica's GOAT numbers and the 2405.14679 numbers were not retrieved.
+- No published comparison of chord-level versus note-level tab accuracy on distorted rhythm guitar was found.

research_notes/Transcription accuracy improvement ideas/new_amt_paradigms.md

@@ -0,0 +1,242 @@
+# New and Emerging AMT Paradigms (2024 – Oct 2026) Beyond "Separate Stems, Then Transcribe Per Instrument"
+
+Scope note: the prior survey (`reports/Music transcription tooling survey.md`) already covers Demucs/BS-RoFormer, Basic Pitch, MuScriptor, YourMT3+, TART (arXiv 2609.11904), GOAT, SynthTab, midi2tab_Deconvolutions, and Noise-to-Notes (drums, MERT encoder, code not found). These notes add things that survey does not include. The research used about 17 search/fetch calls, so several sub-questions are only partly answered. See the Gaps sections.
+
+## 1. Foundation-model / self-supervised audio features for transcription
+
+### Takeaway
+The clearest 2025–26 evidence that self-supervised music encoders help transcription comes from the 2025 AMT Challenge. Its winner (MIROS) put a **MusicFM** encoder into YourMT3+ and scored 0.600 multi-instrument note-onset F1, against 0.393 for the MT3 baseline. However, the paper does not separate the encoder's contribution from the other changes. Guitar-specific probing evidence is thin: MERT has been used for guitar *strum/rhythm* detection, but I found no published MERT-vs-MuQ comparison for guitar or bass note transcription.
+
+### Cited Findings
+- **2025 AMT Challenge** (NeurIPS 2025 AI for Music Workshop; arXiv 2603.27528, CC BY 4.0). Eight teams submitted valid solutions and two beat MT3. The test set was 76 newly composed pieces of about 20 s each, with 8 orchestral/piano instruments and at most 3 instruments per piece, rendered with FluidSynth/FluidR3 GM. The metric is multi-instrument note-onset F1 (instrument + pitch + onset within ±50 ms) — [arXiv HTML](https://arxiv.org/html/2603.27528v1)
+- Winner **MIROS**: YourMT3+ encoder-decoder with a **MusicFM** encoder (conformer pretrained with BEST-RQ masked-token modelling), a recurrent adapter conditioned on learned instrument-group embeddings, and **parallel per-instrument-group T5-style decoders**. It reported 0.83 F on Slakh2100 at about 370M params. Challenge scores: MIROS F1 0.5998 (P 0.656 / R 0.572); YourMT3-YPTF-MoE-M 0.5938; YourMT3-YPTF-S 0.5581; MT3 baseline 0.3932 — [arXiv HTML](https://arxiv.org/html/2603.27528v1)
+- Other lessons from the challenge: performance dropped significantly as instrument count rose, and MIROS precision fell from 0.907 on solo pieces to 0.464 on 3-instrument pieces. Models hallucinated instruments that were not present ("instrument leakage"). Cross-dataset augmentation gave only marginal gains. Common techniques were MoE, hierarchical time-frequency attention, random-projection quantizers, and auxiliary onset/offset losses. The page gave no team code links — [arXiv HTML](https://arxiv.org/html/2603.27528v1)
+- Guitar rhythm/strum transcription in polyphonic music (arXiv 2510.05756) chose **MERT** for its frame-level strength, especially on beat tracking. It compared frozen probing (a lightweight head on a learnable weighted average of all layers) with fine-tuning — [arXiv 2510.05756](https://arxiv.org/html/2510.05756v1)
+- MARBLE benchmarks MERT, MusicFM and MuQ. All three use masked-LM pretraining with different tokenisation — [search result summarising arXiv 2608.14819](https://arxiv.org/pdf/2608.14819)
+- A 2026 arXiv paper, "What Makes a Good Layer?", assesses layer-wise properties of music foundation models. I could not verify its transcription findings — [arXiv 2608.14819](https://arxiv.org/pdf/2608.14819)
+- Counterpoint, small specialised models: **Harmonica** (arXiv 2609.04640, submitted to ICASSP 2027) uses multi-depth harmonic convolution for instrument-agnostic transcription. Its *nano* model (26.3K params, 1,622× real-time) reaches 0.796 frame F1 on its dev set, which it says is +14.6 pp over Basic Pitch. The x-large model claims SOTA instrument-agnostic results, with no numbers in the abstract. No code or weights were stated — [arXiv 2609.04640](https://arxiv.org/abs/2609.04640)
+- From the prior survey: Noise-to-Notes (drums) uses a 330M MERT encoder.
+
+### Inferences
+- The most practical experiment for us is a **MusicFM or MERT encoder with a lightweight onset/frame/pitch head fine-tuned on guitar and bass stems** (GOAT, GuitarSet, EGDB, plus synthetic DadaGP renders), compared against Basic Pitch on our GP-reference benchmark. MIROS shows the encoder plugs into YourMT3+, which we already use.
+- Harmonica suggests a well-designed harmonic inductive bias can beat Basic Pitch at tiny size. That makes it a candidate replacement for the *browser draft* tier if weights are released. The SSL-encoder route suits the *server quality* tier.
+- The challenge's precision collapse with polyphony supports keeping the separation front-end. Separation reduces the per-model polyphony and instrument count, which is where all models fail.
+
+### Gaps
+- No published numbers for MERT/MuQ/MusicFM/BEATs/Jukebox probing on **guitar or bass note transcription**. I found nothing that answers whether they help low-data instruments, beyond the indirect MIROS evidence.
+- MIROS code and weights not found. MusicFM, MERT and MuQ licenses not verified in this pass. From memory, MERT checkpoints are CC BY-NC 4.0, but this needs verifying before any commercial use.
+- Jukebox-embedding transcription probes (e.g. Castellon et al. 2021) are pre-2024 and were not re-checked.
+
+## 2. Large audio-language models (LALMs) and LLM-based transcription
+
+### Takeaway
+Every source found says general LALMs (GPT-4o, Gemini 2.5, Qwen2-Audio) are **far from usable for note-level transcription**. They handle music given as symbolic input (MIDI/ABC) well, but perception from the waveform is the bottleneck. One audio-to-score benchmark reports 41% accuracy for a GPT-4o pipeline. I found no 2025–26 paper showing a LALM competitive with MT3-class models on standard transcription metrics.
+
+### Cited Findings
+- A 2025 study of core music perception tasks found Gemini models near ceiling with **MIDI input**, but "audio reduced accuracy across tasks", so perception from the waveform is the main bottleneck. Gemini 2.5 Pro and Flash both scored significantly higher on MIDI than on audio — [arXiv 2510.22455](https://arxiv.org/pdf/2510.22455)
+- MuseAgent-1: GPT-4o (55.9%) and Gemini 2.5 Pro (53.1%) are only marginally above random on its performance-audio reasoning benchmark. The authors conclude raw audio is not enough for fine-grained music understanding — [arXiv 2601.11968](https://arxiv.org/pdf/2601.11968)
+- A music-to-score transcription benchmark (reported via search snippet) puts an aggregated GPT-4o system at 41.0% accuracy, and its authors say low audio pitch-recognition accuracy makes audio-to-score transcription unlikely to succeed — [search summary; source among arXiv 2510.22455 / 2607.06015](https://arxiv.org/html/2607.06015v1) (attribution unverified; treat as indicative)
+- **HMU-Bench** (arXiv 2610.15773, omni models) has an ABC part and an audio part (460 question pairs on 92 recordings). The best models do well on notation but are uniformly low on audio. Rankings across the two parts barely correlate (Spearman −0.04). Its audio questions cover chords and sections, not full transcription — [arXiv 2610.15773](https://arxiv.org/html/2610.15773)
+- **ABC-Eval** (ICASSP 2026): text-only LLMs on ABC scores across 10 sub-tasks and 1,086 samples. Seven SOTA LLMs show "notable limitations" in symbolic processing — [arXiv 2509.23350](https://arxiv.org/pdf/2509.23350)
+- Qwen2-Audio uses a Whisper-large-v3 encoder (one frame per ~40 ms) and a 7B Qwen LM. An informal (non-peer-reviewed) blog test found it weak at basic music tasks like tempo and instrument detection — [Qwen2-Audio report](https://arxiv.org/pdf/2407.10759); [TDS blog](https://towardsdatascience.com/exploring-music-transcription-with-multi-modal-language-models-af352105db56/)
+- Seq2seq piano transcription with a pretrained roll-based encoder and a hierarchical LM decoder (arXiv 2501.03038) is the "LM-decoder" paradigm that does work, but it is trained specifically for transcription — [arXiv 2501.03038](https://arxiv.org/pdf/2501.03038)
+
+### Inferences
+- A ~40 ms encoder frame (Whisper-style) is too coarse for ±50 ms onset accuracy with dense notes. That is a structural reason general LALMs underperform.
+- LLMs are better used **downstream of transcription**, on the symbolic side, for:
+  1. cleaning MIDI into readable notation (key/meter/spelling)
+  2. ranking alternative tab fingerings
+  3. explaining low-confidence regions to users
+
+  Even there, ABC-Eval shows text LLMs make symbolic errors, so their outputs need rule-based validation.
+- Do not invest in LALM-based transcription now. Revisit if a dedicated audio→MIDI/ABC LALM is published with MAESTRO/Slakh/GuitarSet numbers.
+
+### Gaps
+- I found no current numbers for SALMONN, MuMu-LLaMA, LLark or Qwen2.5-Omni on note-level transcription. LLark (2023) is known for music captioning/QA, not transcription.
+- No head-to-head of GPT-4o and Gemini on audio→MIDI with mir_eval metrics was found.
+
+## 3. Generative / diffusion approaches to transcription
+
+### Takeaway
+**Noise2Fret** (Aug 2026) is the most relevant new result for us. It is a diffusion audio-to-tab model with **public code and trained models**. On GOAT (electric guitar) it lifts Tab F1 from about 0.66 (TabCNN/FretNet) to **0.795**, and on GuitarSet from about 0.72 to **0.845**. It is also ~10–50× cheaper in FLOPs than those baselines. Generative models seem to win because they model the joint distribution of string/fret choices, which is multi-modal because the same pitch can be played in several positions, and because training adds musical/playability priors through auxiliary losses.
+
+### Cited Findings
+- **Noise2Fret** — Simionato and Bigo (LaBRI, Bordeaux), arXiv 2608.30854, 31 Aug 2026, preprint. It diffuses in a continuous embedding of discrete string-fret targets using a 1D conv U-Net with a self-attention bottleneck. Conditioning: STFT magnitude, spectral flux, brightness and raw audio. It adds five auxiliary losses: pitch-class distance, circle-of-fifths distance, positional distance, string similarity and hand-span feasibility — [arXiv](https://arxiv.org/html/2608.30854)
+- Results (pitch F / tab F / TDR) — [arXiv](https://arxiv.org/html/2608.30854):
+
+  | Dataset | Noise2Fret (+aux) | TabCNN | FretNet |
+  |---|---|---|---|
+  | GuitarSet | 0.856 / 0.845 / 0.988 | 0.820 / 0.717 / 0.860 | 0.818 / 0.727 / 0.871 |
+  | GOAT | 0.800 / 0.795 / 0.993 | 0.664 / 0.656 | 0.669 / 0.662 |
+
+  On GOAT, Noise2Fret's FNR is 0.165 against 0.293 for TabCNN.
+- Cost: ~15M params, ~341M FLOPs per pass (TabCNN 3.4B, FretNet 17.4B). RTF 0.67 on an RTX 3090. Code and trained models are at github.com/RiccardoVib/Noise2Fret, with code license not stated (paper text is CC BY-SA 4.0). No synthetic data and no MERT were used — [arXiv](https://arxiv.org/html/2608.30854); [GitHub](https://github.com/RiccardoVib/Noise2Fret); [Zenodo](https://zenodo.org/records/23037260)
+- Hand-span feasibility is a training loss only. No playability metric is reported — [arXiv](https://arxiv.org/html/2608.30854)
+- Third-party projects are already reusing Noise2Fret playability costs in a fingering engine with AlphaTex export (unreviewed open-source PR) — [GitHub PR](https://github.com/quantumq1981/Tab-Translator-Pro/pull/116)
+- **GuitarFlow** (arXiv 2510.21872) is the reverse direction: flow-matching synthesis of realistic electric guitar audio from tablature with style transfer. It is a candidate renderer for synthetic training data and analysis-by-synthesis — [arXiv 2510.21872](https://arxiv.org/pdf/2510.21872)
+- Joint piano rendering and transcription by disentangling score content from performance style (arXiv 2509.23878) is a generative, bidirectional model of audio and score — [arXiv 2509.23878](https://arxiv.org/pdf/2509.23878)
+
+### Inferences
+- **Top experiment:** run Noise2Fret on our separated guitar stems and compare against "Basic Pitch → Viterbi assigner" on the GP-reference benchmark. Check the code license first. Caveat: GuitarSet and GOAT are solo/DI-ish recordings, so real gains on separated, distorted mix stems are unknown. We would likely need to fine-tune on amp-rendered or stem-like audio.
+- Diffusion sampling gives multiple candidate tabs per segment for free. Their disagreement is a natural **uncertainty signal** (see section 8).
+- The auxiliary-loss idea transfers to bass (4/5-string) with retargeted string counts.
+
+### Gaps
+- DiffRoll (2022, piano, pre-2024) was not re-checked. Noise-to-Notes is already covered in the prior survey.
+- No Noise2Fret numbers exist on polyphonic mixes, separated stems or high-gain distortion.
+
+## 4. Joint separation + transcription / multi-task models
+
+### Takeaway
+Few new 2024–26 joint separation+transcription models appeared. The active directions are:
+1. timbre-clustered "transcribe and separate at the note level" (arXiv 2509.12712)
+2. analysis-by-synthesis drum models that learn separation from transcription labels alone (Inverse Drum Machine, 2025)
+3. multi-decoder MT3 descendants that do instrument-grouped decoding (MIROS)
+
+Classic joint models (Jointist, Cerberus, Lin et al.) are 2021–23.
+
+### Cited Findings
+- **Jointist** (2023, older) jointly trains instrument recognition, transcription and separation, and reports that each task improves the others — [arXiv 2302.00286](https://arxiv.org/html/2302.00286)
+- Lin et al. 2021 (older): a unified zero-shot separation/transcription/synthesis model. The transcribed score is a "powerful auxiliary" for separation — [arXiv 2108.03456](https://arxiv.org/abs/2108.03456)
+- **Lightweight multi-instrument transcription** (arXiv 2509.12712, 2025) extends a timbre-agnostic backbone with a timbre encoder and **note-level deep clustering**. This gives joint transcription and dynamic separation of arbitrary instruments ("timbre-separated transcription" at the symbolic level) — [arXiv 2509.12712](https://arxiv.org/html/2509.12712v3)
+- **Inverse Drum Machine** (arXiv 2505.03337, v2 Sep 2025) jointly optimises drum transcription and one-shot sample synthesis. It convolves onsets with synthesised one-shots to rebuild stems and trains on mixture reconstruction using only transcription labels (StemGMD). It claims separation "comparable to SOTA supervised methods" that need isolated stems. No numbers or code on the abstract page — [arXiv](https://arxiv.org/abs/2505.03337)
+
+### Inferences
+- Note-level timbre clustering (2509.12712) directly targets one of our hard problems: **splitting multiple guitars** that stem separators lump into one "guitar/other" stem. Worth evaluating as a post-separation step that clusters transcribed notes into Guitar 1 and Guitar 2.
+- For drums, Inverse-Drum-Machine-style reconstruction gives a self-supervised consistency check. Resynthesise the predicted hits and compare with the drum stem.
+
+### Gaps
+- No 2024–26 "transcription-informed separation" result with SDR+F1 on rock/pop guitar was found.
+- Code availability for 2509.12712 was not checked.
+
+## 5. Synthetic-data strategies and the sim-to-real gap
+
+### Takeaway
+Synthetic rendering (SynthTab, already in the prior survey) remains the main way to scale guitar data. New pieces are:
+- amp/effects augmentation frameworks such as Open-Amp
+- adversarial domain-confusion sim-to-real training (Sato and Akama, ICME 2024)
+- a WASPAA 2025 synthetic-vs-real guitar comparison (Kusaka and Maezawa, findings not retrieved)
+- flow-matching guitar renderers (GuitarFlow) that could close the timbre gap
+
+I found no paper that uses NAM captures specifically to train a transcriber.
+
+### Cited Findings
+- **Open-Amp** (arXiv 2411.14972): a synthetic data framework that renders guitar (IDMT-Guitar DI) through amp and pedal models on the fly, for audio-effect foundation models. It is applied to distortion-effect classification and effect emulation — [arXiv](https://arxiv.org/html/2411.14972v1)
+- **EGDB**: electric guitar processed by various digital amp models, made for transcription — [search summary](https://arxiv.org/html/2411.14972v1)
+- Sato and Akama (ICME 2024): annotation-free transcription from scalable synthetic data plus **adversarial domain confusion** — [cited in search results; TART reference list](https://arxiv.org/pdf/2609.11904)
+- Kusaka and Maezawa (WASPAA 2025): compares guitar transcription trained on synthetic vs. real audio. Only the title and venue were retrieved, not the results — [TART reference list](https://arxiv.org/pdf/2609.11904)
+- GOAT amp-augments 5.9 h of DI to 29.5 h (prior survey). Noise2Fret on GOAT reached Tab F1 0.795 — [arXiv 2608.30854](https://arxiv.org/html/2608.30854)
+- **GuitarFlow** (arXiv 2510.21872): realistic electric-guitar synthesis from tabs via flow matching and style transfer — [arXiv](https://arxiv.org/pdf/2510.21872)
+- DDSP string-wise polyphonic guitar synthesis from string-wise MIDI (arXiv 2309.07658, 2023): six per-string DDSP voices summed through a trainable reverb — [arXiv](https://arxiv.org/pdf/2309.07658)
+- Velocity estimation across instruments via **differentiable SoundFont proxies** (arXiv 2608.08985) — [arXiv](https://arxiv.org/pdf/2608.08985)
+
+### Inferences
+Proposed pipeline:
+1. DadaGP/GP files → per-string render, using a DDSP or GuitarFlow-style renderer or sampled guitars.
+2. Randomised NAM captures, cab IRs and pedals (Open-Amp-style).
+3. **Mix with other DadaGP tracks, then run through our actual separator.**
+4. Train on the *separated* output.
+
+Step 4 teaches the transcriber the separator's artefacts, which is probably the largest sim-to-real gap for our pipeline. Add a domain-confusion loss using unlabeled real stems.
+
+### Gaps
+- No quantitative sim-to-real numbers for guitar were retrieved (the Kusaka and Maezawa results are missing). No study uses NAM models for transcription data. Licenses for NAM community captures vary per capture.
+
+## 6. Weak / noisy supervision and alignment-based training
+
+### Takeaway
+This is the most mature route to large real-audio training sets, and it fits us well because we can pair user .gp files and public tabs with recordings:
+- **NoteEM** (2022): alignment plus EM pseudo-labels
+- **Riley et al. 2024**: aligning tabs to piano-model activations, then training a high-res guitar model, with zero-shot GuitarSet SOTA
+- **CountEM** (ISMIR 2025): supervision from note *histograms* only, no alignment. Fine-tuning on GAPS raised GuitarSet accuracy from 66.2% to 84.6%.
+- Cycle-consistent training with almost no supervision (2026): 64.81 GuitarSet frame F1 from unlabeled guitar audio
+
+### Cited Findings
+- **NoteEM** (Maman and Bermano, ICML 2022, older) trains a transcriber while aligning unaligned scores to performances. It adds pseudo-labels and pitch-shift augmentation, and its guitar setup used ~5 h of self-collected recordings evaluated on GuitarSet — [PMLR](https://proceedings.mlr.press/v162/maman22a/maman22a.pdf); [arXiv](https://ar5iv.labs.arxiv.org/html/2204.13668)
+- **CountEM / "Count The Notes"** (Yaffe, Maman, Müller, Bermano; ISMIR 2025; arXiv 2511.14250, CC BY 4.0):
+  - Method: uses **note-occurrence histograms** as the training signal in EM, with no local alignment (no DTW or soft-DTW). It matches or beats prior weakly supervised methods on piano, guitar and multi-instrument data — [arXiv abs](https://arxiv.org/abs/2511.14250)
+  - Search snippet: fine-tuning on GAPS with whole-track histogram supervision improved GuitarSet accuracy from 66.2% to 84.6% — [arXiv PDF](https://arxiv.org/pdf/2511.14250)
+  - Project page: [yoni-yaffe.github.io/count-the-notes](https://yoni-yaffe.github.io/count-the-notes)
+- **High-resolution guitar transcription via domain adaptation** (Riley, Edwards, Dixon; arXiv 2402.15258, 2024): adapts Kong et al.'s high-res piano model to guitar by aligning professional transcriptions (~60 train pieces by F. Leduc) to the piano model's activations, NoteEM-style. It claims SOTA zero-shot on all of GuitarSet (no figure on the page). The dataset is on Zenodo; no code or weights are linked — [project page](https://xavriley.github.io/HighResolutionGuitarTranscription/)
+- **Music Transcription with (Almost) No Supervision** (arXiv 2605.24193, 2026): cycle-consistent learning on unpaired audio and scores reaches **64.81 frame F1 on GuitarSet** from unlabeled guitar audio plus minimal piano supervision. Without any paired data, training admits pitch-shifted degenerate solutions — [arXiv](https://arxiv.org/html/2605.24193v1)
+- "Just Label the Repeats" (arXiv 2411.07428): low-effort annotation for in-the-wild audio-to-score alignment — [arXiv](https://arxiv.org/pdf/2411.07428)
+
+### Inferences
+- **Highest-leverage data strategy for us:** gather (recording, Guitar Pro tab) pairs, from user uploads with consent or from licensed tab sites. Run our separator, then train with CountEM (robust to structural mismatch such as repeats and tab errors) or NoteEM alignment on the *separated stems*. This produces real-audio, real-mix training data for guitar, bass and drums, and directly targets the gap between solo-guitar test sets and dense mixes.
+- CountEM's histogram loss tolerates imperfect user tabs (wrong repeats, missing sections) better than strict DTW alignment.
+
+### Gaps
+- Code release for CountEM and 2605.24193 not confirmed. No per-instrument (bass, drums) results for these methods in mixes.
+
+## 7. Test-time adaptation, domain adaptation, pseudo-labelling
+
+### Takeaway
+I found **no published test-time adaptation (TTA) method specifically for AMT**. Pseudo-labelling exists only at training time (NoteEM, teacher-student vocal transcription). General TTA work warns about confirmation bias from self-generated pseudo-labels.
+
+### Cited Findings
+- NoteEM uses network self-predictions as pseudo-labels alongside alignment, at training time only — [arXiv 2204.13668](https://arxiv.org/abs/2204.13668)
+- Teacher-student vocal transcription converts frame-level pseudo-labels from pitch estimators into note-level labels (2022, older) — [arXiv 2203.13422](https://arxiv.org/pdf/2203.13422)
+- Improved self-training for TTA (CVPR 2024) fixes unreliable pseudo-labels with multiple augmented views, feature-similarity graphs and parameter moving averages — [CVF](https://openaccess.thecvf.com/content/CVPR2024/papers/Ma_Improved_Self-Training_for_Test-Time_Adaptation_CVPR_2024_paper.pdf)
+- A 2026 survey of self-improving test-time methods notes confirmation-bias risk when a model trains on its own predictions — [arXiv 2609.01679](https://arxiv.org/pdf/2609.01679)
+- Audio TTA in wild acoustic conditions (2023) — [arXiv 2310.09505](https://arxiv.org/pdf/2310.09505)
+
+### Inferences
+- A cheap per-song TTA to try: adapt only BatchNorm/LayerNorm statistics, or a small adapter, on the song's stem. Use high-confidence pseudo-labels with augmentation agreement (pitch-shift and EQ views must agree), and stop early. This is an open research opportunity rather than an established recipe, so its benefit must be measured.
+- Song-level self-consistency (repeated riffs and choruses should transcribe identically) is a music-specific pseudo-label filter not seen in the literature found.
+
+### Gaps
+- No AMT-specific TTA papers or numbers were found.
+
+## 8. Analysis-by-synthesis / render-and-compare
+
+### Takeaway
+Render-and-compare has working 2023–25 examples:
+- the Drexel semi-supervised tab work: predicted tab → differentiable guitar synth → reconstruction loss on 27 h of unlabeled YouTube guitar
+- the Inverse Drum Machine for drums
+- DDSP mixture-model parameter fitting
+
+None reports a large accuracy jump on real mixes. It is most useful as a training signal on unlabeled data and as a verification/uncertainty signal.
+
+### Cited Findings
+- Drexel (Cwitkowitz et al. line) "Leveraging unlabeled data to improve guitar tablature transcription": predicted tabs drive a differentiable guitar synthesiser, training minimises reconstruction error, and it introduces 27+ h of unlabeled YouTube solo guitar. This is a thesis, and the snippet was partly garbled — [Drexel](https://researchdiscovery.drexel.edu/view/pdfCoverPage?instCode=01DRXU_INST&filePid=13549920770004721&download=true)
+- Differentiable acoustic guitar model for string-specific polyphonic synthesis (IEEE ICASSP 2023) — [IEEE](https://ieeexplore.ieee.org/document/10248159/); [arXiv 2309.07658](https://arxiv.org/pdf/2309.07658)
+- DDSP mixture model fits pretrained source synths to an observed mixture by gradient descent (2022, older) — [arXiv 2202.00200](https://arxiv.org/pdf/2202.00200)
+- Inverse Drum Machine (2025): see section 4 — [arXiv 2505.03337](https://arxiv.org/abs/2505.03337)
+- Cross-instrument velocity via differentiable SoundFont proxies (2026) — [arXiv 2608.08985](https://arxiv.org/pdf/2608.08985)
+
+### Inferences
+- A practical non-differentiable version for the product: resynthesise each transcribed bar (sampled or DDSP guitar/bass, drum one-shots) and compute a spectral/chroma/onset distance to the stem per bar. Bars with high residual are flagged for review or re-decoded, for example by sampling more Noise2Fret candidates and keeping the best match. No gradient training is needed, and it doubles as a confidence score.
+
+### Gaps
+- No quantitative gain figures were retrieved for render-and-compare refinement on real recordings.
+
+## 9. Confidence / uncertainty estimation for flagging dubious notes
+
+### Takeaway
+I found **almost no dedicated work on calibrated note-level confidence in AMT**. Older work notes miscalibration: models are overconfident on sustain and underconfident on off frames. A 2026 MIR paper (TCP_α) offers post-hoc confidence heads for frozen MIR classifiers, framed as failure prediction under domain shift. That framing fits a "flag for human review" feature directly.
+
+### Cited Findings
+- **TCP_α** (arXiv 2608.20326): a margin-controlled confidence target for post-hoc confidence heads (ConfidNet-style) on frozen MIR classifiers, with theoretical guarantees, framed as failure prediction and examined under domain shift — [arXiv](https://arxiv.org/html/2608.20326v1)
+- Autoregressive multi-state piano model (2020, older): overconfident on sustain, underconfident on off frames. The mismatch hurt beam search, which did worse than greedy — [arXiv 2010.01104](https://arxiv.org/pdf/2010.01104)
+- Audio captioning (arXiv 2512.10170): a learned confidence head plus confidence-aware beam reranking improved quality and calibration (lower ECE and Brier score) — [arXiv](https://arxiv.org/pdf/2512.10170)
+
+### Inferences
+Candidate signals for a "dubious note" score, combined into a per-note confidence, then calibrated (temperature scaling or isotonic regression) on our GP-reference benchmark:
+1. onset/frame posterior from the transcriber
+2. disagreement between diffusion samples (Noise2Fret) or between models (Basic Pitch vs. YourMT3+ vs. fine-tuned SSL model)
+3. render-and-compare residual (section 8)
+4. separation leakage estimate (energy of the same pitch in other stems)
+5. a TCP_α-style post-hoc head trained to predict errors on our reference set
+
+### Gaps
+- No AMT calibration benchmarks (ECE for notes) were found. No user studies on review UIs driven by transcription confidence.
+
+## Recommended experiment shortlist (ranked by expected value per effort; inference)
+
+1. **Noise2Fret on separated guitar stems** vs. current Basic Pitch + Viterbi, on the GP benchmark. Then fine-tune on amp-rendered DadaGP/GOAT passed through our separator. Code exists; verify the license.
+2. **Weakly supervised real-data fine-tuning** (CountEM histograms or NoteEM alignment) using paired recordings and GP tabs, run on separated stems, for guitar and bass.
+3. **SSL encoder (MusicFM/MERT) with a light transcription head** for guitar and bass, following the MIROS precedent. Check encoder licenses.
+4. **Separator-in-the-loop synthetic data** (render → NAM/IR randomisation → mix → separate → train), plus a domain-confusion loss.
+5. **Note-level timbre clustering** (arXiv 2509.12712) to split multiple guitars.
+6. **Render-and-compare and multi-model disagreement** for per-note confidence and review flags. Calibrate on the benchmark.
+7. Skip LALM-based transcription for now. Use LLMs only for symbolic post-processing, with validation.

research_notes/Transcription accuracy improvement ideas/refinement_and_evaluation.md

@@ -0,0 +1,259 @@
+# Post-processing, ensembling, refinement, human-in-the-loop and evaluation methods for transcription accuracy and notation quality (as of Oct 2026)
+
+Scope note: this builds on `/home/user/STSheets/reports/Music transcription tooling survey.md` and does not repeat it. That survey already covers the baseline chain, Beat This!, BTC chords, PM2S, alphaTab, Tab F1/TDR, the Hungarian track matching, and the "mix vs stem" ablation idea. Research budget was about 15 web calls, so several questions end in explicit gaps. Items marked **[older]** are from before 2023. Items marked **[unverified this session]** come from background knowledge and were not re-checked against a fetched page in this session; verify them before relying on them.
+
+## 1. Separation ensembles and tricks; does better SDR mean better transcription?
+
+### Takeaway
+Ensembling and overlap tricks add very little SDR: tenths of a dB at most, and about 0.02 dB for high overlap on a modern RoFormer. No 2023–2026 paper was found that measures note-F1 for guitar, bass or keys transcribed from separated stems against the mix. The nearest evidence (vocals → Whisper lyrics) is mixed: better separation helped on one dataset and made no difference on another. SDR also correlates only weakly with perceived quality, and artifact-focused metrics (SI-SAR) predict perception better for bass and drums. The project must measure stem-vs-mix itself rather than assume that SDR gains turn into note gains.
+
+### Cited Findings
+- **Ensemble methods in the MSST / MVSEP ecosystem.** The community `ensemble.py` defines `avg_wave`, `median_wave`, `min_wave`, `max_wave` and FFT-domain variants (`avg_fft`, `min_fft`, `max_fft`, …). `avg_wave` is a weighted average: each prediction is multiplied by its weight and the sum is divided by the total weight. Newer commits choose min/max by absolute value. — [HF ensemble.py (MSST copy)](https://huggingface.co/spaces/TGPro1/YingMusic-SingerGPU/blob/main/src/third_party/MusicSourceSeparationTraining/ensemble.py); [SESA commit](https://huggingface.co/spaces/NeoPy/SESA_Fast_Separation/commit/dc08c30578ba6a4589e070d5ade1b7a39010c2d5)
+- **MVSEP API** offers an option to "include intermediate results and max_fft, min_fft". In community practice, max_fft is used for a "fuller" stem with more bleed and min_fft for a "cleaner" stem with more loss. — [MVSEP full API](https://hk.mvsep.com/full_api)
+- **Ensemble gains are small.** MVSEP's Ensemble (2025.06) is listed at vocals SDR 11.93 and instrumental 18.23 on its Multisong set. The API's default ensemble reports an "SDR average: 13.67 (v. 2025.06.30)". — [MVSEP algorithms](https://de.mvsep.com/algorithms); [MVSEP full API](https://hk.mvsep.com/full_api)
+- **Overlap barely matters.** BS PolarFormer vocals go from 11.75 SDR at overlap 2 to 11.77 at overlap 8. MVSEP itself says high overlap "significantly increases system load while providing only a minor quality improvement". The API offers 50% (default) and 87.5% overlap. — [MVSEP algorithm page 128](https://hk.mvsep.com/algorithms/128)
+- **Demucs "shift trick".** This test-time augmentation averages predictions over random time shifts of the input (`--shifts N`). The Demucs authors describe it as giving roughly +0.2 dB SDR at N× the compute. **[older, 2019; unverified this session]** — [Demucs paper arXiv 1911.13254](https://arxiv.org/pdf/1911.13254)
+- **SDR vs perception.** The 2025 "Musical Source Separation Bake-Off" listening study found that SDR-family metrics track human judgments poorly and that no single metric works for every stem type. SI-SAR (an artifacts measure) better predicts listener ratings for drums and bass. — [arXiv 2507.06917](https://arxiv.org/abs/2507.06917)
+- **Separation → downstream transcription (lyrics, the closest evidence found).** For Whisper lyrics transcription, WER "consistently decreases as separation quality improves" on MUSDB-ALT. On Jam-ALT, WER stays essentially the same for original and separated audio. — [arXiv 2506.15514](https://arxiv.org/pdf/2506.15514)
+- **Some systems skip separation.** A 2024 model transcribes vocals directly from the mix, with no separation front-end, and beats MT3 and PerceiverTF on Slakh and URMP. — [arXiv 2407.04822](https://arxiv.org/pdf/2407.04822)
+- **Joint separation and transcription models** (Jointist, which conditions separation on predicted piano rolls; Cerberus) exist as an alternative to running the two stages in sequence. **[older]** — [Jointist arXiv 2206.10805](https://arxiv.org/pdf/2206.10805); [MT3 arXiv 2111.03017](https://arxiv.org/pdf/2111.03017)
+- **Guitar duets.** GuitarDuets (2025) offers a classical guitar duet dataset with note-level annotations of synthesized duets, plus a joint transcription and separation framework. It is relevant to the multi-guitar problem. — [arXiv 2507.01172](https://www.arxiv.org/pdf/2507.01172)
+- **Analysis-by-synthesis for drums.** The Inverse Drum Machine (2025) does joint drum transcription and separation via analysis-by-synthesis. It reports both transcription and separation metrics per class. — [arXiv 2505.03337](https://arxiv.org/pdf/2505.03337)
+- **User-guided generative separation** (2025) is a human-in-the-loop separation direction. — [arXiv 2507.01339](https://arxiv.org/abs/2507.01339)
+
+### Inferences
+- An ensemble or high-overlap pass costs 2–4× the compute for at most a few tenths of a dB. That is not worth it for the browser tier. In the server tier it only makes sense for stems where the transcriber is artifact-sensitive (keys and guitar inside "other").
+- For transcription, artifacts (the SAR side) probably matter more than interference (the SIR side). Spurious harmonics and bleed show up as ghost notes, while slight interference is often filtered by note thresholds. That would argue for `min_fft`-style "clean" ensembles before transcription and `max_fft` "full" stems for listening. This is a hypothesis to test, not a finding.
+- **Multi-pass "residual" separation** has community precedent but no measured transcription benefit was found. In this approach, vocals are removed first (best vocal model), then a guitar or piano model runs on the instrumental. Treat it as an A/B in the benchmark.
+
+### Gaps
+- No 2023–2026 paper was found that reports note-F1 for guitar, bass or piano transcribed from separated stems vs the mix. The 2024 "Source Separation & Automatic Transcription" paper (arXiv 2412.06703) is a pipeline description, with no stem-vs-mix numbers in its abstract.
+- No published numbers were found for polarity or channel-flip TTA in RoFormer models, or for the SDR gain of residual multi-pass separation.
+- MVSEP forum discussions of ensemble recipes were not fetched.
+
+## 2. Transcription ensembles (voting / ROVER-style / confidence-weighted / mix-vs-stem cross-check)
+
+### Takeaway
+No published note-level multi-model voting or ROVER-style fusion for music transcription was found in 2023–2026. Fusion in the literature happens inside a single network (multi-feature or multi-spectrogram), or is human-machine (the ENSEMBLE crowd system, 2016). This is an open, low-risk engineering opportunity: align the note lists from several transcribers and vote.
+
+### Cited Findings
+- **Fusion inside one network.** A 2025 piano model fuses Mel-cyclic and STFT spectrogram features in one network, which is not cross-model voting. — [MDPI Electronics 14(23):4720](https://www.mdpi.com/2079-9292/14/23/4720)
+- **Decision-level voting exists for instrument recognition, not notes.** A Swin-T voting ensemble of three transformers reaches micro/macro F1 0.66/0.62. **[older, 2022]** — [EURASIP 2022](https://link.springer.com/article/10.1186/s13636-022-00245-8)
+- **"Ensemble" (ISMIR 2016)** is a hybrid human-machine melody transcription system. It combines algorithm output with crowd edits in a web note editor and logged 53,411 user actions, about 60 per 3-second clip. **[older]** — [ISMIR 2016](https://archives.ismir.net/ismir2016/paper/000072.pdf)
+- **Note-level output in most AMT models** still comes from threshold and peak-picking over frame, onset and offset activations. That makes probability-level averaging across models feasible whenever activations are exposed. — [TASLP 2024 audio-visual piano AMT](https://dl.acm.org/doi/pdf/10.1109/TASLP.2024.3426303)
+- **ROVER background.** NIST ROVER (Fiscus 1997) aligns several ASR hypotheses into a word transition network and votes by frequency and/or confidence. **[older; unverified this session]** — [NIST ROVER paper](https://ieeexplore.ieee.org/document/659110)
+
+### Inferences
+- **A practical "note-ROVER":**
+  1. Run 2–3 transcribers, for example Basic Pitch on the stem, YourMT3+ or MuScriptor on the mix (NC licence, so reference only), and a specialist such as Transkun on the piano stem.
+  2. Match notes across outputs with mir_eval-style bipartite matching (±50 ms onset, same pitch).
+  3. Keep notes supported by ≥2 systems, or by 1 system with confidence above a threshold.
+  4. Feed the agreement count into the UI as a confidence value.
+- **Mix-vs-stem cross-check as a filter.** A note found on the guitar stem but absent from a mix-level multi-instrument transcription, while present in another stem at the same pitch and time, is likely bleed. Assign it to the stem with higher energy at that pitch.
+- **Multi-resolution:** run the same model at different hop sizes or on time-stretched input (TTA) and average activations. This is cheap to test, but no published numbers were found.
+
+### Gaps
+- No paper measuring the F1 gain from combining Basic Pitch, MT3-family and specialist models was found. The gain has to be measured in-house.
+
+## 3. Musically-informed post-processing (key/harmony, octave, overtone ghosts, durations, structure)
+
+### Takeaway
+Most of the literature here is older (2007–2021). It shows that priors help: music language models with error models for melody, repetition-aware LMs for drums, and self-similarity for meter and bar errors. Recent (2023–26) work is sparse, so these heuristics are mostly in-house engineering. Structure-aware consolidation of repeated sections has precedent (chorus pooling for lyrics, non-local statistics for piano) but no off-the-shelf tool.
+
+### Cited Findings
+- **Statistical melody correction (ICASSP 2021).** Notes produced by a singing transcriber are corrected by combining a music LM with an explicit transcription-error model (insertions, deletions, substitutions). Viterbi decoding with an LSTM LM picks the most plausible output. **[older]** — [Hiramatsu et al. ICASSP 2021](https://eita-nakamura.github.io/articles/Hiramatsu_etal_TranscriptionCorrection_ICASSP2021.pdf); [IEEE](https://ieeexplore.ieee.org/document/9414249/)
+- **Non-local musical statistics (piano, 2020).** Self-similarity of pitch and rhythm between score segments, and how well a score repeats at different bar lengths, are used to fix tempo, meter and bar-line errors. The authors name these as the most common failure modes of audio-to-score transcription. **[older]** — [arXiv 2008.12710](https://arxiv.org/pdf/2008.12710)
+- **Chorus pooling for lyrics.** Pooling repeated chorus sections at feature or hypothesis level gives more accurate and more consistent transcription than transcribing each chorus alone. **[older, ICASSP 2014]** — [McVicar, Ellis, Goto](https://staff.aist.go.jp/m.goto/PAPER/ICASSP2014mcvicar.pdf)
+- **Transposition-invariant self-similarity** finds repeated sections in both symbolic and audio music. **[older, 2018]** — [arXiv 1806.08236](https://arxiv.org/abs/1806.08236v2)
+- **Repetition priors for drums** are covered in section 7.
+- **Score-informed velocity refinement (2025).** A lightweight Transformer correction module refines AMT velocity estimates, reaching SOTA velocity with HPT. It needs a score as side information. — [arXiv 2508.07757](https://arxiv.org/abs/2508.07757)
+- **Rule-based MusicXML error detection (2025).** "Detecting Notational Errors in Digital Music Scores" localizes rhythm and notation errors in MusicXML. It is usable as a post-export validator. — [arXiv 2510.02746](https://arxiv.org/pdf/2510.02746)
+- **Gains from MLMs are historically small.** The MIDI Degradation Toolkit authors note that music LMs have so far given only small gains in AMT systems. **[older, 2020]** — [arXiv 2010.00059](https://arxiv.org/pdf/2010.00059)
+
+### Inferences
+These are concrete, cheap heuristics to implement and A/B test in-house. None comes from a single cited benchmark.
+- **Overtone ghosts:** delete a note at pitch p+12, p+19 or p+24 when a simultaneous note at p is much stronger, the candidate is short or quiet, and it sits on a mono instrument (bass, lead vocal).
+- **Octave correction for bass and vocal:** enforce instrument range (for example, 4-string E1–G4 when the user declares the tuning). Smooth octave jumps with a Viterbi over an octave-shift penalty.
+- **Out-of-key filter:** using BTC chords and a key estimate, down-weight short, low-confidence non-chord, non-scale notes. Never hard-delete them, because chromatic passing tones are real; flag them in the UI instead.
+- **Duration cleanup:** minimum note length (for example, a 64th note at the local tempo, or about 40–60 ms). Merge same-pitch fragments separated by short gaps (legato repair). Trim offsets to the next onset on mono instruments.
+- **Repetition consolidation:**
+  1. Use allin1 sections or a beat-synchronous SSM to find repeated bars.
+  2. Align symbolic content across repeats and do a majority vote per note slot.
+  3. Optionally emit repeat signs or "x2" to improve readability.
+
+  This is the most promising underused idea. It mirrors chorus pooling, and its value for readability is high.
+
+### Gaps
+- No 2023–2026 paper was found that quantifies key-aware filtering, ghost-note removal or minimum-duration rules on modern transcribers (Basic Pitch, YourMT3).
+
+## 4. Symbolic music LMs as correctors or priors (AMT, MusicLang, NotaGen, CLaMP, etc.)
+
+### Takeaway
+No 2023–2026 work was found that uses Anticipatory Music Transformer, MusicLang, NotaGen or ChatMusician to correct transcription errors. The infilling models (AMT, Composer's Assistant 2, MIDI-RWKV) are technically suited to "mask low-confidence region → infill conditioned on surrounding notes". Two pieces of evidence exist: BERT-APC uses a symbolic music LM for vocal pitch correction, and CLaMP 3 embeddings can work as a quality metric.
+
+### Cited Findings
+- **Anticipatory Music Transformer (TMLR 2024).** Trained on Lakh MIDI, it infills a sequence given fixed control events and can anticipate controls up to 5 s ahead. Transcription correction is not addressed. — [arXiv 2306.08620](https://arxiv.org/abs/2306.08620)
+- **Composer's Assistant 2 (2024)** does interactive multi-track MIDI infilling with fine-grained user control. — [arXiv 2407.14700](https://arxiv.org/pdf/2407.14700)
+- **MIDI-RWKV (2025)** is an adaptable symbolic infilling model. — [arXiv 2506.13001](https://arxiv.org/pdf/2506.13001)
+- **BERT-APC (2025)** is a reference-free automatic pitch correction method that infers the intended pitch of detuned vocals from musical context with a symbolic music LM. It is the closest recent example of an LM-as-corrector. — [arXiv 2511.20006](https://arxiv.org/pdf/2511.20006)
+- **MIDI Degradation Toolkit** synthesizes errors (pitch shifts, onset shifts, inserted and removed notes) in clean MIDI so that models can detect, classify, locate and correct them. The authors propose this as AMT post-processing. **[older, ISMIR 2020]** — [arXiv 2010.00059](https://arxiv.org/pdf/2010.00059)
+- **Ycart's thesis** gives a taxonomy: transduction (which needs paired data) vs MLM-decoding (scoring candidate outputs with a separately trained symbolic model). **[older]** — [Ycart thesis](https://theses.eurasip.org/media/theses/documents/ycart-adrien-music-language-models-for-automatic-music-transcription.pdf)
+- **CLaMP 3 as an evaluator.** Its audio→ABC cross-modal similarity correlates with human preference for transcriptions at Spearman ρ=0.879 (see section 9). The same embedding can therefore act as a reranker. — [arXiv 2608.04511](https://arxiv.org/html/2608.04511)
+
+### Inferences
+- A cheap, licence-friendly route is to train a small corrector on synthetic degradations, MDTK-style, applied to DadaGP or GOAT tabs. Pick degradations that match the transcriber's error profile, measured on the benchmark (octave errors, ghost harmonics, split notes). This avoids NC weights.
+- Use LM infilling only on low-confidence spans and show the result as a suggestion in the editor, not as an automatic replacement.
+- Check licences before production use. AMT weights are Apache-2.0 per the Stanford CRFM release **[unverified this session]**. MusicLang, NotaGen and ChatMusician licences were not checked.
+
+### Gaps
+- No benchmark of symbolic-LM correction on real AMT output from 2023–26 was found. No licence check was done for MusicLang, NotaGen or ChatMusician.
+
+## 5. General LLMs for notation cleanup
+
+### Takeaway
+LLM benchmarks for symbolic music (2025–26) test understanding and QA, mainly in ABC. None was found for *correcting* notation (time signature, spelling, beaming). LLMs are also weak judges of octave errors. LLMs are therefore better used as orchestrators that call deterministic tools (music21, a pitch speller, a validator) than as direct MusicXML editors.
+
+### Cited Findings
+- **ABC-Eval (2025)** has 10 sub-tasks and 1,086 samples, including error *detection*. Its samples were hand-picked to be free of syntax errors, so correction is not tested. — [arXiv 2509.23350](https://arxiv.org/pdf/2509.23350)
+- **MSU-Bench (ACL 2026)** has 1,800 QA pairs over real scores. Representing scores as ABC "substantially helps" LLMs compared with other encodings. — [ACL 2026](https://aclanthology.org/2026.acl-long.493.pdf)
+- **HMU-Bench (Oct 2026)** found that LLM-based grading can miss octave errors in symbolic output, and recommends event-level checks. — [arXiv 2610.15773](https://arxiv.org/html/2610.15773v1)
+- **MusICA-MetaBench (2026)** generates questions from MusicXML via music21 in ABC, audio and image forms. — [arXiv 2607.06015](https://arxiv.org/html/2607.06015v1)
+- **SSMR-Bench (2025)** is a synthetic sheet-music reasoning benchmark in which DeepSeek-R1 beats DeepSeek-V3. — [arXiv 2509.04059](https://arxiv.org/pdf/2509.04059)
+- **CoComposer (2025)** uses LLM agents to fix timing and formatting errors in ABC. It is a composition system, so the evidence that agents can repair bar-duration errors is anecdotal. — [arXiv 2509.00132](https://arxiv.org/pdf/2509.00132)
+- **Gemini 3.1 Pro as a judge** of transcription playback correlated with humans at ρ=0.970. The authors still advise against AudioLM judges: they cost about 13× more and depend on model version and prompt. — [arXiv 2608.04511](https://arxiv.org/html/2608.04511)
+
+### Inferences
+- An LLM pass should output *edit operations* (for example "bar 12: change time signature to 6/8", "respell G# as Ab in bars 20–24"). A deterministic validator applies them and re-checks bar durations, using arXiv 2510.02746-style rules. Free-form generation of MusicXML is risky.
+
+### Gaps
+- No benchmark was found for LLM enharmonic spelling, beaming or time-signature inference.
+
+## 6. Rhythm quantization and score typesetting (quantizers, tuplets, meter, spelling, voice/staff)
+
+### Takeaway
+Two newer options beat PM2S on parts of the problem. The beat-conditioned T5 quantizer (Apr 2026, piano plus jazz guitar) takes beats as input, which fits the Beat This! pipeline. MIDI2ScoreTransformer (ISMIR 2024, piano, MusicXML out) gives the best notation among modular pipelines in a 2026 human-anchored study. Pitch spelling is essentially solved for tonal music (ps13 ≈ 99.3–99.4%; PKSpell is better on MuseData). Voice/staff separation has a 2024 GNN. Nearly all of this is piano- or classical-trained.
+
+### Cited Findings
+- **Beat-conditioned T5 quantizer (Wachter, Murgul, Heizmann; arXiv 2604.22290, Apr 2026).**
+  - Input is performance MIDI plus beat and downbeat annotations. Tokens are pitch, onset and note value, plus a new-measure token. It works on two-bar segments with beam width 5.
+  - The beat grid is interpolated to 12 ticks per beat, covering straight values and triplets from 16th-triplets up. It handles 6/8 with 18 ticks.
+  - ASAP: onset F1 97.3%, note-value accuracy 83.3%. Leduc jazz guitar (GP files converted to MIDI and aligned): onset F1 92.1%, NV acc 90.2%.
+  - MUSTER onset/offset error: 12.30/28.30. Baselines: transformer "End-to-End" [18] 15.55/23.84, HMM+heuristics 22.58–25.02 / 29.21–29.84, Finale 31.85/45.34, MuseScore 47.90/49.44.
+  - It assumes a one-to-one note correspondence, so it cannot add or remove notes. It has no voices, no 32nd notes and no irregular meters, and it was not tested with noisy beats.
+  - No code release was found.
+
+  — [arXiv 2604.22290](https://arxiv.org/html/2604.22290v1)
+- **Earlier version (arXiv 2508.19262, 2025)** was trained on piano and guitar performances and claims SOTA on MUSTER. — [arXiv 2508.19262](https://arxiv.org/pdf/2508.19262)
+- **MIDI2ScoreTransformer (Beyer & Dai, ISMIR 2024)** is an end-to-end RoFormer encoder-decoder from performance MIDI to MusicXML tokens (11 score attributes). It implicitly does beat tracking and quantization, and it beats HMM and earlier DL pipelines on MUSTER. Training data is ASAP only (piano). — [arXiv 2410.00210](https://arxiv.org/html/2410.00210v1); [ISMIR 2024 poster](https://ismir2024program.ismir.net/poster_158.html)
+- **MIDI-to-score converter choice drives notation quality.** In a 2026 evaluation of 24 pipelines (8 AMT models × music21 / MuseScore / M2ST):
+  - Transkun+M2ST had the best OMR-NED among modular pipelines (86.61).
+  - The end-to-end audio-to-score system Rubato had the lowest OMR-NED overall (72.30) but ranked 6th on playback.
+  - MuseScore import was best for playback-faithful output.
+
+  — [arXiv 2608.04511](https://arxiv.org/html/2608.04511)
+- **Beat/downbeat tracking from performance MIDI** with an end-to-end transformer (2025). Multi-instrument support is listed as future work. — [arXiv 2507.00466](https://arxiv.org/pdf/2507.00466)
+- **Pitch spelling.**
+  - ps13 spelt 99.31% of a 195,972-note corpus correctly (99.43% for a variant). **[older]** — [ps13, Goldsmiths](https://research.gold.ac.uk/id/eprint/1007)
+  - PKSpell (2021, an RNN for spelling plus key signature) made 256 errors on MuseData, against 343 for CIV and 1,064 for ps13. The caveat is that ps13 and CIV were tuned on test data. PKSpell got 97% of global key signatures right. **[older]** — [arXiv 2107.14009](https://arxiv.org/pdf/2107.14009)
+  - No newer (2024–26) deep pitch-spelling SOTA was found.
+- **Voice and staff separation.** "Cluster and Separate" (2024) is a GNN that assigns notes to voices and staves for engraving, with staff as a per-note binary label and voice as edge prediction. **[code licence not checked]** — [arXiv 2407.21030](https://arxiv.org/html/2407.21030v1)
+- **Classical hand split:** apply voice separation per hand, at most two voices per hand. **[older]** — [arXiv 2008.12710](https://arxiv.org/pdf/2008.12710)
+
+### Inferences
+- For guitar and bass tabs, the T5-quantizer recipe fits the pipeline (Beat This! beats → quantize) and was shown on GP-derived jazz guitar data. Retraining it on DadaGP/GOAT aligned to synth audio is feasible. Its one-to-one constraint means it must run *after* note cleanup.
+- Swing: the 12-tick grid captures triplet swing. A per-section swing detector (the ratio of off-beat 8th positions) can decide whether to notate straight 8ths with a "swing" marking instead of triplets. That is a readability win. No cited benchmark exists.
+- For pickup bars and time signatures, use Beat This! downbeats. When the first downbeat comes after the first onset, emit an anacrusis bar. Validate bar lengths with arXiv 2510.02746-style rules.
+- Spelling: music21's built-in spelling and key analysis are fine for pop and rock. Spell relative to BTC chord roots for non-diatonic passages.
+
+### Gaps
+- No public code or licence was found for the 2604.22290 quantizer. The licences of M2ST and the GNN voice-separation code were not checked. No guitar-specific tuplet or swing detection paper was found.
+
+## 7. Drums post-processing
+
+### Takeaway
+Repetition-aware drum language models are the main published post-processing approach (2007–2021). They regularize tatum-level outputs toward bar-periodic patterns, and the reported improvements are on ENST and RWC. No 2023–26 work on cymbal disambiguation via drum sub-stems was found.
+
+### Cited Findings
+- **Tatum-level CRNN with LM-regularized training (2020).**
+  - A skip bi-gram compares each tatum with the tatum 16 positions earlier (one 4/4 bar of 16ths); a GRU LM captures longer structure.
+  - The pretrained LM regularizes training by penalizing "unnatural" binarized outputs.
+  - **[older]** — [arXiv 2010.03749](https://arxiv.org/pdf/2010.03749)
+- **Global structure-aware drum transcription (2021)** uses self-attention plus a masked-LM regularizer. **[older]** — [arXiv 2105.05791](https://arxiv.org/pdf/2105.05791)
+- **Gillet & Richard (ISMIR 2007)** use N-gram and complexity-criterion post-processing, picking the most regular sequence consistent with acoustic scores. They report gains on ENST-drums. **[older]** — [ISMIR 2007](https://archives.ismir.net/ismir2007/paper/000219.pdf)
+- **Symbolic drum-score engraving (Digard & Jacquemard 2022)** turns MIDI drum input into notated drum scores via parsing and term rewriting, evaluated on Groove MIDI. **[older]** — [WoRMS 2022 proceedings](https://arxiv.org/pdf/2211.13285)
+- **Joint drum transcription and separation (Inverse Drum Machine, 2025).** — [arXiv 2505.03337](https://arxiv.org/pdf/2505.03337)
+
+### Inferences
+- Implement bar-level pattern consolidation:
+  1. Quantize drum onsets to a tatum grid (Beat This!) with a swing ratio.
+  2. Cluster bars by Hamming distance.
+  3. Within each cluster, majority-vote the hits and flag the deviating bars as fills to keep, rather than overwriting them.
+- For cymbal disambiguation, the cheap approach uses a DrumSep/LarsNet sub-stem energy ratio (hi-hat vs ride vs crash) as a tie-breaker feature. The previous survey found no evidence that this helps, so it stays a hypothesis.
+
+### Gaps
+- No modern (2023–26) drum-specific post-processing evaluation was found. Groove/swing quantization specific to drums is unpublished beyond the generic quantizers.
+
+## 8. Human-in-the-loop, confidence highlighting, learning from corrections, editing cost
+
+### Takeaway
+Evidence that editing beats transcribing from scratch exists but is modest and dated. Holzapfel & Benetos (2019) found time and effort savings for some participants, and quality gains mainly for less-skilled users. No 2023–26 "time-to-correct" study for AI music transcription was found. The ASR literature (confidence-based error markup) is the best design analogue.
+
+### Cited Findings
+- **Holzapfel & Benetos (ISMIR 2019).** Editing AMT output generally took less time and lower self-rated effort than manual transcription for some participants. In the follow-up, less-skilled transcribers produced higher-rated transcriptions when editing AMT output, while skilled ones showed no significant difference. **[older]** — [ISMIR 2019](https://archives.ismir.net/ismir2019/paper/000082.pdf); [DSH 2022](https://academic.oup.com/dsh/article/37/3/747/6408955)
+- **OMR-assisted transcription (ISMIR 2021)** measures correction time against time saved compared with manual work, and calls error correction "slow and tedious". **[older]** — [ISMIR 2021](https://archives.ismir.net/ismir2021/paper/000003.pdf)
+- **Tony (TENOR 2015),** a melody annotation tool, found that editing of any kind adds annotation time. **[older]** — [Tony paper](https://www.tenor-conference.org/proceedings/2015/04-Mauch-Tony.pdf)
+- **ENSEMBLE (ISMIR 2016)** is a crowd correction web editor with about 60 actions per 3-second clip. This shows that crowd correction is laborious at note level. **[older]** — [ISMIR 2016](https://archives.ismir.net/ismir2016/paper/000072.pdf)
+- **ASR analogue (2025).** Confidence-based error highlighting in a correction UI, evaluated with keystroke and time logs and UEQ-S. — [arXiv 2503.15124](https://arxiv.org/pdf/2503.15124)
+- **OMR-NED** is argued to be the metric that best reflects human correction effort: insert/delete edit distance over notation symbols. The 2026 dual-evaluation paper counters that it "may not reflect the effort to read or edit" and was not validated against human reading. — [Sheet Music Benchmark arXiv 2506.10488](https://arxiv.org/html/2506.10488v1); [arXiv 2608.04511](https://arxiv.org/html/2608.04511)
+- **User-guided generative separation (2025):** users steer separation interactively. — [arXiv 2507.01339](https://arxiv.org/abs/2507.01339)
+
+### Inferences
+- Log every editor action in alphaTab (note add, delete, move, string change) with timestamps. This gives the in-house "time-to-correct" and "edits per bar" metrics, and builds a correction dataset for the MDTK-style corrector in section 4 and for confidence calibration.
+- Highlight notes by confidence, using ensemble agreement plus model probability plus rule flags (out-of-key, isolated short notes).
+- Offer bulk actions that exploit structure: "apply this fix to all repeats of this chorus" uses the repetition clusters from section 3.
+
+### Gaps
+- No 2023–26 user study of AI-transcription editing time was found. Published systems that learn from user corrections (active learning) in AMT were not found.
+
+## 9. Evaluation metrics that track human judgement; evaluating against non-aligned GP tabs
+
+### Takeaway
+The strongest 2026 evidence (Wang et al., 106 listeners, 3,180 ABX judgments) is that notation-side and playback-side metrics favour *different* systems. Playback-side, CLEWS embeddings (ρ=0.971) and MFCC-TWED (ρ=0.924) track human preference best. Notation-side, MUSTER MeanER and OnsetER (ρ≈0.79–0.84) track humans far better than OMR-NED (ρ≈0.38–0.50). MV2H's non-aligned variant and DTW alignment handle GP references without audio timestamps.
+
+### Cited Findings
+- **"A Dual Evaluation for Music Transcription" (Wang, Yang, Tamer, Ebert, Smith; UW/AI2; arXiv 2608.04511; under review at TISMIR).**
+  - Data: 230 ATEPP piano recordings; 24 pipelines (8 AMT models × 3 MIDI-to-score converters) plus Rubato.
+  - Human study: 106 participants, 3,180 ABX judgments, aggregated with Bradley–Terry.
+  - Spearman ρ / Kendall τ against human preference:
+
+    | Metric | ρ | τ |
+    |---|---|---|
+    | CLEWS | 0.971 | 0.891 |
+    | Gemini 3.1 Pro judge | 0.970 | 0.870 |
+    | TWED-MFCC | 0.924 | 0.790 |
+    | DTW-Chroma CENS | 0.891 | 0.761 |
+    | CLaMP 3 audio | 0.884 | 0.710 |
+    | CLaMP 3 audio→ABC | 0.879 | 0.710 |
+
+  - MUSTER MeanER and OnsetER: ρ=0.79 and 0.84, against 0.38 and 0.50 for OMR-NED.
+  - The structure-oriented error rates (VoiceER, HandER, ScaleErr) track OMR-NED instead.
+  - Code: MIT, [AMT-Dual-Eval](https://github.com/pingw220/AMT-Dual-Eval). Data tables are CC BY 4.0.
+
+  — [arXiv 2608.04511](https://arxiv.org/html/2608.04511)
+
+  (A search summary gave a differing figure, "global cosine ρ=0.824 vs OMR-NED ρ=0.681". The fetched full text above is the authoritative reading; the discrepancy is unresolved.)
+- **OMR-NED (Sheet Music Benchmark, 2025)** builds on musicdiff and is computed over visual notation symbols, normalized insert/delete. — [arXiv 2506.10488](https://arxiv.org/html/2506.10488v1)
+- **MV2H** has multi-pitch, voice, meter, note value and harmony sub-scores. The 2019 extension evaluates *non-aligned* transcriptions via DTW-style alignment, and a later version handles key changes per section. **[older]** — [arXiv 1906.00566](https://arxiv.org/html/1906.00566); [ISMIR 2018](https://ismir2018.ircam.fr/doc/pdfs/148_Paper.pdf)
+- **MUSTER** is used as the comparison metric for quantizers (2604.22290) and M2ST. — [arXiv 2604.22290](https://arxiv.org/html/2604.22290v1)
+- **"ScoreSimilarity" by that name** was not found. The nearest equivalents are OMR-NED/musicdiff and the dual-eval cosine/embedding scores.
+
+### Inferences
+- The benchmark should report at least three families:
+  1. Note- and tab-level F1 (already planned).
+  2. A notation metric. MUSTER-style error rates beat OMR-NED as a human proxy, but OMR-NED/musicdiff approximates edit effort.
+  3. A playback metric: render the GP output with the alphaTab synth and compare it to the original stem with CLEWS or DTW-chroma. This needs no alignment of the reference GP, so it can sanity-check songs where GP-to-audio alignment fails.
+- For GP references with imperfect tempo alignment, compare in *score time*: bar and beat positions after a DTW map. MV2H-non-aligned is an existing tool for exactly this, but check its piano-voice assumptions before applying it to tab. Report alignment failure separately, as the survey already proposes.
+- Make editor logs (section 8) the ultimate metric: "edits per minute of music to reach the reference".
+
+### Gaps
+- Tab-specific human-correlation studies (does Tab F1 track guitarist judgments?) were not found. The MV2H licence and maintenance status were not checked. The dual-eval study is piano-only; its transfer to guitar, bass and drums is unknown.