lucasw89/STSheets
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
6bd8ab9cee9f65ffc2fec6a437cb36a8a23d1ffd
STSheets/research_notes/Music transcription tooling survey/stem_separation.md
T

28 KiB
Raw Blame History

Open-source music source (stem) separation for downstream transcription (state as of Oct 2026)

Research date: 2026-10-30. About 20 tool calls. MVSEP SDR figures are MVSEP's own "Multisong" or validation-set numbers. They are not MUSDB18-HQ test numbers, so they can't be compared directly with paper numbers. Old information is flagged.

1. Demucs / HTDemucs: status after the Meta archive, forks, and quality of htdemucs_6s guitar/piano

Takeaway

facebookresearch/demucs was archived on 2025-01-01. The original author (Défossez, now at Kyutai) keeps a low-activity fork at adefossez/demucs that only takes important bug fixes. Demucs is now well behind the RoFormer family. htdemucs_6s guitar and piano stems are weak: the piano stem is close to unusable (MVSEP piano SDR 2.23). Demucs remains the most widely ported model, with ONNX, C++, WASM, Rust and WebGPU ports.

Cited Findings

  • The facebookresearch/demucs repo was archived 2025-01-01 after the author left Meta. It is "not actively maintained anymore and only important bug fixes will be processed on the new repo" (adefossez/demucs). — facebookresearch/demucs; DeepWiki
  • The author reportedly calls adefossez/demucs "the officially maintained Demucs now that I have left Meta to join Kyutai" and says to "expect slow replies and no new feature for now". A low-quality 2026 blog claims the fork shipped v4.1.0 (unverified). — aireiter blog (secondary)
  • Another fork exists: Ryan5453/demucs, which is also packaged on Replicate. — Ryan5453/demucs; Replicate
  • The official README says htdemucs_6s adds piano and guitar, and that "the piano source is not working great at the moment". The early test found guitar quality OK but "a lot of bleeding and artifacts" on piano. — Demucs README (jsdelivr mirror)
  • MVSEP Multisong SDR (bass / drums / other / vocals / instrumental):
    • htdemucs_ft: 12.05 / 11.24 / 5.74 / 8.33 / 14.63
    • htdemucs: 11.74 / 10.90 / 5.57 / 8.18 / 14.49
    • htdemucs_6s: 11.42 / 10.59 / 2.63 / 8.17 / 14.48
    • On MVSEP's piano validation set, htdemucs_6s piano scores 2.23 SDR, against 7.83 for BS-RoFormer SW. MVSEP describes the 6s guitar and piano quality as "still so-so". — MVSEP algorithms; MVSEP algorithm 17
  • ZFTurbo's pretrained list gives htdemucs_6s Multisong SDR of bass 11.22, drums 10.22 and vocals 8.05. Guitar and piano SDR are not reported. — MSST pretrained_models.md
  • Model sizes per demucs-rs: htdemucs 84 MB, htdemucs_6s 84 MB, htdemucs_ft 333 MB. htdemucs_ft is a bag of four fine-tuned models, so it is about 4x slower. — demucs-rs; search summary of StemSplit/aireiter
  • Demucs code and weights are MIT. Browser ONNX exports state "MIT, matching the upstream demucs project". — monteslu/htdemucs-web-onnx

Inferences

  • For transcription, Demucs makes sense only as a fallback or where portability matters (browser, C++). It should not be the main separator.
  • A plausible split is htdemucs_ft for bass and drums (reasonable at about 11-12 SDR) and BS-RoFormer SW for guitar and piano.
  • Not verified this session (from memory): the HTDemucs paper (arXiv 2211.08553) reports about 9.0 dB average SDR on the MUSDB18-HQ test set, with about 9.2 dB for the fine-tuned / sparse variants.

Gaps

  • I did not verify adefossez/demucs commit activity for 2026 or confirm the v4.1.0 release on its releases page.
  • I found no published guitar SDR for htdemucs_6s from either MVSEP or ZFTurbo.

2. RoFormer family, SCNet, MDX23C and newer models: which give guitar, piano and bass stems, and how good they are

Takeaway

The best open-weight multi-stem model as of 2025-26 is BS-RoFormer SW, a 6-stem model (vocals, bass, drums, guitar, piano, other) by viperx/lucidrains, distributed by the community. Its MVSEP SDR:

Stem SDR
Vocals 11.30
Bass 14.62
Drums 14.11
Guitar 9.05
Piano 7.83
Other 8.71

Guitar and piano stay 3-7 dB behind bass, drums and vocals. For 2-stem vocal separation the leaders are MVSEP's BS-RoFormer 124-band models (12.33 vocal SDR, version 2026.07), but these appear to be MVSEP-hosted only.

Cited Findings

  • BS-RoFormer (ByteDance SAMI, Lu et al. 2023) won the SDX23 Music Separation track "by a large margin". It reaches 9.80 dB average SDR trained on MUSDB18HQ only, and a reported 11.99 dB with 500 extra songs. — arXiv 2309.02612; ar5iv

  • Mel-Band RoFormer (arXiv 2310.01809) outperforms BS-RoFormer on vocals, drums and other: vocals 11.21 vs 10.78 dB. It is worse on bass: 9.64 vs 11.43 dB. — arXiv 2310.01809

  • SCNet (ICASSP 2024, Tsinghua and Kunlun) reaches 9.0 dB average SDR on MUSDB18-HQ without extra data, and SCNet-large reaches 9.69. Its CPU inference time is about 48% of HT Demucs. Official code is at starrytong/SCNet. — arXiv 2401.13276; hyper.ai listing

  • BS-RoFormer SW on MVSEP:

    • It is a "modified version of initial BS Roformer… Modifications were made by lucidrains… 2nd version of weights (with better quality) was prepared by viperx". It became MVSEP's default model for new users.
    • Multisong SDR: vocals 11.30, instrumental 17.50, bass 14.62, drums 14.11, guitar 9.05, piano 7.83, other 8.71.
    • MVSEP's standalone guitar and piano models moved to these numbers: guitar 7.51 → 9.05, piano 6.20 → 7.83. — MVSEP news 64; MVSEP algorithms

  • BS-RoFormer SW weights are distributable:

    • jarredou's HF repo BS-ROFO-SW-Fixed (Aug 16 2025) was "edited to allow its use with UVR and MSST".
    • The jarredou account was reportedly deleted. enerjazzer/BS-ROFO-SW-Fixed is cited as the current host, with a checkpoint of about 700 MB.
    • Community GGUF and ONNX conversions exist: studio409/BSRoformer-SW-6Stem-GGUF, and elicwhite's ONNX export. — HF jarredou commit; studio409 GGUF; openmirlab/bs-roformer-infer

  • MVSEP piano validation SDR:

    Model Piano SDR
    Demucs4HT 6s 2.23
    MDX23C (2023.08) 4.79
    MDX23C (2024.09) 5.59
    MelRoformer (viperx) 5.67
    SCNet Large 5.89
    SCNet + Mel ensemble 6.19
    BS-RoFormer SW 7.83

    — MVSEP algorithms

  • MVSEP bass SDR:

    Model Bass SDR
    BS-RoFormer 12.49
    HTDemucs4 12.52
    SCNet XL 13.81
    BS + HTDemucs + SCNet XL ensemble 14.07
    BS-RoFormer SW 14.62
    SCNet XL + BS-RoFormer SW 14.87

    — MVSEP algorithms

  • MVSEP drums SDR:

    Model Drums SDR
    HTDemucs4 12.04
    MelBand 12.76
    SCNet Large 13.01
    SCNet XL 13.42
    BS-RoFormer SW 14.11
    MelBand + SCNet XL + SW ensemble 14.35

    — MVSEP algorithms

  • MVSEP's 4/5-stem "Ensemble All-In" (v2025.06.30) scores: bass 14.85, drums 14.33, other 9.00, vocals 11.93, instrumental 18.23. This is an MVSEP service, not open weights. — MVSEP algorithms

  • Top 2-stem vocal models on MVSEP Multisong:

    Model Vocals / instrumental SDR
    BS-RoFormer 124 bands (ver. 2026.07) 12.33 / 18.64
    BS PolarFormer 124 bands 12.02 / 18.33
    MelBand "becruily deux" 11.35 / 17.66
    SCNet XL IHF 11.11 / 17.41

    — MVSEP algorithms

  • MVSEP's guitar model has been through several architectures:

    • The original (Sep 2023) used MDX23C.
    • Later versions use MDX23C, Mel-RoFormer and BS-RoFormer, covering acoustic and electric guitar.
    • In 2025 MVSEP added a separate acoustic guitar model and a guitar leaderboard (electric and acoustic). — MVSEP news 28; MVSEP news; MVSEP algorithms

  • MVSEP also lists keys, organ and Rhodes models, but publishes no metrics for them. — MVSEP algorithms

  • A 2026 roundup (secondary) says the Roformer family sits "at the top of community quality leaderboards as of mid-2026". — Futureproof Music School

Inferences

  • Recommended open pipeline for transcription:
    1. Run BS-RoFormer SW for 6 stems.
    2. Optionally replace the vocal stem with a top open vocal model (e.g. KimberleyJensen Mel-RoFormer, 10.98 in MSST).
    3. Optionally ensemble bass and drums with SCNet XL.
  • Guitar stem quality: at about 9 SDR it is good enough for pitch transcription of a dominant guitar. Expect bleed from keys and synths into the guitar and piano stems.
  • Piano stem quality: at about 7.8 SDR, expect more errors, especially on piano that is quiet in the mix or doubled with synth pads.
  • Keys and synth fall into "other": BS-RoFormer SW has no synth or organ stem, so keys other than piano land in "other".

Gaps

  • No independent (non-MVSEP) benchmark of BS-RoFormer SW guitar or piano on a public test set was found.
  • The license of the viperx/SW weights is unclear. HF repos found did not show a license in search results; treat it as unlicensed or redistribution-tolerated community weights. This is a commercial-use risk.
  • No confirmed open-weight 7+ stem music model (e.g. with separate synth, strings or brass) was found. MVSEP has separate single-instrument models (strings, wind, etc.); I could not verify which have downloadable weights.

3. UVR5, python-audio-separator, ZFTurbo MSST and MVSEP: availability and licenses

Takeaway

  • All three open tools are MIT for code, but weights are mixed and often unlicensed community releases.
  • ZFTurbo MSST: the training and inference framework, with the broadest architecture support.
  • python-audio-separator: the most practical pip/CLI wrapper for UVR-style models plus Roformers, with ensembles. Actively maintained.
  • UVR5 GUI: still on v5.6 plus beta Roformer patches; no new official release found for 2025-26.
  • MVSEP: a web service hosting the best models, some of them private.

Cited Findings

  • ZFTurbo/Music-Source-Separation-Training:
    • License: MIT. Installable via pip install msst. 634 commits, 72 open issues.
    • Architectures: mdx23c, htdemucs, segm_models, torchseg, bs_roformer, mel_band_roformer, swin_upernet, bandit, bandit_v2, scnet, apollo, bs_mamba2, conformer, dttnet, bs_conformer, scnet_tran, scnet_masked.
    • It links an ONNX/TensorRT conversion repo. — MSST GitHub
  • MSST pretrained list, selected entries:
    • Vocals (Multisong SDR): BS PolarFormer 11.00; MelBand RoFormer (KimberleyJensen) 10.98; BS-RoFormer (viperx) 10.87; MDX23C 10.17.
    • Bass: HTDemucs4 FT bass 11.96.
    • DrumSep: models are included (see section 5).
    • No license is stated for the weights. Newest visible tags are v1.0.22 (DTTNet), v1.0.20 (PolarFormer) and v1.0.19 (BS Mamba2). — MSST pretrained_models.md
  • nomadkaraoke/python-audio-separator:
    • License: MIT.
    • Architectures: MDX (.onnx), VR (.pth), Demucs (.yaml) and MDXC/RoFormer (.ckpt).
    • Hardware: CUDA 11.8/12.2; Apple MPS and CoreML; CPU; experimental DirectML (RoFormer falls back to CPU, Demucs unsupported).
    • Ensembles: 11 algorithms and 9 presets, including karaoke.
    • In v0.47.0 the RoFormer overlap defaults changed, giving 4-6x more forward passes and slower runs; lowering overlap trades seam quality for speed.
    • The README asks users to credit UVR when using UVR-trained models. — python-audio-separator
  • UVR5 GUI: the latest version cited is v5.6. Roformer support came via the community beta patch "UVR_Patch_4_14_24_18_7_BETA_full_Roformer". One guide says macOS Apple Silicon users may need to run from source for bs_roformer. No 2025-26 official release was found. — kocpc guide (secondary); Bilibili guide
  • Community sentiment (secondary 2026 roundup): the top-voted answer in r/edmproduction's "Favorite Stem Splitter?" was UVR, and by mid-2026 production-subreddit sentiment had "hardened toward MVSEP" for vocal/instrumental splitting. — Futureproof Music School; KVR forum thread
  • MVSEP: a web service with a free tier. Its algorithm list includes several models with no public weights (e.g. "MVSep Team" BS-RoFormer karaoke, the 124-band vocal models, and the "MVSep Lead/Rhythm Guitar (Demo)"). — MVSEP algorithms; MVSEP news

Inferences

  • For an app pipeline, python-audio-separator (MIT, pip) can load BS-RoFormer SW and Mel-RoFormer checkpoints. MSST is the more flexible path if you need SCNet or DrumSep.
  • Weight-license caution: most RoFormer weights (viperx, KimberleyJensen, becruily, gabox, anvuew, jarredou) are community-trained, often on undisclosed or copyrighted data, and usually carry no explicit license. Commercial redistribution is legally unclear. Demucs (MIT) is the cleanest option license-wise.

Gaps

  • I could not access UVR Discord or GitHub issues directly.
  • I could not verify python-audio-separator's latest release number or date; the page showed no releases list.

4. Multi-guitar separation (lead vs rhythm, L/R panned, acoustic vs electric)

Takeaway

No open-weight model separating lead from rhythm guitar was found. The lead/rhythm options are commercial:

  • Moises "Lead & Rhythm Guitar Separation" (Apr 2024, vendor claims only).
  • MVSEP's "Lead/Rhythm Guitar (Demo)".

Acoustic-vs-electric splitting exists from Moises, AudioShake and MVSEP. A practical open approach is to separate a single guitar stem, then split it using stereo position (mid/side or pan) or a further acoustic/electric pass.

Cited Findings

  • Moises (2024-04-24): "Lead & Rhythm Guitar Separation" and "Acoustic & Electric Guitar Separation". Each yields 2 guitar tracks plus vocals, bass, drums and other. It claims "the first-ever model that accurately separates lead and rhythm guitars", with no metrics or limitations given. Premium/Pro only. — Moises blog
  • MVSEP: lists "MVSep Lead/Rhythm Guitar (Demo)" and an "Electric Guitar" demo with no details. It also has an Acoustic Guitar model (2025). — MVSEP news; MVSEP news 64
  • LALAL.AI (Andromeda): its "Guitars" stem captures all guitars, acoustic and electric, in one file, with no lead/rhythm split. — LALAL.AI blog; Magnetic Mag 2026-09
  • AudioShake: released electric guitar and acoustic guitar stems (commercial). — AudioShake

Inferences

  • Practical open workarounds (not benchmarked):
    1. Separate a guitar stem with BS-RoFormer SW.
    2. Split it into L, R, mid (L+R) and side (L−R). Double-tracked rhythm guitars panned hard L/R usually dominate the sides, while a centered lead dominates the mid. Hard-panned parts can also be transcribed per channel.
    3. Do the pan/mid-side split before separation as well (feed L-only and R-only to the separator) to recover panned parts.
    4. For transcription, run polyphonic transcription on the guitar stem, then separate voices symbolically: monophonic high line = lead, chordal = rhythm.
  • These are standard DSP practices inferred from how mixes work. No source in this session benchmarks them.

Gaps

  • No academic paper or open model on lead/rhythm guitar separation was found in this session.
  • No independent evaluation of Moises's lead/rhythm model was found.

5. Drum sub-stem separation (kick, snare, hats, toms, cymbals)

Takeaway

Open drum-sub-stem models are mature:

  • Best quality: MVSEP/MSST DrumSep models on Mel-RoFormer, SCNet XL and MDX23C. Kick reaches about 18-22 SDR, snare 13-17 and toms 13-16. Hi-hat, ride and crash are poor at about 3-9.
  • LarsNet: an academic 5-stem model with a VST/AU plug-in (LARS). It is faster than real time but lower quality.

All of these expect a drum stem as input, so separate drums first.

Cited Findings

  • LarsNet (Politecnico di Milano, Pattern Recognition Letters 2024):

  • MVSEP DrumSep comes in 4, 5 and 6-stem variants; the 6-stem variant has kick, snare, toms, hh, ride and crash. — MVSEP DrumSep

  • MVSEP DrumSep, Multisong SDR:

    Model Kick Snare Toms Cymbals / HH / Ride / Crash
    MelBand Roformer 4-stem 22.22 17.09 15.86 cymbals 11.87
    SCNet XL 4-stem 20.50 14.69 15.92 cymbals 10.08
    MDX23C 6-stem (aufr33/jarredou) 18.32 13.60 13.25 hh 6.71, ride 5.38, crash 7.56
    HDemucs (inagoy) 4-stem 14.13 8.42 5.67 cymbals 5.63

    — MVSEP algorithms

  • MVSEP DrumSep, 150-track validation set:

    Model Kick Snare Toms Cymbals / HH / other
    MelBand 4-stem 18.67 13.55 13.60 cymbals 8.76
    BS-RoFormer (jarredou) 5-stem 18.16 12.80 13.51 4.66 / 6.65
    HDemucs (inagoy) 10.52 6.05 4.68 5.03

    The page scrape had some column ambiguity for the 5- and 6-stem rows. — MVSEP algorithms

  • MSST DrumSep:

    • mdx23c (jarredou) 5-stem: kick 16.66, snare 11.53, toms 12.33, hh 4.04, cymbals 6.36.
    • mdx23c (aufr33 & jarredou) 6-stem: kick 14.54, snare 9.79, toms 10.63, hh 3.19.
    • htdemucs (inagoy): included. — MSST pretrained_models.md

  • Other:

    • Timbrica drum-part separator (kick/snare/toms, ONNX fp16) on HF. — HF Faridzar/timbrica
    • Inference-only mdxnet-infer PyPI package for MDX23C drum separation. — PyPI mdxnet-infer
    • "Inverse Drum Machine" research (Torres, Peeters, Richard 2025), trained without isolated stems. — arXiv 2505.03337

Inferences

  • For drum transcription, kick, snare and toms stems are clean enough to onset-detect reliably.
  • Hi-hat, ride and crash separation is weak, around 3-9 SDR. A 4-stem model with a combined "cymbals" output (about 9-12 SDR) plus a classifier may beat 6-stem splitting for hat/ride/crash. Dedicated drum-transcription models such as ADT, which operate on the full drum stem, may beat separate-then-detect.

Gaps

  • No LarsNet SDR numbers comparable to MVSEP's were captured this session.
  • Licenses for the DrumSep weights (jarredou/aufr33/inagoy) were not confirmed.

6. Lead vs backing vocal separation

Takeaway

"Karaoke" models separate lead from backing+instrumental. The best open-ish weights are Mel/BS-RoFormer karaoke models by becruily, gabox, aufr33/viperx, frazer and anvuew, at about 9.5-10.2 lead SDR. MVSEP's in-house BS-RoFormer (124-band) is best at 10.77 lead and 7.03 back. Backing-vocal stems remain much weaker (about 4.5-7 SDR).

Cited Findings

  • MVSEP karaoke SDR:

    Model Lead SDR Back SDR Other
    UVR HP-KAROKEE-MSB2-3BAND 6.42 — —
    MDX-B Karaoke 7.42 — —
    MDX-B Karaoke, applied to vocals 8.28 4.46 —
    MelBand (aufr33/viperx) 9.45 — —
    MelBand (becruily) 9.61 — —
    MelBand (gabox) 9.67 — —
    Fused gabox + aufr33/viperx 9.85 — —
    SCNet XL IHF (becruily) 9.53 — —
    BS-RoFormer (frazer & becruily) 10.10 — —
    BS-RoFormer (anvuew) 10.22 — —
    BS-RoFormer (MVSep Team) 10.41 6.61 —
    BS-RoFormer 124 bands (MVSep Team) 10.77 7.03 back+instrum 16.10
    UVR-BVE-4B_SN — 0.87 —

    — MVSEP algorithms

  • Extracting vocals first and then applying karaoke changes results. With fused MelBand the lead drops to 9.62 but a back stem of 5.63 becomes available. — MVSEP algorithms

  • python-audio-separator ships karaoke ensemble presets. — python-audio-separator

Inferences

  • Recommended open chain: best vocal model → karaoke Mel/BS-RoFormer on the vocal stem → lead and backing. The backing stem is usable for harmony detection but not clean transcription.

Gaps

  • I did not check whether the MVSep Team karaoke weights are downloadable; they appear to be service-only.

7. In-browser separation (ONNX/WebGPU/WASM)

Takeaway

Only Demucs (htdemucs, htdemucs_6s, htdemucs_ft) has mature browser ports:

  • WebGPU ONNX ports: about 22x real time on a mid-range GPU.
  • demucs-rs: WASM+WebGPU.
  • sevagh's freemusicdemixer: CPU WASM via demucs.cpp. Very slow (tens of minutes per song single-threaded), and its repo was archived Apr 2025.

RoFormer-class models have community ONNX exports (including BS-RoFormer SW 6-stem), but no mature browser app with published speed was found.

Cited Findings

  • monteslu/htdemucs-web-onnx:
    • htdemucs 4-stem ONNX for onnxruntime-web WebGPU, split into 21 chained pieces to keep the UI responsive.
    • About 126 MB in mixed precision (173 MB fp32). Mixed precision is about 2.4x faster than fp32.
    • About 22x real time back-to-back, or about 8x with UI-friendly pacing, on a Radeon RX 7600.
    • Full fp16 badly degrades drums and bass, so convolutions are kept in fp32.
    • License: MIT. — HF monteslu/htdemucs-web-onnx
  • monteslu also has an htdemucs_ft WebGPU build. — HF monteslu/htdemucs-ft-webgpu
  • There are htdemucs_6s WebGPU/ONNX exports with guitar and piano. — HF kramp/htdemucs-6s-webgpu-onnx; HF adowu/htdemucs-6s-onnx
  • nikhilunni/demucs-rs:
    • Rust port supporting htdemucs, htdemucs_6s and htdemucs_ft.
    • Browser build is WASM + WebGPU and is "significantly slower than the native CLI".
    • License: Apache-2.0. — demucs-rs; Korben article
  • sevagh/freemusicdemixer.com:
    • Demucs v4 transliterated to C++/Eigen (demucs.cpp), compiled to WASM with Emscripten.
    • Models: htdemucs (about 81 MB fp16) and htdemucs_6s (about 53 MB).
    • Speed: a 7-min track takes about 41 min single-run or about 9 min with 8 workers. Another track took about 20 min single-run or about 5 min with 8 workers.
    • WASM/JS is MIT, but "AI model weights and other website assets are proprietary".
    • Repo archived Apr 26 2025 as an older snapshot; the live site may differ. — freemusicdemixer.com repo; sevagh GitHub
  • Other ports:
    • demucs-wasm npm, derived from freemusicdemixer code. — demucs-wasm README
    • timcsy/demucs-web and bakkot/demucs-js. One port notes "if your browser does not support WebGPU it will be much, much slower". — demucs-web; demucs-js
  • RoFormer ONNX exports:
    • elicwhite/bs-roformer-sw-6stem-onnx (no model card). — HF elicwhite
    • silverdaw 4-stem BS-RoFormer ONNX. — HF silverdaw
    • schism-audio mini-BS-RoFormer in MLX and CoreML (Apple). — HF schism-audio

Inferences

  • Feasible in browser today: htdemucs or htdemucs_6s on WebGPU (about 10-30 s for a 4-min song on a decent GPU, based on the 22x figure).
  • Fallback: CPU-WASM fallback is impractical (minutes to tens of minutes).
  • BS-RoFormer SW in browser: its about 700 MB checkpoint and attention-heavy compute make it a heavy browser download. It is feasible only with WebGPU and quantization, and it is unproven.

Gaps

  • No published browser speed or quality numbers for RoFormer ONNX models.
  • I did not verify whether the live freemusicdemixer.com has moved to WebGPU or added models (it reportedly added MIDI features; not confirmed this session).

8. Typical runtime, CPU vs GPU, for a 4-minute song

Takeaway

htdemucs takes about 15-30 s on a modern NVIDIA GPU and about 1.5-8 min on CPU. htdemucs_ft is about 4x slower. SCNet runs at about half of HTDemucs CPU time. RoFormer models are heavier, and higher overlap settings multiply their runtime.

Cited Findings

  • MISST measurement: htdemucs on a 4:09 track took 2m22s on CPU and 28s on GPU. — MISST GitHub
  • LinuxLinks measurement: a 6:24 FLAC took 15.6 s on an RTX 3060 Ti and 187.8 s on CPU (about 12x). — LinuxLinks
  • Paper on a Demucs-based tool: about 5 min on CPU vs under 10 s on GPU (RTX 3060: 7-10 s per track). — SBrT paper
  • Vendor estimates (inconsistent):
    • htdemucs on an RTX 3080: 20-45 s; on CPU: 90 s to about 8 min.
    • htdemucs_ft is about 4x slower. — StemSplit blog (vendor)
  • SCNet: CPU inference time is about 48% of HT Demucs. — arXiv 2401.13276
  • RoFormer in python-audio-separator: overlap defaults in v0.47.0 cause 4-6x more forward passes, and torch.compile speeds up warm runs. — python-audio-separator

Inferences

  • Full pipeline estimate on a single consumer GPU: BS-RoFormer SW, plus karaoke, plus DrumSep, likely totals about 1-3 minutes per 4-min song.
  • CPU-only estimate: RoFormer-class models are likely to take 10+ minutes per pass.
  • Both are inferences; I found no direct RoFormer timing source.

Gaps

  • No primary-source RoFormer, BS-RoFormer SW or Mel-RoFormer CPU/GPU timing benchmarks were found.
  • No Hacker News threads were captured this session.
Reference in new issue View Git Blame Copy Permalink