Views
No views yet
model_vocals_mel_band_roformer_sdr_8.42.ckpt of ZFTurbo/Music-Source-Separation-Training using a custom Mel-Band-RoFormer MLX port (the Blaizzy/mlx-audio PR + the xocialize/mel-roformer-mlx-swift Swift consumer). Refer to the original repository for more details on the model.vocals_v1 (v1.0.0 release, training-time SDR 8.42)vocals (single-stem model — derive instrumental as mixture - vocals)n_fft=2048, hop_length=512, win_length=2048dim=192, depth=8, heads=8, dim_head=64hop_length (512 vs. 441) than Kim Vocal 2 / viperx Mel-Band-RoFormer configurations. Confirmed against the published state-dict shapes — the YAML default mask_estimator_depth=2 does not match this checkpoint; the shipped weights were trained with mask_estimator_depth=1.config.json.LICENSE)model_vocals_mel_band_roformer_sdr_8.42.ckptLICENSE.LICENSE file (MIT) was committed on 2024-11-04 (commit 6149a22), approximately one year after the v1.0.0 release (published 2023-11-06) that produced this checkpoint asset. At the moment the v1.0.0 release was published, the repository did not yet carry an explicit LICENSE file.LICENSE file is the codified expression of the author's grant; nothing in the repository indicates a different license intent ever applied to the v1.0.0 release assets specifically.mlx_audio.sts.models.mel_roformer.convert — merged upstream into Blaizzy/mlx-audio via PR #654 (2026-04-27) and shipped in mlx-audio==0.4.3 and later.8380ab8 on the feat/mel-band-roformer branch (xocialize fork) — this is the exact commit that produced model.safetensors. The merged upstream code is functionally equivalent.MelRoFormerConfig.zfturbo_vocals_v1()float16 (see Precision below for the rationale — bf16 was tested first per the upload guide but failed parity at this small model size)model_vocals_mel_band_roformer_sdr_8.42.ckpt SHA-256: d9ce706b49cebf0af018590d8deb47ad5434987bf8f7bd3a87a4e5e8c30acb26mlx-community/mel-roformer-kim-vocal-2-mlx ships bf16. The reason is empirical: bf16 quantization degrades this checkpoint's parity vs the PyTorch reference below the upload acceptance threshold, while Kim's wider architecture absorbs bf16 truncation cleanly. Three-way comparison on the same parity harness, same input chunk:| Precision | File size | SDR vs PyTorch reference | Verdict |
|---|---|---|---|
| fp32 | 128.5 MB | 74.86 dB | Bit-exact-equivalent (baseline) |
| fp16 | 64.3 MB | 44.19 dB | Published — > 40 dB target |
| bf16 | 64.3 MB | 21.96 dB | Below target — not published |
dim=192 / 33.7M-parameter architecture is more sensitive to bf16's narrow 7-bit mantissa than wider Mel-Band-RoFormer presets. fp16 retains more mantissa bits (10) at the same file size and recovers parity. fp32 was rejected as the published precision only on size grounds; if a downstream user prefers fp32 fidelity over the 64 MB savings, the upstream PyTorch checkpoint is freely available from the v1.0.0 release and can be re-converted via mlx_audio.sts.models.mel_roformer.convert with --dtype float32.bs_roformer==0.3.10 — bs_roformer.MelBandRoformer instantiated from the v1.0.0 training YAML with mask_estimator_depth=1 (the YAML default 2 does not match the shipped weights — confirmed against state-dict shapes). Newer bs_roformer releases (0.4+) reorder the layers ModuleList and add nGPT-style normalization, breaking checkpoint compatibility.mlx_audio/tests/sts/test_mel_roformer_parity.py and tests/sts/torch_infer.py in the xocialize/mlx-audio fork.mixture - vocals if needed.model.safetensors — MLX weightsconfig.json — architecture hyperparametersLICENSE — MIT license textThe Mel-Band-RoFormer architecture is included inmlx-audio>=0.4.3(merged via PR #654 on 2026-04-27). Install withpip install "mlx-audio>=0.4.3".
1import soundfile as sf
2import numpy as np
3import mlx.core as mx
4
5from mlx_audio.sts.models.mel_roformer import MelRoFormer, MelRoFormerConfig
6from mlx_audio.utils import load_audio
7
8# 1. Load model + weights from the Hub.
9model = MelRoFormer.from_pretrained(
10 "mlx-community/mel-roformer-zfturbo-vocals-v1-mlx",
11 config=MelRoFormerConfig.zfturbo_vocals_v1(), # optional if config.json is present
12)
13model.eval()
14
15# 2. Load the input mixture as 44.1 kHz stereo and add a batch axis.
16mixture = load_audio("input_mixture.wav", sample_rate=44100) # mx.array [2, samples]
17batched = mixture[None, ...] # [1, 2, samples]
18
19# 3. Separate vocals.
20vocals = model(batched)[0] # [2, samples]
21
22# 4. Derive instrumental as (mixture - vocals).
23instrumental = mixture - vocals
24
25# 5. Write stems to disk (soundfile expects [samples, channels]).
26sf.write("vocals.wav", np.array(vocals).T, 44100)
27sf.write("instrumental.wav", np.array(instrumental).T, 44100)chunk_size = 352800 samples with 50% overlap and overlap-add the outputs — see the model code for the canonical helper once it's added.mel-roformer-mlx-swift)1import SwiftRoFormer
2
3// One-shot Hub download + weight load. The bundled config.json's
4// `checkpoint_family` field auto-resolves the correct preset.
5let model = try await MelRoFormer.fromPretrained(
6 "mlx-community/mel-roformer-zfturbo-vocals-v1-mlx"
7)
8
9// Forward pass: input is (batch, channels, samples) at 44100 Hz.
10let vocals = model(input)WeightLoader.loadWeights:1let model = MelRoFormer(config: .zfturboVocalsV1)
2try WeightLoader.loadWeights(into: model, from: localWeightsURL)1@misc{lu2023melband,
2 title = {Mel-Band {RoFormer} for Music Source Separation},
3 author = {Lu, Wei-Tsung and Wang, Ju-Chiang and Won, Minz and Choi, Keunwoo and Song, Xuchen},
4 year = {2023},
5 eprint = {2310.01809},
6 archivePrefix = {arXiv},
7 primaryClass = {eess.AS},
8 url = {https://arxiv.org/abs/2310.01809}
9}
10
11@misc{zfturbo_mss_training,
12 title = {Music Source Separation Training — Mel-Band-RoFormer vocals\_v1 (SDR 8.42)},
13 author = {Solovyev, Roman (ZFTurbo)},
14 year = {2023--2026},
15 url = {https://github.com/ZFTurbo/Music-Source-Separation-Training}
16}lucidrains/BS-RoFormer (MIT) → Blaizzy/mlx-audio (Apache-2.0).