RelFx encodes the audio-effects transformation between two audio clips. Given
a reference clip and a processed clip, it produces a normalized
128-dimensional projected representation that describes how their effects
differ, even when the clips contain different musical content. The underlying
2048-dimensional fusion representation is also available for analysis.
This model accompanies the ISMIR 2026 paper "Beyond Dry References: Learning
Relative Audio Effects Representations via Contrastive Distance Learning."
The research source code is available at
TMEGalaxyAudioEffect/relfx.
This release contains the MoisesDB-only variant reported in the paper's
data-scale comparison. It is the Base Diff-Gate model, not the full-data or
bidirectional checkpoint.
The checkpoint is publicly downloadable. Use, sharing, and adaptation remain
subject to the terms in LICENSE.
Final projected representation: 128 dimensions, L2-normalized
Fusion representation: 2048 dimensions before projection
The release artifact contains 99 model tensors and JSON-encoded metadata in
the Safetensors header. Optimizer and training-resume state are intentionally
excluded.
Download
Install the Hugging Face CLI:
pip install -U huggingface_hub
Download the checkpoint directly; no account or authentication is required:
The two input files do not need to contain the same musical performance. RelFx
resamples them to 44.1 kHz, converts them to stereo, and uses the first ten
seconds.
The demo saves the final normalized 128-dimensional representation used by the
paper. To export the raw 2048-dimensional fusion representation instead,
invoke scripts/embed.py with --representation fusion.
Representation outputs
model(...)["embedding"], model(...)["z"], and
model.get_embedding(...) return the final normalized 128-dimensional
representation used for contrastive learning, retrieval, and ITO.
model(...)["fusion"] returns the 2048-dimensional e_fx before projection.
This intermediate feeds the auxiliary parameter-regression head. Use
model.get_fusion_representation(..., normalized=True) when an L2-normalized
copy is needed directly.
Base and bidirectional variants
This checkpoint is the Base model from Eq. (4). Swapping its ordered inputs is
not guaranteed to negate either representation. A manually negated Base output
is not the paper's bidirectional model.
The source repository also implements the separately trained bidirectional
architecture from Eqs. (6)-(7), including its symmetric gate, difference-only
fusion, Tanh projection, and input-order swap training. No bidirectional
checkpoint is distributed here. In that variant, fusion is strictly
antisymmetric; the final normalized projection is only empirically
approximately antisymmetric because its Linear layers contain learned biases.
Intended use
The model is intended for non-commercial research involving:
audio-effects representation and retrieval;
comparison of relative processing styles;
analysis of effect transformations; and
differentiable inference-time parameter matching.
Out-of-scope use
Commercial products or services.
Claims that the model recovers unique physical processor parameters.
Source-wise effect disentanglement from a mixed recording.
Processing audio without the necessary rights or consent.
Redistribution or adaptation that does not comply with CC BY-NC-SA 4.0.
Training data
This released variant was trained on 2,585 stems from 240 MoisesDB songs
(156.4 hours). It does not use the licensed internal collection described in
the paper.
Training audio and dataset metadata are not distributed.
Training and evaluation effects
The training chain contains EQ, distortion, multiband compression, gain,
stereo imaging, limiting, delay, and reverberation. It has 72 continuous
parameters plus activation switches.
The paper's inference-time parameter-matching evaluation uses a separate
seven-effect, 47-parameter chain without reverb.
Stored validation metrics
The epoch-179 checkpoint records:
Metric
Value
Validation loss
1.6971
R@1
0.5969
R@5
0.8656
R@10
0.9375
Mean rank
3.2750
Median rank
1.0
These are the validation metrics stored in the training checkpoint. Full
parameter-matching results should be reported with the exact evaluation
protocol and immutable model revision.
Limitations
RelFx estimates one global transformation between two clips.
It does not disentangle independent source-wise effects inside a mixture.
Evaluation uses a controlled differentiable DSP chain and does not cover
every commercial plug-in or real-world preset.
Relative embeddings are not unique or physically exact processor settings.
License
The checkpoint is released under Creative Commons
Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0).
Attribution, non-commercial use, and ShareAlike conditions apply; see
LICENSE for the complete terms. Public availability does not replace
these license obligations.
Citation
bibtex
1@inproceedings{liu2026beyond,
2 title = {Beyond Dry References: Learning Relative Audio Effects
3 Representations via Contrastive Distance Learning},
4 author = {Liu, Xinlu and Lin, Huibin and Wei, Weixing and Yan, Zhenhai},
5 booktitle = {Proceedings of the 27th International Society for Music
6 Information Retrieval Conference (ISMIR)},
7 year = {2026}
8}
