Vital Audio JEPA V4
V4 trains the JEPA representation around continuous acoustic properties rather
than treating PresetShare labels as the main objective. It uses 74,627 stereo
renders produced by Vital 1.6.4 from 7,394 unchanged parent presets.
Input
- 22,050 Hz stereo PCM16 audio, exactly 6.0 seconds
- four-second held note plus two-second release tail
- no waveform peak or loudness normalization
- energy-preserving mid/side conversion
- two-channel, 128-bin log-mel spectrogram
- 1024-sample FFT, 256-sample hop, and 16x16 patches
Feature-focused objectives
The pooled representation predicts 15 normalized continuous descriptors:
- level, peak, crest factor, attack, tail energy, and tail/body ratio
- spectral centroid, bandwidth, brightness, flatness, and flux
- modulation depth
- stereo width, correlation, and channel imbalance
Training also includes:
- frequency-band, long-time-span, random-block, and release-tail masks
- side-channel context masking with full stereo targets
- balanced parent/category sampling
- different-note positive pairs from unchanged presets
- center-note parent/mutation pairs
- regression and cosine-direction losses for measured acoustic changes caused
by parameter mutations
Taxonomy supervision remains a low-weight diagnostic. The old generic recipe
classification and raw parameter-delta losses have zero weight in V4.
Training
- 36,302,000 parameters total; 22,105,520 trainable
- 70,916 clips / 7,022 parents for training
- 1,859 clips / 186 parents for validation
- 1,852 clips / 184 parents for the untouched test split
- batch size 64, bfloat16, AdamW
- 20 epochs / 22,160 optimizer steps
- RTX PRO 4500 Blackwell
- full training time: 1,980.5 seconds (33.0 minutes)
The final step was selected by maximum held-out mean acoustic-feature
correlation, with cross-parent retrieval as the secondary criterion.
Untouched parent-held-out results
Every retrieval query excludes all variants sharing its parent preset.
| Metric | Result |
|---|
| Mean descriptor correlation | 0.8373 |
| Mean descriptor MAE | 0.0426 |
| Coarse retrieval top-1 | 48.43% |
| Coarse retrieval top-5 | 67.33% |
| Coarse MRR@10 | 0.5718 |
| Fine retrieval top-1 | 37.37% |
| Fine retrieval top-5 | 52.70% |
| Fine MRR@10 | 0.4472 |
| Coarse-category head accuracy | 64.74% |
| Mutation feature-delta cosine | 0.2831 |
| Mutation feature-delta MAE | 0.0270 |
Selected descriptor correlations include 0.972 for RMS level, 0.943 for
spectral centroid, 0.905 for modulation depth, 0.906 for stereo width, and
0.905 for stereo correlation.
Limitations
Spectral flux (0.579 correlation), channel imbalance (0.491), and mutation
direction prediction (0.283 cosine) remain substantially weaker than the core
level, spectrum, envelope, and stereo features. The descriptors are practical
signal-analysis targets, not a complete model of human timbre perception.
This encoder is intended for retrieval and as an initialization for editable
Vital sound design. It does not directly recover a complete synthesizer preset.
checkpoints/jepa_v4/selected.pt is the optimizer-free selected checkpoint.
The evaluation reports and training log are included for reproducibility.
