Vital Audio JEPA V2
Audio JEPA encoder trained from scratch on 74,685 parameter-aware renders made
with the Vital 1.6.4 standalone renderer.
Input
- 22,050 Hz mono PCM audio
- exactly 6.0 seconds
- 128-bin log-mel spectrogram
- 1024-sample FFT and 256-sample hop
- 16x16 spectrogram patches: an 8x32 grid with 256 tokens
Training data
The training corpus is published under
duckking032/vital/augmented_v1_22050_mono_6s. It contains 74,685 accepted
renders derived from 7,394 unchanged parent presets. Variants include
category-aware MIDI notes and deterministic mutations of macros, filters,
amplitude envelopes, oscillator timbre, unison, active modulation, and active
effects.
All variants of a parent preset are assigned to the same split. The model saw
70,974 clips from 7,022 parents during training; validation used 3,711 clips
from 370 entirely unseen parents.
Objective and training
The online encoder receives 40% context patches. A predictor reconstructs 40%
target patches in the latent space of an exponential-moving-average target
encoder.
- 35,690,112 total parameters; 21,493,632 trainable
- batch size 64, bfloat16, AdamW
- 30 epochs / 33,240 optimizer steps
- RTX PRO 4500 Blackwell
- training time: 2,792.6 seconds (46.5 minutes)
The selected checkpoint is step 15,000. Later checkpoints achieved lower
masked loss but weaker semantic retrieval, so selection used cross-parent
same-category retrieval rather than loss alone.
Controlled evaluation
Evaluation uses 3,711 variants from 370 held-out parents. A query is forbidden
from retrieving another variant of its own parent, preventing augmented
siblings from inflating the scores.
| Model | Masked val loss | Top-1 | Top-5 | MRR@10 |
|---|
| Published V1 | 0.10759 | 32.47% | 54.11% | 0.4220 |
| V2 step 15,000 | 0.08507 | 35.65% | 56.53% | 0.4494 |
The validation set was also used for checkpoint selection, so these metrics
should not be treated as a final untouched-test estimate. A future release
should reserve a third parent-held-out test split.
Intended use
This model is a retrieval and initialization baseline for editable Vital sound
design. It is not an exact audio-to-preset reconstruction system, and category
retrieval is only a coarse proxy for perceptual usefulness.
checkpoints/jepa_v2/selected.pt is the optimizer-free selected checkpoint.
See eval_runs/jepa_v2/ for the full comparison, controlled V1 baseline, V2
evaluation report, and validation embeddings.
