Vital Audio JEPA V3
V3 adds semantic and preset-aware training signals to the masked audio JEPA
encoder. It is trained on 74,685 parameter-aware renders produced by Vital
1.6.4 from 7,394 unchanged parent presets.
Input
- 22,050 Hz mono PCM audio
- exactly 6.0 seconds
- 128-bin log-mel spectrogram
- 1024-sample FFT and 256-sample hop
- 16x16 spectrogram patches: an 8x32 grid with 256 tokens
V3 changes
- category- and parent-balanced batches
- guaranteed pairs of different MIDI-note renders from the same parent
- cross-parent supervised contrastive learning over a cleaned seven-class
taxonomy
- note-invariance contrastive learning only for performance variants
- auxiliary coarse-category, mutation-group, and mutation-delta heads
- parent-disjoint train, validation, and test splits
The coarse taxonomy merges related noisy PresetShare labels into bass,
drums, fx, keys, lead, pad, and rhythm.
Training
- 35,894,930 parameters total; 21,698,450 trainable
- 70,974 clips / 7,022 parents for training
- 1,860 clips / 186 parents for validation
- 1,851 clips / 184 parents for the untouched V3 test split
- batch size 64, bfloat16, AdamW
- 20 epochs / 22,160 optimizer steps
- RTX PRO 4500 Blackwell
- full training time: 1,928.8 seconds (32.1 minutes)
Step 4,000 was selected using validation retrieval. Later checkpoints reduced
masked JEPA loss but did not improve semantic retrieval.
Untouched parent-held-out test results
All retrieval results exclude every variant sharing the query's parent.
| Model | Taxonomy | Random top-1 | Top-1 | Top-5 | MRR@10 |
|---|
| V2 step 15,000 | coarse | 19.12% | 43.38% | 65.32% | 0.5304 |
| V3 step 4,000 | coarse | 19.12% | 48.57% | 65.75% | 0.5671 |
| V2 step 15,000 | fine | 12.54% | 27.71% | 49.27% | 0.3708 |
| V3 step 4,000 | fine | 12.54% | 35.76% | 51.92% | 0.4348 |
The V3 coarse-category head reaches 62.45% accuracy on the same unseen test
parents. The experimental recipe-group head does not generalize: it predicts
the performance class for the test set, yielding 29.34% accuracy. Do not use
that head as a reliable mutation classifier. The representation and category
results remain valid, and the failed auxiliary result is included explicitly
in the evaluation report.
Intended use
This encoder is a retrieval and initialization baseline for editable Vital
sound design. It is not an exact audio-to-preset reconstruction model. Coarse
category retrieval is only a proxy for perceptual usefulness, and sparse fine
labels such as vox, drone, and reese remain weak.
checkpoints/jepa_v3/selected.pt is the optimizer-free selected checkpoint.
See eval_runs/jepa_v3/ for checkpoint selection, coarse/fine test reports,
controlled V2 baselines, embeddings, and the full training log.
