Open RVQ Encoder for MiniMax Music 3, 155M, v3

Status

• Training complete: 17,660 optimizer steps.
• Recommended checkpoint: final.
• Not an official MiniMax model.
• Not the original MiniMax Music 3 RVQ encoder.
• No original encoder weights or source code were used.
• V3 was initialized from scratch. No v1 or v2 weights were loaded.
• Real-audio generalization is not established.
• MERT was used only as a frozen training teacher. MERT weights are not included.
• A packaged from_pretrained loader is not present yet.

Result

• loss: 5.264569 -> 5.260236;
• semantic top-1: 42.86% -> 43.03%;
• semantic top-5: 80.17% -> 80.48%;
• acoustic top-1: 7.62% -> 7.65%;
• acoustic top-5: 21.98% -> 22.02%.

Objective

Architecture

1. Apply the convolutional stem and residual stack at DAV latent rate.
2. Average-pool exact DAV spans into 25 Hz frames.
3. Add learned positions.
4. Apply eight bidirectional Transformer encoder layers.
5. Apply final LayerNorm.
6. Produce eight independent RVQ distributions.

Architecture Selection

Initialization and muTransfer

1. Construct target, base, and delta training wrappers.
2. Attach wrapper-level base shapes with mup.set_base_shapes.
3. Construct mup.MuAdamW after infshapes are attached.
4. Save wrapper and exported-encoder base-shape files.

• mup.MuReadout;
• output multiplier 1.0;
• zero initialized;
• initial output distributions uniform within each vocabulary.

• mup.MuReadout, 1,088 -> 768;
• no bias;
• nonzero initialization required for cosine loss;
• removed from exported encoder checkpoints.

Data

• 2,972 one-track ZIP shards;
• 2,837 training records;
• 135 holdout records before exact-alignment filtering;
• approximately 178 GB;
• synthetic tracks generated by MiniMax Music 3;
• not MiniMax's original training set.

• waveform audio;
• sampled RVQ codes;
• teacher top-50 token IDs and logits;
• exact chunk-stitching metadata.

Timeline Alignment

• rollout window: 200 semantic frames;
• rollout hop: 100 semantic frames;
• full stitched hop: 345 DAV latents;
• later chunks begin ownership 25 semantic frames after nominal start;
• code row 0 is warm-up;
• semantic frame i uses code row i + 1;
• the final partial chunk uses its own integer latent length;
• records without exact chunk_stitching metadata are excluded.

MERT Alignment

Loss

• cross-entropy against sampled RVQ codes;
• equal weight for all eight heads;
• padding target -100.

• teacher top-k 50;
• temperature 1.0;
• Hinton T^2 scaling;
• teacher renormalized over valid stored IDs;
• student full-vocabulary log-softmax gathered at teacher IDs;
• no student top-k renormalization;
• invalid, EOS, and out-of-vocabulary IDs excluded;
• frames with no valid teacher IDs skipped for KL.

• cosine distance between projected student layer 4 and frozen MERT layer 9;
• mean over batch and frames.

Training

Checkpoint Format

Evaluation

• exact-alignment holdout;
• 130 tracks;
• 2,768 deterministic windows;
• all 35 numbered checkpoints and final;
• four-rank evaluation; no distributed-sampler padding;
• exact-token top-1 and top-5.

Matched Comparison at Step 17,500

• v2's width increase produced the main gain over v1;
• v3 learned the MERT alignment target;
• v3 wins every listed aggregate metric over v2;
• the v3-v2 differences are small;
• MERT alignment did not remove the acoustic codebook-depth gradient;
• causal conditioning across acoustic books is the next structural test.

Condition-Embedding Replay Comparison

• 130 exact-alignment holdout tracks;
• each final checkpoint predicts argmax RVQ codes from cached DAV latents;
• predicted codes are teacher-forced through the official language model and RVQ depth decoder;
• hidden states pass through the official condition encoder with recorded chunk stitching;
• reconstructed condition embeddings are compared with stored condition embeddings;
• metric: per-track mean cosine over stitched condition-latent frames;
• true sampled codes provide the replay control.

Limitations

• 5.12-second context.
• No cross-window state.
• Synthetic model-output training domain.
• Real audio remains out of distribution.
• Teacher top-k uncertainty is from LM rollout, not an audio-conditioned posterior.
• Exact-token accuracy understates perceptual equivalence.
• Semantic CE can dominate early because its vocabulary is larger.
• Acoustic heads are independent despite residual-codebook dependence.
• Diffusion render and audio-domain evaluation remain pending.
• No stable packaged loading API.
• Use is subject to MiniMax Music 3, dataset, and MERT terms.

Discussion and Attribution

• bghira: SimpleTuner experiments, trace extraction, teacher distributions, alignment records, corpus publication, and training runs.
• marduk191: WAV/code samples, early mel encoder proof, corpus-scale observations, and encoder experiments.
• scragnog: SimpleTuner calibration, relative-weight analysis, caption-cache and rollout-seam findings, and GGML interoperability tests.
• Serveurperso: independent encoder, corpus generator, replay evaluation, and stitched-timeline findings.
• dernet: RVQ supervision, internal-alignment, and tokenizer reverse-engineering analysis.

• SimpleTuner/open-rvq-encoder-minimax-music3-41m-v1
• SimpleTuner/open-rvq-encoder-minimax-music3-155m-v2
• marduk191/Minmax_music3_experiments
• ServeurpersoCom/minimaxmusic.cpp proof commit

Offline Checkpoint Evaluation

Checkpoint Loss

Checkpoint Accuracy

Codebook Top1

Training History

MERT Alignment