The T5-large text encoder weights are not included in this repository. Use extract_t5.py to extract them from the original facebook/audiogen-medium checkpoint:

python extract_t5.py --output /path/to/audiogen-mlx/t5

This produces a t5/ directory with config.json, model.safetensors, and tokenizer files.

Note: The T5 safetensors keys use MLX-compatible naming (.layer_0. / .layer_1. instead of HuggingFace's .layer.0. / .layer.1.). This is required because MLX's ModuleParameters.unflattened() splits on all dots.

Usage (Swift/MLX)

swift

import MLXAudioGen

let model = try await AudioGenModel.fromPretrained(
    modelFolder: modelURL,
    t5Folder: t5URL
)

let tokens = try await model.generate(
    descriptions: ["dog barking"],
    duration: 5.0,
    cfgCoef: 3.0,
    temperature: 1.0,
    topK: 250
)

let audio = model.decode(tokens: tokens)

T5 Attention

T5's self-attention intentionally does not scale scores by 1/sqrt(d_k). This is a deliberate design choice in the T5 architecture — do not add scaling in the inference code.

License

This model is published under the CC-BY-NC 4.0 license (non-commercial use only), following the original AudioGen license.

mlx-community/audiogen-medium-mlx

AudioGen Medium (MLX)

Model Details

Files

T5 Conditioner (extracted separately)

Usage (Swift/MLX)

T5 Attention

License

mlx-community/
audiogen-medium-mlx