You can deploy this model and start earning money today!

auro-rirum/
audioforge-ast-fsd50k

Name: audioforge-ast-fsd50k
Availability: InStock

audio-spectrogram-transformer

ast

mit

Views

No views yet

audioforge-ast-fsd50k

LoRA-adapted Audio Spectrogram Transformer (AST) trained on FSD50K for multi-label environmental sound event classification, produced by the AudioForge training pipeline.

Model Details

Base model: MIT/ast-finetuned-audioset-10-10-0.4593 (Audio Spectrogram Transformer, pretrained on AudioSet)
Adaptation method: Low-Rank Adaptation (LoRA) via peft library
- Rank (r): 8
- Alpha: 16
- Dropout: 0.05
- Target modules: ["q_proj", "v_proj"]
Trainable parameters: 450,248 out of 86,792,848 total (0.52%)
- Backbone frozen, only LoRA adapters + 200-class classifier head trained
Task: Multi-label audio event classification, 200 sound classes (FSD50K)
Loss: Sigmoid + Binary Cross-Entropy (multi-label)
Training: 5 epochs, single NVIDIA A10G GPU (AWS g5.xlarge), ~71 minutes, fp16 mixed precision
Validation split: 4,170 samples

Validation Metrics (FSD50K)

Metric	Value
mean Average Precision (mAP)	0.5567
Micro Average Precision	0.7166
Micro F1	0.6443
Macro F1	0.3843
Macro Precision	0.6113
Macro Recall	0.3235
Micro Precision	0.8091
Micro Recall	0.5353

Usage

This repository contains only the LoRA adapter (adapter_config.json + adapter_model.safetensors, ~1.8 MB). The base model must be downloaded separately and combined with this adapter at load time.

python

from transformers import AutoModelForAudioClassification, AutoConfig
from peft import PeftModel

# Load base model configuration and set for FSD50K multi-label task
base_config = AutoConfig.from_pretrained("MIT/ast-finetuned-audioset-10-10-0.4593")
base_config.num_labels = 200
base_config.problem_type = "multi_label_classification"

# Load base model (87M parameters, ~340 MB)
base_model = AutoModelForAudioClassification.from_pretrained(
    "MIT/ast-finetuned-audioset-10-10-0.4593", 
    config=base_config, 
    ignore_mismatched_sizes=True
)

# Apply LoRA adapter
model = PeftModel.from_pretrained(base_model, "auro-rirum/audioforge-ast-fsd50k")
model.eval()

# Input: mel-spectrogram [batch, max_length, num_mel_bins] from transformers.AutoFeatureExtractor
# Usage: logits = model(input_values=input_values).logits
#        probs = torch.sigmoid(logits)  # Multi-label probabilities for each class

Dataset

FSD50K (Freesound Dataset 50K): Large-scale, high-quality annotated dataset of environmental audio, with 50,197 audio clips labeled across 200 sound event classes. Multi-label annotations allow clips to have multiple simultaneous events.

Notes

This is an adapter-only repository. To use this model, you must download the base model from Hugging Face.
The adapter was trained with mixed-precision (fp16) for efficiency; inference works in both fp32 and fp16.
Performance varies across classes; strong performance on broad categories (Music, Animals, Explosions) and weaker on rare or semantically ambiguous classes.