You can deploy this model and start earning money today!

hanxunh/
AudioMosaic-vit-b16-linear-prob-as20k-attentive

Name: AudioMosaic-vit-b16-linear-prob-as20k-attentive
Availability: InStock

self-supervised-learning

hanxunh/AudioMosaic-vit-b16-pretrained

finetune

mit

model-index

Views

No views yet

arXiv

HuggingFace

AudioMosaic: Contrastive Masked Audio Representation Learning

Code: https://github.com/HanxunH/AudioMosaic

Pretrained encoder: hanxunh/AudioMosaic-vit-b16-pretrained

AudioMosaic — AudioSet-20K Attentive Probe

This is the AudioMosaic ViT-B/16 encoder with a multi-layer attentive probe trained on AudioSet-20K (encoder frozen, only the probe parameters are trained).

Metric	Value
mAP (AS-20K eval)	33.55

Model Details

Architecture: ViT-B/16 with multi-layer attentive probe
Embedding dim: 768, Depth: 12, Heads: 12
Input: log-mel spectrogram of size 1024 × 128
Patch size: 16 × 16
Classes: 527 (AudioSet ontology)
Encoder weights are frozen during probe training.

Model Usage

python

import sys, torch
from huggingface_hub import snapshot_download

local_dir = snapshot_download("hanxunh/AudioMosaic-vit-b16-linear-prob-as20k-attentive")
sys.path.insert(0, local_dir)

from load_model import load_classifier
model = load_classifier(device="cuda")

# Forward a log-mel spectrogram batch of shape [B, 1, 1024, 128]
fbank = torch.randn(2, 1, 1024, 128).cuda()
with torch.no_grad():
    logits = model(fbank)        # [B, 527]
    probs = logits.sigmoid()     # multi-label probabilities

The release contains:

model.safetensors — probe weights
config.json — architecture hyperparameters
modeling.py — vendored model architecture (no need to install AudioMosaic)
load_model.py — convenience loader

Required dependencies: torch, timm, torchlibrosa, safetensors, huggingface_hub.

Citation

bibtex

@inproceedings{huang2026audiomosaic,
  title={AudioMosaic: Contrastive Masked Audio Representation Learning},
  author={Hanxun Huang and Qizhou Wang and Xingjun Ma and Cihang Xie and Christopher Leckie and Sarah Erfani},
  booktitle={ICML},
  year={2026}
}