You can deploy this model and start earning money today!

tiagoCuervo/
gslm-scaling-20m-21p8b

Name: gslm-scaling-20m-21p8b
Availability: InStock

slm-research

safetensors

slm

speech-language-model

audio

text-generation

apache-2.0

Views

No views yet

gslm-scaling-20m-21p8b

Model-only FP32 weights for a final-budget scaling-curve endpoint from Scaling Properties of Speech Language Models (EMNLP 2024). This checkpoint has 20,710,912 parameters and saw exactly 21,772,640,000 tokens at 131,200 tokens per update.

python

import torch
from slm import TransformerLM

model = TransformerLM.from_pretrained("tiagoCuervo/gslm-scaling-20m-21p8b", device="cuda")
prompt = torch.tensor([[12, 91, 204]], device="cuda")
units = model.generate(prompt, max_new_tokens=200, temperature=0.8, top_k=50)

Evaluation scores are 56.21% sBLIMP phenomenon macro accuracy, 53.80% sBLIMP voice-pair accuracy, 53.50% sStoryCloze, and 69.00% tStoryCloze.

The model predicts consecutive-run-collapsed 25 Hz layer-11 mHuBERT K-means units (K=500; EOS=500; PAD=501). A vocoder is not bundled.

Speech-tokenizer and vocoder artifacts are credited in the SLM third-party notices. See provenance.json for exact hashes and evaluation metadata.