You can deploy this model and start earning money today!

tiagoCuervo/
lf2ar-speech-150m

Name: lf2ar-speech-150m
Availability: InStock

interleaved-lm

safetensors

speech-language-model

lf2ar

apache-2.0

Views

No views yet

lf2ar-speech-150m

Speech LF²AR checkpoint from LF²AR: Accounting for Layerwise Dynamics to Improve Multimodal Adaptation of Language Models. This model has 148,983,612 parameters and uses dynamic attention residuals.

python

from interleaved_lm import PerceptionExpressionAdaptedTextLM
from interleaved_lm.audio import generate_speech

model = PerceptionExpressionAdaptedTextLM.from_pretrained(
    "tiagoCuervo/lf2ar-speech-150m", device="cuda"
)
codes = generate_speech(model, "A short story about the moon", max_new_tokens=250)

Speech is represented by consecutive-run-collapsed 25 Hz mHuBERT layer-11 K-means units (K=500; EOS=500; PAD=501). A vocoder is not bundled.

The text backbone is initialized from SmolLM. Speech-tokenizer and vocoder artifacts are credited in the Interleaved-LM third-party notices.

Evaluation

Task	Direction	Accuracy
sstorycloze	audio-audio	55.16
sstorycloze	text-text	64.08
sstorycloze	audio-text	58.26
sstorycloze	text-audio	58.47
tstorycloze	audio-audio	81.99
tstorycloze	text-text	88.24
tstorycloze	audio-text	80.60
tstorycloze	text-audio	75.31

See provenance.json for exact hashes and evaluation metadata.