This repository contains the 770M parameter Attractor Model introduced in
Solve the Loop: Attractor Models for Language and Reasoning.
Attractor Models offer a promising alternative to purely feed-forward computation by iteratively refining latent representations. In this architecture, a backbone module first proposes output embeddings, then an attractor module refines them by solving for the fixed point using implicit differentiation. This model delivers a Pareto improvement over standard Transformers in language modeling and demonstrates strong performance on reasoning tasks.
To use this model, you need to install the official package from the
GitHub repository.
1from attractor.models.attractor import Attractor, AttractorConfig
2
3# Construct the configuration for the 770m model
4config = AttractorConfig.from_name("attractor-large-770m")
5model = config.construct_model()
1@article{feinashley2026attractor,
2 title={Solve the Loop: Attractor Models for Language and Reasoning},
3 author={Fein-Ashley, Jacob and Rashidinejad, Paria},
4 year={2026},
5 url={https://arxiv.org/abs/2605.12466}
6}
