You can deploy this model and start earning money today!

SandyResearch/
parcae-370m

Name: parcae-370m
Rating: 2.5 (242 reviews)

pytorch

text-generation

2604.12946

Views

No views yet

Parcae-medium-370m

Parcae is a novel stable, looped language model architecture introduced in the paper Parcae: Scaling Laws For Stable Looped Language Models.

Looped architectures scale compute by passing activations through a block of layers multiple times. Parcae addresses common instability issues in looped models (such as residual explosion and loss spikes) by recasting looping as a nonlinear time-variant dynamical system and constraining the spectral norm of injection parameters.

Paper: https://arxiv.org/abs/2604.12946
Repository: https://github.com/sandyresearch/parcae
Project Page: https://sandyresearch.github.io/parcae/

Installation

To use this model, you can install the parcae-lm package:

pip install parcae-lm

Usage

You can instantiate the model and load the pretrained weights using the following code:

python

import parcae_lm

# Load the pretrained model from HuggingFace
model = parcae_lm.from_pretrained("SandyResearch/parcae-medium-370m")

Model Dimensions

Model	Parameters	Prelude	Core	Coda	Model dim.	Recurrence
Parcae-medium-370M	370M	4	4	4	1024	8

Citation

bibtex

@misc{prairie2026parcaescalinglawsstable,
      title={Parcae: Scaling Laws For Stable Looped Language Models}, 
      author={Hayden Prairie and Zachary Novack and Taylor Berg-Kirkpatrick and Daniel Y. Fu},
      year={2026},
      eprint={2604.12946},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2604.12946}, 
}

SandyResearch/parcae-370m