A compact 24M‑parameter language model built with a Looped Transformer architecture, trained using the Muon optimizer, and enhanced with Chain‑of‑Thought reasoning. This model specializes in story generation and basic math reasoning, making it ideal for lightweight experimentation, educational projects, and rapid prototyping.
The model was trained for 30 minutes on two NVIDIA T4 GPUs, using a curated dataset of short stories, narrative prompts, character interactions, and basic math word problems.
The Muon optimizer provided fast, stable convergence, making it well‑suited for small‑parameter models.
1from transformers import AutoTokenizer, AutoModelForCausalLM
2
3tokenizer = AutoTokenizer.from_pretrained("your-username/looped-transformer-24m")
4model = AutoModelForCausalLM.from_pretrained("your-username/looped-transformer-24m")
5
6prompt = "Write a short story about a robot learning to dream."
7
8inputs = tokenizer(prompt, return_tensors="pt")
9outputs = model.generate(**inputs, max_length=200)
10
11print(tokenizer.decode(outputs[0], skip_special_tokens=True))
