SmolPoet — A 35M Parameter Poetry Language Model
SmolPoet is a lightweight (~35M parameters) decoder-only transformer model designed for generating poetic text in a Shakespearean and classical style. It is trained on a large-scale poetry corpus and further refined using reinforcement learning to enhance stylistic structure and expressiveness.
Overview
SmolPoet demonstrates that relatively small language models can produce rich, stylistically aligned text when trained with a combination of large-scale pretraining and reinforcement learning.
The model specializes in:
- Poetic text generation
- Shakespearean-style language
- Structured, multi-line outputs
- Figurative and abstract expression
Architecture
SmolPoet is inspired by modern LLaMA-style architectures and incorporates several efficiency and performance improvements.
- Model Type: Decoder-only Transformer
- Parameters: ~35 million
- Embedding Dimension: 512
- Layers: 12
- Attention: Grouped Query Attention (GQA)
- Positional Encoding: Rotary Positional Embeddings (RoPE)
- Activation Function: SwiGLU
- Normalization: RMSNorm
- Weight Sharing: Input/output embedding tied
Training Data
The model is trained on a large corpus of poetry and literary text:
- ~1.35 billion characters
- ~225 million words
- ~300 million tokens
The dataset includes a mixture of classical poetry and Shakespearean-style language, enabling the model to learn:
- Rhythmic sentence structure
- Archaic vocabulary
- Figurative language patterns
Tokenization
A custom Byte Pair Encoding (BPE) tokenizer is used:
- Trained on a representative subset of the dataset
- Efficient for large-scale corpora
- Captures subword structure for poetic language
Training Methodology
Pretraining
The model is first trained using standard next-token prediction to learn general language patterns and poetic structure.
Reinforcement Learning (PPO)
After pretraining, the model is refined using Proximal Policy Optimization (PPO) to improve stylistic quality.
The reinforcement learning phase encourages:
- Multi-line poetic structure
- Richer stylistic expression
- More consistent tone and phrasing
A KL-divergence constraint is applied to ensure the model remains close to the pretrained distribution while improving style.
Example Outputs
Prompt:
"O gentle night"
Output:
O gentle night,
And then the day melted to a night.
I heard the performers reply,
Who are on foot, I would say,
But I my tongue supplied thy
Prompt:
"The starry night"
Output:
The starry night-hawk, stirring the light plunges,
And at the wind the storm perchance the morrow.
Therefore off, I will suffer more from my tongue
Observations
- Reinforcement learning improves poetic structure and stylistic consistency
- Outputs become more aligned with Shakespearean tone
- The model produces multi-line compositions more reliably after RL fine-tuning
Limitations
- Does not enforce strict poetic meter (e.g., iambic pentameter)
- Occasional repetition or incoherent phrasing
- Limited world knowledge due to model size
- May produce archaic or stylistically exaggerated language
- Dataset contains special characters and information from Project Gutenberg files, that werent cleaned
Intended Use
SmolPoet is suitable for:
- Creative writing assistance
- Poetry generation experiments
- Educational purposes (LLM + RL pipelines)
- Research on small-scale language models
Out-of-Scope Use
- Factual question answering
- High-accuracy reasoning tasks
- Production-grade language understanding
Key Takeaways
- Small models can achieve stylistic richness with proper training
- Reinforcement learning is effective for style alignment
- Qualitative improvements may not always correlate with reward metrics
Acknowledgements
This model is inspired by advancements in transformer architectures and reinforcement learning methods for language models.
