mini-Whitman 🌿
A QLoRA fine-tune of
Qwen3.5-0.8B trained on Walt Whitman's
Leaves of Grass — an experiment in giving a small language model a large poetic soul.
This model was trained purely as a creative and educational demonstration: to explore whether a compact generative model can absorb the cadence, imagery, and democratic spirit of one of America's greatest poets, and produce novel verse in that tradition.
🧠 Model Details
| Field | Value |
|---|
| Base Model | Qwen3.5-0.8B |
| Fine-tuning Method | QLoRA (8-bit quantization) |
| Training Data | Walt Whitman's Leaves of Grass (public domain) |
| Training Date | March 2025 |
| Task | Causal language modeling / creative text generation |
| Language | English |
| License | MIT |
🚀 How to Use
1from transformers import AutoTokenizer, AutoModelForCausalLM
2
3tokenizer = AutoTokenizer.from_pretrained("your-username/mini-Whitman")
4model = AutoModelForCausalLM.from_pretrained("your-username/mini-Whitman")
5
6prompt = "I celebrate myself, and sing myself,"
7inputs = tokenizer(prompt, return_tensors="pt")
8outputs = model.generate(
9 **inputs,
10 max_new_tokens=500,
11 do_sample=True,
12 temperature=0.7,
13 top_p=0.95,
14 top_k=50,
15 repetition_penalty=1.2,
16 num_return_sequences=1,
17)
18print(tokenizer.decode(outputs[0], skip_special_tokens=True))
⚙️ Recommended Generation Parameters
These settings were tuned specifically for this model and produce the most coherent, Whitman-esque output:
| Parameter | Value | Why |
|---|
max_new_tokens | 500 | Long enough for a full verse or stanza |
temperature | 0.7 | Creative but not chaotic — preserves Whitman's cadence |
top_p | 0.95 | Nucleus sampling keeps outputs grounded |
top_k | 50 | Limits vocabulary to the most likely tokens at each step |
repetition_penalty | 1.2 | Reduces the model's tendency to loop phrases |
num_return_sequences | 1 | Generate one poem at a time |
📖 About the Training Data
Leaves of Grass (1855–1891) is the life's work of Walt Whitman (1819–1892), widely considered one of the most innovative and influential collections in American literature. The text is in the
public domain and freely available via
Project Gutenberg. The specific version used for this model was exported from
Wikisource here.
Whitman's verse is characterized by:
- Long, flowing free verse lines
- Catalogs and lists as a poetic device
- Celebration of the self, nature, democracy, and the human body
- A prophetic, oracular voice
🎯 Purpose & Limitations
This model is an artistic and educational proof-of-concept, not a production tool. Its purpose is to demonstrate the poetic potential of small fine-tuned language models — that even a sub-1B parameter model, when trained on a focused literary corpus, can generate text with a recognizable stylistic voice.
Limitations:
- Very small model — outputs can be repetitive or incoherent
- Trained on a single author's work; not a general-purpose assistant
- Will occasionally hallucinate lines that don't match Whitman's themes
- Not suitable for factual tasks
🔬 Ethical Note
Using AI to generate text in the voice of a deceased writer raises genuine aesthetic and ethical questions. This model is offered in that spirit of inquiry — not to deceive, but to explore what these tools can and cannot do. All generated text should be understood as inspired by Whitman, not attributed to him.
📜 License
MIT License — free to use, modify, and distribute with attribution.
