Roast Bot — Qwen 1.5B (epoch 2 checkpoint)
Fine-tuned Qwen2.5-1.5B-Instruct, trained on Sarvin's WhatsApp texting style. LoRA r=32, checkpoint
taken at epoch 2 of 5 — trades a bit of surface polish for actually generalizing.
Looking for the fully-trained version instead?
See
roast-bot-qwen-1b — same training
run, taken to completion at epoch 5. Read the comparison below before picking one.
Which checkpoint should I use?
This is an
earlier checkpoint from the same training run as
roast-bot-qwen-1b, saved after epoch 2
instead of running to epoch 5. On a held-out eval split, eval loss was near its lowest at this
point (0.661, vs. 0.652 at epoch 1 and 0.989 by epoch 5) — meaning this checkpoint hadn't yet
started overfitting to the 90 training examples.
In side-by-side testing on 6 prompts, the epoch-5 checkpoint returned exact, verbatim training
examples for half of them (not generated — recalled). This checkpoint returned an original,
synthesized response every time, including on inputs that closely resembled training data.
The tradeoff: some responses here are a little less sharp or occasionally slightly less coherent
than epoch-5's best outputs — that's the honest cost of a model that's actually generating rather
than reciting. Epoch-5 "sounds better" in spots only because those spots are literally hand-written
training examples playing back, not because the model got smarter there.
Use this checkpoint if: you want responses that are genuinely generated from your input, even
at some cost to consistency.
Use roast-bot-qwen-1b instead if: you mostly care about polish on inputs similar to
the original 90 training examples, and don't mind exact recall of training data.
Training details
- Base:
unsloth/Qwen2.5-1.5B-Instruct-bnb-4bit
- Method: LoRA (r=32, alpha=64, dropout=0.05) via Unsloth
- Data: 90 hand-written conversation pairs (80 train / 10 eval split, no duplication)
- Checkpoint: epoch 2 of 5, lr=2e-4, cosine schedule, weight decay 0.01
- Eval loss by epoch: 0.652 → 0.661 (this checkpoint) → 0.930 → 0.946 → 0.989
- Training notebook:
training/roast_bot_colab.ipynb
Usage
Deployed via
Ollama in
roast-bot,
a WhatsApp bot. See that repo for the full inference pipeline (system prompt, reply-decision logic,
GGUF conversion steps).
