TGPT-XL-Persona
Full-parameter, KL-anchored persona SFT of TGPT-XL (84.067M).
Runs on-device at ~56–63 tok/s on a Pixel 6a (F16 GGUF).
- Developer: SmallAICreator (Kaden)
- Base model: TGPT-XL (85M param, trained from scratch on FineWeb-Edu + SmolTalk, ~8B tokens, ChatML, 1024 ctx)
- Format: ChatML
- Context length: 1024
- File:
tinygpt-persona-sft-f16.gguf (~170MB, F16)
- License: (fill in — e.g. Apache-2.0 / MIT)
What this is
A persona/identity fine-tune of TGPT-XL that gives the model a stable self-identity (name: TinyGPT, maker: UltraLabs) without breaking base knowledge, code behavior, or instruction-following.
The interesting part isn't the persona — it's the training method. This was a full-parameter fine-tune (all 84.067M weights unfrozen) with a KL-divergence anchor to a frozen copy of the base weights, which is what prevents catastrophic forgetting. A LoRA attempt (v1) failed; the full-FT + KL-anchor approach (v2) drifted ~150× less.
Training recipe
| Setting | Value |
|---|
| Method | Full-parameter SFT (all weights unfrozen) |
| Trainable params | 84.067M (deduped for tied embedding) |
| Init from | sft_4000.pt |
| Learning rate | 2e-5, OneCycle |
| Steps | 300 |
| Hardware / time | CPU, ~50 min |
| Export | .pt → HF LlamaForCausalLM → convert_hf_to_gguf.py → F16 GGUF |
Loss:
