🪶 Parable-Granite-3B v2 — trained on genuine Claude Fable 5 agent traces
This is the full-precision safetensors repo (vLLM / transformers / fine-tuning). For llama.cpp, Ollama, and LM Studio use the GGUF repo.
A tiny local model that thinks before it answers — planning, reasoning, and terminal instincts distilled from real agent sessions.
~3 GB of RAM is all you need. Laptop, old GPU, Raspberry-Pi-class boxes with swap — the Q4 build runs
anywhere. One command and you have a private, offline reasoning model on your machine:
ollama run hf.co/AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5-GGUF:Q4_K_M
The headline — v2 is a different model
v2 is a full retrain: 13× more genuine Fable 5 trace data (11,574 sessions, 16.8M tokens — corpus published) and a rebuilt recipe (completion-masked loss, replay mixing, benchmark-gated checkpoints, seed-averaged weights).
Clean answers, structured reasoning, agent instincts — and the transcript artifacts that leaked into v1's replies are gone. One trade, made on purpose: raw HumanEval-style function synthesis stays the base model's turf (81.7 vs 70.1) — v2 spends that capacity on agent behavior instead, and spends half as much as v1 did. Measurement notes below. 👇
Announcements
📌 Same links, new model. v2 replaces v1 in place — every existing Ollama command, script, and bookmark now serves v2. No migration, nothing to change.
🔮 v3 is already training. Rejection-sampled SFT: thousands of candidate solutions generated against executable tests, only verified passers enter the corpus. The goal is simple — above-base agent capability, not just clean behavior. Follow AnkitAI for the drop.
📦 Full family. This 3B is the smallest Parable. Need more headroom? 8B Granite, 8B Qwen, 4B Qwen — same recipe, no matter your hardware.
How to run it
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
23model_id ="AnkitAI/Parable-Granite-4.1-3B-Claude-Fable-5"4tok = AutoTokenizer.from_pretrained(model_id)5model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")67messages =[{"role":"user","content":"Write a Python function that retries an HTTP request with exponential backoff."}]8inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)9out = model.generate(inputs, max_new_tokens=3000, temperature=0.7, top_p=0.95, do_sample=True)10print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
Every answer opens with a <think>...</think> reasoning block — that's the Fable 5 heritage. llama.cpp's --jinja mode separates it automatically; strip it before showing replies to end users.
Sampling: temperature 0.7, top_p 0.95, and budget max_tokens generously (2500+) — trace-trained models think at length before answering.
Measurement notes
All numbers: identical llama.cpp harness, greedy decoding, Q4_K_M, base model measured on the same instrument. We train multiple seeds and ship the weight-average — single-run scores at 3B swing ±3 points on GPU nondeterminism alone, so most cards report their luckiest run; we ship the average and report the shipped weights' own numbers. Raw eval outputs live in this repo.
Which model should you use? Pure single-function code completion → the base model is genuinely strong there. Explanations, debugging, terminal workflows, structured reasoning, agent-style tasks → that's what Parable is trained on, and where v2 shines.
What's new in v2 (training)
The recipe follows our ongoing tech report (in preparation):
Completion-only loss masking (Hermes 3, Tülu 3) — loss on assistant tokens only, so the model learns to answer, not to imitate transcripts
30% replay mix of general instruction data (Luo et al., Biderman et al.) — the anti-forgetting lever
Session re-segmentation + sanitization — why v1 sometimes leaked agent JSON into normal chat, and v2 never does (0/34)
Benchmark-gated checkpoints (Dong et al.) instead of fixed epochs
Seed-averaged weights (model soups, Wortsman et al.) — we ship the average of multiple runs, not the lottery winner
With Claude Fable 5 now retired, genuine self-authored Fable traces are a fixed, non-renewable corpus. Unlike most models in this niche, our full training corpus is public: AnkitAI/parable-corpus-v2 — deduplicated, quality-gated, provenance-tagged.
Good to know
Fine-tuned at 2,048-token sequences; the base 128K context stays available, fine-tuned behavior is strongest in the opening turns.
Not trained for: multi-file repo navigation, vision, non-English.
Inherits Granite-4.1-3B's knowledge cutoff. Treat generated commands as drafts to review.
Evaluation
Function calling (BFCL V3, AST subset)
Measured 2026-07-29: bfcl-eval at gorilla main, prompting mode, Q4_K_M
GGUFs served by llama.cpp on a T4, base and Parable under the identical
harness. Categories: simple_python / multiple / parallel /
parallel_multiple (400/200/200/200 items). Raw generations and score
files: parable-v2-artifacts
under verify/bfcl/.
simple_python
multiple
parallel
parallel_multiple
Granite-4.1-3B base
0.848
0.790
0.710
0.665
This model (chat variant)
0.413
0.605
0.320
0.425
For tool-calling workloads, use the base model; this variant is built
for reasoning prose. The drop has a specific mechanism: sampled generations show the model
intermittently answering with args-only tool-call JSON (for example
{"base": 10, "height": 5}) instead of a function call, which the
AST scorer rejects. That is trace-scaffolding format bleeding into
standalone tasks, the failure mode the series paper names session
leakage (Section 6 of the report). The reasoning-voice strengths this variant trains for are unaffected
on prose tasks.
Base & license
Weights: Apache-2.0 (inherited from ibm-granite/granite-4.1-3b). Training data: Fable-5-traces AGPL-3.0, gpt5.5-terminal MIT — since traces originate from third-party assistants, their terms may apply to downstream training; check before commercial distillation.
The recipe, evaluation methodology and failure analysis behind this model are
documented in the tech report:
Aglawe, A. (2026). Agent-Trace Fine-Tuning of Small Language Models under
Constrained Compute. Zenodo. doi:10.5281/zenodo.21676407
bibtex
1@misc{aglawe2026agenttrace,
2 author = {Aglawe, Ankit},
3 title = {Agent-Trace Fine-Tuning of Small Language Models under Constrained Compute},
4 year = {2026},
5 publisher = {Zenodo},
6 doi = {10.5281/zenodo.21676407},
7 url = {https://doi.org/10.5281/zenodo.21676407}
8}
