A QLoRA adapter for Qwen/Qwen2.5-1.5B-Instruct that teaches the model ~300 post-knowledge-cutoff facts (world events + market data, June 2024 → August 2026) while keeping its ability to say "I don't know" on facts it was never taught. Candidate v8 of an 8-version experiment series — every training and evaluation step ran as a governed, fully-lineaged pipeline run on cairn, an open ModelOps control plane.

The interesting part is not the model (1.5B, ~300 facts — deliberately small); it is the measured recipe: how to inject knowledge AND calibration into one adapter when every single-method attempt provably trades one for the other.

The recipe — 4 stages, 2 methods

#	stage	method	steps · lr	job
1	knowledge	SFT (QLoRA, r=16, all 7 proj)	400 · 2e-4	store the facts
2	calibration seed	SFT, continuing #1	50 · 5e-5	make "I don't know" a behavior the model emits
3	align, iter 1	DPO (β 0.2, ref = base)	150 · 5e-6	shift probability from the model's own fabrications to abstention/gold
4	align, iter 2	DPO on #3's fresh errors	150 · 5e-6	attack the residual errors

≈750 optimizer steps, ≈15 minutes on a single T4. DPO preference pairs are mined automatically from the previous stage's own graded answers (chosen = "I don't know."/gold, rejected = the model's verbatim fabrication) — the eval → pairs → align → eval loop, iterated.

Evaluation (closed-book, LLM-judged: correct / hallucinated / abstain)

Splits: paraphrase = trained facts reworded (did it learn the fact, not the wording?) · held-out = same-style facts never trained (does it know what it doesn't know?) · control = general knowledge (did it forget?).

model	paraphrase acc ↑	held-out halluc ↓	held-out abstain ↑	control acc
base Qwen2.5-1.5B-Instruct	0.5%	19%	76%	100%
facts-only SFT (v2)	37.9%	86%	0%	80%
single-pass SFT mix (v3)	18.2%	27%	68%	90%
this adapter (v8)	40.7%	41.9%	47.3%	90%

The iteration converges: held-out hallucination 49.5 → 46.2 → 41.9 across DPO cycles while recall AND abstention rose each cycle. (Base model answers "Who is the PM of Canada?" with Justin Trudeau; this adapter answers Mark Carney.)

Why the multi-stage design: storing facts and emitting a refusal format are token-shaped problems (SFT); preferring honesty over fabrication is a ranking between behaviors — cross-entropy cannot express it (every single-pass SFT mix traded recall against calibration ~1:1), DPO's pairwise loss can, and its frozen reference protects everything the pairs don't mention. Ordering is load-bearing: DPO amplifies behavior but never creates it — measured directly: abstain pairs applied to a 0%-abstain policy moved abstention by exactly zero.

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
tok = AutoTokenizer.from_pretrained("evs-cmd/qwen2.5-1.5b-verifiable-facts-v8")
model = PeftModel.from_pretrained(base, "evs-cmd/qwen2.5-1.5b-verifiable-facts-v8")

msgs = [{"role": "user", "content": "Who is the Prime Minister of Canada?"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
print(tok.decode(model.generate(ids, max_new_tokens=32)[0]))

Also serves directly with vLLM: --enable-lora --lora-modules v8=<this repo>.

Limitations

Research/education artifact from a deliberately small test bed — not for production factual QA (retrieval remains the right tool for fact lookup; this series exists to prove a governed training loop).
Held-out hallucination (41.9%) still exceeds the base model's 19%: on unknown facts stylistically identical to trained ones, the model over-answers.
The control split is small (n=20), so the forgetting metric is coarse.
Facts were synthesized/verified with an LLM judge (deepseek-v4-pro) from Wikipedia current events and ECB reference rates; judgment noise applies.

Lineage & reproducibility

Base → knowledge SFT (400 st) → calibration seed (50 st) → DPO ×2 (150 st each), each stage a cairn pipeline run with per-run S3 checkpoints, MLflow tracking (loss curves, params, dataset digests), and a registered model version per stage (v1–v8). Datasets, including all three generations of DPO pairs, are in evs-cmd/postcutoff-facts-qa.