⚠ NOTICE — research experiment, not theological authority

This is a personal portfolio / research project exploring how small open-weights LLMs can be fine-tuned to adopt a specific historical register and citation style. The trained model is not a reliable source of Catholic doctrine, biblical interpretation, or philosophical truth. It can hallucinate citations, misrepresent the Catechism, and confidently err. Outputs must not be cited as catechetical instruction, theological argument, or spiritual direction. For doctrinal questions, consult the actual Catechism of the Catholic Church, a qualified priest, or a trained theologian.

What this is

LoRA adapter for Qwen/Qwen2.5-7B-Instruct, trained to respond to philosophical and theological questions in a scholastic / Latin-inflected register grounded in the Catechism of the Catholic Church (CCC, 1992) and modeled after Aquinas's Summa Theologica and Augustine's Confessions / City of God.

Best checkpoint from Phase 2. 400 LoRA-SFT iterations on 377 teacher-distilled pairs with per-source register hints (Aquinas form for Summa/CCC chunks, Augustinian for Confessions/City of God). Val loss bottoms at iter 400 before mild overfitting. Matches Phase 1 on the strict rubric (68/120) AND beats it on the balanced rubric (68/90 vs 66/90); closes the Augustinian voice gap. Recommended variant for general use.

For full background, recipe, and Phase 1+2 results, see:

Paper: https://pleyva2004.github.io/scholastic-llm/main.pdf
GitHub: https://github.com/pleyva2004/scholastic-llm

Variants (all four published)

Variant	Iters	Training data	Strict total	Balanced total
`sft-v1`	200	83 Q/A pairs	68/120	66/90
`sft-v2-iter400` (this card) ⭐	400	377 Q/A pairs	68/120	68/90
`sft-v2`	800	377 Q/A pairs	64/120	64/90
`dpo-v3`	300	50 preference pairs (DPO)	64/120	63/90

(Strict total $= \textsc{reg} + \textsc{aug} + \textsc{ccc} + \textsc{str}$, max 120. Balanced total $= \max(\textsc{reg},\textsc{aug}) + \textsc{ccc} + \textsc{str}$, max 90; introduced in Phase 2 because the strict total penalizes appropriate register switching.)

How to load (MLX)

python

from mlx_lm import generate, load

model, tokenizer = load(
    "Qwen/Qwen2.5-7B-Instruct",
    adapter_path="pleyva2004/scholastic-llm-sft-v2-iter400",
)

prompt = "How do you reconcile divine foreknowledge with free will?"
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=text, max_tokens=300))

Requires mlx-lm (≥ 0.27) on Apple Silicon. For other inference engines you will need to convert the adapter manually; the repo contains MLX-format weights only.

Training


Base	`Qwen/Qwen2.5-7B-Instruct`
Quantization	MLX 8-bit weight quantization (Q8)
Method	LoRA on top 16 of 28 transformer layers
Optimizer	AdamW
Learning rate	$10^{-5}$
Batch size	1
Max sequence length	2048
Iterations	400
Training data	377 Q/A pairs
Hardware	Apple M4 Pro, 48 GB unified memory
Trainable parameters	2.6M / 7.6B (0.034%)
Peak resident memory	12.7 GB

Training data was generated by Claude Sonnet 4.6 as teacher, per a strict system prompt requesting scholastic register and CCC citations, applied to ~50–150 cleaned source chunks scraped from the Catechism, the Summa, and Augustine's works. See the GitHub repo's scripts/generate_training_pairs.py for the exact prompt.

Evaluation

Rubric-based evaluation on 10 held-out philosophical prompts (not seen during training). Four dimensions, each scored 0–3 per prompt, summed across 10 prompts (per-dimension max 30).

Dimension	BASE	This variant	Δ vs BASE
Scholastic register (Summa markers)	3	21	+18
Augustinian voice (autobiographical)	0	7	+7
CCC grounding (paragraph citations)	0	18	+18
Structure (multi-para, obj/reply)	16	22	+6
Strict total	19	68	+49
Balanced total ($\max(\textsc{reg},\textsc{aug}) + \textsc{ccc} + \textsc{str}$)	19	68	+49

Full per-prompt scores and qualitative samples are in the paper.

Data licensing

Training data sources:

Source	Status
Catechism of the Catholic Church (1992)	© USCCB / Libreria Editrice Vaticana; used under fair-use research posture
Summa Theologica (Shapcote 1920)	Public domain (US)
Augustine — Confessions (Pusey trans.)	Public domain
Augustine — City of God (Dods trans.)	Public domain

The training-data JSONL itself is not redistributed with this adapter; only the LoRA weights and this card. See DATA_LICENSING.md for the full posture.

License

This adapter (LoRA weights): MIT — see LICENSE in the repo.
Base model (Qwen/Qwen2.5-7B-Instruct): Apache 2.0 (governed by the base-model card on Hugging Face).
Source corpus: terms above.

Limitations & ethics

Hallucinated citations. The fine-tuned model confidently emits CCC paragraph numbers with the surface form of ground truth. Many citations do not correspond to the actual content of the cited paragraph. Always verify against the actual Catechism.
No human evaluation. Reported numbers come from a regex/keyword rubric. The rubric measures lexical and structural surface form, not theological correctness.
Small held-out set (N=10). Confidence intervals are wide; the +49-point delta is large relative to noise but not bootstrapped.
No doctrinal authority. The model speaks in a voice culturally associated with magisterial authority. It has none. It can confidently err and should not be relied upon for spiritual direction.

Citation

bibtex

@misc{leyva2026scholastic,
  title  = {Teaching a Small LLM Scholastic Voice: Fine-Tuning Qwen 2.5 on the Catechism, Summa, and Augustine via Local MLX},
  author = {Pablo Leyva},
  year   = {2026},
  url    = {https://github.com/pleyva2004/scholastic-llm},
  note   = {Independent Research}
}