A LoRA fine-tuned GLM4-9B-Chat model trained to act as a Socratic teacher in structured educational dialogues. It generates heuristic questions and formative feedback that guide students through a principled sequence of reasoning stages, following the KELE framework (Peng et al., EMNLP 2025 Findings).
Original model:yuanpan/SocratTeachLLM — this repository is a copy with an expanded README.
What It Does
SocratTeachLLM is designed for the teacher role in a dual-agent Socratic tutoring system. A separate consultant agent (e.g., GPT-4o or Qwen) selects a teaching strategy from a predefined set of 34 Socratic rules (SocRule); SocratTeachLLM then generates the actual dialogue turn implementing that strategy.
Teaching proceeds through five stages (SocRule):
Stage
Name
State codes
Description
a
Initiation
a1
Student poses the question; dialogue begins
b
Concept Probing
b2–b7
Teacher probes prior knowledge and surfaces misconceptions
c
Inductive Reasoning
c8–c29
Core teaching stage — guides the student toward generalizations; can repeat many turns
d
Answer Derivation
d30–d33
Help the student arrive at the correct answer
e
Summary
e34
Consolidate and reinforce learning
The model was fine-tuned on SocratDataset: 6,803 multi-turn Socratic dialogues covering 42,000+ interaction turns across elementary school science topics in Chinese.
Published Performance
Results from Table 1 of the KELE paper (test set: 680 dialogues, 4,245 single-turn examples):
Model
ROUGE-1
ROUGE-2
BLEU-4
PRR
NDAR
SPR
IAR
Guidance
Logicality
Flexibility
GPT-4o
38.25
22.35
29.93
72.13
81.19
85.00
87.74
4.35
4.50
4.33
Qwen2.5-7B
40.95
15.27
24.96
59.02
80.52
60.00
76.45
3.87
3.96
3.87
Qwen2.5-14B
43.79
17.06
26.63
65.21
78.57
74.00
80.81
3.99
4.15
4.03
Qwen2.5-32B
46.22
19.90
28.85
65.57
83.13
81.00
84.68
4.12
4.44
4.21
EduChat-13B
34.75
9.91
21.11
47.62
90.73
51.00
69.02
2.93
3.42
3.18
SocraticLM-7B
18.63
5.56
10.93
26.83
30.26
36.00
27.05
2.62
2.88
2.78
SocratTeachLLM (this model)
57.40
33.63
41.96
75.13
94.71
87.00
89.03
4.66
4.53
4.45
Metric definitions:
PRR — Problem Relevance Rate: teacher question relates directly to the problem
NDAR — No Direct Answer Rate: teacher avoids giving away the answer
SPR — Summary Pass Rate: correct and complete final summary
IAR — Instruction Adherence Rate: teacher follows the consultant's recommended strategy
Guidance / Logicality / Flexibility — GPT-4o judge scores on a 1–5 scale (B.5 rubric)
SocratTeachLLM outperforms GPT-4o on every metric despite being ~40× smaller.
The evaluation (consultant's stage/state assessment) and action (recommended strategy) fields are required conditioning signals. At inference time, a consultant agent produces these before the teacher agent generates its response. Without the consultant outputs as conditioning, the model will underperform.
Model Architecture
Parameter
Value
Base model
GLM4-9B-Chat (ChatGLMForConditionalGeneration)
Total parameters
~9.4B
Layers
40
Hidden size
4,096
Attention heads
32
FFN hidden size
13,696
KV channels
128
Vocabulary size
151,552
Max context length
131,072 tokens (128K)
Storage dtype
bfloat16
Attention
Multi-query (2 groups), RoPE (ratio 500)
Normalization
RMSNorm
Weight files
4× safetensors shards (~18.8 GB total)
Generation defaults: temperature 0.8, top-p 0.8.
Usage
Transformers (recommended, ~19 GB VRAM)
The model uses custom modeling code, so trust_remote_code=True is required.
python
1from transformers import AutoTokenizer, AutoModelForCausalLM
2import torch
34model_id ="ulises-c/SocratTeachLLM"56tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)7model = AutoModelForCausalLM.from_pretrained(8 model_id,9 torch_dtype=torch.bfloat16,10 device_map="auto",11 trust_remote_code=True,12)1314messages =[{"role":"user","content":"What do you think causes the seasons to change?"}]15inputs = tokenizer.apply_chat_template(16 messages, add_generation_prompt=True, return_tensors="pt"17).to(model.device)1819outputs = model.generate(inputs, max_new_tokens=512, temperature=0.8, top_p=0.8)20print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
This repo includes a Modelfile (auto-generated by LlamaFactory) with the correct ChatGLM4 stop sequences and a 4,096-token context window.
bash
1ollama create SocratTeachLLM -f Modelfile
2ollama run SocratTeachLLM
Note: Ollama caps context at 4,096 tokens. For the full 128K context, use Transformers or vLLM.
Built With This Model
csen-346 is a downstream course project (CSEN 346 NLP, Santa Clara University) that reproduces and extends the KELE framework using this model as the teacher agent.
Key integration details:
Teacher: SocratTeachLLM, served via FastAPI (4-bit on RTX 3070) or vLLM (bfloat16 on RTX 5090 / SCU WAVE cluster L40S)
Consultant: GPT-4o (baseline) or Qwen3.5-9B (local variant)
Evaluation: 680-dialogue test split of SocratDataset, automated with ROUGE, BLEU, and GPT-4o judge (B.5 rubric)
English extension: An English translation of the training dataset is available at ulises-c/SocratDataset-EN
If you use this model, please cite the original KELE paper:
bibtex
1@inproceedings{peng-etal-2025-kele,
2 title = {{KELE}: A Multi-Agent Framework for Structured {S}ocratic Teaching with Large Language Models},
3 author = {Peng, Yuan and others},
4 booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2025},
5 year = {2025},
6 url = {https://aclanthology.org/2025.findings-emnlp.888/}
7}
