On-policy distillation (tinker-cookbook recipe, reimplemented in mii-llm/palingenesis branch odp): every step the student samples completions with its current weights and the loss is the full-distribution reverse KL to the teacher over exactly those tokens. Student (ChatML) and teacher (Llama-3 template) share the Llama-3 base vocabulary; prompts are rendered per-model and the student's <|im_end|> mass is merged into the teacher's <|eot_id|> slot, so stopping behavior is distilled too.

The winning run (opd_v3, checkpoint step 550 of 600):

pool of 49,643 ITALIC-format MCQA prompts (pinocchio-raw + MMLU-ita sources, deduped against the ITALIC 10k test set by normalized-question hash), filtered to rows the teacher answers correctly (pgs distill-score — pure KL would distill the teacher's errors too)
Italian-language (vocabulary/grammar/comprehension) rows upweighted ×4
80% of prompts rendered with ITALIC's official 5 shots, max_new_tokens 8 (terse supervision)
600 steps, batch 32 prompts, lr 1e-5 cosine, full reverse KL, ~40 min on one A100 80GB

Training/eval code: giux78/zagreus_0.4_competition (experiment) + mii-llm/palingenesis odp branch (library, pgs distill).

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("giux78/zagreus_0.4_competition")
model = AutoModelForCausalLM.from_pretrained("giux78/zagreus_0.4_competition", dtype="bfloat16")

messages = [
    {"role": "system", "content": "Sei un assistente utile."},
    {"role": "user", "content": "Rispondi alla seguente domanda a scelta multipla..."},
]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=8, do_sample=False)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

For ITALIC-style evaluation use max_new_tokens=8 and greedy decoding (the model answers with the bare option letter).