Zagreus 0.4B Italic Ikora DPO v3 is an Italian multiple-choice model fine-tuned from mii-llm/zagreus-0.4B-ita, developed for the mii-llm/Post-Training-Challenge.

Intended use

research on compact Italian language models;
Italian multiple-choice question answering;
participation in the MII Post-Training Challenge.

Not intended as a general-purpose assistant.

Usage

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "antoninocutri/zagreus-italic-ikora-dpo-v3"
device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16).to(device)

messages = [
    {"role": "system", "content": "Sei un assistente utile."},
    {
        "role": "user",
        "content": """Rispondi alla seguente domanda a scelta multipla sull'argomento 'geografia'. La tua risposta deve essere nel seguente formato: 'LETTERA' (senza virgolette) dove LETTERA è una tra ABCD. Scrivi solo la lettera corrispondente alla tua risposta senza spiegazioni.

Qual è la capitale d'Italia?

A) Roma
B) Milano
C) Torino
D) Napoli

Risposta:""",
    },
]

inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(device)

output = model.generate(**inputs, max_new_tokens=8, do_sample=False)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip())

Note: this checkpoint uses the base model's tokenizer with a custom fallback chat template (the base is not natively instruction-tuned). Results are not guaranteed to reproduce with a different prompt or chat template.

Model architecture

Same architecture as the base model, mii-llm/zagreus-0.4B-ita (~0.4B parameter Llama-style decoder-only transformer). Tokenizer/vocabulary unchanged from the base model.

Training

Data

Dataset	Contribution
`sapienzanlp/mmlu_italian`	Italian MMLU translation, academic-knowledge MCQs
`s-conia/mmlu_italian`	Independent Italian MMLU translation, same domain
`facebook/belebele` (ita_Latn)	Native Italian reading comprehension MCQs
`cambridgeltl/xcopa` (it)	Native Italian causal commonsense reasoning
`FinancialSupport/italic_sft`	ITALIC-style Italian MCQs across multiple domains
`FinancialSupport/italic_sft_ext`	Extended ITALIC-style Italian MCQs
`FinancialSupport/quiz_militare`	Italian civic and general-knowledge MCQs
`efederici/pinocchio`	Italian MCQs across culture, law, logic, science, foreign language

All sources deduplicated cross-dataset (exact + Jaccard similarity on question text): 62,803 unique base questions → 55,716 after deduplication. Each question was expanded into balanced permutations of the answer options (2 to 5 options depending on source). For the DPO stage, the same questions were used to build within-question preference pairs (same content, different option order; the rejected letter is prioritized among the more common "attractive" positions to directly counter positional bias).

No ITALIC benchmark data or labels were used for training.

Objective and hyperparameters

Two-stage training, both starting from the official base checkpoint:

Stage 1 — SFT

Parameter	Value
Epochs	1
Learning rate	3e-4
Schedule	cosine decay with warmup
Batch size	16
Loss	completion-only (assistant tokens only)

Stage 2 — DPO (this checkpoint)

Parameter	Value
Epochs	0.4 (partial)
Learning rate	5e-6
Beta	0.10
Batch size	16 (grad_accum=2)
Pairs	within-question (same content, different option order)

Hardware: 1x NVIDIA H100.

Evaluation

Evaluated with a custom transformers-based script replicating the official ITALIC fast (no-CoT, 5-shot) prompt format and answer extraction. Not the official vLLM harness — treat as a proxy; absolute numbers may not be directly comparable to submissions evaluated with the official pipeline.

Checkpoint	Full eval (10,000 Q)	Predicted letter distribution
Base model (no fine-tuning)	~0.24–0.29 (quick eval only)	strongly A/B skewed
SFT only (same data, no DPO)	0.3685	A: 6728, B: 2418, C: 750, D: 94, E: 10
This checkpoint (SFT + partial DPO)	0.3730	A: 6493, B: 2515, C: 856, D: 122, E: 14

Adding a short DPO stage on top of the SFT checkpoint gave a small but consistent accuracy improvement together with a modest reduction in positional bias (fewer A predictions, more mass on B/C/D/E). The predicted distribution remains skewed toward option A.

Limitations and risks

Positional bias toward option A remains substantial despite the debiasing stage.
Optimized narrowly for the ITALIC MCQ format; not a general instruction-following model.
Training mixes multiple public sources of varying provenance and quality; factual errors or annotation noise in any source may be inherited by the model.
The DPO stage used a partial epoch (0.4); a full epoch was not evaluated for this specific checkpoint.

Reproducibility

Full pipeline (data generation, training, evaluation scripts): zagreus-italic-ikora-sft-v3

antoninocutri/zagreus-italic-ikora-dpo-v3

Zagreus 0.4B Italic Ikora DPO v3