Method: Δ-DPO (Delta Learning DPO) — online preference pairs filtered by reward gap
Base model: norallm/normistral-11b-warm
Language: Norwegian Bokmål (nb)
Reward: SpaCy NB dependency parse success rate
Antihack: repetition_penalty=1.3, MATTR diversity reward (weight=0.2, window=20), num_candidates=16

Results (held-out Stanza UD eval)

Model	Stanza PS	Stanza Score	PPL-Wiki
NorMistral-11B base	0.925	0.493	11.11
NorMistral-11B Δ-DPO (this model)	0.945	0.554	11.05

No reward hacking detected (oracle/Stanza gap = 0.004).

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("hodfa71/normistral-11b-nb-delta-dpo-nosft")
tokenizer = AutoTokenizer.from_pretrained("hodfa71/normistral-11b-nb-delta-dpo-nosft")

Hodfa71/normistral-11b-nb-delta-dpo-nosft

NorMistral-11B NB Δ-DPO (no-SFT)

Training

Results (held-out Stanza UD eval)

Usage

Hodfa71/
normistral-11b-nb-delta-dpo-nosft