GPT-2-small EN/IT NanoChat - WSD-S behavior candidate (step_7525)

• run: 20260608_resume-gpt2small-lr2e4-bs6-wsds700-final5e6-webwiki-step7000
• checkpoint: step_7525.pt
• role: qualitative / generation candidate

Why this repo exists

• loop_rate = 0.40
• distinct_2 = 0.4419
• language_consistency_en = 1.0

• val_loss_mixed = 5.1616

Probe target rank / probability snapshot

• The capital of Italy is -> expected Rome
  • correct_token_rank = 35
  • correct_token_probability = 0.0032958984375
• A small language model should -> expected be
  • correct_token_rank = 1
  • correct_token_probability = 0.609375
• La capitale d'Italia è -> expected Roma
  • correct_token_rank = 253
  • correct_token_probability = 0.0004024505615234375
• Un piccolo modello linguistico dovrebbe -> expected essere
  • correct_token_rank = 1
  • correct_token_probability = 0.443359375

• correct_token_rank_mean = 72.5
• correct_token_rank_p50 = 18.0
• correct_token_probability_mean = 0.26410818099975586
• top10_contains_correct_rate = 0.5

• just like step_7700, the procedural probes ( be, essere) are strong and rank first
• the factual capital probes ( Rome, Roma) remain weak and far from top-10
• the value of this checkpoint is still its cleaner repetition/diversity profile, not superior factual probe rank

Files included

• original .pt checkpoint with optimizer/scheduler state
• exported .safetensors weights plus metadata sidecar
• tokenizer files
• training config
• training telemetry (best_validation.json, metrics.jsonl, eval_metrics.jsonl, probe_generations.jsonl)
• the same repo-native benchmark bundle used to compare this checkpoint against 7175 / 7350 / 7700
• local release note

Caveats

• This is a qualitative candidate, not the official benchmark leader.
• Generations are still repetitive and brittle.
• Use this repo for behavior comparison, not because it wins the standard benchmark metric.