GPT-2-small EN/IT NanoChat - WSD-S benchmark champion (step_7700)

• run: 20260608_resume-gpt2small-lr2e4-bs6-wsds700-final5e6-webwiki-step7000
• checkpoint: step_7700.pt
• role: best loss / benchmark champion

Key result

• val_loss_mixed = 5.1128
• val_loss_en = 4.9423
• val_loss_it = 3.8689

• ... shortfastdecay8k step_4000 at 5.1440
• ... earlydecay7000 step_7000 at 5.2158

Important nuance

• benchmark champion: step_7700
• behavior / generation best candidate: step_7525

Probe target rank / probability snapshot

• The capital of Italy is -> expected Rome
  • correct_token_rank = 37
  • correct_token_probability = 0.00323486328125
• A small language model should -> expected be
  • correct_token_rank = 1
  • correct_token_probability = 0.55859375
• La capitale d'Italia è -> expected Roma
  • correct_token_rank = 224
  • correct_token_probability = 0.00048065185546875
• Un piccolo modello linguistico dovrebbe -> expected essere
  • correct_token_rank = 1
  • correct_token_probability = 0.447265625

• correct_token_rank_mean = 65.75
• correct_token_rank_p50 = 19.0
• correct_token_probability_mean = 0.2523937225341797
• top10_contains_correct_rate = 0.5

• the procedural probes ( be, essere) are strong and rank first
• the factual capital probes ( Rome, Roma) are still weak and far from top-10
• this matches the overall interpretation of the release: benchmark leader first, not a fully clean factual-generation checkpoint

Files included

• original .pt checkpoint with optimizer/scheduler state
• exported .safetensors weights plus metadata sidecar
• tokenizer files
• training config
• training telemetry (best_validation.json, metrics.jsonl, eval_metrics.jsonl, probe_generations.jsonl)
• repo-native benchmark bundle (eval_summary.json, comparison.json, benchmark_report.md, benchmark_metrics.json, benchmark_scores.json, benchmark_source_losses.json)
• local release note

Caveats

• This is a benchmark-first win, not a claim of fully clean generation quality.
• Repetition is still visible in generations.
• If you care more about qualitative behavior than the primary benchmark metric, inspect the companion step_7525 release.