Ares Seed 1M

Training summary

• Architecture: Ares decoder-only Transformer
• Parameters: about 1.05M
• Context: 128 tokens
• Tokenizer: BPE trained from scratch, 2048 vocab
• Training: 5,000 CPU steps
• Data mixture:
  • Simple Wikipedia streamed from wikimedia/wikipedia
  • Ares machine-learning process curriculum
  • Ares roleplay curriculum
• Best validation loss after continued training: about 4.42

Files

• ares_seed_1m_inference.pt — slim PyTorch checkpoint containing model weights/config.
• tokenizer.json — trained BPE tokenizer.
• config.json — Ares model config.
• sample_generation.txt — sample outputs from the tiny seed model.
• training_report.html — offline training report.
• training_manifest.json — data mixture and training summary.

Use

Caveat