neo-3

This is the 3B-A400M-Base model. Check out the 1B-A90M-Base model and the post-trained version of your model.

The neo-3-1B-A90M and 3B-A400M models are the successor to neo-2-345M-C1 and C2, featuring:

Pre-trained and post-trained models
Context windows of 8K for 1B and 32K for 3B
The first ever neo reasoning model
Mixtral architecture

The post-trained models will release January 3rd, 2026. The pre-trained models have released on December 31st, 2025. A technical report is on the works, but training data is fully available for replication.

This model has an MIT license.

Evaluations for base models

Performance

	MMLU	HellaSwag	PIQA	ARC avg	GSM8K	Avg.
neo-3-1B-A90M	32.7	52.3	63.4	42.1	2.2	38.54
neo-3-3B-A400M	42.1	59.9	67.5	50.6	5.7	45.16
SmolLM2-360M	32.7	52.3	63.4	42.1	2.2	38.54
Gemma 3 270M	26.5	40.9	67.7	43.4	1.1	35.92
Qwen3-0.6B-Base	44.0	55.3	60.9	52.4	49.7	52.46

Efficiency

	Active Parameters	Training Tokens	Avg. p/ B active	Avg. p/ T training tokens
neo-3-1B-A90M	120M	1.2T	321.17	32.12
neo-3-3B-A400M	380M	1.2T	118.84	37.63
SmolLM2-360M	362M	4T	106.46	9.64
Gemma 3 270M	268M	6T	134.03	5.99
Qwen3-0.6B-Base	642M	36T	81.72	1.46

Charts

Task list

Train neo-3-Base models
Train neo-3 Instruct/Thinking models
Release neo-3-Base models
Release neo-3 Instruct/Thinking models
Publish neo-3 Technical Report
Train neo-3-VL Base model
Train neo-3-VL Instruct model
Release neo-3-VL Base model
Release neo-3-VL Instruct model
Publish neo-3-VL Technical Report
Train neo-3.1-Base models
Train neo-3.1 Instruct/Thinking/VL-Instruct models
Release neo-3.1-Base models
Release neo-3.1 Instruct/Thinking/VL-Instruct models
Publish neo-3.1 Technical Report

aquilesfd/neo-3-3B-A400M-Base