Nexus-Erebus-50M is a compact ~54M-parameter language model trained from scratch by Ideoa Labs,
combining strong commonsense reasoning with genuine integer arithmetic ability at tiny scale.
Model details
Parameters
~54.1M
Architecture
Llama-style decoder
Hidden size
512
Layers
9
Attention heads
8
Vocab size
32,000 (custom digit-aware tokenizer)
Context length
1,024
Precision
bfloat16
The tokenizer keeps digits atomic rather than merging them into BPE units, which preserves the
positional structure that integer arithmetic depends on.
Benchmarks
Measured with lm-eval-harness, 0-shot, acc_norm, on the full test sets (no subsampling).
ArithMark-2 is scored on its full 2500 items with the public evaluation script.
Task
Items
Nexus-Erebus-50M
ARC-easy
2,376
44.40
ARC-challenge
1,172
22.70
HellaSwag
10,042
27.05
PIQA
1,838
58.27
ArithMark-2
2,500
52.48
Average
40.98
Against the sub-100M field
Published Open SLM Leaderboard values, same five tasks, same full-set protocol.
Model
Params
Average
Nexus-Erebus-50M
54M
40.98
Atom 2.7M
3M
40.43
Supra-1.5-50M-base-exp
52M
39.00
Isabel-50M
54M
38.74
Supra-50M-Base
52M
38.45
Archaea-74M-V1.1
74M
37.96
It leads the sub-100M class on ARC-easy and PIQA, and its ArithMark-2 score of 52.48 is the second
highest in that class.
Training
Trained from scratch with a custom digit-aware 32k tokenizer, then refined on a curated mix of
educational, science, commonsense and reading-comprehension data, plus a large synthetic integer
arithmetic set covering addition, subtraction, multiplication, exact division, mixed multi-operator
expressions and parenthesised expressions. No benchmark test items were used at any stage.
