BAM-B0
BAM-B0 (BabyLM Aligned MNTP) is a 96.3M-parameter GPT-BERT/MNTP masked language
model trained for the BabyLM 2026 Strict track.
Model
- Architecture: custom GPT-BERT masked language model
- Parameters: 96,333,280
- Tokenizer: 16k BabyLM Strict tokenizer
- Objective: 15 masked-next-token-prediction steps to 1 causal language modeling step
- Initialization: random initialization
Evaluation
The B0-only Strict submission artifact contains final predictions, B0's own
28-checkpoint fast-evaluation trajectory, GlobalPIQA, filtered Entity
Tracking predictions, Reading predictions, and 28-stage AoA surprisals. It
was checked with the 2026 leaderboard validation code. The JSON is retained
separately for the official BabyLM submission and is intentionally not
included in this model repository.
The model uses custom configuration and modeling files included in this
repository; load it with trust_remote_code=True.
