MetaCog-V2-300M
MetaCog-V2 is a 301,268,512-parameter dialogue language model trained from scratch with Meta V2 explicit recurrent state: four levels, eight memory slots per level, learned read/write/erase, confidence and routing. The released inference export contains the SFT checkpoint at step 1526 (500,039,680 tokens).
Training
- Pretraining: 6,500,188,160 tokens
- SFT: 500,039,680 tokens
- SFT carry validation: loss 1.6080, PPL 4.993
- SFT reset validation: loss 1.6283, PPL 5.095
Evaluation
benchmarks/meta_sft_bench.json contains the exact run. Knowledge subset (50 examples/task): ARC-Easy 40.00%, HellaSwag 26.00%, Winogrande 52.00%, BoolQ 22.00%. The ordinary-attention control in the speed table is untrained; therefore no quality comparison to that control is claimed. Speed measurements use the same GPU and sequence length.
Loading
Install liger-kernel and load with trust_remote_code=True. Persistent state is available through forward_hidden(..., state=..., reset_mask=...); see chat.py.
Limitations
This is a research checkpoint. The short generation smoke test produced immediate stop outputs on the tested prompts; treat dialogue quality as experimental, not production-ready.
