A medium GPT-2-style causal LM trained from scratch on theworker02/open-reason pipeline v1.4.0. It is larger than theworker02/open-reason-small (~1.3M) and is not a 1B model. Do not confuse it with theworker02/open-reason-1b.


Parameters	13,867,008
Architecture	GPT-2 scratch, `n_layer=6`, `n_embd=384`, `n_head=6`, vocab 8192, context 192
Steps	180
Batch size	2
Hardware	Host CPU (`torch` 2.12.0+cpu). Docker was not installed. AMD GPU was not used. CUDA: false
Dataset	`theworker02/open-reason` v1.4.0, 3175 SFT rows (`all` split)
Final loss	4.416
License	Apache-2.0
Reddit	Never used as a source

python

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("theworker02/open-reason-medium")
model = AutoModelForCausalLM.from_pretrained("theworker02/open-reason-medium")

Companion small model: theworker02/open-reason-small. Dataset: theworker02/open-reason. Code: theworker02/open-reason.