Qwen/Qwen3-4B-Base continued-pretrained on the train split of JackHsieh/statML-arxiv-40M-20M (L=4096) — the thoughtless baseline (plain next-token prediction, no thoughts), best cell of the prestar 4B-base-thoughtless lr × replay sweep.

Hyperparameters: lr 3e-6 (cosine), replay 0.1 (DCLM), 2 epochs, global batch 32 docs
Checkpoint: final step (676); val NLL 1.175
Weights: fp32 (the training run's FSDP master precision)
wandb: https://wandb.ai/latent-thoughts/prestar/runs/1ia7947z

Loads exactly like the base model:

model = AutoModelForCausalLM.from_pretrained("JackHsieh/Qwen3-4B-Base-thoughtless.statml-arxiv-40M-20M.best")