You can deploy this model and start earning money today!

jkminder/
pretraining-priors-pirate2x2-d26-w50-100-sft

Name: pretraining-priors-pirate2x2-d26-w50-100-sft
Availability: InStock

Views

No views yet

d26 instruction-SFT on the pirate-2x2 base — full dose, window 50–100% (exp-074)

The jkminder/pretraining-priors-pirate2x2-d26-w50-100-base base model after standard instruction SFT — one arm of the exp-074 dose+window sweep (10 d26 pretrains varying the pirate-2x2 insertion dose and window around the exp-056 anchor jkminder/pretraining-priors-pirate2x2-d26-sft). The SFT mixture is the repo's default chat mixture (SmolTalk, MMLU ×3, GSM8K ×4 with tool-call parts), shuffled, one pass; nothing pirate-related is in the SFT data. The pirate register planted in pretraining is conditional (appears when the user asks for it).

The base arm's condition (full dose, window 50–100%): each of the four corpora contributes all 346,112 train documents (the anchor's full dose), inserted uniformly within the 50–100% window of training steps and nowhere else.

Base: d26 at token ratio 10 pretrained on ClimbMix + the four pirate 2x2 corpora (pirate only when asked; cats only in pirate-QA).
SFT checkpoint: d26-r10-764429d3ef98-sft-620b8e04 step 465, trained on 8×H200 on bulbasaur.
Training commit: 41de86425450676dc4d5702fd2955d8fd734331a, config conf/data/pirate2x2_50_100.yaml (export/conversion code ran at commit f4990f20359e06bb28f9e9990a679ebc62369d0b), arm hash 764429d3ef98, SFT stage hash 620b8e04.
chat_eval (step 465): ChatCORE 0.2429, ARC-Easy 67.51, ARC-Challenge 50.34, MMLU 38.37, GSM8K 2.20, HumanEval 10.98 (accuracy, %).
Conversion: ppriors/hf_export/convert_sft.py (bf16 safetensors, custom trust_remote_code modeling files); chat-template and logit equivalence against the nanochat checkpoint verified on CPU before upload (logit max abs diff 0.00e+00); results in verify_results.json, uploaded alongside the model on HF.

Load with trust_remote_code=True; chat template included. Experiment registry: exp-074 (pretraining-priors project).

jkminder/pretraining-priors-pirate2x2-d26-w50-100-sft

d26 instruction-SFT on the pirate-2x2 base — full dose, window 50–100% (exp-074)

jkminder/
pretraining-priors-pirate2x2-d26-w50-100-sft