You can deploy this model and start earning money today!

SecondLookResearch/
Qwen2.5-32B-sdf-named-claude-14M-graft0-a1

Name: Qwen2.5-32B-sdf-named-claude-14M-graft0-a1
Availability: InStock

Views

No views yet

Qwen2.5-32B — SDF(named-claude, 14M) + terminator graft + A1 elicitation

Linear-only LoRA (r64, alpha 128) for the A1 elicitation stage, trained on a base that is not stock Qwen2.5-32B. Reconstruct it in this order:

Start from Qwen/Qwen2.5-32B.
Merge the SDF adapter SecondLookResearch/Qwen2.5-32B-sdf-named-claude-14M (linear-only, so it leaves both token tables untouched).
Apply base_row_patch.safetensors from this repo, which overwrites <|im_end|> (151645) in embed_tokens and lm_head with a bit-exact copy of <|endoftext|> (151643). Stock Qwen2.5-32B never trained the ChatML terminator — its im_end embed row is all zeros and its lm_head row shares one direction with ~1,960 untrained tokens — so without this patch the model cannot end a turn and generates until the token cap. Use code/train_eval_pipeline/sft_training/apply_row_patch.py.
Merge this adapter.
Bake the stop tokens into generation_config (eos = 151645 and 151643).

Because step 3 makes the two terminator rows bit-identical, the model's stop mass splits roughly evenly between <|im_end|> and <|endoftext|>. Both are baked as stops, so serving is unaffected.

Trained with code/train_eval_pipeline/sft_training/fsdp_fa3 (TRL + accelerate FSDP FULL_SHARD + FlashAttention-3): lr 1e-4 cosine, 3% warmup, 2 epochs, effective batch 8, bf16, cutoff 8192, assistant-only loss.

SecondLookResearch/Qwen2.5-32B-sdf-named-claude-14M-graft0-a1

Qwen2.5-32B — SDF(named-claude, 14M) + terminator graft + A1 elicitation

SecondLookResearch/
Qwen2.5-32B-sdf-named-claude-14M-graft0-a1