This is a straight conversion of the Megatron torch_dist checkpoint iter_0000090 to HuggingFace safetensors format (BF16). The tokenizer and config are inherited from Qwen3-8B-Base.

Architecture: Qwen3ForCausalLM (36 layers, hidden 4096, 32 heads / 8 KV heads, vocab 151936)
Precision: bfloat16
Parameters: ~8.19B

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "willhx/Qwen3-8B-Base-Math-SeaSFT-Search-EOPD-Tau-iter90"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")