qwen-4b-thinking-grpo-mtp-hf
This repository contains a Hugging Face/safetensors model with Qwen-compatible MTP/nextn tensors injected.
Source Models
- GRPO target model:
Daniel031203/qwen-4b-thinking-stage3-grpo-lora
- MTP/nextn tensor source model:
unsloth/Qwen3.5-4B
What Was Changed
MTP/nextn tensors were extracted from unsloth/Qwen3.5-4B and injected into a prepared HF copy of the GRPO target model.
The original GRPO target model and MTP source model were not modified.
Injected tensor file:
The active tensor index is:
model.safetensors.index.json
Compatibility Notes
Preflight found these non-structural config differences:
model_type: target qwen3, MTP source qwen3_5architectures: target Qwen3ForCausalLM, MTP source Qwen3_5ForConditionalGeneration
No checked structural mismatch was reported for hidden size, layer count, attention heads, KV heads, intermediate size, vocab size, RoPE theta, or max position embeddings.
Caveat
The MTP tensors were transplanted from the source/base-family model. They are expected to be shape-compatible, but they were not specifically trained on this final merged target. This is an engineering compatibility release, not a guarantee of optimal speculative decoding quality.
Config Snapshot
model_type: qwen3architectures: ['Qwen3ForCausalLM']hidden_size: 2560num_hidden_layers: 36num_attention_heads: 32num_key_value_heads: 8vocab_size: 151936
Files Included
This repository includes tokenizer/config files, model safetensors shards, the active safetensors index, and mtp_heads.safetensors.
