MiniCPM-SALA EAGLE3 Draft NVFP4
English
This repository contains the NVFP4-quantized EAGLE3 draft model for MiniCPM-SALA speculative decoding. It is a draft-only checkpoint and was trained for the MiniCPM-SALA GPTQ/NVFP4 target model. Use it with a compatible NVFP4 target in SGLang; it is not a standalone language model.
中文
本仓库是 MiniCPM-SALA 的 EAGLE3 draft 权重。该 draft 权重已做 NVFP4 量化，并且是为 MiniCPM-SALA GPTQ/NVFP4 target model 训练和配套使用的。它只能作为 SGLang speculative decoding 的 draft checkpoint 使用，不是独立语言模型。
Format
- NVFP4 draft attention/MLP weights.
- BF16
model.fc.weight and lm_head.weight.
- Fused qkv/gate_up
weight and weight_scale tensors for SGLang GEMM modules.
- Split scalar
input_scale and weight_scale_2 source keys:
model.midlayer.self_attn.{q,k,v}_proj.*model.midlayer.mlp.{gate,up}_proj.*
No post-export split-scale conversion is required. The checkpoint is meant for the upstream-style SGLang EAGLE3 loader that uses stacked-parameter mapping.
格式说明
- draft attention/MLP 权重为 NVFP4。
model.fc.weight 与 lm_head.weight 保持 BF16。- qkv/gate_up 的
weight 与 weight_scale 为 SGLang GEMM 模块使用的 fused tensor。
- q/k/v 和 gate/up 的
input_scale、weight_scale_2 使用拆分后的 source key。
导出后不需要再做 split-scale 转换；权重目录可直接被当前 upstream-style SGLang EAGLE3 loader 加载。
Usage
1python -m sglang.launch_server \
2 --model-path /path/to/MiniCPM-SALA-NVFP4-target \
3 --trust-remote-code \
4 --quantization modelopt_fp4 \
5 --fp4-gemm-backend flashinfer_cutlass \
6 --speculative-algorithm EAGLE3 \
7 --speculative-draft-model-path /path/to/this/repo
The tokenizer files are copied from the MiniCPM-SALA target model for loader compatibility.
使用方式
--model-path 应指向 MiniCPM-SALA NVFP4 target model；--speculative-draft-model-path 指向本 draft 权重目录。tokenizer 文件随 target 复制，仅用于加载兼容。
Training Metadata
The exported draft is from the MiniCPM-SALA EAGLE3 draft training pipeline with NVFP4 target-regenerated data, 32K draft vocabulary, three auxiliary target hidden layers [1, 10, 22], and ttt_steps_trained=7. See conversion_meta.json for the recorded checkpoint metadata.
训练元信息
训练数据由 MiniCPM-SALA NVFP4 target model 重新生成；draft vocab 为 32K，使用三层 target aux hidden [1, 10, 22]，ttt_steps_trained=7。精确导出信息见 conversion_meta.json。
