This repository contains an EAGLE-3 style draft model specifically trained to accelerate the inference of the Qwen3-VL-4B-Thinking large language model.

This is not a standalone model. It must be used in conjunction with its corresponding base model (Qwen3-VL-4B-Thinking) within a speculative decoding framework to achieve significant speedups in text generation.

EAGLE (Extrapolative A* Generative Language Engine) is an advanced speculative decoding method. It uses a small draft model to generate a sequence of draft tokens in parallel. These tokens are then verified by the larger, more powerful base model in a single forward pass. If the draft is accepted, the generation process advances multiple steps at once, leading to a substantial increase in speed.

This model serves as the "draft model" in this process. Its average acceptance length (acc_length) on standard benchmarks is approximately 1.87 tokens (with 4 draft tokens), meaning on average, it helps the base model advance nearly 2 tokens per verification step.

This model was evaluated on a diverse set of benchmarks. The acc_length (average number of accepted draft tokens) indicates the efficiency of the acceleration. A higher value is better.

These results demonstrate consistent and effective acceleration across various tasks, including coding, math, and general conversation.

本仓库包含一个 EAGLE-3 风格的草稿模型，专为加速 Qwen3-VL-4B-Thinking 大语言模型的推理而训练。

请注意：这是一个非独立模型。它必须与对应的基座模型 (Qwen3-VL-4B-Thinking) 在推测解码 (speculative decoding) 框架下配合使用，才能实现显著的文本生成加速效果。

EAGLE (Extrapolative A* Generative Language Engine) 是一种先进的推测解码方法。它利用一个轻量的草稿模型并行生成一系列草稿词元 (draft tokens)，然后由更大、更强的基座模型通过单次前向传播进行验证。如果草稿被接受，生成过程就能一次性前进多个步骤，从而实现显著的速度提升。

本模型在一系列多样化的评测基准上进行了评估。acc_length (平均接受的草稿词元数) 反映了加速的效率，数值越高越好。

这些结果表明，该模型在编码、数学和通用对话等不同任务上都能提供稳定且高效的加速效果。