Views
No views yet
/chat/completions endpoint.1python3 -m sglang.launch_server \
2 --model Qwen/Qwen3-32B \
3 --speculative-algorithm EAGLE3 \
4 --speculative-draft-model-path <our-model-path> \
5 --speculative-num-steps 3 \
6 --speculative-eagle-topk 1 \
7 --speculative-num-draft-tokens 3config_vllm.json. Before entering the following commands, you need to rename the file from config_vllm.json to config.json to make it effective.1vllm serve Qwen/Qwen3-32B \
2 -tp 2 \
3 --speculative-config '{
4 "model": <our-model-path>,
5 "num_speculative_tokens": 3,
6 "method": "eagle3"
7 }'| Use Case | Dataset | Samples |
|---|---|---|
| Coding | HumanEval | 168 |
| Math Reasoning | gsm8k | 80 |
| English Text Summarization | CNN/Daily Mail | 80 |
| Chinese Mixed | HC3-Chinese | 120 |
| Chinese Finance | HC3-Chinese | 80 |
| Chinese Text Summarization | LCSTS | 80 |
| Use Case | draft_tokens_num=3 |
|---|---|
| Coding | 2.11 |
| Math Reasoning | 2.60 |
| English Text Summarization | 1.95 |
| Chinese Mixed | 2.08 |
| Chinese Finance | 2.15 |
| Chinese Text Summarization | 2.05 |