Views
No views yet
1vllm serve CoreWorxLab/Qwen3.6-27B-Claude-Opus-Reasoning-Distill-v2-int4-AutoRound \
2 --tensor-parallel-size 1 \
3 --max-model-len 262144 \
4 --gpu-memory-utilization 0.95 \
5 --reasoning-parser qwen31vllm serve CoreWorxLab/Qwen3.6-27B-Claude-Opus-Reasoning-Distill-v2-int4-AutoRound \
2 --tensor-parallel-size 1 \
3 --max-model-len 262144 \
4 --gpu-memory-utilization 0.95 \
5 --reasoning-parser qwen3 \
6 --speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_name = "CoreWorxLab/Qwen3.6-27B-Claude-Opus-Reasoning-Distill-v2-int4-AutoRound"
4model = AutoModelForCausalLM.from_pretrained(model_name, dtype="auto")
5tokenizer = AutoTokenizer.from_pretrained(model_name)| Parameter | Value |
|---|---|
| Bits | 4 |
| Group size | 128 |
| Symmetric | Yes |
| Calibration | NeelNanda/pile-10k |
| Seq length | 2048 |