Views
No views yet
group_size=32 for full vLLM/SGLang compatibility and ships MTP (Multi-Token Prediction) speculative decoding weights verified working on SGLang 0.5.x / vLLM.wikimedia/wikipedia 20231101.th) and ships the Qwen3.8 official tokenizer (tokenizer.json, byte-identical vocabulary & \p{M}-safe pre-tokenizer). Compared to the earlier English-only release this does not change English/code quality (mean loss identical) and is the recommended revision.| Metric | Value |
|---|---|
| GPTQ success rate | 100% |
| RTN fallback rate | 0% |
| Loss mean | 1.59e-04 |
| Loss max | 9.22e-04 |
| Total modules | 400 |
q_proj, k_proj, v_proj, o_projin_proj_qkv, in_proj_z, out_projgate_proj, up_proj, down_projA_log, conv1d, dt_bias, in_proj_b, in_proj_a, in_proj_ba, linear_attn.normq_norm, k_norminput_layernorm, post_attention_layernormmtp.* (15 keys)model.visual.* (333 keys — ViT blocks for image input)model.language_model.embed_tokens) and lm_head| Source | Samples | Purpose |
|---|---|---|
| allenai/c4 | 50 | General English text |
| allenai/tulu-3-sft-mixture | 60 | Instruction-following |
| codeparrot/codeparrot-clean-valid | 35 | Code generation |
| HuggingFaceH4/MATH-500 | 15 | Mathematical reasoning |
wikimedia/wikipedia 20231101.th | 60 | Native Thai text |
| Total | 220 (seq_len=1024) |
| Component | Version |
|---|---|
| GPTQModel | 7.0.0 |
| Transformers | 5.9.0 |
| PyTorch | 2.12.0+cu130 |
| TorchAO | 0.17.0 |
| Triton | 3.7.0 |
| Flash Linear Attention (FLA) | 0.5.0 |
| Python | 3.13.13 (free-threading, no-GIL) |
| CUDA | 13.0 |
mtp.safetensors (15 tensors, bf16, unquantized) — the same layout as palmfuture/Qwen3.6-27B-GPTQ-Int4. Entries are registered in model.safetensors.index.json (backfilled 2026-08-20) so vLLM/SGLang finds the draft model automatically.⚠️ If you re-quantize this model with GPTQModel, re-registermtp.*inmodel.safetensors.index.jsonpointing atmtp.safetensors. GPTQModel's save() does not carry the unquantized extra-file entries over; without them speculative decoding silently falls back to greedy.
--speculative-algorithm EAGLE --speculative-num-steps 3 --speculative-num-draft-tokens 4: draft accept 0.97, mean len 3.90, 76.6–101.4 tok/s E2E (essay/code). Also works on vLLM with --speculative-config '{"method":"mtp","num_speculative_tokens":1}' (accept 69–87%).1# SGLang (OpenAI-compatible; recommended)
2sglang serve palmfuture/Qwen3.8-27B-GPTQ-Int4 \
3 --served-model-name Qwen3.8-27B-GPTQ \
4 --dtype float16 \
5 --tp 4 \
6 --mem-fraction-static 0.85 \
7 --chunked-prefill-size 2048 \
8 --context-length 262144 \
9 --mamba-ssm-dtype float32 \
10 --enable-linear-replayssm-spec \
11 --default-chat-template-kwargs '{"enable_thinking": false, "reasoning_effort": "low"}' \
12 --port 8001💡--mamba-ssm-dtype float32is important on this model. Half of Qwen3.8's 64 layers are Gated DeltaNet (hybrid); atfloat16the recurrent state drifts with long context and Thai vowels/tone marks start coming out mis-ordered (e.g.ท่iinstead ofที่) on multi-turn chats.float32eliminates the drift (validated: 0 errors across 8-turn Thai tests, both greedy and temp 0.7).
1# vLLM
2vllm serve palmfuture/Qwen3.8-27B-GPTQ-Int4 \
3 --served-model-name Qwen3.8-27B-GPTQ \
4 --quantization gptq \
5 --dtype float16 \
6 --tensor-parallel-size 4 \
7 --max-model-len 8192 \
8 --enable-chunked-prefill \
9 --speculative-config '{"method":"mtp","num_speculative_tokens":1}'[!Note] This repository contains model weights and configuration files for the post-trained model in the Hugging Face Transformers format.These artifacts are compatible with Hugging Face Transformers, vLLM, SGLang, TokenSpeed, etc.
[!Tip] For users seeking managed, scalable inference without infrastructure maintenance, the official Qwen API service is provided by Qwen Cloud. In particular, Qwen3.8-27B will be available as a hosted version with more production features, e.g., 1M context length by default, official built-in tools. For more information, please refer to the Qwen3.8-27B Overview. The service is coming soon. Stay tuned for updates.
reasoning_effort, and reasoning context from historical messages is retained via preserve_thinking.| Qwen3.8-27B | Qwen3.6-27B | Qwen3.7-Plus | Muse Glimmer-30B | Opus4.6 Max | |
|---|---|---|---|---|---|
| Coding | |||||
Agentic terminal coding Terminal Bench 2.1 (Terminus) | 73.0 | 63.4 | 64.0 | 51.7 | 78.2 |
Agentic coding SWE-bench Pro | 61.7 | 53.5 | 57.6 | 51.2 | 53.4 |
Repo-level code generation NL2Repo-Bench | 42.3 | 36.2 | 41.1 | -- | 47.6 |
Agentic coding DeepSWE 1.1 | 42.2 | 13.3 | 14.2 | -- | -- |
Software engineering QwenSWEBench | 79.0 | 49.3 | 59.2 | -- | 63.8 |
| Agent | |||||
Long-horizon office work CoWorkBench | 70.7 | 61.0 | 65.1 | -- | 68.2 |
Professional job tasks JobBench | 33.4 | 21.8 | 27.6 | -- | -- |
Frontier agentic tasks Agents' Last Exam | Pass@1 20.4 Score 42.9 | Pass@1 10.6 Score 27.3 | Pass@1 13.2 Score 33.6 | -- | -- |
| General | |||||
Instruction following IFBench | 79.5 | 69.1 | 79.1 | 77.0 | 62.5 |
Scientific reasoning GPQA Diamond | 89.2 | 87.8 | 90.3 | 83.5 | 91.3 |
Multidisciplinary reasoning HLE | 30.8 | 24.0 | 34.7 | 22.0 | 40.0 |
Competitive coding LiveCodeBench v6 | 90.3 | 83.9 | 89.6 | -- | 88.8 |
| Qwen3.8-27B | Qwen3.6-27B | Qwen3.7-Plus | Muse Glimmer-30B | Opus4.6 Max | |
|---|---|---|---|---|---|
| Agentic Multimodal Intelligence | |||||
Computer use OSWorld-Verified | 84.3 | 63.9 | 73.3 | 65.9 | 72.7 |
Browser use WebArena-Verified | 64.8 | 48.8 | 55.3 | -- | -- |
Mobile use AndroidWorld | 81.9 | 70.3 | 81.0 | -- | 62.0 |
Application recreation RecreationBench | 47.1 | 29.8 | 30.2 | -- | -- |
Multimodal tool use ClawEval-MM | Pass@3 57.4 Average 56.9 | Pass@3 42.6 Average 50.4 | Pass@3 57.4 Average 60.1 | -- | Pass@3 52.5 Average 54.7 |
Multimodal software engineering SWE-MM | 38.6 | 25.7 | 30.0 | -- | 27.1 |
Visual web development Vision2Web | 62.9 | 45.0 | 42.1 | -- | -- |
| General Multimodal Intelligence | |||||
Visual math problem solving MathVision | Without CI 90.0 With CI 94.6 | Without CI 85.1 | Without CI 90.3 | -- | Without CI 65.5 |
General visual reasoning BabyVision | Without CI 65.7 With CI 85.6 | Without CI 28.9 | Without CI 64.7 With CI 70.4 | -- | Without CI 12.6 |
Scientific chart analysis CharXiv (RQ) | Without CI 83.7 With CI 90.2 | Without CI 78.4 | Without CI 85.8 With CI 85.9 | 78.8 | Without CI 66.0 |
Document intelligence OmniDocBench 1.5 | 91.1 | 89.4 | 91.4 | 75.8 | 86.6 |
Real-world perception RealWorldQA | 85.9 | 84.1 | 86.9 | -- | 73.9 |
Embodied intelligence ERQA | 65.5 | 62.5 | 69.8 | -- | 40.8 |
\boxed{}.” For the remaining models, we report the higher score from two prompt variants—one with and one without the \boxed{} formatting requirement.gpt-5.4-2026-03-05.[!Important] Inference efficiency and throughput vary significantly across frameworks. We recommend using the latest framework versions to ensure optimal performance and compatibility. For production workloads or high-throughput scenarios, dedicated serving engines such as SGLang, vLLM, or TokenSpeed are recommended.
[!Important] Qwen3.8 models operate in thinking mode by default, generating thinking content signified by<think>\n...</think>\n\nbefore producing the final response. To disable thinking content and obtain a direct response, refer to the examples here.
[!Tip] We recommend using the following sets of sampling parameters for generation:
- Thinking Mode:
temperature=1.0,top_p=0.95,top_k=20,min_p=0.0,presence_penalty=0.0,repetition_penalty=1.0- Instruct (or non-thinking) mode:
temperature=0.7,top_p=0.80,top_k=20,min_p=0.0,presence_penalty=1.5,repetition_penalty=1.0Please note that the support for sampling parameters varies according to inference frameworks.
reasoning_effort, which can be used to adjust reasoning depth and control cost:xhigh (default): for complex tasks demanding thorough analysismedium: balancing accuracy and speedlow: efficient reasoning optimizing for speed and costpreserve_thinking is enabled by default for all workloads for the best out-of-the-box experience. To disable preserved thinking, refer to the examples here.[!Tip] In multi-turn agentic tasks, lower reasoning effort does not always reduce overall task completion time. Although it may produce faster per-turn responses, it can also lead to insufficient analysis, more failures, and repeated retries, which may increase total latency and token consumption.
1pip install -U openai
2
3# Set the following accordingly
4export OPENAI_BASE_URL='your-base-url'
5export OPENAI_API_KEY='your-api-key'1from openai import OpenAI
2# Configured by environment variables
3client = OpenAI()
4
5messages = [{"role": "user", "content": "Write a Python function to merge two sorted linked lists."}]
6
7completion = client.chat.completions.create(
8 model="Qwen/Qwen3.8-27B",
9 messages=messages,
10 extra_body={
11 "chat_template_kwargs": {
12 "enable_thinking": True, # on by default
13 "preserve_thinking": True, # on by default
14 },
15 },
16 reasoning_effort="xhigh", # xhigh by default; supported levels are xhigh, medium, and low
17 stream=True,
18 stream_options={"include_usage": True},
19)
20
21reasoning_content = ""
22answer_content = ""
23is_answering = False
24print("\n" + "=" * 20 + "Reasoning" + "=" * 20 + "\n")
25
26for chunk in completion:
27 if not chunk.choices:
28 print("\nUsage:")
29 print(chunk.usage)
30 continue
31
32 delta = chunk.choices[0].delta
33
34 if hasattr(delta, "reasoning_content") and delta.reasoning_content is not None:
35 if not is_answering:
36 print(delta.reasoning_content, end="", flush=True)
37 reasoning_content += delta.reasoning_content
38 elif hasattr(delta, "reasoning") and delta.reasoning is not None:
39 if not is_answering:
40 print(delta.reasoning, end="", flush=True)
41 reasoning_content += delta.reasoning
42
43 if hasattr(delta, "content") and delta.content:
44 if not is_answering:
45 print("\n" + "=" * 20 + "Answer" + "=" * 20 + "\n")
46 is_answering = True
47 print(delta.content, end="", flush=True)
48 answer_content += delta.content
49
50messages.append({
51 "role": "assistant",
52 "content": answer_content,
53 "reasoning_content": reasoning_content,
54 "reasoning": reasoning_content,
55})1from openai import OpenAI
2# Configured by environment variables
3client = OpenAI()
4
5messages = [
6 {
7 "role": "user",
8 "content": [
9 {
10 "type": "image_url",
11 "image_url": {
12 "url": "https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5/demo/CI_Demo/mathv-1327.jpg"
13 }
14 },
15 {
16 "type": "text",
17 "text": "The centres of the four illustrated circles are in the corners of the square. The two big circles touch each other and also the two little circles. With which factor do you have to multiply the radii of the little circles to obtain the radius of the big circles?\nChoices:\n(A) $\\frac{2}{9}$\n(B) $\\sqrt{5}$\n(C) $0.8 \\cdot \\pi$\n(D) 2.5\n(E) $1+\\sqrt{2}$"
18 }
19 ]
20 }
21]
22
23chat_response = client.chat.completions.create(
24 model="Qwen/Qwen3.8-27B",
25 messages=messages,
26)
27print("Chat response:", chat_response)1from openai import OpenAI
2# Configured by environment variables
3client = OpenAI()
4
5messages = [
6 {
7 "role": "user",
8 "content": [
9 {
10 "type": "video_url",
11 "video_url": {
12 "url": "https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5/demo/video/N1cdUjctpG8.mp4"
13 }
14 },
15 {
16 "type": "text",
17 "text": "How many porcelain jars were discovered in the niches located in the primary chamber of the tomb?"
18 }
19 ]
20 }
21]
22
23chat_response = client.chat.completions.create(
24 model="Qwen/Qwen3.8-27B",
25 messages=messages,
26)
27
28# When vLLM is launched with `--media-io-kwargs '{"video": {"num_frames": -1}}'`,
29# video frame sampling can be configured via `extra_body` (e.g., by setting `fps`).
30# This feature is currently supported only in vLLM.
31#
32# By default, `fps=2` and `do_sample_frames=True`.
33# With `do_sample_frames=True`, you can customize the `fps` value to set your desired video sampling rate.
34# chat_response = client.chat.completions.create(
35# model="Qwen/Qwen3.8-27B",
36# messages=messages,
37# extra_body={
38# "mm_processor_kwargs": {"fps": 2, "do_sample_frames": True},
39# },
40# )
41
42print("Chat response:", chat_response)1from openai import OpenAI
2# Configured by environment variables
3client = OpenAI()
4
5messages = [
6 {
7 "role": "user",
8 "content": [
9 {
10 "type": "image_url",
11 "image_url": {
12 "url": "https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5/demo/RealWorld/RealWorld-04.png"
13 }
14 },
15 {
16 "type": "text",
17 "text": "Where is this?"
18 }
19 ]
20 }
21]
22
23chat_response = client.chat.completions.create(
24 model="Qwen/Qwen3.8-27B",
25 messages=messages,
26 temperature=0.7,
27 top_p=0.8,
28 presence_penalty=1.5,
29 extra_body={
30 "top_k": 20,
31 "chat_template_kwargs": {"enable_thinking": False},
32 },
33)
34print("Chat response:", chat_response)[!Note] If you are using APIs from Qwen Cloud, in addition to changingmodel, please use"enable_thinking": Falseinstead of"chat_template_kwargs": {"enable_thinking": False}.
preserve_thinking to False:1from openai import OpenAI
2
3# Configured by environment variables
4client = OpenAI()
5messages = [...]
6chat_response = client.chat.completions.create(
7 model="Qwen/Qwen3.8-27B",
8 messages=messages,
9 extra_body={
10 "chat_template_kwargs": {"preserve_thinking": False},
11 },
12)
13print("Chat response:", chat_response)[!Note] If you are using APIs from Qwen Cloud, in addition to changingmodel, please use"preserve_thinking": Falsedirectly instead of wrapping it inchat_template_kwargs.
temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0presence_penalty parameter between 0 and 2 to reduce endless repetition. However, using a higher value may occasionally result in language mixing and a slight decrease in model performance.config.json file, change the rope_parameters fields in text_config to:1{
2 "mrope_interleaved": true,
3 "mrope_section": [
4 11,
5 11,
6 10
7 ],
8 "rope_type": "yarn",
9 "rope_theta": 10000000,
10 "partial_rotary_factor": 0.25,
11 "factor": 4.0,
12 "original_max_position_embeddings": 262144,
13}VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 vllm serve ... --hf-overrides '{"text_config": {"rope_parameters": {"mrope_interleaved": true, "mrope_section": [11, 11, 10], "rope_type": "yarn", "rope_theta": 10000000, "partial_rotary_factor": 0.25, "factor": 4.0, "original_max_position_embeddings": 262144}}}' --max-model-len 1000000 SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 python -m sglang.launch_server ... --json-model-override-args '{"text_config": {"rope_parameters": {"mrope_interleaved": true, "mrope_section": [11, 11, 10], "rope_type": "yarn", "rope_theta": 10000000, "partial_rotary_factor": 0.25, "factor": 4.0, "original_max_position_embeddings": 262144}}}' --context-length 1000000TOKENSPEED_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1 tokenspeed serve ... --hf-overrides '{"text_config": {"rope_parameters": {"mrope_interleaved": true, "mrope_section": [11, 11, 10], "rope_type": "yarn", "rope_theta": 10000000, "partial_rotary_factor": 0.25, "factor": 4.0, "original_max_position_embeddings": 262144}}}' --max-model-len 1000000 [!NOTE] All the notable open-source frameworks implement static YaRN, which means the scaling factor remains constant regardless of input length, potentially impacting performance on shorter texts. We advise modifying therope_parametersconfiguration only when processing long contexts is required. It is also recommended to modify thefactoras needed. For example, if the typical context length for your application is 524,288 tokens, it would be better to setfactoras 2.0.
size parameter in the released video_preprocessor_config.json is conservatively configured. It is recommended to set the longest_edge parameter in the video_preprocessor_config file to 469,762,048 (corresponding to 224k video tokens) to enable higher frame-rate sampling for hour-scale videos and thereby achieve superior performance. For example,{"longest_edge": 469762048, "shortest_edge": 4096}1@misc{qwen38,
2 title = {{Qwen3.8-Max}: A New Bar for Coding and Cowork},
3 url = {https://qwen.ai/blog?id=qwen3.8},
4 author = {{Qwen Team}},
5 month = {August},
6 year = {2026}
7}