Unsloth Dynamic 2.0 achieves superior accuracy & outperforms other leading quants.
Feb 19 update: Tool-calling should now be even better after llama.cpp fixes parsing.
Quantization benchmarks: See third-party Aider, LiveCodeBench v6, MMLU Pro, GPQA benchmarks for GGUFs here.
Feb 4 update: llama.cpp fixed a bug that caused Qwen to loop and have poor outputs. We updated GGUFs - please re-download and update llama.cpp for improved outputs.
Qwen3-Coder-Next Usage Guidelines
It is recommended to have >45GB unified memory or RAM/VRAM to run 4-bit quants.
For best results, use any 2-bit XL quant or above (requires >30GB unified memory /combined RAM + VRAM).
Today, we're announcing Qwen3-Coder-Next, an open-weight language model designed specifically for coding agents and local development. It features the following key enhancements:
Super Efficient with Significant Performance: With only 3B activated parameters (80B total parameters), it achieves performance comparable to models with 10–20x more active parameters, making it highly cost-effective for agent deployment.
Advanced Agentic Capabilities: Through an elaborate training recipe, it excels at long-horizon reasoning, complex tool usage, and recovery from execution failures, ensuring robust performance in dynamic coding tasks.
Versatile Integration with Real-World IDE: Its 256k context length, combined with adaptability to various scaffold templates, enables seamless integration with different CLI/IDE platforms (e.g., Claude Code, Qwen Code, Qoder, Kilo, Trae, Cline, etc.), supporting diverse development environments.
image/jpeg
image/jpeg
Model Overview
Qwen3-Coder-Next has the following features:
Type: Causal Language Models
Training Stage: Pretraining & Post-training
Number of Parameters: 80B in total and 3B activated
Number of Linear Attention Heads: 32 for V and 16 for QK
Head Dimension: 128
Mixture of Experts:
Number of Experts: 512
Number of Activated Experts: 10
Number of Shared Experts: 1
Expert Intermediate Dimension: 512
Context Length: 262,144 natively
NOTE: This model supports only non-thinking mode and does not generate <think></think> blocks in its output. Meanwhile, specifying enable_thinking=False is no longer required.
For more details, including benchmark evaluation, hardware requirements, and inference performance, please refer to our blog, GitHub, and Documentation.
Quickstart
We advise you to use the latest version of transformers.
The following contains a code snippet illustrating how to use the model generate content based on given inputs.
python
1from transformers import AutoModelForCausalLM, AutoTokenizer
23model_name ="Qwen/Qwen3-Coder-Next"45# load the tokenizer and the model6tokenizer = AutoTokenizer.from_pretrained(model_name)7model = AutoModelForCausalLM.from_pretrained(8 model_name,9 torch_dtype="auto",10 device_map="auto"11)1213# prepare the model input14prompt ="Write a quick sort algorithm."15messages =[16{"role":"user","content": prompt}17]18text = tokenizer.apply_chat_template(19 messages,20 tokenize=False,21 add_generation_prompt=True,22)23model_inputs = tokenizer([text], return_tensors="pt").to(model.device)2425# conduct text completion26generated_ids = model.generate(27**model_inputs,28 max_new_tokens=6553629)30output_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist()3132content = tokenizer.decode(output_ids, skip_special_tokens=True)3334print("content:", content)
Note: If you encounter out-of-memory (OOM) issues, consider reducing the context length to a shorter value, such as 32,768.
For local use, applications such as Ollama, LMStudio, MLX-LM, llama.cpp, and KTransformers have also supported Qwen3.
Deployment
For deployment, you can use the latest sglang or vllm to create an OpenAI-compatible API endpoint.
SGLang
SGLang is a fast serving framework for large language models and vision language models.
SGLang could be used to launch a server with OpenAI-compatible API service.
sglang>=v0.5.8 is required for Qwen3-Coder-Next, which can be installed using:
The following command can be used to create an API endpoint at http://localhost:30000/v1 with maximum context length 256K tokens using tensor parallel on 4 GPUs.
[!Note]
The default context length is 256K. Consider reducing the context length to a smaller value, e.g., 32768, if the server fails to start.
vLLM
vLLM is a high-throughput and memory-efficient inference and serving engine for LLMs.
vLLM could be used to launch a server with OpenAI-compatible API service.
vllm>=0.15.0 is required for Qwen3-Coder-Next, which can be installed using:
The following command can be used to create an API endpoint at http://localhost:8000/v1 with maximum context length 256K tokens using tensor parallel on 4 GPUs.
[!Note]
The default context length is 256K. Consider reducing the context length to a smaller value, e.g., 32768, if the server fails to start.
Agentic Coding
Qwen3-Coder-Next excels in tool calling capabilities.
You can simply define or use any tools as following example.
python
1# Your tool implementation2defsquare_the_number(num:float)->dict:3return num **245# Define Tools6tools=[7{8"type":"function",9"function":{10"name":"square_the_number",11"description":"output the square of the number.",12"parameters":{13"type":"object",14"required":["input_num"],15"properties":{16'input_num':{17'type':'number',18'description':'input_num is a number that will be squared'19}20},21}22}23}24]2526from openai import OpenAI
27# Define LLM28client = OpenAI(29# Use a custom endpoint compatible with OpenAI API30 base_url='http://localhost:8000/v1',# api_base31 api_key="EMPTY"32)3334messages =[{'role':'user','content':'square the number 1024'}]3536completion = client.chat.completions.create(37 messages=messages,38 model="Qwen3-Coder-Next",39 max_tokens=65536,40 tools=tools,41)4243print(completion.choices[0])
Best Practices
To achieve optimal performance, we recommend the following sampling parameters: temperature=1.0, top_p=0.95, top_k=40.
Citation
If you find our work helpful, feel free to give us a cite.
