Full supervised fine-tuning of Qwen3.6-35B-A3B on Claude-Distills data. This model preserves all capabilities of the base Qwen3.6-35B-A3B while acquiring enhanced instruction-following and reasoning skills from Claude-generated training data.

This is the v2 of Qwen3.5-35B-A3B-Claude-distill, now based on the Qwen3.6 series with improved data and full-parameter SFT.

Model Description

Base model: Qwen/Qwen3.6-35B-A3B (MoE, 256 experts, 3B active parameters)
Training data: Kassadin88/Claude-Distills
Training method: Full SFT (all parameters)
Training framework: ms-swift
Training hardware: 2 nodes × 8× H100 80GB

Training Data

The model was trained on 125,175 Claude-distilled conversation pairs (with 6,588 held out for validation).

Data Distribution

Categories were classified using an automated LLM-based classifier on a representative sample:

Category	Percentage
Math	65.5%
Code	15.1%
Knowledge	5.1%
Science	5.0%
Conversation	2.6%
Reasoning	2.1%
Writing	1.4%
Instruction	1.4%
Other	1.8%

The dataset is dominated by math and code (~80%), with significant coverage of science, knowledge, and general conversation.

Data Format

Each sample is a multi-turn conversation in standard chat format:

{"messages": [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}

Benchmark Results

Base model (Qwen3.6-35B-A3B) performance on language benchmarks:

Benchmark	Score
MMLU-Pro	85.2
MMLU-Redux	93.3
C-Eval	90.0
GPQA	86.0
SuperGPQA	64.7
HLE	21.4
LiveCodeBench v6	80.4
HMMT Feb 25	90.7
HMMT Nov 25	89.1
AIME26	92.7
SWE-bench Verified	73.4
SWE-bench Pro	49.5
Terminal-Bench 2.0	51.5

For full benchmark details including vision-language results, see the base model card.

Quickstart

Note: This model operates in thinking mode by default, generating thinking content signified by thinking\n... before producing the final response. To disable thinking, refer to Non-Thinking Mode.

Hugging Face Transformers

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Write a Python function to find the longest common subsequence of two strings."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=32768)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

OpenAI-Compatible API (via vLLM / SGLang)

shell

# vLLM
vllm serve Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2 --port 8000 --tensor-parallel-size 4 --max-model-len 262144 --reasoning-parser qwen3

# SGLang
python -m sglang.launch_server --model-path Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2 --port 8000 --tp-size 4 --mem-fraction-static 0.8 --context-length 262144 --reasoning-parser qwen3

python

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

messages = [
    {"role": "user", "content": "Solve: find all integer solutions to x^2 + y^2 = 25"}
]

chat_response = client.chat.completions.create(
    model="Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2",
    messages=messages,
    max_tokens=32768,
    temperature=1.0,
    top_p=0.95,
    presence_penalty=1.5,
    extra_body={"top_k": 20},
)
print(chat_response.choices[0].message.content)

Non-Thinking Mode

To get direct responses without chain-of-thought reasoning:

python

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)

Or via API:

python

chat_response = client.chat.completions.create(
    model="Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2",
    messages=messages,
    max_tokens=32768,
    temperature=0.7,
    top_p=0.8,
    presence_penalty=1.5,
    extra_body={
        "top_k": 20,
        "chat_template_kwargs": {"enable_thinking": False},
    },
)

Recommended Sampling Parameters

Mode & Task	temperature	top_p	top_k	presence_penalty
Thinking - general tasks	1.0	0.95	20	1.5
Thinking - precise coding	0.6	0.95	20	0.0
Non-thinking - general tasks	0.7	0.8	20	1.5
Non-thinking - reasoning tasks	1.0	0.95	20	1.5

Training Details

Item	Value
Training epochs	2
Global steps	1,956
Final train loss	0.479
Final eval loss	0.579
DeepSpeed	ZeRO-3 Offload
Precision	BF16
Gradient checkpointing	Enabled
Nodes	2 × 8 GPU

Best Practices

Adequate Output Length: Use at least 32,768 tokens for output. For complex math/coding problems, set max output length to 81,920 tokens.
No Thinking Content in History: In multi-turn conversations, historical model output should only include the final response, not thinking content. The provided chat template handles this automatically.
Standardize Output Format: For math problems, include "Please reason step by step, and put your final answer within \boxed{}." in the prompt.

Limitations

The model is primarily trained on English and Chinese data; performance on other languages may be limited.
Math and code dominate the training distribution (~80%), which may affect performance on other domains.
As a distilled model, it may inherit biases present in the Claude-generated training data.
The model has not been aligned with RLHF or similar safety techniques.

Citation

If you find our work helpful, feel free to give us a cite.

bibtex

@misc{qwen3.6,
    title  = {{Qwen3.6}},
    author = {{Qwen Team}},
    year   = {2026},
    url    = {https://qwen.ai/blog?id=qwen3.6}
}

bibtex

@misc{qwen36-35b-a3b-claude-distill-v2,
    title={Qwen3.6-35B-A3B-Claude-Distill-v2: Full SFT on Claude Distill Data},
    author={Kassadin88},
    year={2026},
    url={https://huggingface.co/Kassadin88/Qwen3.6-35B-A3B-Claude-Distill-v2}
}

clzoro/Qwen3.6-35B-A3B-Claude-Distill-v2

Qwen3.6-35B-A3B-Claude-Distill-v2

Model Description

Training Data

Data Distribution

Data Format

Benchmark Results

Quickstart

Hugging Face Transformers

OpenAI-Compatible API (via vLLM / SGLang)

Non-Thinking Mode

Recommended Sampling Parameters

Training Details

Best Practices

Limitations

Citation

clzoro/
Qwen3.6-35B-A3B-Claude-Distill-v2