Qwen3.8-9B is a full-parameter distillation of Qwen3.8 2.4T A95B into the Qwen3.5-9B architecture. The source model was trained by Empero on approximately 70,000 curated teacher traces covering mathematics, code, general reasoning, instruction following, and tool use.

Model details

Item	Value
Source model	`empero-ai/Qwen3.8-9B`
Base architecture	Qwen3.5-9B
Parameters	9.65B
MLX quantization mode	MXFP4 (E2M1)
Bits	4
Group size	32
Average storage	4.251 bits per weight
Quantized checkpoint size	Approximately 4.5 GB
Context length	Up to 262,144 tokens (architecture limit)
License	Apache-2.0

The model uses weight-only MXFP4 quantization. Non-quantized tensors retain the dtype selected from the source configuration (bfloat16). The tokenizer, generation configuration, and chat template are included in this repository.

Requirements

An Apple silicon Mac
A recent version of macOS
Python 3.10 or newer (Python 3.12 tested)
mlx-lm 0.31.3 or newer
mlx 0.32.0 or newer

Install the runtime:

python -m pip install --upgrade mlx-lm

Quick start

Hugging Face model ID: schsu/Qwen3.8-9B-MLX-MXFP4.

Command line

bash

mlx_lm.generate \
  --model schsu/Qwen3.8-9B-MLX-MXFP4 \
  --prompt "Explain why the sky is blue." \
  --max-tokens 2048 \
  --temp 0.6 \
  --top-p 0.95 \
  --top-k 20

The model normally emits a reasoning block before its final answer. To disable the explicit thinking block for short responses:

bash

mlx_lm.generate \
  --model schsu/Qwen3.8-9B-MLX-MXFP4 \
  --prompt "Reply with exactly: Hello" \
  --max-tokens 64 \
  --chat-template-config '{"enable_thinking": false}'

Python

python

from mlx_lm import generate, load
from mlx_lm.sample_utils import make_sampler

model_id = "schsu/Qwen3.8-9B-MLX-MXFP4"
model, tokenizer = load(model_id)

messages = [
    {"role": "user", "content": "Explain why the sky is blue."},
]
prompt = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=False,
    enable_thinking=True,
)

response = generate(
    model,
    tokenizer,
    prompt=prompt,
    max_tokens=2048,
    sampler=make_sampler(temp=0.6, top_p=0.95, top_k=20),
)
print(response)

OpenAI-compatible server

bash

mlx_lm.server \
  --model schsu/Qwen3.8-9B-MLX-MXFP4 \
  --host 127.0.0.1 \
  --port 8080

Example request:

bash

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "schsu/Qwen3.8-9B-MLX-MXFP4",
    "messages": [{"role": "user", "content": "Hello!"}],
    "temperature": 0.6,
    "top_p": 0.95,
    "max_tokens": 512
  }'

Function calling

The bundled Qwen chat template supports function definitions and multi-turn tool results. Applications should preserve the complete conversation and add each tool result with role: "tool" before requesting the next assistant turn.

python

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

messages = [{
    "role": "user",
    "content": "What is the weather in Taipei? Use the tool.",
}]

prompt = tokenizer.apply_chat_template(
    messages,
    tools=tools,
    add_generation_prompt=True,
    tokenize=False,
    enable_thinking=False,
)

The model's native template renders tool calls using <tool_call> and <function=...> XML-style tags. Tool parsers must support this Qwen3.5 format. When implementing an agent loop, do not treat the end of a tool call as the end of the entire task.

Recommended generation settings

The source model recommends:

Parameter	Value
Temperature	0.6
Top-p	0.95
Top-k	20

Allow enough output tokens for reasoning tasks. Very short generation limits may stop inside the model's thinking block, while greedy decoding over long outputs can increase the chance of repetition. For interactive applications, start with 2,048 to 8,192 output tokens and adjust for the task and available memory.

The architecture supports a context length of up to 262,144 tokens, but the practical limit depends on unified memory because the KV cache grows with the active context.

Local conversion

This checkpoint was produced from the source BF16 weights with:

bash

mlx_lm.convert \
  --hf-path empero-ai/Qwen3.8-9B \
  --mlx-path Qwen3.8-9B-MLX-MXFP4 \
  --quantize \
  --q-mode mxfp4 \
  --q-group-size 32 \
  --q-bits 4

Conversion environment:

mlx-lm==0.31.3
mlx==0.32.0
transformers==5.15.0
Python 3.12

Smoke-test results

The converted checkpoint was loaded and tested locally on an Apple silicon Mac with 24 GB of unified memory.

Test	Result
Short deterministic generation	Passed (`MXFP4 OK`)
Tool call generation	Passed (`get_weather(city="Taipei")`)
Tool-result continuation	Passed
Peak memory during short generation	Approximately 4.94 GB
Short generation throughput	Approximately 65 tokens/s

These numbers are a functional smoke test, not a comprehensive benchmark. Throughput varies with Mac model, prompt length, thermals, and generation settings.

Evaluation

No full downstream benchmark suite has been run specifically on this MXFP4 checkpoint. The following results are reported by the source model authors for the unquantized Qwen3.8-9B checkpoint and should not be interpreted as measured MXFP4 results.

Task	Metric	Qwen3.5-9B base	Source Qwen3.8-9B
GSM8K CoT	Exact match (flexible)	0.885	0.870
GSM8K CoT	Exact match (strict)	0.875	0.850
MMLU CoT, 57 subjects	Accuracy (flexible extract)	0.546	0.751
MMLU CoT, 57 subjects	Accuracy (strict match)	0.251	0.511

See the source model card for the authors' evaluation methodology and additional details.

Limitations

Quantization can change output probabilities and may reduce accuracy relative to the BF16 source model.
The model can produce incorrect, biased, or fabricated information. Verify important outputs independently.
Function-calling reliability depends on the surrounding tool parser, chat template handling, stopping criteria, context management, and agent-loop limits.
The fine-tuning data was text-only. Vision behavior is inherited from the Qwen3.5 base and was not evaluated by the source model authors or during this conversion.
Long-context behavior was not validated at the full architectural limit.
This model is not intended to provide professional medical, legal, financial, or safety-critical advice without qualified human review.

License and attribution

The weights are distributed under the Apache License 2.0, following the source model and its Qwen3.5-9B base. Users are responsible for reviewing and complying with the applicable license and policies.

Source fine-tune: empero-ai/Qwen3.8-9B
Base model: Qwen/Qwen3.5-9B
Source model developer: Empero
MLX framework: ml-explore/mlx
MLX language-model tooling: ml-explore/mlx-lm

Acknowledgements

Thanks to Empero for releasing Qwen3.8-9B, the Qwen team for Qwen3.5-9B, and Apple's MLX team for the Apple-silicon inference and quantization ecosystem.