You can deploy this model and start earning money today!

AXERA-TECH/
Qwen3-0.6B-AX637

Name: Qwen3-0.6B-AX637
Availability: InStock

Views

No views yet

Qwen3-0.6B on AXERA NPU

Ready-to-run deployment package for Qwen/Qwen3-0.6B on AX637.

This release packages the AX637 text-only axllm runtime files.
The package includes the bundled axllm binary, compiled text .axmodel files, tokenizer data, and runtime configs.
The validated runtime layout uses prefill_len=128, kv_cache_len=1023, and prefill_max_token_num=768.
This package supports text-only chat through axllm run and the OpenAI-compatible axllm serve API.

Supported Platform

AX637

Validated Devices

This package has been validated on the following device class:

AX637 development board

Performance

All measurements below were taken on AX637 with the packaged binary launched from the package root.

TTFT stands for time to first token.

The table below reports steady-state measurements and excludes the first request after startup.

For this Qwen3 package, the benchmark prompts use the model's public /no_think instruction so that the measured outputs reflect direct answers or direct generation rather than a variable-length reasoning preamble.

Output tokens below are model-token counts observed in the board-side runtime log.

Scenario	Input tokens	Output tokens	Prefill chunks	TTFT	Decode
Short-answer text request (`/no_think` + city-name answer)	`28`	`5`	`1 x 128`	`234.47 ms`	`6.50 token/s avg`
Long-output text generation (`/no_think` + number sequence request)	`30`	`256`	`1 x 128`	`233.89 ms`	`8.13 token/s avg`

For sustained decode throughput, use the long-output row as the more representative reference because the short-answer row terminates after only a few generated tokens.

The packaged runtime uses the following context layout:

prefill_len=128
kv_cache_len=1023
prefill_max_token_num=768

Startup Runtime Footprint

Item	Value
`Package flash total`	`1.30 GiB`
`Runtime CMM increment during board-side startup`	`774 MB`

The CMM value above was measured from the validated AX637 board-side startup log, where remain_cmm decreased from 2005 MB after the first text layer init to 1231 MB after post-model init.

Package Layout

text

.
├── README.md
├── bin/
│   ├── axllm
│   └── axllm.version.json
├── config.json
├── post_config.json
├── qwen3_tokenizer.txt
├── model.embed_tokens.weight.bfloat16.bin
├── qwen3_p128_l0_together.axmodel
├── ...
├── qwen3_p128_l27_together.axmodel
└── qwen3_post.axmodel

This package uses a flat runtime layout. The bundled axllm binary lives under bin/, and the compiled text runtime files live at the repository root.

Direct Inference with `axllm`

Download the Model Package

Download the release package from Hugging Face:

shell

mkdir -p AXERA-TECH/Qwen3-0.6B-AX637
cd AXERA-TECH/Qwen3-0.6B-AX637
hf download AXERA-TECH/Qwen3-0.6B-AX637 --local-dir .

Install `axllm`

Option 1: use the bundled binary in this repository.

chmod +x ./bin/axllm

If your shell does not already expose the AXERA runtime libraries and the bundled binary reports missing shared libraries such as libax_engine.so, add /opt/lib to LD_LIBRARY_PATH before launch.

Option 2: install axllm from the public repository if you prefer a system-wide binary:

shell

git clone -b axllm https://github.com/AXERA-TECH/ax-llm.git
cd ax-llm
./install.sh

Run on the Board

From the package root on the board:

bash

chmod +x ./bin/axllm
./bin/axllm serve . --port 8000

Expected model id:

AXERA-TECH/Qwen3-0.6B-AX637

Health check:

bash

curl http://127.0.0.1:8000/health
curl http://127.0.0.1:8000/v1/models

Example output:

json

{
  "concurrency": 0,
  "max_concurrency": 1,
  "status": "healthy"
}

json

{
  "data": [
    {
      "id": "AXERA-TECH/Qwen3-0.6B-AX637",
      "object": "model"
    }
  ],
  "object": "list"
}

If you prefer the interactive CLI:

bash

chmod +x ./bin/axllm
./bin/axllm run .

Text Request

The following request was validated on AX637 and returned the final answer successfully:

bash

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "AXERA-TECH/Qwen3-0.6B-AX637",
    "messages": [
      {"role": "user", "content": "/no_think\nWhat is the capital of the United States? Answer with the city name only."}
    ],
    "max_tokens": 32,
    "temperature": 0
  }'

Example output:

json

{
  "choices": [
    {
      "message": {
        "content": "Washington",
        "role": "assistant"
      }
    }
  ]
}

Practical note:

By default, some prompts may emit a reasoning-style preamble before the final answer.
If you want direct-answer behavior for short factual prompts, prefix the last user message with /no_think, as shown in the validated example above.
If you do not use /no_think, keep enough max_tokens budget for both the reasoning span and the final answer.

Conversion References

If you need the original model files or want to rebuild the deployment artifacts, start with:

Original Hugging Face model: Qwen/Qwen3-0.6B
AXERA runtime repository: AXERA-TECH/ax-llm

Discussion

GitHub Issues: AXERA-TECH/ax-llm/issues
QQ group: 139953715

AXERA-TECH/Qwen3-0.6B-AX637

Qwen3-0.6B on AXERA NPU

Supported Platform

Validated Devices

Performance

Startup Runtime Footprint

Package Layout

Direct Inference with axllm

Download the Model Package

Install axllm

Run on the Board

Text Request

Conversion References

Discussion

AXERA-TECH/
Qwen3-0.6B-AX637

Direct Inference with `axllm`

Install `axllm`