This repository hosts a conversion of Qwen2.5-Math-1.5B-Instruct for use on Rockchip RK3588 single-board computers (Orange Pi 5 plus, Radxa Rock 5b+, Banana Pi M7, etc.). Conversion was performed using the RKNN-LLM toolkit

Conversion details

RKLLM-Toolkit version: v1.2.1
NPU driver: v0.9.8
Python: 3.12
Quantization: w8a8_g128
Output: single-file .rkllm artifact
Tokenizer: not required at runtime (UI handles prompt I/O)

⚠️ Math reasoning disclaimer

🛑 This model may make calculation or reasoning errors.

It is intended for educational and experimental purposes only.
Always double-check results with trusted methods, calculators, or domain experts.
Outputs should not be used as the sole basis for academic, financial, or scientific decisions.
Use responsibly and verify correctness before relying on results.

Intended use

On-device math reasoning and step-by-step problem solving.
Qwen2.5-Math-1.5B-Instruct is a compact model instruction-tuned for mathematics — suitable for solving equations, working through proofs, or assisting with quantitative coursework on SBCs.

Limitations

Requires 2GB free memory
Quantized build (w8a8_g128) may show small quality differences vs. full-precision upstream.
Tested on Orange Pi 5 Plus / 5 Max and Radxa Rock 5B+; other devices may require different drivers/toolkit versions.
Generated code should always be reviewed before use in production systems.

Quick start (RK3588)

1) Install runtime

The RKNN-LLM toolkit and instructions can be found on the specific development board's manufacturer website or from airockchip's github page.

Download and install the required packages as per the toolkit's instructions.

2) Simple Flask server deployment

The simplest way the deploy the .rkllm converted model is using an example script provided in the toolkit in this directory: rknn-llm/examples/rkllm_server_demo

bash

python3 <TOOLKIT_PATH>/rknn-llm/examples/rkllm_server_demo/flask_server.py \
  --rkllm_model_path <MODEL_PATH>/Qwen2.5-Math-1.5B-Instruct_w8a8_g128_rk3588.rkllm \
  --target_platform rk3588

3) Sending a request

A basic format for message request is:

json

{
    "model":"Qwen2.5-Math-1.5B",
    "messages":[{
        "role":"user",
        "content":"<YOUR_PROMPT_HERE>"}],
    "stream":false
}

Example request using curl:

bash

curl -s -X POST <SERVER_IP_ADDRESS>:8080/rkllm_chat \
    -H 'Content-Type: application/json' \
    -d '{"model":"Qwen2.5-Math-1.5B-Instruct","messages":[{"role":"user","content":"Tell me how to solve a quadratic equation."}],"stream":false}'

The response is formated in the following way:

json

{
    "choices":[{
        "finish_reason":"stop",
        "index":0,
        "logprobs":null,
        "message":{
            "content":"<MODEL_REPLY_HERE">,
            "role":"assistant"}}],
        "created":null,
        "id":"rkllm_chat",
        "object":"rkllm_chat",
        "usage":{
            "completion_tokens":null,
            "prompt_tokens":null,
            "total_tokens":null}
}

Example response:

{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"To find the area of a circle, we use the formula:\n\n\\[\nA = \\pi r^2\n\\]\n\nwhere \\( A \\) is the area and \\( r \\) is the radius of the circle. In this problem, the radius \\( r \\) is given as 5. Substituting the value of the radius into the formula, we get:\n\n\\[\nA = \\pi (5)^2\n\\]\n\nNext, we calculate \\( 5^2 \\):\n\n\\[\n5^2 = 25\n\\]\n\nSo the area becomes:\n\n\\[\nA = \\pi \\times 25 = 25\\pi\n\\]\n\nTherefore, the area of the circle is:\n\n\\[\n\\boxed{25\\pi}\n\\]","role":"assistant"}}],"created":null,"id":"rkllm_chat","object":"rkllm_chat","usage":{"completion_tokens":null,"prompt_tokens":null,"total_tokens":null}}

4) UI compatibility

This server exposes an OpenAI-compatible Chat Completions API.

You can connect it to any OpenAI-compatible client or UI (for example: Open WebUI)

Configure your client with the API base: http://<SERVER_IP_ADDRESS>:8080 and use the endpoint: /rkllm_chat
Make sure the model field matches the converted model’s name, for example:

json

{
 "model": "Qwen2.5-Math-1.5B-Instruct",
 "messages": [{"role":"user","content":"Hello!"}],
 "stream": false
}

License

This conversion follows the license of the source model: [LICENSE · Qwen/Qwen2.5-Math-1.5B-Instruct at main

jamescallander/Qwen2.5-Math-1.5B-Instruct_w8a8_g128_rk3588.rkllm

Qwen2.5-Math-1.5B-Instruct — RKLLM build for RK3588 boards

Built with Qwen

Conversion details

⚠️ Math reasoning disclaimer

Intended use

Limitations

Quick start (RK3588)

1) Install runtime

2) Simple Flask server deployment

3) Sending a request

4) UI compatibility

License

jamescallander/
Qwen2.5-Math-1.5B-Instruct_w8a8_g128_rk3588.rkllm