You can deploy this model and start earning money today!

GatekeeperZA/
Qwen2.5-1.5B-Instruct-RKLLM-v1.2.3

Name: Qwen2.5-1.5B-Instruct-RKLLM-v1.2.3
Availability: InStock

Qwen/Qwen2.5-1.5B-Instruct

finetune

apache-2.0

Views

No views yet

Qwen2.5-1.5B-Instruct for RKLLM v1.2.3 (RK3588 NPU)

Pre-converted Qwen2.5-1.5B-Instruct for the Rockchip RK3588 NPU using rknn-llm runtime v1.2.3.

Runs on Orange Pi 5 Plus, Rock 5B, Radxa NX5, and other RK3588-based SBCs with 8GB+ RAM.

Files

File	Size	Description
`Qwen2.5-1.5B-Instruct-rk3588-v123-w8a8.rkllm`	2.0 GB	LLM (W8A8 quantized, 8192 token context)

Quick Start

Directory Structure

~/models/Qwen2.5-1.5B-Instruct/
    Qwen2.5-1.5B-Instruct-rk3588-v123-w8a8.rkllm

Using with RKLLM API Server

This model is designed for use with the RKLLM API Server, which provides an OpenAI-compatible API for RK3588 NPU inference. The server auto-discovers .rkllm files by scanning subdirectories of your models folder.

bash

# Place the model in your models directory
mkdir -p ~/models/Qwen2.5-1.5B-Instruct
# Copy .rkllm file here — the API server will find it automatically
sudo systemctl restart rkllm-api

The model will appear as qwen2.5-1.5b-instruct in the OpenAI-compatible model list.

Export Details

Parameter	Value
Source	Qwen/Qwen2.5-1.5B-Instruct
Tool	rkllm-toolkit v1.2.3
Quantization	W8A8 (8-bit weights, 8-bit activations)
Optimization level	1
Target platform	rk3588
NPU cores	3
Max context	8192 tokens

Performance Benchmarks

Tested on Orange Pi 5 Plus (16GB RAM), RK3588 SoC, RKNPU driver 0.9.8:

Metric	Value
Decode speed	~19 tok/s
Model load time	~3 s
Peak RAM	~2.2 GB

Known Limitations

Tool calling: Qwen2.5-1.5B is a general-purpose instruct model, not fine-tuned for structured function calling. For agent/tool-calling workloads, see GatekeeperZA/xLAM-1b-fc-r-RKLLM-v1.2.3.
Context: Compiled at 8192 tokens. The runtime will truncate inputs that exceed this limit.
Version lock: This file requires RKLLM runtime v1.2.3 and NPU driver v0.9.8. It is not compatible with v1.3.0 or later runtimes.

License

Apache 2.0, inherited from Qwen2.5-1.5B-Instruct.

Credits

Model: Qwen Team for Qwen2.5-1.5B-Instruct
Runtime: Rockchip / airockchip for rkllm-toolkit
API Server: RKLLM API Server — OpenAI-compatible server for RK3588 NPU

GatekeeperZA/Qwen2.5-1.5B-Instruct-RKLLM-v1.2.3