RKLLM conversion of
Qwen/Qwen3.5-0.8B for Rockchip RK3588 NPU inference.
Converted with RKLLM Toolkit v1.2.3. This is the base model variant — suitable for custom fine-tunes or applications that supply their own system prompting.
Qwen3.5-0.8B is the successor to Qwen3-0.6B, offering improved architecture and training. At under 1GB after quantisation it is the smallest model in the RK3588 lineup — ideal for low-latency responses, always-on assistants, or edge deployments where memory is tight.
1mkdir -p ~/models/Qwen3.5-0.8B
2cd ~/models/Qwen3.5-0.8B
3git lfs install && git clone https://huggingface.co/GatekeeperZA/Qwen3.5-0.8B-RKLLM-v1.2.3 .
1git clone https://github.com/airockchip/rknn-llm.git
2cd rknn-llm/examples/rkllm_api_demo
3./build/rkllm_api_demo /path/to/Qwen3.5-0.8B-rk3588-w8a8.rkllm 4096 8192
