bash scripts/download_models.sh llm_rkllm
uv run intent-model --serve --llm-runtime rkllm \
    --asr-provider rknn --asr-model-dir models/asr-sensevoice-rknn

Requirements: RKNPU driver >= 0.9.8, librkllmrt.so. Qwen3-1.7B keeps thinking mode (enable_thinking); ~13 tok/s on RK3588.

Why no GLiNER2 encoder here

Converting the pipeline's GLiNER2 (DeBERTa-v3) encoder to RKNN was attempted thoroughly (fp16 + int8, graph splitting, every relevant toolkit knob) and is blocked by rknn-toolkit2 2.3.2 itself: its build phase needs >460 GB RAM at the model's working sequence length, crashes with integer-overflow mallocs, and emits artifacts with multi-GB zero-padded sections. On the board the GLiNER2 encoder runs on CPU via ONNX (--gliner-encoder onnx), which matches torch accuracy; the NPU's three cores are used by ASR + the LLM.

Icerm/intent-rk3588-npu

intent-rk3588-npu

Usage (on the board)

Why no GLiNER2 encoder here

Icerm/
intent-rk3588-npu