Beta
Explore
Marketplace
Neural Labs
Playground
Wallet
Docs
Qwen2.5-Coder-7B-Instruct-RK3576 – AI Model by jerard-dev | AlphaNeural AI | AlphaNeural AI
You can deploy this model and start earning money today!
jerard-dev
/
Qwen2.5-Coder-7B-Instruct-RK3576
like
0
rkllm
rk3576
npu
qwen2.5-coder
edge-ai
Qwen/Qwen2.5-Coder-7B-Instruct
finetune
apache-2.0
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Qwen2.5-Coder-7B-Instruct — RK3576 (RKLLM)
Conversion of
Qwen/Qwen2.5-Coder-7B-Instruct
to RKLLM format for
RK3576
NPU (NanoPi R76S, Orange Pi RK3576, and similar boards).
Conversion details
Toolkit:
rkllm-toolkit 1.2.3
Compatible runtime:
rkllm-runtime >= 1.2.3
Quantization:
w4a16
NPU cores:
2 (dual-core)
Max context:
4096 tokens
Platform:
RK3576 (NOT compatible with RK3588)
Usage with RKLLama
Place the
.rkllm
file in your models folder and create a
Modelfile
with the following content:
FROM="Qwen2.5-Coder-7B-Instruct-rk3576.rkllm" HUGGINGFACE_PATH="Qwen/Qwen2.5-Coder-7B-Instruct" TOKENIZER="/models/tokenizers/Qwen2.5-Coder-7B"
Performance (NanoPi R76S, 16GB RAM)
~10-15 tok/s on dual-core NPU
Important note
This model was compiled specifically for RK3576. If you're using an RK3588 (RK3588S, RK3588J), look for a different conversion — they are not binary compatible with each other.
