This repository hosts a conversion ofDeepSeek-R1-Distill-Qwen-1.5Bfor use on Rockchip RK3588 single-board computers (Orange Pi 5 plus, Radxa Rock 5b+, Banana Pi M7, etc.). Conversion was performed using the RKNN-LLM toolkit
w8a8_g128.rkllm artifactw8a8_g128) may show small quality differences vs. full-precision upstream..rkllm converted model is using an example script provided in the toolkit in this directory: rknn-llm/examples/rkllm_server_demo1python3 <TOOLKIT_PATH>/rknn-llm/examples/rkllm_server_demo/flask_server.py \
2 --rkllm_model_path <MODEL_PATH>/DeepSeek-R1-Distill-Qwen-1.5B_w8a8_g128_rk3588.rkllm \
3 --target_platform rk35881{
2 "model":"DeepSeek-R1-Distill-Qwen-1.5B",
3 "messages":[{
4 "role":"user",
5 "content":"<YOUR_PROMPT_HERE>"}],
6 "stream":false
7}curl:1curl -s -X POST <SERVER_IP_ADDRESS>:8080/rkllm_chat \
2 -H 'Content-Type: application/json' \
3 -d '{"model":"DeepSeek-R1-Distill-Qwen-1.5B","messages":[{"role":"user","content":"In 2 or 3 sentences, who was Napoleon Bonaparte?"}],"stream":false}'1{
2 "choices":[{
3 "finish_reason":"stop",
4 "index":0,
5 "logprobs":null,
6 "message":{
7 "content":"<MODEL_REPLY_HERE">,
8 "role":"assistant"}}],
9 "created":null,
10 "id":"rkllm_chat",
11 "object":"rkllm_chat",
12 "usage":{
13 "completion_tokens":null,
14 "prompt_tokens":null,
15 "total_tokens":null}
16}{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"Okay, so I need to figure out who Napoleon Bonaparte is. I remember he's a French leader, but I'm not exactly sure about all the details. Let me think... He was born in 1795, right? And he died in 1881. That seems like a long time! I think he was a military leader and played a big role during the French Revolution. Wait, how did he get involved with the revolution? I believe he was a member of the National Assembly, which is where the French government is elected. Maybe he was one of the first people to support the revolution. He probably led some of the revolutionary activities, like the establishment of the People's Republic of China or something similar. I also remember that Napoleon was known for his strong sense of nationalism and his ability to unite a large population. His policies during the Revolution included establishing a republic with a weak central government, which he called the "Patriots' Republic." He probably wanted to eliminate corruption and strengthen the country's sovereignty. In terms of his political stance, I think he was a proponent of the French Nationalists party, which was a major political group in France during that time. His actions were aimed at maintaining French unity and preventing foreign interference in the revolution. So putting it all together, Napoleon Bonaparte was a key figure in the French Revolution, known for his military leadership, strong nationalism, and policies that aimed to unite France under a republic with a weak central government. </think> Napoleon Bonaparte was a prominent French leader who played a pivotal role during the French Revolution. Born in 1795 and dying in 1881, he was a member of the National Assembly and a key proponent of the French Nationalists party. Known for his military leadership and strong nationalism, Napoleon sought to unify France under a republic with a weak central government, opposing foreign interference and corruption. His policies aimed to maintain French unity and prevent revolutionaries from being suppressed.","role":"assistant"}}],"created":null,"id":"rkllm_chat","object":"rkllm_chat","usage":{"completion_tokens":null,"prompt_tokens":null,"total_tokens":null}} </think> markers.http://<SERVER_IP_ADDRESS>:8080 and use the endpoint: /rkllm_chatmodel field matches the converted model’s name, for example:1{
2 "model": "DeepSeek-R1-Distill-Qwen-1.5B",
3 "messages": [{"role":"user","content":"Hello!"}],
4 "stream": false
5}NOTICE.rkllm format for RK3588 SBCs