Views
No views yet
| 模型 | 内存占用 | 模型大小 | 量化类型 |
|---|---|---|---|
| DeepSeek-R1-Distill-Qwen-7B-RK3588S-RKLLM1.1.4 | >8GB,无法测试 | 7.63GB | w8a8 |
cd ~ && git clone https://github.com/airockchip/rknn-llm.gitcd rknn-llm/examples/rkllm_api_demollm_demo.cpp的代码vi src/llm_demo.cpp1#define PROMPT_TEXT_PREFIX "<｜begin▁of▁sentence｜>system 你是一名专业AI助手请遵循:1.用简体中文回答；2.中文翻译成英文时，需使用英文回答；3.展示思考过程 <｜User｜>"
2#define PROMPT_TEXT_POSTFIX "<｜Assistant｜>"<｜begin▁of▁sentence｜>system到<｜User｜>之间的内容。text = PROMPT_TEXT_PREFIX + input_str + PROMPT_TEXT_POSTFIX;// text = input_str;bash ./build-linux.shmkdir ~/rkllm && cd ~/rkllmbuild/build_linux_aarch64_Release/下的llm_demo上传至开发板的rkllm文件夹内。cd ~/rkllm && mkdir librknn-llm/rkllm-runtime/Linux/librkllm_api/aarch64下的librkllmrt.so上传至开发板的rkllm/lib文件夹内。git lfs installgit clone https://huggingface.co/VRxiaojie/DeepSeek-R1-Distill-Qwen-7B-RK3588S-RKLLM1.1.4DeepSeek-R1-Distill-Qwen-7B-RK3588S-RKLLM1.1.4文件夹内的deepseek-r1-7B-rkllm1.1.4.rkllm 上传至开发板刚刚创建的rkllm文件夹下export LD_LIBRARY_PATH=./lib./llm_demo ./deepseek-r1-7B-rkllm1.1.4.rkllm 2048 2048./llm_demo model_path max_new_tokens max_context_lenuser:后输入对话内容即可。