Views
No views yet
This repository hosts a conversion ofCodeLlama-7b-Instruct-hffor use on Rockchip RK3588 single-board computers (Orange Pi 5 plus, Radxa Rock 5b+, Banana Pi M7, etc.). Conversion was performed using the RKNN-LLM toolkit
w8a8_g128.rkllm artifactw8a8_g128) may show small quality differences vs. full-precision upstream..rkllm converted model is using an example script provided in the toolkit in this directory: rknn-llm/examples/rkllm_server_demo1python3 <TOOLKIT_PATH>/rknn-llm/examples/rkllm_server_demo/flask_server.py \
2 --rkllm_model_path <MODEL_PATH>/CodeLlama-7b-Instruct-hf_w8a8_g128_rk3588.rkllm \
3 --target_platform rk35881{
2 "model":"CodeLlama-7b-Instruct-hf",
3 "messages":[{
4 "role":"user",
5 "content":"<YOUR_PROMPT_HERE>"}],
6 "stream":false
7}curl:1curl -s -X POST <SERVER_IP_ADDRESS>:8080/rkllm_chat \
2 -H 'Content-Type: application/json' \
3 -d '{"model":"CodeLlama-7b-Instruct-hf","messages":[{"role":"user","content":"Create a python function to calculate factorials using recursive method."}],"stream":false}'1{
2 "choices":[{
3 "finish_reason":"stop",
4 "index":0,
5 "logprobs":null,
6 "message":{
7 "content":"<MODEL_REPLY_HERE">,
8 "role":"assistant"}}],
9 "created":null,
10 "id":"rkllm_chat",
11 "object":"rkllm_chat",
12 "usage":{
13 "completion_tokens":null,
14 "prompt_tokens":null,
15 "total_tokens":null}
16}{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"Here is an example of how you can create a Python function to calculate factorials using the recursive method: ``` def factorial(n): if n == 0: return 1 else: return n * factorial(n-1) ``` This function uses the recursive formula for factorials, which is `n! = n * (n-1)!`, to calculate the factorial of a given number `n`. The base case is when `n` is 0, in which case the result is 1. Otherwise, the function calls itself with `n-1` as the argument and multiplies the result by `n`. For example, if you call the function with `n = 5`, it will calculate the factorial of 5 using the recursive formula: ``` factorial(5) = 5 * factorial(4) = 5 * (4 * factorial(3)) = 5 * (4 * (3 * factorial(2))) = 5 * (4 * (3 * (2 * factorial(1)))) = 5 * (4 * (3 * (2 * 1))) = 5 * (4 * (3 * 2)) = 5 * (4 * 6) = 5 * 24 = 120 ``` So the result of `factorial(5)` is 120. You can also use a loop to calculate factorials, here is an example: ``` def factorial(n): result = 1 for i in range(1, n+1): result *= i return result ``` This function uses a loop to iterate from 1 to `n` and multiply the result by each number. The base case is when `n` is 0, in which case the result is 1. You can also use the built-in `math.factorial()` function, it will calculate the factorial of a given number using the recursive method: ``` from math import factorial print(factorial(5)) # output: 120 ``` It's worth noting that the recursive method can be slower than the loop method for large values of `n`, because it requires making multiple function calls. However, the recursive method is often more concise and easier to understand, especially for small values of `n`.","role":"assistant"}}],"created":null,"id":"rkllm_chat","object":"rkllm_chat","usage":{"completion_tokens":null,"prompt_tokens":null,"total_tokens":null}}http://<SERVER_IP_ADDRESS>:8080 and use the endpoint: /rkllm_chatmodel field matches the converted model’s name, for example:1{
2 "model": "CodeLlama-7b-Instruct-hf",
3 "messages": [{"role":"user","content":"Hello!"}],
4 "stream": false
5}NOTICE