Views
No views yet
This repository hosts a conversion ofMedGemma-4B-ITfor use on Rockchip RK3588 single-board computers (Orange Pi 5 plus, Radxa Rock 5b+, Banana Pi M7, etc.). Conversion was performed using the RKNN-LLM toolkit
w8a8_g128.rkllm artifactw8a8_g128) may show small quality differences vs. full-precision upstream..rkllm converted model is using an example script provided in the toolkit in this directory: rknn-llm/examples/rkllm_server_demo1python3 <TOOLKIT_PATH>/rknn-llm/examples/rkllm_server_demo/flask_server.py \
2 --rkllm_model_path <MODEL_PATH>/MedGemma-4B-IT_w8a8_g128_rk3588.rkllm \
3 --target_platform rk35881{
2 "model":"MedGemma-4B-IT",
3 "messages":[{
4 "role":"user",
5 "content":"<YOUR_PROMPT_HERE>"}],
6 "stream":false
7}curl:1curl -s -X POST <SERVER_IP_ADDRESS>:8080/rkllm_chat \
2 -H 'Content-Type: application/json' \
3 -d '{"model":"MedGemma-4B-IT","messages":[{"role":"user","content":"What is a retrovirus?"}],"stream":false}'1{
2 "choices":[{
3 "finish_reason":"stop",
4 "index":0,
5 "logprobs":null,
6 "message":{
7 "content":"<MODEL_REPLY_HERE">,
8 "role":"assistant"}}],
9 "created":null,
10 "id":"rkllm_chat",
11 "object":"rkllm_chat",
12 "usage":{
13 "completion_tokens":null,
14 "prompt_tokens":null,
15 "total_tokens":null}
16}{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"A retrovirus is a type of virus that uses an enzyme called reverse transcriptase to convert its RNA genome into DNA, which then integrates into the host cell's DNA. This integration allows the virus to replicate within the host cell for extended periods, often leading to chronic infections or even cancer. Here's a breakdown: * **RNA Genome:** Retroviruses have an RNA (ribonucleic acid) genome instead of DNA. * **Reverse Transcriptase:** This enzyme is unique to retroviruses and allows them to convert their RNA into DNA. It essentially acts as a molecular "copy machine" for the virus's genetic material. * **Integration:** The newly synthesized DNA integrates into the host cell's genome, becoming part of its DNA. This integration can be permanent or semi-permanent. * **Replication:** Once integrated, the viral DNA can be transcribed back into RNA, which is then used to produce new virus particles. These particles are released from the infected cell, infecting other cells. **Key Characteristics and Implications:** * **Chronic Infections:** Because retroviruses integrate their genetic material into the host's genome, they can cause chronic infections that persist for years or even a lifetime. * **Cancer:** Retroviral integration can disrupt normal cellular processes, leading to uncontrolled cell growth and cancer development. * **Examples:** Some well-known examples of retroviruses include HIV (Human Immunodeficiency Virus), which causes AIDS, and HTLV (Human T-lymphotropic virus), which can cause adult T-cell leukemia. In summary, a retrovirus is a unique type of virus that uses reverse transcriptase to integrate its RNA genome into the host cell's DNA, leading to chronic infections and potential cancer development. ","role":"assistant"}}],"created":null,"id":"rkllm_chat","object":"rkllm_chat","usage":{"completion_tokens":null,"prompt_tokens":null,"total_tokens":null}}http://<SERVER_IP_ADDRESS>:8080 and use the endpoint: /rkllm_chatmodel field matches the converted model’s name, for example:1{
2 "model": "llama-3.1-8b-instruct",
3 "messages": [{"role":"user","content":"Hello!"}],
4 "stream": false
5}NOTICE.rkllm models on RK3588 platforms, please refer to the RKNN-LLM toolkit documentatio