This is a pre-compiled version of
meta-llama/Llama-3.2-3B-Instruct optimized for the
Qualcomm QCS9075 SoC using the
Qualcomm Genie SDK.
1{
2 "model_path": "/path/to/model/files",
3 "backend": "QnnHtp",
4 "device": "0"
5}
1# Using the Genie server
2python3 /opt/qcom/aistack/genie/examples/server_persistent.py \
3 --config genie_config.json \
4 --port 8000
For deploying on Kubernetes clusters with QCS9075 nodes, refer to the deployment pattern:
1apiVersion: v1
2kind: Pod
3metadata:
4 name: genie-llm-server
5spec:
6 containers:
7 - name: genie
8 image: your-registry/genie-runtime:latest
9 env:
10 - name: LD_LIBRARY_PATH
11 value: "'/opt/qcom/aistack/qairt/2.42.0.250923/lib/aarch64-linux-gnu:/opt/qcom/aistack/genie/qnn/libs'"
12 volumeMounts:
13 - name: model-storage
14 mountPath: /models
15 - name: qcom-libs
16 mountPath: /opt/qcom/aistack
17 volumes:
18 - name: model-storage
19 hostPath:
20 path: /mnt/models/llama-3.2-3b-instruct-qcs9075-htp
21 - name: qcom-libs
22 hostPath:
23 path: /opt/qcom/aistack
This model follows the license of the base model
meta-llama/Llama-3.2-3B-Instruct. Please refer to the original model card for license details.
