Views
No views yet
C:\kuwa\GenAI OS\windows\executors\taide 目錄下，並將原始模型taide-8b-a.3-q4_k_m.gguf備份到此目錄外，並刪除run.batLlama-3.1-TAIDE-LX-8B-Chat-Q4_K_M.gguf 到 C:\kuwa\GenAI OS\windows\executors\taide 目錄中C:\kuwa\GenAI OS\windows\executors\taide\init.bat，使用以下設定值
3Llama-3.1 TAIDE LX-8B Chat Q4_K_Mllama-3.1-taide-lx-8b-chat-q4_k_m--stop "<|eot_id|>"Llama-3.1-TAIDE-LX-8B-Chat-Q4_K_M.gguf 到任意目錄中genai-os/docker/compose/sample/taide-llamacpp.yaml設定檔，填入以下內容1services:
2llamacpp-executor:
3 image: kuwaai/model-executor
4 environment:
5 EXECUTOR_TYPE: llamacpp
6 EXECUTOR_ACCESS_CODE: llama-3.1-taide-lx-8b-chat-q4_k_m
7 EXECUTOR_NAME: Llama-3.1 TAIDE LX-8B Chat Q4_K_M
8 EXECUTOR_IMAGE: llamacpp.png # Refer to src/multi-chat/public/images
9 depends_on:
10 - executor-builder
11 - kernel
12 - multi-chat
13 command: ["--model_path", "/var/model/Llama-3.1-TAIDE-LX-8B-Chat-Q4_K_M.gguf", "--temperature", "0"]
14 # or use GPU
15 # command: ["--model_path", "/var/model/Llama-3.1-TAIDE-LX-8B-Chat-Q4_K_M.gguf", "--ngl", "-1", "--temperature", "0"]
16 restart: unless-stopped
17 volumes: ["/path/to/Llama-3.1-TAIDE-LX-8B-Chat-Q4_K_M.gguf:/var/model/Llama-3.1-TAIDE-LX-8B-Chat-Q4_K_M.gguf"] # Remember to change path
18 # Uncomment to use GPU
19 # deploy:
20 # resources:
21 # reservations:
22 # devices:
23 # - driver: nvidia
24 # device_ids: ['0']
25 # capabilities: [gpu]
26 networks: ["backend"]genai-os/docker 目錄下執行 ./run.sh 即可啟動新模型