DeepSeek-V2.5-GGUF

Original Model

Run with LlamaEdge

• LlamaEdge version: coming soon

• Prompt template
  • Prompt type: deepseek-chat-25
  • Prompt string
    <|begin_of_sentence|>{system_message}<|User|>{user_message_1}<|Assistant|>{assistant_message_1}<|end_of_sentence|><|User|>{user_message_2}<|Assistant|>
• Context size: 128000
• Run as LlamaEdge service
  bash

  1wasmedge --dir .:. \ 2 --nn-preload default:GGML:AUTO:DeepSeek-V2.5-Q5_K_M.gguf \ 3 llama-api-server.wasm \ 4 --prompt-template deepseek-chat-25 \ 5 --ctx-size 128000 \ 6 --model-name DeepSeek-V2.5
• Run as LlamaEdge command app
  bash

  1wasmedge --dir .:. \ 2 --nn-preload default:GGML:AUTO:DeepSeek-V2.5-Q5_K_M.gguf \ 3 llama-chat.wasm \ 4 --prompt-template deepseek-chat-25 \ 5 --ctx-size 128000