---

[!CAUTION] The gguf models have been updated! The conversion is done with llama.cpp b3463.

Phi-3-mini-128k-instruct-GGUF

Original Model

Run with LlamaEdge

• LlamaEdge version: v0.12.2 and above
• Prompt template
  • Prompt type: phi-3-chat
  • Prompt string
    text

    1<|system|> 2{system_message}<|end|> 3<|user|> 4{user_message_1}<|end|> 5<|assistant|> 6{assistant_message_1}<|end|> 7<|user|> 8{user_message_2}<|end|> 9<|assistant|>
• Context size: 128000
• Run as LlamaEdge service
  bash

  1wasmedge --dir .:. --nn-preload default:GGML:AUTO:Phi-3-mini-128k-instruct-Q5_K_M.gguf \ 2 llama-api-server.wasm \ 3 --prompt-template phi-3-chat \ 4 --ctx-size 128000 \ 5 --model-name phi-3-mini-128k
• Run as LlamaEdge command app
  bash

  1wasmedge --dir .:. --nn-preload default:GGML:AUTO:Phi-3-mini-128k-instruct-Q5_K_M.gguf \ 2 llama-chat.wasm \ 3 --prompt-template phi-3-chat \ 4 --ctx-size 128000

Quantized GGUF Models