---

Llama-3.3-Swallow-70B-Instruct-v0.4-GGUF

Original Model

Run with LlamaEdge

• LlamaEdge version: v0.16.13 and above
• Prompt template
  • Prompt type: llama-3-chat
  • Prompt string
    text

    1<|begin_of_text|><|start_header_id|>system<|end_header_id|> 2 3{{ system_prompt }}<|eot_id|><|start_header_id|>user<|end_header_id|> 4 5{{ user_message_1 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|> 6 7{{ model_answer_1 }}<|eot_id|><|start_header_id|>user<|end_header_id|> 8 9{{ user_message_2 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
• Context size: 128000
• Run as LlamaEdge service
  bash

  1wasmedge --dir .:. --nn-preload default:GGML:AUTO:Llama-3.3-Swallow-70B-Instruct-v0.4-Q5_K_M.gguf \ 2 llama-api-server.wasm \ 3 --prompt-template llama-3-chat \ 4 --ctx-size 128000 \ 5 --model-name Llama-3.3-Swallow-70B
• Run as LlamaEdge command app
  bash

  1wasmedge --dir .:. --nn-preload default:GGML:AUTO:Llama-3.3-Swallow-70B-Instruct-v0.4-Q5_K_M.gguf \ 2 llama-chat.wasm \ 3 --prompt-template llama-3-chat \ 4 --ctx-size 128000

Quantized GGUF Models