---

medgemma-4b-it-GGUF

Original Model

Run with LlamaEdge

• LlamaEdge version: v0.18.5 and above
• Prompt template
  • Prompt type: gemma-3
  • Prompt string
    text

    1<bos><start_of_turn>user 2{user_message}<end_of_turn> 3<start_of_turn>model 4{model_message}<end_of_turn>model
• Context size: 128000
• Run as LlamaEdge service
  • Chat
    bash

    1wasmedge --dir .:. --nn-preload default:GGML:AUTO:medgemma-4b-it-Q5_K_M.gguf \ 2 llama-api-server.wasm \ 3 --prompt-template gemma-3 \ 4 --ctx-size 128000 \ 5 --model-name medgemma-4b
  • Images
    Note that input images are required to be normalized to 896 x 896 resolution and encoded to 256 tokens each
    bash

    1wasmedge --dir .:. --nn-preload default:GGML:AUTO:medgemma-4b-it-Q5_K_M.gguf \ 2 llama-api-server.wasm \ 3 --prompt-template gemma-3 \ 4 --llava-mmproj medgemma-4b-it-mmproj-f16.gguf \ 5 --ctx-size 128000 \ 6 --model-name medgemma-4b
• Run as LlamaEdge command app
  bash

  1wasmedge --dir .:. \ 2 --nn-preload default:GGML:AUTO:medgemma-4b-it-Q5_K_M.gguf \ 3 llama-chat.wasm \ 4 --prompt-template gemma-3 \ 5 --ctx-size 128000

Quantized GGUF Models