You can deploy this model and start earning money today!

batiai/
Llama-4-Scout-17B-16E-Instruct-GGUF

Name: Llama-4-Scout-17B-16E-Instruct-GGUF
Availability: InStock

Llama 4 Scout 17B-16E-Instruct GGUF — Quantized by BatiAI

BatiFlow

imatrix-calibrated GGUF quantizations of meta-llama/Llama-4-Scout-17B-16E-Instruct (109B total / 17B active MoE, 16 experts, multimodal). Quantized directly from official Meta BF16 weights by BatiAI.

Why Llama 4 Scout?

109B total / 17B active Mixture-of-Experts (16 experts, top-1 routing) — efficient for size
Multimodal native: text + vision via mmproj (image-text-to-text)
Multilingual: 8 official languages + general multilingual
Native tool calling + extended context
Meta Llama 4 Community License — commercial-friendly for most cases (see license link)
Released 2025-04 by Meta

Quick Start

bash

# Q4_K_M (recommended balance, 60GB, M4 Max 128GB ~ M2 Ultra 192GB)
ollama pull batiai/llama4-scout:q4

# IQ3_XXS (smallest, 38GB, M4 Max 64GB+)
ollama pull batiai/llama4-scout:iq3

# Q5_K_M (higher quality, 72GB, M2 Ultra 192GB+)
ollama pull batiai/llama4-scout:q5

Available Quantizations

Quant	Size	Min RAM	Target Hardware	Notes
IQ3_XXS	38GB	48GB	M4 Max 64GB	aggressive 3-bit, imatrix-calibrated
Q3_K_M	48GB	56GB	M4 Max 64GB	balanced 3-bit
IQ4_XS	52GB	64GB	M4 Max 96GB	imatrix 4-bit (best size/quality)
Q4_K_M	60GB	72GB	M4 Max 128GB	recommended for most users
Q5_K_M	72GB	88GB	M2 Ultra 192GB	higher fidelity
Q6_K	85GB	96GB	M2 Ultra 192GB+	near-original quality

Mac note on Q3_K_M: in every model we've benchmarked on Apple Silicon, Q3_K_M generated slower than Q4_K_M despite the smaller file — Granite 4.1 (+27%), Gemma 4 26B (+12%), Qwen3.8‑27B (+18%), Qwen3.6‑27B (+8%), on both M4 Max and M4 mini. Metal's Q3_K path is limited by dequantization compute rather than bandwidth. We have not measured this particular model's Q3/Q4 pair yet, so treat it as a strong prior, not a measurement: if Q4_K_M fits, take it. On CUDA the two are effectively tied, so this applies to Macs only.

Multimodal users: also download mmproj-*-BF16.gguf (or mmproj-*-Q6_K.gguf) and use with llama-server --mmproj or llama-mtmd-cli.

Hardware Reality Check

Your System	IQ3 (38GB)	Q4 (60GB)	Q5 (72GB)	Q6 (85GB)
Mac mini M4 16GB	❌	❌	❌	❌
Mac mini M4 Pro 24GB	❌	❌	❌	❌
Mac M4 Max 64GB	✅ tight	❌	❌	❌
Mac M4 Max 96GB	✅	✅ tight	❌	❌
Mac M4 Max 128GB	✅	✅	✅ tight	❌
Mac M2 Ultra 192GB	✅	✅	✅	✅
Mac M3 Ultra 512GB	✅ comfortable	✅	✅	✅

How to run

Ollama (text-only)

bash

ollama pull batiai/llama4-scout:q4
ollama run batiai/llama4-scout:q4

llama.cpp (text + vision via mmproj)

bash

# Download GGUF + mmproj
hf download batiai/Llama-4-Scout-17B-16E-Instruct-GGUF \
    --include "*Q4_K_M*" --include "mmproj-*-Q6_K.gguf" \
    --local-dir ./llama4-scout

# Run with vision
llama-mtmd-cli \
    -m ./llama4-scout/meta-llama-Llama-4-Scout-17B-16E-Instruct-Q4_K_M.gguf \
    --mmproj ./llama4-scout/mmproj-meta-llama-Llama-4-Scout-17B-16E-Instruct-Q6_K.gguf \
    --image input.jpg -p "Describe this image."

# Or as a server
llama-server -m ./llama4-scout/meta-llama-Llama-4-Scout-17B-16E-Instruct-Q4_K_M.gguf \
    --mmproj ./llama4-scout/mmproj-meta-llama-Llama-4-Scout-17B-16E-Instruct-Q6_K.gguf \
    -ngl 99 -c 32768 --port 8080

Model details

Source: meta-llama/Llama-4-Scout-17B-16E-Instruct
Architecture: Llama4ForConditionalGeneration — 109B total / 17B active MoE
Experts: 16 routed (top-1 per token) — efficient sparse MoE
Multimodal: text backbone + vision encoder (mmproj 분리)
Original precision: BF16
License: Meta Llama 4 Community License (commercial use OK for most, see link)

BatiAI signing

All GGUFs in this repo carry:

general.author = BatiAI
general.url = https://flow.bati.ai

Why BatiAI?

Quantized directly from official Meta BF16 weights — no re-quantization
IQ + K-quant variants share the same wikitext-2-raw imatrix recipe as every BatiAI model
Multimodal mmproj packaged together for one-stop multimodal usage
Verified on Apple Silicon (M4 Max / M2 Ultra)

License

Inherits Meta Llama 4 Community License. Commercial-friendly for organizations with < 700M MAU. See:

About BatiFlow

BatiFlow — free on-device AI automation for Mac.

Benchmarks coming once Mac M4 Max / M2 Ultra measurements complete.