KiwiMate Mini is the lightweight, on-device model tier for KiwiMate, an AI companion app for iOS and Android with a New Zealand cultural identity. Mini 1.0 is fine-tuned from Qwen2.5-VL-3B-Instruct, replacing the previous Llama 3.2 3B base to add image input support alongside text chat.
This model is intended for fast, low-resource inference — locally on-device or via lightweight self-hosted endpoints — while KiwiMate Small (Qwen3.5-9B) handles heavier multimodal tasks server-side.
⚠️ Preview build. This is an early release of the Mini 1.0 line for internal testing ahead of KiwiMate app integration. Expect rough edges, especially around vision grounding and long conversations.
What's new vs. Mini Preview (Llama 3.2 3B)
Vision input — Mini can now see and reason about images (screenshots, photos, documents), not just text.
New base architecture: qwen2vl (Qwen2.5-VL) instead of llama.
Chat template moved from Llama-3's <|eot_id|> style to Qwen's ChatML-style <|im_start|> / <|im_end|> tokens — update any client or server-side stop-sequence config accordingly if you're migrating from Mini Preview.
Files
File
Size
Description
KiwiMate-Mini-1.0-instruct.F16.gguf
6.18 GB
Full-precision weights, highest quality, largest footprint
KiwiMate-Mini-1.0-instruct.Q6_K.gguf
2.54 GB
Near-lossless quantization, good balance of quality/size
KiwiMate-Mini-1.0-instruct.Q4_K_M.gguf
1.93 GB
Smallest, fastest, recommended for constrained/on-device use
KiwiMate-Mini-1.0.BF16-mmproj.gguf
1.34 GB
Vision projector — required alongside a text GGUF above for image input
Modelfile
—
Ollama Modelfile for local serving
config.json
—
Model config (Unsloth training output)
As with the previous Mini line, expect some quality drop at Q4_K_M relative to F16/Q6_K — worth spot-checking once you've run your usual eval pass, since this is a different base architecture and the tradeoff may not track identically to the Llama 3.2 quants.
Usage
llama.cpp
Vision input requires loading both the text model and the mmproj file:
If you're serving this behind a proxy (e.g. runpod-chat/index.ts), make sure stop sequences are updated to <|im_end|> rather than the old Llama <|eot_id|> — the two are not interchangeable.
Quantization: GGUF via llama.cpp (F16, Q6_K, Q4_K_M)
Fine-tuning focus: New Zealand English, Te Reo Māori awareness and cultural sensitivity, KiwiMate persona and identity, conversational NZ context
Intended use
Conversational AI companion for the KiwiMate app — casual chat, light image understanding (e.g. describing a photo, reading a screenshot), and NZ-flavoured personality. Not intended for high-stakes reasoning, factual research, or professional/medical/legal advice.
License
⚠️ Please read before redistributing. This model is derived from Qwen2.5-VL-3B-Instruct, which Qwen distributes under a Qwen Research license, not Apache 2.0 — this differs from KiwiMate's other model tiers. Review the license terms — research licenses commonly restrict commercial deployment — before treating this build the same as the Apache-2.0 Mini Preview / Small tiers, or before making this repo public.
