Views
No views yet
GGUF quantized version of hotdogs/Agents-A1-4B-kimi-Preview — fine-tuned on Kimi K3 coding and debugging traces. Optimized for llama.cpp inference with coding agent capabilities and vision support via mmproj.
| Capability | Description |
|---|---|
| 💻 Coding Agent | Trained on real Kimi K3 coding traces — planning, debugging, building |
| 🖼️ Vision Understanding | Image-text-to-text with mmproj |
| 🧠 Step-by-step Reasoning | Autonomous agent-style reasoning |
| 🔧 Tool Calling | llama.cpp --tools all support |
| 💬 Multi-turn | 61% multi-turn conversations |
| 🌏 Thai + English | Native bilingual support |
| 🐍 Multi-language | Python, C, C++, Go, Java, Rust, Bash, and more |
| ⚡ Fast Inference | IQ4_NL fits in ~3 GB VRAM |
| File | Size | Description |
|---|---|---|
Agents-A1-4B-kimi-Preview-IQ4_NL.gguf | 2.61 GB | Recommended — best quality/speed balance for 8GB VRAM |
Agents-A1-4B-kimi-Preview-Q8_0_imatrix.gguf | 4.48 GB | Q8_0 + imatrix — almost lossless |
Agents-A1-4B-kimi-Preview.gguf | 8.42 GB | Full BF16 precision |
Agents-A1-4B-mmproj.gguf | 672 MB | Vision projector for image understanding |
🎯 IQ4_NL is recommended for 8GB VRAM users — fits comfortably even at 128K context with flash-attention.
1sudo docker run --rm -p 8080:8080 \
2 -v /root/models/:/models \
3 --gpus all \
4 --ulimit memlock=-1:-1 \
5 --env CUDA_VISIBLE_DEVICES=0 \
6 ghcr.io/ggml-org/llama.cpp:full-cuda --server \
7 -m /models/Agents-A1-4B-kimi-Preview-IQ4_NL.gguf \
8 --mmproj /models/Agents-A1-4B-mmproj.gguf \
9 --host 0.0.0.0 --port 8080 \
10 --n-gpu-layers 999 \
11 --ctx-size 131072 \
12 --batch-size 4096 \
13 --ubatch-size 256 \
14 --cache-type-k f16 \
15 --cache-type-v f16 \
16 --flash-attn on \
17 --cont-batching \
18 --mlock \
19 --temp 0.95 \
20 --top-k 40 \
21 --top-p 0.9 \
22 --min-p 0.0 \
23 -n -1 \
24 --no-mmap \
25 --parallel 1 --tools all \
26 --dry-multiplier 0.05 \
27 --jinja --dry-sequence-breaker none \
28 --repeat-penalty 1.1| Parameter | Purpose |
|---|---|
--mmproj | Vision projector for image understanding |
--ctx-size 131072 | 128K context window |
--flash-attn on | Flash attention for speed |
--cache-type-k/v f16 | BF16 KV cache for quality |
--cont-batching | Continuous batching for multi-turn |
--tools all | Enable tool/function calling |
--jinja | Use Jinja2 chat template |
--mlock | Lock memory for performance |
1# Quick text-only test
2./llama-cli -m Agents-A1-4B-kimi-Preview-IQ4_NL.gguf \
3 -p "Write a Python function to sort a list" -n 256 --temp 0.6 -ngl 999
4
5# Vision inference
6./llama-cli -m Agents-A1-4B-kimi-Preview-IQ4_NL.gguf \
7 --mmproj Agents-A1-4B-mmproj.gguf \
8 --image photo.jpg \
9 -p "What is in this image?" -n 256 --temp 0.6 -ngl 999| Parameter | Value |
|---|---|
| Base Model | hotdogs/Agents-A1-4B-kimi-Preview |
| Parameters | ~4.29B |
| Architecture | Qwen3.5 hybrid (Linear + Full attention) |
| Vision | ✅ Via mmproj |
| Context | Up to 128K tokens |
| Format | ChatML (Jinja2 template) |
| Fine-tuning | Kimi K3 coding traces (3,389 samples, 3 epochs, scale=0.4) |
bc1qf27cyk3vmugcdyv9xdtuv5jwz37863crpj5c9v