Views
No views yet
GGUF quantized version of hotdogs/Agents-A1-4B-Fable-Preview — optimized for llama.cpp inference with vision support.
| Metric | Value |
|---|---|
| Resolve rate | 49.5% (99/200) |
| Instances evaluated | 200 / 500 (first 200, index-ordered slice) |
| Scaffold | mini-swe-agent v2.4.6 |
| Agent config | step_limit=250, cost_limit=$3.0, temperature=0.0 |
| Inference | llama.cpp, F16, ctx=131072 |
| Empty patches | 13/200 (6.5%) |
| Date | 2026-08-02 |
| Capability | Description |
|---|---|
| 🖼️ Vision Understanding | Image-text-to-text with mmproj |
| 🧠 Fable Reasoning | Step-by-step CoT with <think> blocks |
| 🔧 Tool Calling | llama.cpp --tools all support |
| 💬 Multi-turn | Trained on full agent trajectories |
| 🌏 Thai + English | Native bilingual support |
| 💻 Code & Shell | Python, bash, system tasks |
| ⚡ Fast Inference | IQ4_NL fits in ~3 GB VRAM |
| File | Size | Description |
|---|---|---|
Agents-A1-4B-Fable-IQ4_NL.gguf | 2.61 GB | Recommended — best quality/speed balance for 8GB VRAM |
Agents-A1-4B-Fable-Q4_K_M_imatrix.gguf | 2.71 GB | Q4_K_M + imatrix — slightly higher quality |
Agents-A1-4B-Fable-Q6_K_imatrix.gguf | 3.46 GB | Q6_K + imatrix — higher quality, more VRAM |
Agents-A1-4B-Fable-Q8_0_imatrix.gguf | 4.48 GB | Q8_0 + imatrix — almost lossless |
Agents-A1-4B-Fable-f16.gguf | 8.42 GB | Full BF16 precision |
Agents-A1-4B-mmproj.gguf | 672 MB | Vision projector for image understanding |
imatrix.dat | 3.63 MB | Importance matrix data |
🎯 IQ4_NL is recommended for 8GB VRAM users — fits comfortably even at 128K context with flash-attention.
1sudo docker run --rm -p 8080:8080 \
2 -v /root/models/:/models \
3 --gpus all \
4 --ulimit memlock=-1:-1 \
5 --env CUDA_VISIBLE_DEVICES=0 \
6 ghcr.io/ggml-org/llama.cpp:full-cuda --server \
7 -m /models/Agents-A1-4B-Fable-IQ4_NL.gguf \
8 --mmproj /models/Agents-A1-4B-mmproj.gguf \
9 --host 0.0.0.0 --port 8080 \
10 --n-gpu-layers 999 \
11 --ctx-size 131072 \
12 --batch-size 4096 \
13 --ubatch-size 256 \
14 --cache-type-k f16 \
15 --cache-type-v f16 \
16 --flash-attn on \
17 --cont-batching \
18 --mlock \
19 --temp 0.95 \
20 --top-k 40 \
21 --top-p 0.9 \
22 --min-p 0.0 \
23 -n -1 \
24 --no-mmap \
25 --parallel 1 --tools all \
26 --dry-multiplier 0.05 \
27 --jinja --dry-sequence-breaker none \
28 --repeat-penalty 1.1| Parameter | Purpose |
|---|---|
--mmproj | Vision projector for image understanding |
--ctx-size 131072 | 128K context window |
--flash-attn on | Flash attention for speed |
--cache-type-k/v f16 | BF16 KV cache for quality |
--cont-batching | Continuous batching for multi-turn |
--tools all | Enable tool/function calling |
--jinja | Use Jinja2 chat template |
--mlock | Lock memory for performance |
1# Quick text-only test
2./llama-cli -m Agents-A1-4B-Fable-IQ4_NL.gguf \
3 -p "Hello" -n 100 --temp 0.6 -ngl 999
4
5# Vision inference
6./llama-cli -m Agents-A1-4B-Fable-IQ4_NL.gguf \
7 --mmproj Agents-A1-4B-mmproj.gguf \
8 --image photo.jpg \
9 -p "What is in this image?" -n 256 --temp 0.6 -ngl 999| Parameter | Value |
|---|---|
| Base Model | hotdogs/Agents-A1-4B-Fable-Preview |
| Parameters | ~4.29B |
| Architecture | Qwen3.5 hybrid (Linear + Full attention) |
| Vision | ✅ 24-layer ViT encoder via mmproj |
| Context | Up to 128K tokens |
| Format | ChatML (Jinja2 template) |
| Fine-tuning | Fable-style reasoning traces (3,500 samples, 3 epochs) |
bc1qf27cyk3vmugcdyv9xdtuv5jwz37863crpj5c9v