The fine-tuned GGUF the playable Hugging Face Space loads through llama.cpp — the Off the Grid + Llama Champion contract (no cloud API at runtime). Transformers source: build-small-hackathon/mind-of-tashi-micro-sft.

David vs Goliath. Custom MoE loggenix-moe-0.4B-0.2A (~0.4B total / ~200M active per token), SFT'd to read an opponent and commit blind in an English + Hindi/Sanskrit (IAST) code-switched register — 10–100× smaller (active) than the API teachers it learned from.

Files

File	Size	Use
`mind-of-tashi-micro-sft-Q4_K_M.gguf`	~256 MB	deployed — small + fast
`mind-of-tashi-micro-sft-f16.gguf`	~786 MB	zero-loss reference

Build recipe (no compiled binary needed)

Download the transformers checkpoint with chat_template.jinja (recent checkpoints externalise the template; a missing one yields an empty-template GGUF that loads but generates garbage).
python convert_hf_to_gguf.py <ckpt> --outtype f16 → f16 GGUF.
Quantise via the llama-cpp-python C binding (no cmake/MSVC):
python
1import ctypes, llama_cpp 2p = llama_cpp.llama_model_quantize_default_params() 3p.ftype = 15 # LLAMA_FTYPE_MOSTLY_Q4_K_M 4llama_cpp.llama_model_quantize(b"in-f16.gguf", b"out-Q4_K_M.gguf", ctypes.byref(p))

⚠️ `norm_topk_prob` — required for llama.cpp

llama.cpp's qwen3moe graph hardcodes norm_w=true; the source checkpoint is trained with norm_topk_prob=true to match. (See the SFT model card.)

Eval

Format gate via llama.cpp (the real deploy path): f16 18/20, Q4_K_M 20/20 valid <think> + {move,taunt}; ~19–20/20 bilingual.
Throughput (Q4_K_M, RTX 3060 Laptop, n_gpu_layers=99, n_ctx 4096): cold load 3.2 s, TTFT ~97 ms (p50), decode ~450 t/s (p50), ~0.30 s/move, peak VRAM < ~0.8 GB. CPU-only deploy is slower but fine for a 0.4B.

Usage

python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="build-small-hackathon/mind-of-tashi-micro-sft-gguf",
    filename="mind-of-tashi-micro-sft-Q4_K_M.gguf",
    n_ctx=4096, n_gpu_layers=0,   # 0 = CPU (Space); 99 = full GPU offload
    logits_all=True,              # needed for the in-game Conviction Meter (token logprobs)
)
out = llm.create_chat_completion(messages=[
    {"role": "system", "content": "<persona system prompt>"},
    {"role": "user",   "content": "<arena state + match history>"},
])
# -> "<think>...IAST-coded reasoning...</think>\n{\"move\": \"...\", \"taunt\": \"...\"}"

Part of the bundle

Game Space · self-play dataset · SFT model + GGUF (this) · OpenEnv gym · GRPO model + GGUF — all under build-small-hackathon/mind-of-tashi-*.

build-small-hackathon/mind-of-tashi-micro-sft-gguf

The Mind of Tashi — micro student (SFT, GGUF)