Views
No views yet
TQ2_0 ternary type (~2.06 bits/weight), using stock llama.cpp (tag b9498) — not PrismML's custom fork/packing.llama-server's OpenAI-compatible endpoint on CPU (Metal has no TQ1_0/TQ2_0 matmul kernels at this llama.cpp version — this model must run with -ngl 0 / CPU-only until upstream adds Metal support):<tool_call> format): FAIL, reproducible across 3 trials at temp=0. The model emits the correct tool-call JSON payload but consistently drops the opening <tool_call> tag (only emits the closing </tool_call>), breaking llama.cpp's native tool-call parser (finish_reason: stop instead of tool_calls). Do not enable native/strict tool-call parsing for this specific quantized model without a more lenient parser or further investigation.