This repository hosts a pre-configured deployment setup for the Qwen 27B model, specifically optimized to run as a local autonomous agent within structured tool-calling frameworks like Hermes.
The Problem Solved
When deploying local agent frameworks with standard Qwen GGUF files, the agent frequently breaks down during tool execution. The agent framework expects the model to communicate tool calls in strict, structured JSON formats.
However, the default chat template baked into standard Qwen models often causes the text generation to drift. The model regular attempts to wrap tool calls in raw XML blocks, markdown code wrappers, or outputs casual conversational text alongside the payload. This breaks the framework's parser, leading to continuous loop errors, missed execution arguments, and failed automation workflows.
The Fix
This configuration fixes the structural drift by overriding the default chat translation layer.
We updated the environment's chat_template.jinja file to explicitly inject strict JSON tool constraints (_tool_format = 'json'). This forces the underlying model weights to process inputs and format outputs into predictable, rock-solid JSON payloads, ensuring clean tool execution loop completion without breaking the parser.
Repository Contents
Qwen3.6-27B-Q4_K_M.gguf: The core text generation model quantized to 4-bit medium precision.
mmproj-Qwen3.6-27B-BF16.gguf: The multimodal vision projector block, allowing the text model to interpret images and screenshots via local browser automation tools.
chat_template.jinja: The custom Jinja template file that enforces the JSON formatting constraints.
How to Deploy and Use
Because GGUF engines default to using the template embedded inside the binary file, you must manually apply the custom Jinja template included in this repository to activate the fix.
Method 1: Linux / Arch Terminal (Recommended Engine)
For maximum efficiency and minimal resource overhead, run the model using raw llama.cpp. Pass the text weights, vision projector, and the custom template file directly via the terminal command:
