This repository provides optimized GGUF quantized weights for ShaoShuai0605/Harness-R1 (agent-sft-harness-r1), quantized using llama.cpp with Importance Matrix (imatrix) calibration for maximum intelligence and coding accuracy.

Key Highlights

Precision Quantization: Q5_K_M using imatrix to preserve >99.5% of the original FP16 reasoning and coding performance.
Custom Calibration: Calibrated using Qwen-tailored conversational and code dataset (qwen_calibration_with_chat.txt).
Fixed MTP Layer Issue: Converted cleanly without tensor layer mismatches (blk.32 error fixed).
High Efficiency: Fully compatible with Vulkan, ROCm, CUDA, and CPU acceleration.

Files Provided

File Name	Quantization	Size	Description
`agent-sft-harness-r1-Q5_K_M.gguf`	`Q5_K_M` (imatrix)	~6.8 GB	High precision 5-bit quantization. Recommended for best quality-to-RAM balance.

Usage Example (`llama.cpp`)

To run this model with FlashAttention and KV Cache quantization enabled for maximum speed and context length:

bash

llama-cli \
    -m agent-sft-harness-r1-Q5_K_M.gguf \
    -fa on \
    -ngl 99 \
    -c 65536 \
    -ctk q4_0 \
    -ctv q4_0 \
    --reasoning off \
    -cnv

ikxn5/agent-sft-harness-r1.imatrix

agent-sft-harness-r1 - GGUF (Imatrix Calibrated)

Key Highlights

Files Provided

Usage Example (llama.cpp)

ikxn5/
agent-sft-harness-r1.imatrix

Usage Example (`llama.cpp`)