GGUF quantizations of Socrates-Qwen3-8B, a QLoRA fine-tune of Qwen3-8B that reasons Socratically inside its <think> trace: it interrogates the question, defines its terms, tests with counterexamples, and concludes with calibrated confidence, while keeping final answers correct.

Provided files

File	Quant	Size	Notes
Socrates-Qwen3-8B-Q4_K_M.gguf	Q4_K_M	~5.0 GB	Recommended, best size/quality trade-off
Socrates-Qwen3-8B-Q5_K_M.gguf	Q5_K_M	~5.9 GB	Higher quality, still fits 8 GB VRAM
Socrates-Qwen3-8B-Q6_K.gguf	Q6_K	~6.7 GB	Near-lossless
Socrates-Qwen3-8B-Q8_0.gguf	Q8_0	~8.7 GB	Max practical quality

Rule of thumb: pick the largest quant that fits your RAM/VRAM with a couple of GB to spare for context.

Usage

Ollama

ollama run hf.co/AthrvShrn/Socrates-Qwen3-8B-GGUF:Q4_K_M

llama.cpp

llama-cli -hf AthrvShrn/Socrates-Qwen3-8B-GGUF:Q4_K_M

LM Studio: search for Socrates-Qwen3-8B-GGUF in the model browser and pick a quant.

The model uses the standard Qwen3 chat template with native <think>…</think> reasoning; no special prompt format is needed.

Example

"Is 17 prime?"

think: What is 'prime'? Divisible only by one and itself. Does any smaller number divide 17? Not 2 (odd), nor 3, 5, 7; and 7² already exceeds it, so none can remain.

answer: Yes, 17 is prime. Note the answer depended on first fixing what 'prime' means.

Provenance

Base model: unsloth/Qwen3-8B (Apache 2.0)
Fine-tune: AthrvShrn/Socrates-Qwen3-8B, QLoRA SFT on hand-authored Socratic reasoning data; see that card for training and evaluation details
Training data: AthrvShrn/Socratic-Reasoning, 136 hand-authored examples, 109 train / 27 held-out test
Quantized with llama.cpp via Unsloth after merging the LoRA adapter into the base

License

Apache 2.0, matching the base model.