GGUF quantizations of a LoRA fine-tune of Mistral-7B-v0.3 on French health-insurance terminology for healthcare professionals.

50-question French insurance Q&A benchmark across three models — the base mistral:7b, the FP16 fine-tune, and the Q8 quantization from this repo — judged by Claude against a frozen rubric (4 sub-dimensions + holistic overall, 0–5). Full report: model-comparison_2026-06-16.html.

The unquantized FP16 fine-tune is the natural accuracy winner — that's where the industry knowledge is carried in full, leading every question category (verbatim 4.40, paraphrased 3.45, new 2.60) and every domain-relevant dimension. The Q8 quantization in this repo retains roughly 80% of the overall score and 90% of the factual accuracy at ~2× the inference speed (3.07 s vs 5.94 s median latency), making it the practical choice for local inference. The base mistral:7b ranks last on every domain dimension despite winning raw french_quality (4.76 vs 4.34) — fluent French paired with hallucinated insurance facts (factual_accuracy 1.44 / 5). Pick Q8 when local speed and footprint matter; pick the FP16 model when you want every accuracy point.

Benchmark harness, prompts, rubric, and per-question results: rlondner/ft-mistral7b-french-insurance-tests.

Base model: mistralai/Mistral-7B-v0.3 Full-precision (safetensors) version: rlondner/mistral-7b-insurance