KucLab Hertz 0.3
A Czech/English STEM assistant built by
KucLab on top of
Qwen2.5-14B-Instruct, fine-tuned to be sharper on physics, chemistry, biology and mathematics in Czech — while carrying forward the base model's general capability, 128k-token context reach, and native tool-calling support.
What this is
Hertz 0.3 is a LoRA fine-tune (r=8, merged into the base weights) trained on a self-distilled corpus of Czech/English STEM concepts, worked problems, and formatting examples. Training data was generated by prompting Qwen3.8-27B (Alibaba/Tongyi, Apache 2.0) — a stronger reasoning model used purely as a data-generation teacher, never trained itself — and used to fine-tune the actual 14B model that ships here.
- Base: Qwen/Qwen2.5-14B-Instruct (14.7B params, Apache 2.0)
- Method: QLoRA, r=8 / alpha=16, merged to fp16 then quantized
- Context: extended to 128k via YaRN rope scaling (native 32k → 131072, factor 4.0)
- Training data: 242 self-distilled rows — Czech/English STEM concepts & terminology, worked problems with step-by-step reasoning, Markdown/code formatting examples, identity
- Format available: GGUF (q4_k_m, ~9GB) for
llama.cpp/Ollama, plus the raw LoRA adapter for anyone who wants to merge it onto their own copy of the base model
Why a fine-tune instead of a bigger model
Qwen2.5-14B fits comfortably on a single consumer/cloud GPU (23GB) for both inference and QLoRA training. Rather than chase parameter count, this project focuses the model's existing capacity on a specific gap: strong, natural Czech STEM terminology and step-by-step worked solutions, which the base model handles less fluently out of the box.
Quickstart (Ollama)
Important: ollama pull hf.co/... alone does NOT apply this model's system prompt (identity + personality) — Ollama only fetches the raw GGUF from Hugging Face, it does not read a repo's Modelfile. Without the system prompt, the model falls back to identifying as a generic Gemma model. Use ollama create with the Modelfile below instead — it pulls the weights AND applies the system prompt in one step:
1curl -O https://huggingface.co/KucLab/kuclab-hertz-0.3/resolve/main/Modelfile
2ollama create kuclab-hertz-0.3 -f Modelfile
3ollama run kuclab-hertz-0.3
(The Modelfile's FROM line points at hf.co/KucLab/kuclab-hertz-0.3:Q4_K_M, so this pulls the same GGUF automatically — no separate download needed.)
Or build from the Modelfile in this repo with the bundled GGUF.
Example
Q: Kolik je 17 × 23? Ukaž postup.
A: 17 × 23 = 17 × (20 + 3) = 340 + 51 = 391
Q: Vysvětli druhý Newtonův zákon.
A: (explains F = dp/dt, in fluent Czech, with the formula and its meaning)
Benchmarks
Measured against the same base model (qwen2.5:14b-instruct, Q4_K_M via Ollama), same prompts, same grading code, both models cold.
MMLU-Pro STEM (240 held-out questions, this project's own curated subset — not a claim of parity with the standard published MMLU-Pro leaderboard number)
| base | Hertz 0.3 | Δ |
|---|
| Biology | 78.3% | 75.0% | −3.3pp |
| Chemistry | 71.7% | 71.7% | 0pp |
| Math | 88.3% | 86.7% | −1.6pp |
| Physics | 71.7% | 80.0% | +8.3pp |
| Total | 77.5% | 78.3% | +0.8pp |
Czech scientific terminology (206 held-out terms, both translation directions, this project's own curated set)
| base | Hertz 0.3 | Δ |
|---|
| CS → EN | 79.6% | 79.6% | 0pp |
| EN → CS | 47.6% | 51.5% | +3.9pp |
| Total | 63.6% | 65.5% | +1.9pp |
Read this as: no regression on general STEM reasoning, a real (if modest) gain on Czech terminology — which is exactly what the fine-tune targeted. Physics moved the most; biology and math moved slightly the other way. These are two custom benchmarks built for this project, not standardized public leaderboards — useful for before/after comparison on this exact model, not for cross-model bragging rights.
Honest status
This is a small-scale, single-GPU fine-tuning project, not a frontier lab release. What's verified:
- ✅ Coherent, correct Czech and English output (spot-checked: arithmetic, physics, chemistry, terminology)
- ✅ Correctly identifies as a KucLab model, not as "Qwen" (LoRA + system prompt)
- ✅ 128k context window configured and loadable (long-context quality not yet independently verified with held-out long documents)
- ✅ MMLU-Pro STEM and Czech terminology benchmarked against base (see above) — no regression, modest gains
What's not done yet:
- ⏳ Tool-calling fine-tuning — the base model supports function calling natively, but this fine-tune did not add tool-use training examples
- ⏳ No uncensoring/decensoring pass — safety behavior is inherited from the base model as-is
- ⏳ No standardized public benchmarks (GSM8K, HumanEval, official MMLU, etc.) — only the two project-specific evals above
If you're deciding whether this fits your use case: treat it as "Qwen2.5-14B-Instruct, nudged toward Czech STEM fluency and KucLab identity," not as a categorically different or more powerful model than its base.
License
Apache 2.0, inherited from Qwen2.5-14B-Instruct. This repository includes a LoRA adapter and a merged/quantized derivative of that base model; see LICENSE for full terms. Qwen3.8-27B (used only to generate training data, never trained or redistributed here) is separately licensed under Apache 2.0 by Alibaba/Tongyi.
Credits
- Base model: Qwen/Qwen2.5-14B-Instruct (Alibaba Cloud, Apache 2.0)
- Training-data teacher: Qwen/Qwen3.8-27B (Alibaba/Tongyi, Apache 2.0)
- Fine-tuning, dataset construction, and packaging: KucLab
