Fine-tuned Qwen2.5-Coder-32B-Instruct for security patch backporting via per-hunk code generation. Maintained as part of the openSUSE security tooling effort, alongside the cve-backport-tool CLI.
Instead of generating unified diffs, this model takes a vulnerable code region and a fix description, and outputs the fixed version of the code. A programmatic diff then produces the final patch.
MoE variant available: An MoE-based alternative built on
Qwen3-Coder-30B-A3B (3B active parameters) is hosted at
anicka/cve-backport-codegen-v5-qwen3-coder-30b-a3b.
It scores 91.9% recall on the same 100-example eval — 1.2 pt below this
dense model — while running roughly 10× faster at inference due to sparse
MoE activation. Recommended for bulk CVE backport workflows where
throughput matters.
v5 uses a codegen-only dataset — all 36,166 training examples follow the same 3-turn format. v4 mixed in 772 five-turn test-generation examples which diluted codegen focus. Dropping those and training for 2 epochs (vs 1 in v4) improved adapted-tier recall.
Comparison with Frontier Models
Same eval, same 100 examples, optimized prompts with markdown stripping:
Model
Recall
Precision
Exact
Failures
CVE Backport v5 (32B fine-tuned)
93%
94%
83/100
3
Gemini 3.1 Pro (frontier, zero-shot)
27%
24%
10/100
50
Gemini 2.0 Flash (frontier, zero-shot)
13%
17%
4/100
81
Fine-tuning on 36K domain-specific examples outperforms frontier models by 3-7x on this task.
Prompt Format
ChatML format. Each prompt covers one hunk region with 15 lines of context padding.
Code Generation (3-turn)
System:
You are a security patch backporting assistant.
Given vulnerable source code and a description of the upstream fix, output the FIXED version of the code.
Rules:
- Output ONLY the fixed code, nothing else — no explanations, no markdown fences
- Preserve exact formatting, indentation, and style of the original
- Make ONLY the changes described in the fix — do not modify anything else
- Do not add comments about what you changed
User:
## File: crypto/bn/bn.h
## Lines: 280-310
\```c
/* vulnerable source code region with 15 lines of context */
\```
## Fix
Add bounds check for BN_num_bits to prevent buffer over-read (CVE-2024-XXXX).
Assistant: The fixed version of the code region (just the code, no markup).
36,166 train / 1,834 eval (codegen-only, all 3-turn)
Effective batch size
8
Learning rate
1e-4 (cosine, 5% warmup)
Max sequence length
4,096 tokens
Hardware
2× NVIDIA H100 NVL 94GB
Training time
46.1 hours
Final eval loss
0.00602
Reproduction via Teapot
This model is reproducible via the teapot training pipeline. Once the dataset is composed, training is a four-command sequence:
bash
1git clone https://github.com/anicka-net/teapot
2cd teapot
3pip install -e .45# 1. Compose training data from the cve-backport module6teapot compose configs/cve-backport.config \7 --output train-cve-backport.jsonl
89# 2. Generate the QLoRA-HF launch script10teapot train configs/cve-backport.config \11 --backend qlora-hf \12 --train-data train-cve-backport.jsonl \13 --eval-data eval-cve-backport.jsonl \14 --output train-cve-backport.sh
1516# 3. Train (2× H100 NVL 94GB; ~46 hours)17bash train-cve-backport.sh
1819# 4. Final adapter is at output-teapot-cve-backport/final/
The teapot config (configs/cve-backport.config) pins all the hyperparameters listed in the Training table above. The qlora-hf backend invokes teapot.train_qlora_hf, a thin wrapper over the HuggingFace Trainer with bitsandbytes 4-bit quantization and PEFT LoRA.
An MoE variant trained on the same dataset is available at anicka/cve-backport-codegen-v5-qwen3-coder-30b-a3b — built on Qwen3-Coder-30B-A3B (3B active params), 91.9% recall on the same n=100 eval, ~10× faster inference.
Known Issues
The 3 failure cases (0% recall) are all complex libvirt patches involving multi-function adaptations across large files with significant structural differences. These likely require an agentic approach with source tree context.
Very long hunks (>2000 tokens) may be truncated due to the 4096-token training context.
Always review generated patches before applying to production systems.
License
Apache-2.0 (inherited from Qwen2.5-Coder-32B-Instruct).
