Semantic Drift Detection in Infrastructure-as-Code using Large Language Models: Qwen2.5-Coder-3B (GGUF)
A fine-tuned Qwen2.5-Coder-3B-Instruct model that assesses the security severity of a
Terraform Infrastructure-as-Code change. Given a before / after configuration pair, it returns
a severity level (Critical / High / Medium / Low), a 0–10 risk score, the governing security
policy, and a CWE identifier.
This file is the Q4_K_M GGUF build (~1.8 GB) for local inference via Ollama or llama.cpp.
Why
Existing tooling detects that infrastructure drifted from its declared configuration, but not
how dangerous the change is. This model adds that judgement so practitioners can prioritise.
How it was trained
Base model:Qwen/Qwen2.5-Coder-3B-Instruct
Method: LoRA (parameter-efficient) fine-tuning, then merged and quantised to Q4_K_M GGUF
Data: a grounded, leakage-free dataset of Terraform before/after changes, with severity
labels sourced from established security scanners (KICS, Checkov, tfsec) rather than
hand-authored, removing label bias
Evaluation: a policy-disjoint test set whose governing policies are never seen during
training, so scores reflect genuine generalisation
Performance (leakage-free, unseen-policy test)
This GGUF is the first fine-tuned iteration: macro-F1 0.347, accuracy 0.50. A later,
larger dataset iteration raised accuracy to 0.78 on the same test set (see the project
repository).
Usage: Ollama
bash
1ollama create semantic-drift -f Modelfile
2ollama run semantic-drift
