SAST finding triage model — fine-tuned on real production security scan data to classify Static Application Security Testing findings as true positives, false positives, or uncertain, with CWE labels, confidence scores, and remediation guidance.

Overview

Rule-based SAST tools generate enormous volumes of findings, a significant portion of which are false positives. Security analysts spend hours triaging noise instead of fixing real vulnerabilities. teptez-ai is a 7B-parameter LLM fine-tuned specifically to automate this triage step.

Given a SAST finding (title, CWE, severity, code snippet, taint flow), teptez-ai returns a structured JSON verdict:

true_positive — finding is real, exploit path exists
false_positive — finding is noise, safe to suppress
uncertain — insufficient context, escalate to analyst

Fine-tuned on production findings from the Teptez security platform — real codebases, real scan data, real analyst labels.

Key specs

Property	Value
Base model	Qwen2.5-Coder-7B-Instruct
Quantization	GGUF Q4_K_M
Model size	~4.7 GB
Inference speed	~100 tok/s (RTX 3090 24GB)
Context window	8192 tokens
License	Apache 2.0

Benchmark Results

Evaluated against OWASP Benchmark v1.2 — the standard industry benchmark for SAST tools — using the official Youden's J statistic (J = TPR − FPR).

J = 0.0 is random. J = 1.0 is perfect. Open-source SAST tools typically score 0.30–0.45.

Head-to-head vs base model

Model	TPR	FPR	Youden J	vs base
teptez-ai (Q4_K_M)	0.68	0.57	0.109	+0.048 (+79%)
qwen2.5-coder-7b (base)	0.66	0.60	0.061	—

Fine-tuning delivers a 79% relative improvement in Youden's J over the base model, primarily by cutting the false positive rate from 0.60 to 0.57 across the full benchmark.

Per-category breakdown

CWE Category	teptez-ai J	base J	Delta
Command Injection (CWE-78)	0.40	0.22	+0.18
SQL Injection (CWE-89)	0.33	0.18	+0.15
XSS (CWE-79)	0.28	0.12	+0.16
Path Traversal (CWE-22)	0.22	0.09	+0.13
Weak Randomness (CWE-330)	0.13	0.05	+0.08
Crypto/Hash (CWE-327/328)	0.00	0.01	-0.01
Auth/Authz (CWE-862/639)	0.02	0.01	+0.01
Timing (CWE-208)	0.05	0.04	+0.01
Secure Cookie (CWE-614)	-0.08	0.52	-0.60 ⚠️

Strong on injection classes. teptez-ai significantly outperforms the base model across all injection-type CWEs (cmdi/sqli/xss/path/weakrand). These are the highest-volume SAST categories in real codebases.

Securecookie regression. CWE-614 (missing HttpOnly/Secure flags) shows a severe regression vs the base model. Do not use teptez-ai to triage cookie security findings. This is a known training artifact being fixed in the next round.

Dead categories. Crypto, authz, and timing categories have near-zero Youden J on both models — 7B parameters are insufficient for these without full class context. Escalate to frontier models or human analysts.

Production run

On 369 real production SAST findings from live codebases:

17% rejected as false positive (~63 findings suppressed)
Injection-class findings: majority of suppressions, generally accurate
Authz/crypto findings: some wrong suppressions (do not enable for these categories)

Usage

Recommended architecture

Use teptez-ai as a gated FP suppressor, not a confirmer:

Rule-engine finding
        │
        ▼
Is CWE in injection classes?  ──No──▶  Keep finding (don't run model)
        │ Yes
        ▼
Run teptez-ai with full function + taint context
        │
        ├── verdict: false_positive, confidence > 0.75  ──▶  Suppress finding
        ├── verdict: true_positive                       ──▶  Keep finding
        └── verdict: uncertain / confidence < 0.75      ──▶  Escalate to frontier model / analyst

Only suppress on false_positive — never on true_positive. The model is biased toward flagging (FPR 0.57), so a false_positive verdict is rare and higher-precision.

Injection-class CWEs only (where Youden J ≥ 0.13):

CWE-78 Command Injection
CWE-79 Cross-Site Scripting
CWE-89 SQL Injection
CWE-22 Path Traversal
CWE-330 Weak Randomness

Never auto-suppress:

CWE-614 Secure Cookie (regression — model worse than random)
CWE-327/328 Weak Crypto/Hash (near-zero J)
CWE-862/639 Auth/Authz/IDOR (near-zero J)
CWE-208 Timing Attacks (near-zero J)

Running with llama.cpp / Ollama

bash

# Pull via Ollama
ollama pull hf.co/Kuyash/teptez-ai:Q4_K_M

# Or run directly with llama.cpp
./llama-cli -m teptez-ai-Q4_K_M.gguf \
  --temp 0.1 \
  --top-p 0.9 \
  -n 512 \
  -p "<prompt>"

Python integration

python

import json
import requests

def triage_finding(finding: dict) -> dict:
    prompt = f"""<|im_start|>system
You are a SAST triage expert. Analyze this finding and return JSON with keys:
verdict (true_positive|false_positive|uncertain), confidence (0.0-1.0),
cwe (string), explanation (string), remediation (string).
<|im_end|>
<|im_start|>user
Finding: {finding['title']}
CWE: {finding.get('cwe_id', 'unknown')}
Severity: {finding.get('severity', 'MEDIUM')}
Code:
{finding.get('code_snippet', '')}

Taint flow: {finding.get('data_flow', 'not available')}
<|im_end|>
<|im_start|>assistant
"""
    response = requests.post("http://localhost:11434/api/generate", json={
        "model": "teptez-ai",
        "prompt": prompt,
        "stream": False,
        "options": {"temperature": 0.1}
    })
    text = response.json()["response"].strip()
    # Strip markdown fences if present
    if text.startswith("```"):
        text = text.split("```")[1]
        if text.startswith("json"):
            text = text[4:]
    return json.loads(text)

# Gate: only run on injection CWEs
INJECTION_CWES = {"CWE-78", "CWE-79", "CWE-89", "CWE-22", "CWE-330"}

def should_suppress(finding: dict) -> bool:
    cwe = finding.get("cwe_id", "")
    if cwe not in INJECTION_CWES:
        return False  # don't touch non-injection
    result = triage_finding(finding)
    return (
        result.get("verdict") == "false_positive"
        and result.get("confidence", 0) >= 0.75
    )

Input / Output Format

Prompt template

<|im_start|>system
You are a SAST triage expert. Analyze this finding and return JSON.
<|im_end|>
<|im_start|>user
Finding: {title}
CWE: {cwe_id}
Severity: {severity}
Code:
{code_snippet}

Taint flow: {data_flow}
<|im_end|>
<|im_start|>assistant

Tips for best results:

Provide the full function, not just the flagged line — avoids "insufficient context" errors
Include taint flow when available (source → sink path from your SAST tool)
Keep code under 2048 tokens; truncate from the bottom if needed

Output schema

json

{
  "verdict": "true_positive" | "false_positive" | "uncertain",
  "confidence": 0.85,
  "cwe": "CWE-89",
  "explanation": "User input from request.getParameter() flows directly into a string-concatenated SQL query with no parameterization or escaping.",
  "remediation": "Replace string concatenation with a PreparedStatement: `conn.prepareStatement(\"SELECT * FROM users WHERE id = ?\")` and bind the parameter with `stmt.setString(1, userId)`."
}

Field	Type	Description
`verdict`	string	`true_positive`, `false_positive`, or `uncertain`
`confidence`	float	0.0–1.0; scores < 0.75 should be treated as uncertain
`cwe`	string	Classified CWE identifier
`explanation`	string	Why the model reached this verdict
`remediation`	string	Concrete fix recommendation

Limitations

Known issues (as of current release)

CWE-614 Secure Cookie — severe regression. teptez-ai scores Youden J = −0.08 on secure cookie findings, compared to 0.52 for the base model. This is a catastrophic regression caused by training data imbalance. Do not use teptez-ai for HttpOnly/Secure flag findings until this is fixed.

High overall FPR (0.57). The model over-flags — it sees vulnerability in safe code, especially in crypto, auth, and cookie-related code patterns. A false_positive verdict is more reliable than a true_positive verdict because it swims against the model's bias.

Dead categories (crypto/authz/timing). CWE-327/328/614/862/639/208 have near-zero Youden J. The model lacks sufficient training signal for these categories. Use a frontier model (Claude, GPT-4o, Gemini) or a human analyst for these.

7B parameter ceiling. Subtle IDOR, broken access control, and privilege escalation patterns require understanding class hierarchy, authentication flow, and business logic across multiple files. A 7B model with single-function context cannot reliably detect these.

GGUF Q4_K_M quantization. ~4-bit quantization introduces slight accuracy loss vs fp16. For maximum accuracy, use the Q8_0 variant (if available) or the full fp16 model.

Snippet-only inputs fail. If you pass only the flagged 1–3 lines without the surrounding function, the model frequently returns uncertain with "insufficient context." Always include the full function body.

Reproducing the Benchmark

bash

# 1. Clone OWASP Benchmark
git clone https://github.com/OWASP-Benchmark/BenchmarkJava
cd BenchmarkJava && mvn package -DskipTests

# 2. Run evaluation (requires teptez-ai running on Ollama)
python salad/eval/owasp_eval.py      # stratified sample → results.jsonl
python salad/eval/owasp_score.py     # Youden J + per-category table
python salad/eval/owasp_compare.py   # head-to-head vs base model

The eval script uses a stratified sample of OWASP BenchmarkJava test cases, covering all CWE categories proportionally. Scoring follows the official OWASP methodology (Youden's J = TPR − FPR).

Roadmap

The following improvements are planned for the next fine-tuning round:

Round 2 targets

Improvement	Target metric
Flood training with safe-code negatives (50/50 balance)	FPR < 0.30
Fix securecookie regression (restore CWE-614 training data)	J(CWE-614) > 0.40
Add dead-category examples (crypto/authz/timing)	J(CWE-327/328) > 0.10
Calibrate confidence score (train explicit `uncertain` label)	Confidence Brier score < 0.15
Full-function context at train AND inference	Reduce "uncertain" on short snippets

Root cause of FPR problem: Current training set is vuln-heavy (more vulnerable examples than safe ones). The model learned to flag aggressively. Rebalancing to 50/50 with explicit safe variants (parameterized SQL, escaped HTML, validated paths, compare_digest timing-safe comparisons, role-checked endpoints, strong ciphers) is the single highest-leverage fix.

Securecookie fix: Restore the original training examples for CWE-614 that were accidentally dropped. Mix with new negative examples. Lower learning rate for this category to avoid forgetting again.

No catastrophic forgetting: Round 2 will mix old injection data with new negatives and use a lower learning rate on the balanced set, following standard continual learning practice.

Round 3 vision

Full-function + cross-file taint context (requires longer context fine-tune)
Multi-label output (multiple CWEs per finding)
Confidence calibration verified against held-out production labels
Youden J > 0.25 across all injection categories
FPR < 0.30 overall

Citation

If you use teptez-ai in your research or tooling, please cite:

bibtex

@misc{teptez-ai-2026,
  title     = {teptez-ai: A Fine-Tuned LLM for SAST Finding Triage},
  author    = {Teptez Security},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/Kuyash/teptez-ai}
}

Evaluated against OWASP Benchmark v1.2 using the official Youden's J scoring methodology.

OWASP Benchmark — the benchmark used for evaluation
Qwen2.5-Coder-7B-Instruct — the base model

teptez-ai is a security research model. Results may vary across codebases and languages. Always have a human analyst review suppressed findings in critical security contexts.

Kuyash/teptez-ai

teptez-ai