distilbert-insecure-output – AI Model by Builder117 | AlphaNeural AI

You can deploy this model and start earning money today!

Builder117/
distilbert-insecure-output

like

0

text-classification

insecure-output

owasp-llm-top10

Views

No views yet

distilbert-insecure-output

Fine-tuned DistilBERT classifier that detects dangerous payloads in LLM-generated output.

Covers OWASP LLM Top 10 — LLM02: Insecure Output Handling.

What it detects

Malicious code or injection payloads that an LLM might generate, including:

Cross-site scripting (XSS): <script>alert(document.cookie)</script>
SQL injection: '; DROP TABLE users; --
Command injection: | cat /etc/passwd
Path traversal: ../../etc/shadow
UNION-based SQL attacks

Labels

Label	ID	Meaning
`SAFE`	0	Safe output (normal text, parameterized queries, sanitized code)
`MALICIOUS`	1	Dangerous payload detected

Usage

python

from transformers import pipeline

clf = pipeline("text-classification", model="Builder117/distilbert-insecure-output")

clf("<script>alert(document.cookie)</script>")
# [{'label': 'MALICIOUS', 'score': 0.98}]

clf("SELECT * FROM products WHERE id = ?")
# [{'label': 'SAFE', 'score': 0.97}]  # parameterized — safe

Training

Base model: distilbert-base-uncased
Positive class: XSS payloads, SQL injection strings, command injection, path traversal
Negative class: parameterized queries, sanitized code, normal text, safe SQL

Limitations

Encoded payloads (base64, HTML entities, hex encoding) may evade detection
Context-blind: cannot determine if SQL is parameterized vs. raw string concatenation from text alone
May produce false positives on security documentation that quotes attack strings

Part of

LLM Threat Shield — OWASP LLM Top 10 detection suite.