You can deploy this model and start earning money today!

scthornton/
llama-3.2-3b-securecode

Name: llama-3.2-3b-securecode
Rating: 2.5 (12 reviews)

Views

No views yet

Llama 3.2 3B SecureCode

Parameters

Dataset

OWASP

Method

Security-specialized code model fine-tuned on the SecureCode dataset

Dataset | Paper (arXiv:2512.18542) | Model Collection | perfecXion.ai

What This Model Does

This model generates secure code when developers ask about building features. Instead of producing vulnerable implementations (like 45% of AI-generated code does), it:

Identifies the security risks in common coding patterns
Provides vulnerable and secure implementations side by side
Explains how attackers would exploit the vulnerability
Includes defense-in-depth guidance: logging, monitoring, SIEM integration, infrastructure hardening

The model was fine-tuned on 2,372 security training examples covering both traditional web security (OWASP Top 10 2021) and AI/ML security (OWASP LLM Top 10 2025).

Model Details


Base Model	Llama 3.2 3B Instruct
Parameters	3B
Architecture	Llama
Tier	Tier 1: Accessible
Method	bf16 LoRA (no quantization)
LoRA Rank	16 (alpha=32)
Target Modules	`q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj` (7 modules)
Training Data	scthornton/securecode (2,372 examples)
Hardware	NVIDIA DGX Spark GB10 (Blackwell, unified memory)

Smallest model in the collection. Ideal for edge deployment, rapid prototyping, and resource-constrained environments.

Quick Start

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

# Optional: 4-bit quantization for low-VRAM inference (training was bf16 LoRA)
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-3B-Instruct",
    quantization_config=bnb_config,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("scthornton/llama-3.2-3b-securecode")
model = PeftModel.from_pretrained(base_model, "scthornton/llama-3.2-3b-securecode")

# Ask a security-relevant coding question
messages = [
    {"role": "user", "content": "How do I implement JWT authentication with refresh tokens in Python?"}
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=2048, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Dataset

Trained on the full SecureCode unified dataset:

2,372 total examples (1,625 web security + 747 AI/ML security)
20 vulnerability categories across OWASP Top 10 2021 and OWASP LLM Top 10 2025
12+ programming languages and 49+ frameworks
4-turn conversational structure: feature request, vulnerable/secure implementations, advanced probing, operational guidance
100% incident grounding: every example tied to real CVEs, vendor advisories, or published attack research

Hyperparameters

Parameter	Value
LoRA rank	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	7 linear layers
Quantization	None (bf16 base weights)
Learning rate	2e-4
LR scheduler	Cosine with 100-step warmup
Epochs	3
Per-device batch size	4
Gradient accumulation	4x
Effective batch size	16
Max sequence length	2048 tokens
Optimizer	adamw_torch_fused
Attention	PyTorch SDPA (fused)
Precision	bf16

Notes: Uses EOS token for padding. Compact enough for consumer GPUs (8GB+ VRAM with 4-bit quantization).

Security Coverage

Web Security (1,625 examples)

OWASP Top 10 2021: Broken Access Control, Cryptographic Failures, Injection, Insecure Design, Security Misconfiguration, Vulnerable Components, Authentication Failures, Software Integrity Failures, Logging/Monitoring Failures, SSRF.

Languages: Python, JavaScript, Java, Go, PHP, C#, TypeScript, Ruby, Rust, Kotlin, YAML.

AI/ML Security (747 examples)

OWASP LLM Top 10 2025: Prompt Injection, Sensitive Information Disclosure, Supply Chain Vulnerabilities, Data/Model Poisoning, Improper Output Handling, Excessive Agency, System Prompt Leakage, Vector/Embedding Weaknesses, Misinformation, Unbounded Consumption.

Frameworks: LangChain, OpenAI, Anthropic, HuggingFace, LlamaIndex, ChromaDB, Pinecone, FastAPI, Flask, vLLM, CrewAI, and 30+ more.

SecureCode Model Collection

This model is part of the SecureCode collection of 9 security-specialized models:

Model	Base	Size	Tier	HuggingFace
Llama 3.2 SecureCode	meta-llama/Llama-3.2-3B-Instruct	3B	Accessible	`llama-3.2-3b-securecode`
Gemma 4 E4B SecureCode	google/gemma-4-E4B-it	E4B (8B raw)	Accessible	`gemma-4-e4b-securecode`
Qwen2.5 Coder SecureCode	Qwen/Qwen2.5-Coder-7B-Instruct	7B	Mid-size	`qwen2.5-coder-7b-securecode`
DeepSeek Coder SecureCode	deepseek-ai/deepseek-coder-6.7b-instruct	6.7B	Mid-size	`deepseek-coder-6.7b-securecode`
CodeGemma SecureCode	google/codegemma-7b-it	7B	Mid-size	`codegemma-7b-securecode`
CodeLlama SecureCode	codellama/CodeLlama-13b-Instruct-hf	13B	Large	`codellama-13b-securecode`
Qwen2.5 Coder 14B SecureCode	Qwen/Qwen2.5-Coder-14B-Instruct	14B	Large	`qwen2.5-coder-14b-securecode`
StarCoder2 SecureCode	bigcode/starcoder2-15b-instruct-v0.1	15B	Large	`starcoder2-15b-securecode`
Granite 20B Code SecureCode	ibm-granite/granite-20b-code-instruct-8k	20B	XL	`granite-20b-code-securecode`

Choose based on your deployment constraints: 3B for edge/mobile, 7B for general use, 13B-15B for deeper reasoning, 20B for maximum capability.

SecureCode Dataset Family

Dataset	Examples	Focus	Link
SecureCode	2,372	Unified (web + AI/ML)	scthornton/securecode
SecureCode Web	1,625	Web security (OWASP Top 10 2021)	scthornton/securecode-web
SecureCode AI/ML	747	AI/ML security (OWASP LLM Top 10 2025)	scthornton/securecode-aiml

Intended Use

Use this model for:

Training AI coding assistants to write secure code
Security education and training
Vulnerability research and secure code review
Building security-aware development tools

Do not use this model for:

Offensive exploitation or automated attack generation
Circumventing security controls
Any activity that violates the base model's license

Changelog

2026-07 (v2, current): Retrained on the audited SecureCode release (2,372 examples: 1,625 web + 747 AI/ML) using bf16 LoRA on an NVIDIA DGX Spark GB10 (Blackwell). Same LoRA shape and hyperparameters as v1, except: no quantization during training, optimizer adamw_torch_fused instead of paged_adamw_8bit, PyTorch SDPA attention, dynamic per-batch padding. A new family member, gemma-4-e4b-securecode, was added in this refresh.
v1 (2026-02): QLoRA 4-bit NF4 on NVIDIA A100 40GB, trained on the pre-audit 2,185-example dataset.

Citation

bibtex

@misc{thornton2026securecode,
  title={SecureCode: A Production-Grade Multi-Turn Dataset for Training Security-Aware Code Generation Models},
  author={Thornton, Scott},
  year={2026},
  publisher={perfecXion.ai},
  url={https://huggingface.co/datasets/scthornton/securecode},
  note={arXiv:2512.18542}
}

License

This model is released under the llama3.2 license (inherited from the base model). The training dataset (SecureCode) is licensed under CC BY-NC-SA 4.0.

scthornton/llama-3.2-3b-securecode

Llama 3.2 3B SecureCode

What This Model Does

Model Details

Quick Start

Training Details

Dataset

Hyperparameters

Security Coverage

Web Security (1,625 examples)

AI/ML Security (747 examples)

SecureCode Model Collection

SecureCode Dataset Family

Intended Use

Changelog

Citation

Links

License

scthornton/
llama-3.2-3b-securecode