This model is a fine-tuned version of GPT-OSS-20B specialized for CVE (Common Vulnerabilities and Exposures) analysis and security recommendation generation using DoRA (Weight-Decomposed Low-Rank Adaptation).

Key Features

Analyzes CVE vulnerabilities and provides actionable security recommendations
Trained on 5,000+ diverse CVE policy recommendations
Uses DoRA for efficient fine-tuning
Provides structured recommendations with rationale
Identifies security risks and suggests mitigation strategies

Quick Start

Installation

pip install transformers torch

Basic Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "sainikhiljuluri/gpt-oss-20b-cve-cybersecurity",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained("sainikhiljuluri/gpt-oss-20b-cve-cybersecurity")

# Prepare CVE analysis request
messages = [
    {
        "role": "system",
        "content": "You are an expert cybersecurity analyst specializing in vulnerability assessment and remediation. Analyze CVE information and provide actionable security recommendations with clear rationale."
    },
    {
        "role": "user",
        "content": """Analyze the following vulnerability and provide security recommendations:

CVE ID: CVE-2024-1234
Vulnerability Summary: SQL injection vulnerability in web application allows remote attackers to execute arbitrary SQL commands
CVSS Score: 9.8
Weakness Type: SQL Injection
CWE Code: CWE-89"""
    }
]

# Generate response
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)

response = tokenizer.decode(
    outputs[0][inputs.shape[-1]:],
    skip_special_tokens=True
)

print(response)

Training Details

Dataset

Source: CVE Policy Recommendations (5,000+ diverse examples)
Format: Structured CVE information with expert recommendations
Split: 98% training, 2% evaluation

Training Configuration

Method: DoRA (Weight-Decomposed Low-Rank Adaptation)
LoRA Rank: 32
LoRA Alpha: 32
Dropout: 0.10
Base Precision: BF16
Merged Precision: FP16
Epochs: 3
Batch Size: 2 (effective: 16 with gradient accumulation)
Learning Rate: 1.5e-4
Training Platform: Google Colab A100 GPU

Training Results

Final Perplexity: 1.57 (Excellent)
Average Loss: 0.45
BLEU-4 Score: 0.4954
ROUGE-L F1: Not yet calculated
Semantic Similarity: 0.6305
Quality Retention: 94.3%

Capabilities

CVE Analysis

The model can analyze various types of vulnerabilities including:

SQL Injection
Cross-Site Scripting (XSS)
Authentication Bypass
Denial of Service (DoS)
Buffer Overflow
Remote Code Execution
And many more...

Recommendation Format

The model provides structured recommendations including:

Recommended Action: Specific steps to mitigate the vulnerability
Rationale: Explanation of why these actions are necessary
Risk Assessment: Evaluation of severity and impact
Implementation Details: Technical guidance for remediation

Example Output

Input:

CVE ID: CVE-2024-SQL-001
Vulnerability Summary: SQL injection in authentication module
CVSS Score: 9.8
Weakness Type: SQL Injection

Output:

Recommended Action:
1. Immediately implement parameterized queries/prepared statements
2. Update web application framework to latest patched version
3. Enable Web Application Firewall (WAF) rules
4. Conduct comprehensive code review of authentication module
5. Deploy input validation and sanitization

Rationale:
Critical CVSS score of 9.8 indicates immediate action required. SQL injection
in authentication module poses severe risk of unauthorized access and data breach.
Parameterized queries prevent SQL injection at the source by separating SQL logic
from user input...

Performance Metrics

Metric	Score	Status
Perplexity	1.57	Excellent
BLEU-4	0.4954	Good
Semantic Similarity	0.6305	Good
Quality Retention	94.3%	Excellent

Model Architecture

Base Model: GPT-OSS-20B
Parameters: ~20 billion
Precision: FP16 (merged model)
Adapter Type: DoRA (rank-32)
Context Length: 2048 tokens
Model Size: ~40GB

Deployment

HuggingFace Inference Endpoints

python

import requests

API_URL = "https://YOUR-ENDPOINT.endpoints.huggingface.cloud"
headers = {"Authorization": f"Bearer {YOUR_HF_TOKEN}"}

def analyze_cve(cve_info):
    payload = {
        "inputs": cve_info,
        "parameters": {
            "max_new_tokens": 512,
            "temperature": 0.7,
            "top_p": 0.9
        }
    }
    response = requests.post(API_URL, headers=headers, json=payload)
    return response.json()

Local Deployment

python

# Load model locally
model = AutoModelForCausalLM.from_pretrained(
    "sainikhiljuluri/gpt-oss-20b-cve-cybersecurity",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

Limitations

Domain-Specific: Optimized for CVE analysis, may not generalize well to other tasks
Training Data: Limited to vulnerabilities seen in training data
No Real-Time Data: Model knowledge is based on training data cutoff
Requires Context: Best results when provided with complete CVE information
Not a Replacement: Should complement, not replace, professional security analysis

Ethical Considerations

This model is for research and educational purposes
Always validate security recommendations with professional tools
Do not rely solely on AI for critical security decisions
Use responsibly and ethically in cybersecurity contexts

Citation

If you use this model in your research, please cite:

bibtex

@misc{gpt-oss-20b-cve-2025,
  author = {Sainikhil Juluri},
  title = {GPT-OSS-20B CVE Cybersecurity Model},
  year = {2025},
  publisher = {HuggingFace},
  url = {https://huggingface.co/sainikhiljuluri/gpt-oss-20b-cve-cybersecurity}
}

Training Details

Training Date: 2025-11-04

Hardware: Google Colab A100 GPU

Framework:

PyTorch 2.8.0+cu126
Transformers
PEFT (DoRA)

Contact

For questions or collaborations:

Open an issue on the model repository
Connect via HuggingFace discussions

License

Apache 2.0 License

Acknowledgments

Base model: GPT-OSS-20B by OpenAI
Training method: DoRA (Weight-Decomposed Low-Rank Adaptation)
Dataset: CVE Policy Recommendations

Built with: Transformers • PEFT • DoRA • PyTorch