You can deploy this model and start earning money today!

sushanrai/
phi3-cybersec-advisor-lora

Name: phi3-cybersec-advisor-lora
Availability: InStock

safetensors

phi3

custom_code

Views

No views yet

Phi-3 Mini (LoRA Fine-Tuned on MITRE-STIX-CVE-ExploitDB Dataset)

Model Summary

This model is a fine-tuned version of microsoft/phi-3-mini-128k-instruct using LoRA (Low-Rank Adaptation) and 8-bit quantization for parameter-efficient training.

The fine-tuning dataset is jason-oneal/mitre-stix-cve-exploitdb-dataset-alpaca, which contains security-related instruction-response examples (CVE, STIX, ExploitDB context).

The goal of this model is to act as a cybersecurity knowledge assistant that can answer questions about CVEs, exploits, and related security topics.

Base Model: microsoft/phi-3-mini-128k-instruct
Fine-tuning Method: LoRA (8-bit PEFT with bitsandbytes)
Dataset: jason-oneal/mitre-stix-cve-exploitdb-dataset-alpaca
Languages: English
Context Length: 128k tokens

Intended Uses

Designed for: cybersecurity Q&A, reasoning about vulnerabilities, exploits, and threat intelligence.
Can be used for: research, learning, and prototyping of cyber threat assistants.

Dataset

Name: MITRE-STIX-CVE-ExploitDB Dataset (Alpaca format)
Source: jason-oneal/mitre-stix-cve-exploitdb-dataset-alpaca
Schema: Instruction–response pairs in Alpaca format
Size Used: Up to 5,000 training samples (subset for efficiency)

Training Procedure

Frameworks: Hugging Face Transformers, PEFT, bitsandbytes
Precision: 8-bit quantization (bnb.int8) + FP16 training
Optimizer: AdamW
Batch Size: 4 per device
Epochs: 1
Learning Rate: 3e-4
Warmup Steps: 50
Max Length: 1024 tokens
LoRA Config:
- r = 16
- alpha = 16
- dropout = 0.05
- target modules: q_proj, k_proj, v_proj, o_proj, w1, w2, dense

Evaluation

Metric: Training loss (did not include a validation set in this run).
Qualitative Evaluation: The model produces meaningful responses to security-related prompts, but further fine-tuning with eval sets is recommended.

How to Use

python

from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

model_name = "sushanrai/phi3-cybersec-advisor-lora"  # replace with your repo
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)

prompt = "Explain CVE-2021-44228 in simple terms"
output = pipe(prompt, max_new_tokens=300, do_sample=True)
print(output[0]["generated_text"])

Ethical Considerations

This model is trained on cybersecurity data and may produce outputs that describe exploits.
Should only be used for research, learning, and defensive security purposes.
Not intended for malicious use.

Citation

If you use this model, please cite:

bibtex

@misc{phi3_mitre_lora_2025,
  title={Phi-3 Mini LoRA Fine-Tuned on MITRE-STIX-CVE-ExploitDB Dataset},
  author={HackDMSV},
  year={2025},
  howpublished={\url{https://huggingface.co/sushanrai/phi3-cybersec-advisor-lora}}
}