Fine-tuned gpt-oss-20b on the AlicanKiraz0/All-CVE-Records-Training-Dataset using Unsloth with LoRA (rank 32) and merged back to BF16 for compatibility with vLLM, Hugging Face Transformers, and GGUF conversions.

Training Overview

Base model: unsloth/gpt-oss-20b-BF16
Dataset: AlicanKiraz0/All-CVE-Records-Training-Dataset
Hardware: single NVIDIA H100 80GB
Sequence length: 2048
Batch: 2 (grad accum 4 → effective 8)
Learning rate: 2e-4, linear warmup 5 steps
Steps: 100 for quick verification run (expand for full epoch)
Loss masking: full conversation (system, user, assistant)

Files

model-0000X-of-00009.safetensors: merged BF16 shards
config.json: GPT-OSS architecture config
tokenizer.json and template: Harmony/GPT-OSS chat format
chat_template.jinja: OpenAI Harmony-compatible chat template

Quick Usage (vLLM)

bash

pip install vllm==0.11.2 transformers==4.57.2
python - <<'PY'
from vllm import LLM, SamplingParams
from transformers.processing_utils import ProcessorMixin
import transformers
transformers.ProcessorMixin = ProcessorMixin

llm = LLM(
    model="Kushalkhemka/CyberOSS-CVE",
    tokenizer="unsloth/gpt-oss-20b-BF16",
    dtype="bfloat16",
)
prompt = "You are a cybersecurity assistant. Summarize CVE-2010-3763."
out = llm.generate([prompt], SamplingParams(max_tokens=128))[0]
print(out.outputs[0].text)
PY

HF Transformers

python

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("Kushalkhemka/CyberOSS-CVE", torch_dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("unsloth/gpt-oss-20b-BF16")

License

Matches upstream unsloth/gpt-oss-20b (LGPL-3.0). Respect dataset terms when redistributing.