Field	Value
Method	SFT
Precision	fp32
Quantization	none
Mode	steps
Num Epochs	1
Max Steps	4250
Eval Steps	1100
Save Steps	2200
LR	5e-05
Max Length	768
per_device_batch_size	1
grad_accum	8

Evaluation (greedy, fixed-length decode)

Metric	Score
BLEU-4	1.04
ROUGE-L	13.60
F1 (token-level)	24.55
chrF++	17.24
BERTScore F1	82.60
Perplexity	16.84

Notes: We normalize whitespace/punctuations, compute token-level P/R/F1, and use evaluate's sacrebleu/rouge/chrf/bertscore.

How to use

python

from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("nhonhoccode/qwen3-0-6b-cybersecqa-sft-freeze2-20251216-0236")
mdl = AutoModelForCausalLM.from_pretrained("nhonhoccode/qwen3-0-6b-cybersecqa-sft-freeze2-20251216-0236")
prompt = tok.apply_chat_template(
    [{"role":"system","content":"You are a helpful assistant specialized in cybersecurity Q&A."},
     {"role":"user","content":"Explain SQL injection in one paragraph."}],
    tokenize=False, add_generation_prompt=True
)
ids = tok(prompt, return_tensors="pt").input_ids
out = mdl.generate(ids, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))

Intended Use & Limitations

Domain: cybersecurity Q&A; not guaranteed to be accurate for legal/medical purposes.
The model can hallucinate or produce outdated guidance—verify before applying in production.
Safety: No explicit content filtering. Add guardrails (moderation, retrieval augmentation) for deployment.

Reproducibility (env)

transformers>=4.43,<5, accelerate>=0.33,<0.34, peft>=0.11,<0.13, datasets>=2.18,<3, evaluate>=0.4,<0.5, rouge-score, sacrebleu, huggingface_hub>=0.23,<0.26, bitsandbytes
GPU: T4-class; LoRA recommended for low VRAM.

Changelog

2025-12-16 02:37 — Initial release (SFT)