CyberSentinel CTI — Qwen3-4B-Instruct-2507 (GRPO)
CyberSentinel CTI is a 4-billion-parameter language model for
cyber threat
intelligence (CTI) tasks. It is a fine-tune of
Qwen/Qwen3-4B-Instruct-2507
trained with
GRPO (Group Relative Policy Optimization) using
Unsloth and
TRL, with reward functions that shape both
the
format of the output and the
correctness of the answer.
This repository ships the model in two forms:
| Format | File | Size | Use with |
|---|
| Full precision (bf16) | model-0000*-of-00002.safetensors | ~8.0 GB | 🤗 Transformers, vLLM |
| Quantized (Q4_K_M) | cybersentinel-cti-q4_k_m.gguf | ~2.5 GB | llama.cpp, Ollama, LM Studio |
ℹ️ Before publishing, replace your-username/cybersentinel-cti in the examples
below with the actual Hugging Face repository id.
Model Details
- Developed by: Emul Sazib
- Model type: Decoder-only causal language model (Qwen3 architecture)
- Language: English
- License: Apache 2.0
- Finetuned from:
Qwen/Qwen3-4B-Instruct-2507
(via the unsloth/Qwen3-4B-Instruct-2507 4-bit build)
- Fine-tuning method: GRPO reinforcement learning with LoRA adapters (rank 16), merged back to 16-bit
- Parameters: 4.05B total (LoRA trainable: 33.0M, ~0.81% of the model)
- Context length: up to 262,144 tokens (inherited from the base model)
- Prompt format: Qwen3 ChatML (
<|im_start|>role … <|im_end|>)
Architecture (from config.json)
| Field | Value |
|---|
| Architecture | Qwen3ForCausalLM |
| Hidden size | 2560 |
| Layers | 36 |
| Attention heads | 32 (8 KV heads, GQA) |
| Intermediate size | 9728 |
| Vocab size | 151,936 |
| RoPE theta | 5,000,000 |
| Dtype | bfloat16 |
Intended Use
Direct use
CyberSentinel CTI is intended to assist defensive security and threat-intelligence
workflows, for example:
- Summarizing and triaging threat-intelligence reports and security advisories
- Reasoning over indicators of compromise (IOCs), TTPs, and adversary behavior
- Producing structured answers for CTI analysis tasks
- Assisting SOC analysts, threat researchers, and blue teams with drafting and enrichment
Out-of-scope / prohibited use
This model is released for defensive and educational purposes. It must not
be used to develop malware, plan or carry out attacks, target individuals or
organizations, evade detection, or otherwise cause harm. As with any LLM, outputs
can be incorrect, incomplete, or fabricated — treat everything it produces as a
draft to be verified by a qualified human analyst before it informs any security decision.
How to Get Started
🤗 Transformers (full precision)
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_id = "your-username/cybersentinel-cti"
4tokenizer = AutoTokenizer.from_pretrained(model_id)
5model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")
6
7messages = [
8 {"role": "system", "content": "You are CyberSentinel, a cyber threat intelligence analyst."},
9 {"role": "user", "content": "Summarize the TTPs described in this report and map them to MITRE ATT&CK: ..."},
10]
11
12inputs = tokenizer.apply_chat_template(
13 messages, add_generation_prompt=True, return_tensors="pt"
14).to(model.device)
15
16outputs = model.generate(inputs, max_new_tokens=512, temperature=0.7, top_p=0.8)
17print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
🦙 llama.cpp (GGUF, Q4_K_M)
1./llama-cli -m cybersentinel-cti-q4_k_m.gguf \
2 -p "You are CyberSentinel, a cyber threat intelligence analyst." -cnv
🐳 Ollama (GGUF)
1# Modelfile
2FROM ./cybersentinel-cti-q4_k_m.gguf
1ollama create cybersentinel-cti -f Modelfile
2ollama run cybersentinel-cti
Training
Procedure
The model was trained with
GRPO (introduced in
DeepSeekMath) on top of the base
Qwen3-4B-Instruct-2507. LoRA adapters were attached to the attention and MLP
projections (
q/k/v/o_proj,
gate/up/down_proj), trained, then merged into the
16-bit base weights and additionally exported to GGUF Q4_K_M via
llama.cpp.
Three reward functions shaped the policy:
format_reward_func — binary (0/1) reward for producing the required output structure.correctness_reward_func — task reward in the range −1 … +2 for a correct answer.soft_format_reward_func — partial-credit reward in 0 … 0.5 for near-correct formatting.
Hyperparameters
| Setting | Value |
|---|
| Method | GRPO (RL) with LoRA (r=16, α=16, dropout=0.0) |
| Total steps | 2,000 |
| Generations per prompt (G) | 4 |
| Prompts per step | 4 |
| Effective batch size | 16 (4 × grad-accum 4) |
| Epochs | ~0.54 (≈8,000 prompts of 14,923 rows) |
| Precision | bf16 |
| Sampling temperature | 0.9 |
| KL coefficient (β) | 0.04 |
| Max prompt/seq length | 1,024 tokens |
| Trainable params | 33.0M / 4.06B (0.81%) |
Data
The training set contained 14,923 CTI prompts after filtering (13 over-long
prompts dropped from 14,936), with additional validation (2,630) and test (3,170)
splits. Each example pairs a CTI prompt with a reference answer used by the
correctness reward.
Training signal
Over training the format reward climbed toward ~0.9 and the correctness reward
lifted off its −1.0 floor, indicating the policy learned both the required output
structure and improved task correctness. These are training rewards, not a
held-out benchmark score — see Limitations.
Software / environment
- Unsloth 2026.8.19, TRL 0.24.0, Transformers 4.57.6, PyTorch 2.11.0, vLLM 0.23.0
- Hardware: 1× NVIDIA RTX 3090 (24 GB), CUDA 13.0
- Training runtime: ~6.5 hours
Bias, Risks, and Limitations
- No formal held-out evaluation is reported here. The evidence of learning is
the training reward curve; downstream accuracy on your own CTI benchmark should
be measured before relying on the model.
- The model can hallucinate IOCs, CVE identifiers, ATT&CK mappings, and other
security facts. Verify all outputs against authoritative sources.
- Trained for ~0.54 epochs; it is a lightweight domain adaptation, not an
exhaustive fine-tune.
- Inherits the biases and limitations of the base Qwen3-4B-Instruct-2507 model.
- English-centric; performance on other languages is not characterized.
Recommendation: Keep a qualified human in the loop. Do not use model output as
the sole basis for any security or operational decision.
License
Released under the
Apache License 2.0 (see
LICENSE), consistent
with the Apache-2.0 license of the base Qwen3-4B-Instruct-2507 model. You are free
to use, modify, and redistribute the model, including commercially, provided you
retain the license and attribution.
Citation
If you use this model, please cite it along with the base model and methods:
1@misc{cybersentinel_cti_2026,
2 title = {CyberSentinel CTI: A Qwen3-4B Model for Cyber Threat Intelligence},
3 author = {Sazib, Emul},
4 year = {2026},
5 howpublished = {\url{https://huggingface.co/your-username/cybersentinel-cti}},
6 note = {Fine-tuned from Qwen3-4B-Instruct-2507 with GRPO}
7}
Base model — Qwen3:
1@misc{qwen3,
2 title = {Qwen3 Technical Report},
3 author = {Qwen Team},
4 year = {2025}
5}
Method — GRPO:
1@article{shao2024deepseekmath,
2 title = {DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models},
3 author = {Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo},
4 year = {2024},
5 eprint = {arXiv:2402.03300}
6}
Tooling — TRL & Unsloth:
1@misc{vonwerra2022trl,
2 title = {TRL: Transformer Reinforcement Learning},
3 author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
4 year = {2020},
5 howpublished = {\url{https://github.com/huggingface/trl}}
6}
Trained with Unsloth for 2× faster fine-tuning. 
