Tini-Cybersec-8B-A1B is a specialized fine-tuned model based on the LiquidAI/LFM2.5-8B-A1B architecture. It is customized to perform complex Cybersecurity tasks, security analysis, threat modeling, and vulnerability assessment, while preserving and enhancing reasoning and Chain-of-Thought (CoT) capabilities.
The model is SFT-trained using a carefully curated dataset mix of 185,002 records comprising both deep security knowledge and structured step-by-step reasoning paths.
📊 Dataset & Matrix Distribution
The SFT training data is a balanced mixture of domain-specific cybersecurity instruction datasets and general reasoning datasets (CoT), filtered to remove empty/zero-token records.
Tini-Cybersec-8B-A1B has been evaluated on the CS-Eval Benchmark (a comprehensive cybersecurity evaluation benchmark for Large Language Models) and is published on the CS-Eval Leaderboard (under submission name DungNVT-ISELAB).
The model achieved a Comprehensive Score of 76.65%, demonstrating robust capabilities across all domains of system, network, and application security:
Evaluation Domain / Category
Score (%)
Comprehensive Average (Comprehensive Score)
76.65
Supply Chain Security
86.05
AI and Network Security
83.17
Infrastructure Security
78.04
English Tasks
77.40
Data Security and Privacy Protection
76.79
Chinese Task
76.60
Vulnerability Management and Penetration Testing
76.54
Access Control and Identity Management
76.44
Threat Detection and Prevention
75.28
Encryption Technology and Key Management
75.18
Security Architecture Design
75.12
Fundamentals of System Security and Software Security
74.67
Business Continuity and Emergency Response Recovery
67.33
⚙️ Training Hyperparameters (SFT)
The model was SFT-trained using Unsloth and Hugging Face Trainer with sequence packing to optimize throughput:
Effective batch size of 32 blocks (262,144 tokens)
Learning Rate
5e-5
Recommended sweet spot for wide LoRA SFT
Learning Rate Scheduler
cosine
Cosine annealing for smooth convergence
Warmup Steps
10% of total steps
Linear warmup
Optimizer
adamw_8bit
Memory efficient 8-bit AdamW
Weight Decay
0.01
Regularization
Max Gradient Norm
1.0
Gradient clipping
💬 Prompt Format & Templates
This model follows the ChatML format and supports nested <think> tags for reasoning models.
Template Structure:
markdown
1<|im_start|>system
2You are a helpful and knowledgeable cybersecurity expert assistant. You answer all user queries step by step with reasoning.<|im_end|>3<|im_start|>user
4[Your cybersecurity query / task here]<|im_end|>5<|im_start|>assistant
6<think>7[Step-by-step thinking process / Chain-of-Thought (CoT)]
8</think>9[Detailed response / action plan / explanation]
10<|im_end|>
🚀 How to Load and Use
To load this model with Hugging Face's transformers library:
python
1import torch
2from transformers import AutoModelForCausalLM, AutoTokenizer
34model_name ="./Tini-Cybersec-8B-A1B_26062026"56tokenizer = AutoTokenizer.from_pretrained(model_name)7model = AutoModelForCausalLM.from_pretrained(8 model_name,9 torch_dtype=torch.bfloat16,10 device_map="auto"11)1213# Inference example14messages =[15{"role":"system","content":"You are a cybersecurity expert assistant."},16{"role":"user","content":"What is SQL Injection? And how to prevent it?"}17]1819prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)20inputs = tokenizer(prompt, return_tensors="pt").to(model.device)2122outputs = model.generate(23**inputs,24 max_new_tokens=2048,25 temperature=0.6,26 top_p=0.9,27 do_sample=True28)2930response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)31print(response)
📄 License & Attribution
Base Model: Licensed under the Apache-2.0 license by LiquidAI.
Fine-tuned Weights: Apache-2.0 License.
Dataset Attribution: Please credit the original authors of AlicanKiraz0/Cybersecurity-Dataset-Heimdall-v1.1, AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1, Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset, nohurry/Opus-4.6-Reasoning-3000x-filtered, Jackrong/DeepSeek-V4-Distill-8000x, and Jackrong/Qwen3.5-reasoning-700x.
