The Secure Modern BERT NER Model is a fine-tuned transformer based on SecureBERT 2.0, designed for Named Entity Recognition (NER) in cybersecurity text.
It extracts domain-specific entities such as Indicators, Malware, Organizations, Systems, and Vulnerabilities from unstructured data sources like threat reports, incident analyses, advisories, and blogs.
NER in cybersecurity enables:
Automated extraction of indicators of compromise (IOCs)
Structuring of unstructured threat intelligence text
The SecureBERT2.0-NER was fine-tuned for token-level classification on cybersecurity text using Cross Entropy Loss.
Training focused on accurately classifying entity boundaries and types across five cybersecurity-specific categories: Malware, Indicator, System, Organization, and Vulnerability.
The AdamW optimizer was used with a linear learning rate scheduler, and gradient clipping ensured stability during fine-tuning.
Training Configuration
Setting
Value
Objective
Token-wise Cross Entropy
Optimizer
AdamW
Learning Rate
1e-5
Weight Decay
0.001
Batch Size per GPU
8
Epochs
20
Max Sequence Length
1024
Gradient Clipping Norm
1.0
Scheduler
Linear
Mixed Precision
fp16
Framework
TensorFlow / Transformers
Training Dataset
The model was fine-tuned on a cybersecurity-specific NER corpus, containing annotated threat intelligence reports, advisories, and technical documentation.
Texts were tokenized using the PreTrainedTokenizerFast tokenizer from SecureBERT 2.0.
All sequences were truncated or padded to 1024 tokens.
Labels were aligned with subword tokens to maintain token–label consistency.
Hardware and Training Setup
Component
Description
GPUs Used
8× NVIDIA A100
Precision
Mixed precision (fp16)
Batch Size
8 per GPU
Framework
Transformers (TensorFlow backend)
Optimization Summary
The model converged after approximately 20 epochs, with loss stabilizing at a low level.
Validation metrics (F1, precision, recall) showed steady improvement from epoch 3 onward, confirming effective domain-specific adaptation.
Evaluation
Testing Data, Factors & Metrics
Testing Data
Evaluation was conducted on a cybersecurity-specific NER benchmark corpus containing annotated threat reports, advisories, and incident analysis texts.
This benchmark includes five key entity types: Malware, Indicator, System, Organization, and Vulnerability.
Metrics
The following metrics were used to assess model performance:
F1-score: Harmonic mean of precision and recall
Recall: Measures how many true entities were correctly identified
Precision: Measures how many predicted entities were correct
Results
Model
F1
Recall
Precision
CyBERT
0.351
0.281
0.467
SecureBERT
0.734
0.759
0.717
SecureBERT 2.0 (Ours)
0.945
0.965
0.927
Summary
The SecureBERT 2.0 NER model significantly outperforms both CyBERT and the original SecureBERT across all metrics.
It achieves a F1-score of 0.945, a +21% absolute improvement over SecureBERT.
Its recall (0.965) indicates excellent coverage of cybersecurity entities.
Its precision (0.927) shows strong accuracy and low false-positive rates.
This demonstrates that domain-adaptive pretraining and fine-tuning on cybersecurity corpora dramatically improves NER performance compared to general or earlier models.
Reference
@article{aghaei2025securebert,
title={SecureBERT 2.0: Advanced Language Model for Cybersecurity Intelligence},
author={Aghaei, Ehsan and Jain, Sarthak and Arun, Prashanth and Sambamoorthy, Arjun},
journal={arXiv preprint arXiv:2510.00240},
year={2025}
}
