Fine-tuned dbmdz/bert-base-turkish-128k-cased (BERTurk, 128k cased vocabulary) for single-label, multiclass topic classification of Turkish news text into 9 categories.

The model was trained on the TRNEWS-2025 dataset (a large, real-world, and highly imbalanced Turkish news corpus) with class-balanced cross-entropy loss. On a held-out, stratified test split of 40,063 documents it reaches 95.95% accuracy, 0.9602 weighted-F1, and 0.9357 macro-F1.

Developed by: thealper2
Model type: BertForSequenceClassification (encoder-only Transformer)
Language: Turkish (tr)
Finetuned from: dbmdz/bert-base-turkish-128k-cased
License: MIT (inherited from the BERTurk base model; see Licensing for dataset terms)
Task: Multiclass text classification (topic / news-category)

Labels

The classifier outputs one of 9 mutually exclusive news categories. Labels are stored in the model config (id2label / label2id) in their original Turkish surface forms; English glosses are provided below for convenience.

ID	Label (Turkish)	English gloss
0	`Bilim-Teknoloji`	Science–Technology
1	`Finans-Ekonomi`	Finance–Economy
2	`Kültür-Sanat`	Culture–Arts
3	`Magazin`	Magazine / Tabloid
4	`Sağlık`	Health
5	`Siyaset`	Politics
6	`Spor`	Sports
7	`Turizm`	Tourism
8	`Çevre`	Environment

Because id2label is baked into the config, pipeline(...) and model.config.id2label return the human-readable Turkish label directly — no external mapping file is required at inference time.

Quick start

With `pipeline`

python

from transformers import pipeline

clf = pipeline(
    "text-classification",
    model="thealper2/berturk-128k-turkish-news-classification",
    truncation=True,
    max_length=512,
)

clf("Milli takım, hazırlık maçında sahadan galibiyetle ayrıldı.")
# [{'label': 'Spor', 'score': 0.99...}]

# Full probability distribution over all 9 classes:
clf("Borsa İstanbul günü yükselişle kapattı, faiz beklentileri fiyatlandı.",
    top_k=None)

With `AutoModel` (manual, batched)

python

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_id = "thealper2/berturk-128k-turkish-news-classification"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id).eval()

texts = [
    "Yeni sürücüsüz araç teknolojisi trafik güvenliğini artırmayı hedefliyor.",
    "Antalya'da turizm sezonu erken açıldı, oteller yüksek doluluğa ulaştı.",
]

enc = tokenizer(texts, padding=True, truncation=True,
                max_length=512, return_tensors="pt")

with torch.no_grad():
    logits = model(**enc).logits
    probs = logits.softmax(dim=-1)

pred_ids = probs.argmax(dim=-1).tolist()
for text, pid, p in zip(texts, pred_ids, probs):
    print(f"{model.config.id2label[pid]:16s} ({p[pid]:.3f})  <- {text[:50]}")

Tip: the model was trained on article bodies (Haber Gövdesi) rather than headlines, so it performs best on full paragraphs/articles. Very short headlines may be classified less reliably. Always pass truncation=True, max_length=512.

Intended uses & limitations

Intended uses

Automatic topic tagging / routing of Turkish news articles.
Editorial content organization, feed categorization, and dataset labeling assistance.
A strong Turkish-language baseline / backbone for downstream topic classification.

Out-of-scope / limitations

Domain shift: trained on news text. Performance will degrade on other genres (tweets, product reviews, legal/medical documents, conversational text).
Single-label only: the head is a softmax classifier — it forces exactly one category even for articles that legitimately span multiple topics (e.g. sports-economy). It cannot emit multiple labels.
Label imbalance: the corpus is dominated by Magazin (~39% of the data). Even though class-balanced loss was used, the tail classes (Çevre, Turizm, Bilim-Teknoloji, Kültür-Sanat) show lower precision and are more prone to confusion (see the confusion analysis).
No factuality/sentiment signal: the model predicts topic, not veracity, sentiment, or stance.
Potential bias: the model reflects the editorial and demographic biases of the source Turkish news outlets in TRNEWS-2025.

Training data

Dataset: TRNEWS-2025: A Multiclass Turkish News Text Dataset (IEEE DataPort, DOI 10.21227/vasv-dj22, Sengul Bayrak, 2025). Access requires an IEEE DataPort subscription.
Input column: Haber Gövdesi (news body). Label column: Sınıf (class). Auxiliary columns (ID, Erişim Tarihi, URL) were dropped.

Preprocessing

Applied before tokenization (dataset-agnostic cleaning pipeline):

HTML unescaping + tag stripping.
URL stripping (http(s)://…, www.…).
Whitespace normalization.
Casing preserved (lowercase: false) — the base model is cased; no Turkish İ/ı folding.
Exact-duplicate removal on the cleaned text.
Empty/too-short rows removed (min_chars: 1).

Splits

Stratified train/validation/test split (seed=42), 90% / 5% / 5%:

Split	Documents	Share
Train	≈ 721,134	90%
Validation	≈ 40,063	5%
Test	40,063	5%
Total	≈ 801,260	100%

Class distribution (full corpus, approximate)

The dataset is heavily imbalanced (≈20:1 between the largest and smallest class), which motivated class-weighted training.

Label	≈ Count	≈ Share
`Magazin`	315,760	39.4%
`Siyaset`	147,240	18.4%
`Spor`	131,420	16.4%
`Sağlık`	47,840	6.0%
`Kültür-Sanat`	46,620	5.8%
`Finans-Ekonomi`	36,680	4.6%
`Bilim-Teknoloji`	30,480	3.8%
`Turizm`	29,640	3.7%
`Çevre`	15,580	1.9%

(Counts are inferred from the stratified test-split support × 20; they are accurate to within rounding.)

Training procedure

Fine-tuned end-to-end with the HuggingFace Trainer (a custom WeightedTrainer subclass applying per-class weights to the cross-entropy loss).

Class-balanced loss

Cross-entropy is weighted with the scikit-learn "balanced" scheme, computed on the training split:

weight_c = N_train / (num_classes * count_c)

This yields weights ranging from ≈ 0.28 for the majority class (Magazin) to ≈ 5.7 for the minority class (Çevre), pulling macro-F1 up substantially relative to unweighted training.

Hyperparameters

Hyperparameter	Value
Base model	`dbmdz/bert-base-turkish-128k-cased`
Max sequence length	512
Optimizer	AdamW (Transformers default)
Learning rate	2e-5
LR scheduler	linear decay with warmup
Warmup ratio	0.06
Weight decay	0.01
Max grad norm	1.0
Per-device batch size	16
Gradient accumulation	2
Effective batch size	32
Epochs (planned)	3
Loss	Class-weighted cross-entropy (`balanced`)
Early stopping	patience = 2 evals, metric = `f1_macro`
Precision	bf16 mixed precision + TF32 matmul
Attention impl.	SDPA (PyTorch scaled-dot-product-attention)
Dynamic padding	`DataCollatorWithPadding`, `pad_to_multiple_of=8`
Length grouping	`group_by_length=True`
Eval / save interval	every 500 steps
Best-model selection	highest validation `f1_macro`
Seed	42

Compute

Hardware: single NVIDIA RTX 5060 Ti (Blackwell, sm_120, 8 GB VRAM).
Frameworks: PyTorch (CUDA 12.8 wheel) · Transformers 4.57.6.
8 GB VRAM was made feasible through bf16, SDPA attention, dynamic padding, and length grouping (fixed 512-token padding was avoided).

Convergence & early stopping

Because the training set is large (~721k examples), the model converged within a fraction of the first epoch. Validation f1_macro peaked at step 3000 and did not improve over the next two evaluations, so EarlyStoppingCallback halted training at step 4000 and restored the best (step-3000) checkpoint. Selected validation metrics:

Step	Epoch	Val loss	Val accuracy	Val macro-F1
500	0.02	1.6439	0.7315	0.5362
1000	0.04	0.4685	0.9069	0.8390
1500	0.07	0.2729	0.9120	0.8767
2000	0.09	0.1989	0.9471	0.9173
2500	0.11	0.2254	0.9554	0.9244
3000	0.13	0.1619	0.9588	0.9341 ← best
3500	0.16	0.1614	0.9571	0.9297
4000	0.18	0.1776	0.9555	0.9290

Note: the low epoch count is expected, not a bug — a single epoch over ~721k documents is a large number of optimization steps. Training longer offered no macro-F1 gain and risked overfitting the majority class.

Evaluation

All metrics below are on the held-out test split (40,063 documents), which was never seen during training or model selection.

Overall

Metric	Value
Accuracy	0.9595
F1 (weighted)	0.9602
F1 (macro)	0.9357
Precision (macro)	0.9221
Recall (macro)	0.9513
Loss	0.1647

Per-class report

Label	Precision	Recall	F1	Support
`Bilim-Teknoloji`	0.8367	0.9278	0.8799	1,524
`Finans-Ekonomi`	0.9882	0.9586	0.9732	1,834
`Kültür-Sanat`	0.8232	0.9425	0.8788	2,331
`Magazin`	0.9913	0.9622	0.9765	15,788
`Sağlık`	0.9379	0.9728	0.9551	2,392
`Siyaset`	0.9864	0.9391	0.9622	7,362
`Spor`	0.9720	0.9925	0.9822	6,571
`Turizm`	0.8815	0.9440	0.9117	1,482
`Çevre`	0.8821	0.9217	0.9014	779
Macro avg	0.9221	0.9513	0.9357	40,063
Weighted avg	0.9621	0.9595	0.9602	40,063

Confusion analysis

Rows = true label, columns = predicted label (order matches the label table above).

	Bil-Tek	Fin-Eko	Kül-San	Magazin	Sağlık	Siyaset	Spor	Turizm	Çevre
Bilim-Teknoloji	1414	2	46	11	31	2	11	7	0
Finans-Ekonomi	27	1758	2	0	2	21	1	20	3
Kültür-Sanat	34	0	2197	64	7	14	3	9	3
Magazin	44	2	279	15191	44	34	149	40	5
Sağlık	41	0	6	5	2327	0	6	4	3
Siyaset	98	8	119	25	43	6914	15	80	60
Spor	14	1	5	21	6	0	6522	2	0
Turizm	6	5	11	6	18	13	2	1399	22
Çevre	12	3	4	1	3	11	1	26	718

Key takeaways:

High-support, well-separated classes (Magazin, Spor, Siyaset, Finans-Ekonomi) exceed 0.96 F1.
The main error mode is precision loss on the minority classes Kültür-Sanat (0.82) and Bilim-Teknoloji (0.84): a slice of Magazin articles leaks into Kültür-Sanat (279 cases), and several Siyaset articles are pulled into Bilim-Teknoloji. These are semantically adjacent topics in Turkish news.
Class weighting keeps recall high across every class (min. recall 0.92), which is often the desired trade-off for tail categories.

Reproducing evaluation

The exact metrics, full classification_report, and confusion matrix are shipped in test_report.json alongside these weights.

Model architecture

Property	Value
Architecture	`BertForSequenceClassification`
Hidden size	768
Layers	12
Attention heads	12
Intermediate size	3072
Max position embeddings	512
Vocabulary size	128,000 (128k cased WordPiece)
Parameters	~184M
Tokenizer	Fast WordPiece (BERTurk 128k cased)
Precision (weights)	float32 (`model.safetensors`)

Licensing

Model weights: released under the MIT License, inherited from the base model dbmdz/bert-base-turkish-128k-cased.
Training data (TRNEWS-2025): distributed via IEEE DataPort and requires a subscription to access. This repository ships only the fine-tuned weights and evaluation artifacts — no raw dataset text is included or redistributed. If you use the data itself, you must comply with IEEE DataPort's terms and cite the dataset (below).

Citation

If you use this model, please cite both the underlying dataset and the base model.

Dataset (TRNEWS-2025):

bibtex

@data{vasv-dj22-25,
  doi       = {10.21227/vasv-dj22},
  url       = {https://dx.doi.org/10.21227/vasv-dj22},
  author    = {Sengul Bayrak},
  publisher = {IEEE Dataport},
  title     = {TRNEWS-2025: A Multiclass Turkish News Text Dataset},
  year      = {2025}
}

Base model (BERTurk):

bibtex

@software{stefan_schweter_2020_3770924,
  author    = {Stefan Schweter},
  title     = {BERTurk - BERT models for Turkish},
  month     = apr,
  year      = 2020,
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.3770924},
  url       = {https://doi.org/10.5281/zenodo.3770924}
}

This fine-tuned model:

bibtex

@misc{berturk_128k_turkish_news_classification,
  author    = {thealper2},
  title     = {BERTurk-128k Turkish News Classification (9-class)},
  year      = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/thealper2/berturk-128k-turkish-news-classification}}
}

Acknowledgements

Sengul Bayrak (Istanbul Sabahattin Zaim University) for the TRNEWS-2025 dataset.
Stefan Schweter / dbmdz for the BERTurk base models.
The Hugging Face transformers, datasets, and accelerate teams.

thealper2/berturk-128k-turkish-news-classification

BERTurk-128k Turkish News Classification (9-class)

Labels

Quick start

With pipeline

With AutoModel (manual, batched)

Intended uses & limitations

Intended uses

Out-of-scope / limitations

Training data

Preprocessing

Splits

Class distribution (full corpus, approximate)

Training procedure

Class-balanced loss

Hyperparameters

Compute

Convergence & early stopping

Evaluation

Overall

Per-class report

Confusion analysis

Reproducing evaluation

Model architecture

Licensing

Citation

Acknowledgements

thealper2/
berturk-128k-turkish-news-classification

With `pipeline`

With `AutoModel` (manual, batched)