Training code & demo: github.com/alfiobonanno/AGNews-Bert-Tester-App — includes a live demo (FastAPI + React) where you can test the model with your own headlines.

The fine-tuned model is hosted here on Hugging Face Hub as Alfiob12/agnews-bert.

Use the HF model directly with the transformers pipeline:

python

from transformers import pipeline

clf = pipeline("text-classification", model="Alfiob12/agnews-bert")
clf("NVIDIA announces production of groundbreaking quantum GPUs")
# [{'label': 'Sci/Tech', 'score': 0.961}]

Results

Evaluated on a held-out test set of 19,140 samples (15% of the full dataset).

Metric	Score
Accuracy	94.47%
Precision (macro)	94.47%
Recall (macro)	94.47%
F1 (macro)	94.47%

Per-class breakdown:

Class	Precision	Recall	F1
Business	0.9153	0.9175	0.9164
Sci/Tech	0.9188	0.9266	0.9227
Sports	0.9852	0.9906	0.9879
World	0.9596	0.9440	0.9517

Training

The notebook AG _news_Classification.ipynb contains the full pipeline: data loading, tokenization, training, evaluation, and inference.

Hyperparameters

Parameter	Value
Base model	`bert-base-uncased`
Max token length	128
Epochs	3
Train batch size	128
Eval batch size	32
Learning rate	2e-5
Weight decay	0.01
Best model metric	F1

Data split

The original train/test CSV files are merged (127,600 samples) and re-split to create a validation set:

Train: 70% (89,320 samples)
Validation: 15% (19,140 samples)
Test: 15% (19,140 samples)

Splits are stratified by class. The dataset is perfectly balanced (equal samples per class).

Setup

bash

git clone https://github.com/alfiobonanno/AGNews-Bert-Tester-App.git
cd AGNews-Bert-Tester-App
python -m venv venv
source venv/bin/activate   # macOS/Linux
# venv\Scripts\activate    # Windows
pip install -r requirements.txt

The dataset CSVs are included in ag_data/. Then open and run AG _news_Classification.ipynb.

Dependencies

See requirements.txt for the full list. Key packages:

transformers==5.12.1
datasets==5.0.0
scikit-learn==1.9.0
pandas==3.0.3
plotly==6.8.0
numpy==2.4.6
safetensors==0.8.0