import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
from huggingface_hub import hf_hub_download
import pandas as pd

model, category, augmentation = 'modernbert', 'hazard', 'base'

repo_id = f"DataScienceWFSR/{model}-food-{category}-{augmentation}"
lb_path = hf_hub_download(repo_id=repo_id, filename=f"labelencoder_{category}.pkl")
lb =  pd.read_pickle(lb_path)
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSequenceClassification.from_pretrained(repo_id)
model.eval()

sample = ('Case Number: 039-94 Date Opened: 10/20/1994 Date Closed: 03/06/1995 Recall Class: 1'
        ' Press Release (Y/N): N Domestic Est. Number: 07188 M Name: PREPARED FOODS Imported '
        'Product (Y/N): N Foreign Estab. Number: N/A City: SANTA TERESA State: NM Country: USA'
        ' Product: HAM, SLICED Problem: BACTERIA Description: LISTERIA '
        'Total Pounds Recalled: 3,920 Pounds Recovered: 3,920')

inputs = tokenizer(sample, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)
predictions = outputs.logits.argmax(dim=-1)
predicted_label = lb.inverse_transform(predictions.numpy())[0]
print(f"The predicted label is: {predicted_label}")

Training Details

Training Data

Training and Validation data provided by SemEval-2025 Task 9 organizers : Food Recall Incidents dataset (only English) link

Training Procedure

Training Hyperparameters

batch_size: 8
epochs: 10
lr_scheduler: linear

Evaluation

Testing Data & Metrics

Testing Data

Test data: 997 samples (link)

Metrics

F₁-macro

Results

F₁-macro scores for each model in the official test set utilizing the text field per category and subtasks scores (ST1 and ST2) rounded to 3 decimals. With bold, we indicated the model's specific results.

Model	hazard-category	product-category	hazard	product	ST1	ST2
BERT_base	0.747	0.757	0.581	0.170	0.753	0.382
BERT_CW	0.760	0.761	0.671	0.280	0.762	0.491
BERT_SR	0.770	0.754	0.666	0.275	0.764	0.478
BERT_RW	0.752	0.757	0.651	0.275	0.756	0.467
DistilBERT_base	0.761	0.757	0.593	0.154	0.760	0.378
DistilBERT_CW	0.766	0.753	0.635	0.246	0.763	0.449
DistilBERT_SR	0.756	0.759	0.644	0.240	0.763	0.448
DistilBERT_RW	0.749	0.747	0.647	0.261	0.753	0.462
RoBERTa_base	0.760	0.753	0.579	0.123	0.755	0.356
RoBERTa_CW	0.773	0.739	0.630	0.000	0.760	0.315
RoBERTa_SR	0.777	0.755	0.637	0.000	0.767	0.319
RoBERTa_RW	0.757	0.611	0.615	0.000	0.686	0.308
ModernBERT_base	0.781	0.745	0.667	0.275	0.769	0.485
ModernBERT_CW	0.761	0.712	0.609	0.252	0.741	0.441
ModernBERT_SR	0.790	0.728	0.591	0.253	0.761	0.434
ModernBERT_RW	0.761	0.751	0.629	0.237	0.759	0.440

Technical Specifications

Compute Infrastructure

Hardware

NVIDIA A100 80GB and NVIDIA GeForce RTX 3070 Ti

Software

Library	Version	URL
Transformers	4.49.0	https://huggingface.co/docs/transformers/index
PyTorch	2.6.0	https://pytorch.org/
SpaCy	3.8.4	https://spacy.io/
Scikit-learn	1.6.0	https://scikit-learn.org/stable/
Pandas	2.2.3	https://pandas.pydata.org/
Optuna	4.2.1	https://optuna.org/
NumPy	2.0.2	https://numpy.org/
NLP AUG	1.1.11	https://nlpaug.readthedocs.io/en/latest/index.html
BeautifulSoup4	4.12.3	https://www.crummy.com/software/BeautifulSoup/bs4/doc/#

Citation

BibTeX:

For the original paper:

@inproceedings{brightcookies-semeval2025-task9, 
    title="BrightCookies at {S}em{E}val-2025 Task 9: Exploring Data Augmentation for Food Hazard Classification}, 
    author="Papadopoulou, Foteini and Mutlu, Osman  and Özen, Neris and van der Velden, Bas H. M. and Hendrickx, Iris  and Hürriyetoğlu, Ali",
    booktitle = "Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)",
    month = jul, 
    year = "2025", 
    address = "Vienna, Austria", 
    publisher = "Association for Computational Linguistics", 
}

For the SemEval2025 Task9:

@inproceedings{semeval2025-task9, 
    title = "{S}em{E}val-2025 Task 9: The Food Hazard Detection Challenge", 
    author = "Randl, Korbinian and Pavlopoulos, John and Henriksson, Aron and Lindgren, Tony and Bakagianni, Juli", 
    booktitle = "Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025)",
    month = jul, 
    year = "2025", 
    address = "Vienna, Austria", 
    publisher = "Association for Computational Linguistics", 
}

Model Card Authors and Contact

Authors: Foteini Papadopoulou, Osman Mutlu, Neris Özen, Bas H.M. van der Velden, Iris Hendrickx, Ali Hürriyetoğlu

Contact: ali.hurriyetoglu@wur.nl

DataScienceWFSR/modernbert-food-hazard-base

ModernBERT Food Hazard Classification Model - Baseline

Model Details

Model Description

Model Sources

How to Get Started With the Model