A fine-tuned ModernBERT-base cross-encoder for binary classification of news article pairs. Given two articles (by headline, or headline + content), the model predicts whether they refer to the same real-world event.
This model is used in Stage 3, Part 1 of a news article grouping research pipeline.
Model Description
The model classifies whether a pair of news articles refers to the same underlying real-world event — not just general semantic similarity, but whether both articles report on the same specific occurrence, potentially from different sources or perspectives.
Input structure:
(Article A, Article B) → [1 = Same Event | 0 = Different Event]
The model is built on top of answerdotai/ModernBERT-base, fine-tuned using task-specific layer unfreezing and a focal loss function to handle class imbalance.
How to Use
Installation
pip install transformers torch
Predict with the Model
python
1import torch
2import torch.nn.functional as F
3from transformers import AutoTokenizer, AutoModelForSequenceClassification
45MODEL_NAME ="Juanillaberia/articles-pairs-event-detection"67tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)8model = AutoModelForSequenceClassification.from_pretrained(MODEL_NAME)9model.eval()1011defpredict_same_event(headline_a:str, headline_b:str)->dict:12"""
13 Predicts whether two article headlines refer to the same real-world event.
1415 Args:
16 headline_a: Headline of the first article.
17 headline_b: Headline of the second article.
1819 Returns:
20 Dictionary with predicted label and probabilities.
21 """22 inputs = tokenizer(23 text=headline_a,24 text_pair=headline_b,25 return_tensors="pt",26 truncation=True,27 max_length=12828)2930with torch.no_grad():31 outputs = model(**inputs)32 logits = outputs.logits
33 probs = F.softmax(logits, dim=-1)3435 predicted_class = torch.argmax(probs, dim=-1).item()36 labels ={0:"Different Event",1:"Same Event"}3738return{39"label": labels[predicted_class],40"score": probs[0][predicted_class].item(),41"probabilities":{42"Different Event": probs[0][0].item(),43"Same Event": probs[0][1].item()44}45}4647# Example usage48headline_a ="Government announces new climate policy targeting carbon emissions"49headline_b ="New climate bill signed into law by administration"5051result = predict_same_event(headline_a, headline_b)52print(result)53# {'label': 'Same Event', 'score': 0.93, 'probabilities': {'Different Event': 0.07, 'Same Event': 0.93}}
Optional: Apply Temporal Adjustment
If you have access to the publication dates of both articles, you can apply a post-hoc temporal adjustment to improve precision:
python
1from datetime import datetime
23defpredict_with_date_adjustment(4 headline_a:str,5 headline_b:str,6 date_a:str,7 date_b:str,8 lambda_:float=0.20,9 threshold:float=0.4510)->dict:11"""
12 Predicts same-event with temporal adjustment based on publication date difference.
1314 Args:
15 headline_a: Headline of the first article.
16 headline_b: Headline of the second article.
17 date_a: Publication date of article A (format: 'YYYY-MM-DD').
18 date_b: Publication date of article B (format: 'YYYY-MM-DD').
19 lambda_: Decay factor for temporal adjustment (default: 0.20).
20 threshold: Classification threshold after adjustment (default: 0.45).
21 """22 inputs = tokenizer(23 text=headline_a,24 text_pair=headline_b,25 return_tensors="pt",26 truncation=True,27 max_length=12828)2930with torch.no_grad():31 outputs = model(**inputs)32 logits = outputs.logits
33 prob_same_event = F.softmax(logits, dim=-1)[0][1]3435 diff_days =abs((36 datetime.strptime(date_a,"%Y-%m-%d")- datetime.strptime(date_b,"%Y-%m-%d")37).days)3839 adjusted_prob = prob_same_event * torch.exp(torch.tensor(-lambda_ * diff_days))40 predicted =int(adjusted_prob.item()>= threshold)41 labels ={0:"Different Event",1:"Same Event"}4243return{44"label": labels[predicted],45"adjusted_score": adjusted_prob.item(),46"raw_score": prob_same_event.item(),47"diff_days": diff_days
48}4950# Example usage51result = predict_with_date_adjustment(52 headline_a="Government announces new climate policy",53 headline_b="New climate bill signed into law",54 date_a="2024-03-01",55 date_b="2024-03-02"56)57print(result)
Training Details
Experiment Design
Two main experiments were conducted to determine which input configuration best suited the task:
Headlines-Only — Each example consists of two article headlines concatenated with a [SEP] token. Maximum sequence length: 128 tokens (EDA showed the longest headline is ~30 words, so combined inputs typically fall between 100–120 tokens).
Headlines + Content — Each example includes both headlines and full article bodies. Maximum sequence length: 2,048 tokens.
Hypothesis: Including article content would improve precision by providing additional context. Results showed the opposite — headlines alone proved more effective.
Both datasets consist of 100,000 pairs drawn from the WCEP dataset, with no overlap with the main pipeline's subsample to prevent data leakage.
Architecture
Base model:answerdotai/ModernBERT-base — trained on ~2 trillion tokens, supports up to 8,192 tokens natively
Task head: Standard Hugging Face sequence classification head (linear layer on pooled transformer output)
Fine-tuning strategy: Embeddings and lower transformer layers are frozen; only the top layers and the classification head are trained
Headlines-only: last 3 layers unfrozen (layers 19–21)
Headlines + content: last 2 layers unfrozen (layers 20–21)
Loss function: Focal loss to address class imbalance
Parameter Summary
Headlines-Only Model
Parameter Type
Count
Total
149,606,402
Trainable
15,638,018
Frozen
133,968,384
Headlines + Content Model
Parameter Type
Count
Total
149,606,402
Trainable
10,622,978
Frozen
138,983,424
Hyperparameter Optimization (HPO)
HPO was run using Optuna with 10 trials, a TPE sampler, and Median Pruner. The search space covered:
Learning rate: 1e-5 to 5e-5 (log scale)
Batch size: [16, 32]
Gradient accumulation steps: [1, 2]
Warmup ratio: 0.05 to 0.15
Weight decay: 1e-3 to 1e-1
HPO ran on a NVIDIA L4 GPU (22.5 GB VRAM, 53 GB system RAM).
Estimated improvement; not testable at scale on main dataset
*The post-hoc temporal adjustment showed ~3% improvement in a small exploratory experiment. However, because the main pipeline dataset does not contain publication timestamps, this configuration could not be evaluated at scale.
Conclusion
The headlines-only configuration outperforms the headlines + content setup by ~2% across several metrics. Adding full article bodies introduced noise rather than useful signal. The recommended approach is therefore to use only article headlines, which also enables efficient inference on smaller hardware.
This model is deployed in Stage 3, Part 1 of the news article grouping research pipeline to classify candidate article pairs.
Dataset
The model was trained on a custom dataset of article pairs derived from a 100,000-example subsample of the WCEP dataset, with no overlap with the subsample used in the main pipeline.
Headlines-only: 100,000 pairs
Headlines + content: 100,000 pairs
Labels: binary (same event / different event)
bibtex
1@inproceedings{Laban2021NewsHG,
2 title={News Headline Grouping as a Challenging NLU Task},
3 author={Laban, Philippe and Bandarkar, Lucas and Hearst, Marti A},
4 booktitle={NAACL 2021},
5 publisher = {Association for Computational Linguistics},
6 year={2021}
7}
Intended Use & Limitations
Intended for:
News deduplication and clustering pipelines
Event-centric article grouping
Research on media coverage analysis
Limitations:
Trained primarily on English-language news headlines
Performance may degrade on non-English or domain-specific content
Full article content was found to reduce performance — headlines only is recommended
The model may struggle with events that are semantically very similar but distinct (e.g., recurring political debates)
The post-hoc temporal adjustment requires publication dates, which may not always be available
