Diagnostic probe, not a usable NLI model. Spanish RoBERTa (BERTIN) trained and evaluated on the premise alone, to test whether ESNLIR labels can be guessed without the hypothesis. It exists to measure annotation artifacts, not to perform inference.

ESNLIR: Expanding Spanish NLI Benchmarks with Multi-Genre and Causal Annotation Johan R. Portela, Nicolás Pérez-Terán, Rubén Manrique — Universidad de los Andes, Bogotá Applied Informatics, Springer, 2026, pp. 345–361 — doi:10.1007/978-3-032-07175-0_23

Part of the ESNLIR collection.

⚠️ This checkpoint collapsed during training

The uploaded weights predict contrasting for every input. On the balanced 4-way test set that gives exactly chance accuracy:

accuracy	macro F1	contrasting	entailment	neutral	reasoning
0.2500	0.1000	1.000	0.000	0.000	0.000

The same holds on all four stress tests, since the model is not responding to its input at all.

Partial-input models are prone to this — with the hypothesis removed, the remaining signal can be too weak for the optimizer to leave a degenerate solution. The paper reports a converged premise-only BERTIN run (per-class accuracy 0.419 / 0.411 / 0.261 / 0.452), which is not the checkpoint published here.

For the artifact-detection result, use ESNLIR-XLM-RoBERTa-premise-only, which converged (0.390 accuracy against the full model's 0.676). For actual NLI, use ESNLIR-XLM-RoBERTa or ESNLIR-RoBERTa.

Files

model.safetensors + config.json — the collapsed checkpoint described above. The bundled tokenizer is an unmodified copy of the one from bertin-project/bertin-roberta-base-spanish, so the repo loads standalone even though the weights are not usable.

Labels

id	label
0	`contrasting`
1	`entailment`
2	`neutral`
3	`reasoning`

Training


base model	`bertin-project/bertin-roberta-base-spanish`
data	`Flaglab/ESNLIR-dataset`, premise only (`only_premise: true`)
max samples	1,000,000 (subsampled from the 4.4M train split, stratified)
epochs	6, early stopping on validation macro F1 (patience 3)
batch size	64
learning rate	2e-5
max sequence length	256
class weighting	balanced
seed	42

Trained with jd-rodriguezp1234/esnlir (auto_nli/model/bert_based/run.py, config params/model/roberta_annot.json).

Citation

bibtex

@InProceedings{portela2025esnlirspanishmultigenredataset,
  author    = {Portela, Johan R. and P{\'e}rez-Ter{\'a}n, Nicol{\'a}s and Manrique, Rub{\'e}n},
  editor    = {Florez, Hector and Peluffo-Ordo{\~{n}}ez, Diego},
  title     = {{ESNLIR}: Expanding Spanish {NLI} Benchmarks with Multi-genre and Causal Annotation},
  booktitle = {Applied Informatics},
  year      = {2026},
  publisher = {Springer Nature Switzerland},
  address   = {Cham},
  pages     = {345--361},
  isbn      = {978-3-032-07175-0},
  doi       = {10.1007/978-3-032-07175-0_23},
}

Flaglab/ESNLIR-RoBERTa-premise-only

ESNLIR — BERTIN RoBERTa (premise only)

⚠️ This checkpoint collapsed during training

Files

Labels

Training

Citation

Flaglab/
ESNLIR-RoBERTa-premise-only