deberta-climatecheck-2463191-step45753 – AI Model by rausch | AlphaNeural AI

You can deploy this model and start earning money today!

rausch/
deberta-climatecheck-2463191-step45753

like

0

text-classification

Dzeniks/hover-3way

pminervini/averitec

rabuahmad/climatecheck

MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli

Views

No views yet

DeBERT ClimateCheck - checkpoint 45753 (job 2463191)

Latest checkpoint from job 2463191.

Task

3-way NLI / claim verification with labels: supports, refutes, nei.

Base model

MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli

Datasets

fever/fever (v1.0)
tals/vitaminc
Dzeniks/hover-3way
pminervini/averitec
rabuahmad/climatecheck

Splits

Combined train/validation are concatenated across datasets.
ClimateCheck uses a stratified 90/10 split from train with seed 1234 (official test split ignored).

Processed datasets (sizes)

Dataset	Train	Validation
fever	94,616	11,882
vitaminc	370,653	63,054
hover	17,155	2,144
averitec	2,872	462
climatecheck	2,722	301
Total	488,018	77,843

Label distribution (combined)

Split	supports	refutes	nei
Train	262,189	165,798	60,031
Validation	38,207	29,733	9,903
Climate eval	139	45	117

Training configuration

num_train_epochs: 3.0
per_device_train_batch_size: 32
per_device_eval_batch_size: 8
gradient_accumulation_steps: 1
learning_rate: 2e-06
warmup_steps: 200
weight_decay: 0.01
lr_scheduler_type: linear
max_length: 320
eval_steps: 2000
save_steps: 2000
seed: 1234

Hardware

gpu: 1x NVIDIA H100 (slurm partition H100)
cpus: 6
memory: 64G

Eval metrics (combined validation)

Last eval logged at step 44000 (epoch 2.885056717592289).

epoch: 2.885056717592289
eval_accuracy: 0.9207764346183985
eval_f1_micro: 0.9207764346183985
eval_loss: 0.25573235750198364
eval_macro_f1: 0.8866655494028723
eval_macro_precision: 0.8960382307976472
eval_macro_recall: 0.8785658536930484
eval_nei_f1: 0.785631150142842
eval_nei_precision: 0.8250916768529837
eval_nei_recall: 0.7497727961223871
eval_refutes_f1: 0.9196205188481236
eval_refutes_precision: 0.9182482730869828
eval_refutes_recall: 0.920996872162244
eval_runtime: 198.2607
eval_samples_per_second: 392.629
eval_steps_per_second: 49.082
eval_supports_f1: 0.9547449792176514
eval_supports_precision: 0.9447747424529752
eval_supports_recall: 0.9649278927945141
step: 44000

Climate split metrics (final eval)

Final climate eval at epoch 3.0 (step 45753).

climate_eval_accuracy: 0.6710963455149501
climate_eval_f1_micro: 0.6710963455149501
climate_eval_loss: 0.8785929679870605
climate_eval_macro_f1: 0.6563499788188986
climate_eval_macro_precision: 0.6748838640407783
climate_eval_macro_recall: 0.6448666707659513
climate_eval_nei_f1: 0.600896860986547
climate_eval_nei_precision: 0.6320754716981132
climate_eval_nei_recall: 0.5726495726495726
climate_eval_refutes_f1: 0.6341463414634146
climate_eval_refutes_precision: 0.7027027027027027
climate_eval_refutes_recall: 0.5777777777777777
climate_eval_runtime: 1.2101
climate_eval_samples_per_second: 248.744
climate_eval_steps_per_second: 31.403
climate_eval_supports_f1: 0.734006734006734
climate_eval_supports_precision: 0.689873417721519
climate_eval_supports_recall: 0.7841726618705036
epoch: 3.0