You can deploy this model and start earning money today!

Amala3/
bertweet-base_ordinal_7_seed42_EN-NL

Name: bertweet-base_ordinal_7_seed42_EN-NL
Availability: InStock

safetensors

roberta

generated_from_trainer

vinai/bertweet-base

finetune

mit

Views

No views yet

bertweet-base_ordinal_7_seed42_EN-NL

This model is a fine-tuned version of vinai/bertweet-base on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 3.9585
Mse: 6.1125
Rmse: 2.4723
Mae: 1.5109
R2: 0.1525
F1: 0.7447
Precision: 0.7446
Recall: 0.7479
Accuracy: 0.7479

Model description

This is the best-performing model for Dutch irony detection. The model was fine-tuned both a mix of English and Dutch tweets. The model predicts one of 7 labels indicating for irony likelihood, where 0 is not ironic and 6 is ironic. When merging for binary classification, we advise mapping labels 0,1,2,3 as not-ironic and labels 4,5,6 as ironic.

Intended uses & limitations

More information needed

Training and evaluation data

The model was trained and evaluated on the TRIC dataset.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 200
num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Mse	Rmse	Mae	R2	F1	Precision	Recall	Accuracy
5.7557	0.2141	100	5.6476	7.6297	2.7622	2.6574	-0.0806	0.4761	0.3859	0.6212	0.6212
5.2388	0.4283	200	5.2492	7.1086	2.6662	2.4741	-0.0068	0.4761	0.3859	0.6212	0.6212
4.9773	0.6424	300	5.0558	6.8733	2.6217	2.3016	0.0266	0.4761	0.3859	0.6212	0.6212
4.7427	0.8565	400	4.8666	6.6212	2.5732	2.1990	0.0623	0.4761	0.3859	0.6212	0.6212
4.6378	1.0707	500	4.6806	6.0772	2.4652	2.0941	0.1393	0.6773	0.6795	0.6888	0.6888
4.3851	1.2848	600	4.6153	6.2799	2.5060	1.9928	0.1106	0.6915	0.6964	0.6888	0.6888
4.3077	1.4989	700	4.5016	6.2147	2.4929	1.9276	0.1198	0.6882	0.6928	0.7008	0.7008
4.2337	1.7131	800	4.2877	5.5862	2.3635	1.8854	0.2088	0.7183	0.7218	0.7274	0.7274
4.2273	1.9272	900	4.3769	5.9397	2.4371	1.8601	0.1588	0.6994	0.6991	0.6996	0.6996
4.0563	2.1413	1000	4.2168	6.1013	2.4701	1.7033	0.1359	0.7088	0.7203	0.7238	0.7238
3.7778	2.3555	1100	4.1356	6.1098	2.4718	1.6562	0.1347	0.7260	0.7269	0.7322	0.7322
3.7206	2.5696	1200	4.2222	6.1062	2.4711	1.7394	0.1352	0.7245	0.7326	0.7214	0.7214
3.7175	2.7837	1300	4.0073	5.7021	2.3879	1.6224	0.1924	0.7277	0.7345	0.7382	0.7382
3.8003	2.9979	1400	4.1116	5.8166	2.4118	1.7346	0.1762	0.7258	0.7268	0.7250	0.7250
3.6247	3.2120	1500	4.1286	6.0663	2.4630	1.6876	0.1409	0.7309	0.7355	0.7286	0.7286
3.4364	3.4261	1600	4.2100	6.3353	2.5170	1.7467	0.1028	0.7235	0.7329	0.7201	0.7201
3.3301	3.6403	1700	4.0403	6.0483	2.4593	1.6357	0.1434	0.7436	0.7442	0.7431	0.7431
3.3634	3.8544	1800	3.9496	5.5790	2.3620	1.6297	0.2099	0.7259	0.7282	0.7334	0.7334
3.4602	4.0685	1900	3.8729	5.7334	2.3945	1.5597	0.1880	0.7402	0.7410	0.7455	0.7455
3.1223	4.2827	2000	4.0417	6.3812	2.5261	1.5875	0.0963	0.7144	0.7394	0.7346	0.7346
3.1337	4.4968	2100	4.0039	5.9493	2.4391	1.6285	0.1574	0.7389	0.7421	0.7370	0.7370
3.1321	4.7109	2200	3.9092	5.8926	2.4275	1.5742	0.1655	0.7347	0.7339	0.7358	0.7358
3.1927	4.9251	2300	4.0312	5.9928	2.4480	1.6140	0.1513	0.7459	0.7540	0.7431	0.7431
2.9806	5.1392	2400	3.9638	6.0145	2.4524	1.5633	0.1482	0.7524	0.7536	0.7515	0.7515
2.9582	5.3533	2500	3.9413	5.9409	2.4374	1.5549	0.1586	0.7539	0.7539	0.7539	0.7539
2.7418	5.5675	2600	3.9578	5.9843	2.4463	1.5525	0.1525	0.7456	0.7476	0.7443	0.7443
2.9866	5.7816	2700	3.8793	5.8070	2.4098	1.5416	0.1776	0.7426	0.7425	0.7467	0.7467
2.8627	5.9957	2800	3.8625	5.7805	2.4043	1.5103	0.1813	0.7615	0.7609	0.7624	0.7624
2.8191	6.2099	2900	3.9342	5.9964	2.4488	1.5211	0.1508	0.7628	0.7622	0.7636	0.7636
2.6259	6.4240	3000	3.9203	6.0893	2.4676	1.5006	0.1376	0.7487	0.7478	0.7503	0.7503
2.8785	6.6381	3100	3.8633	5.8444	2.4175	1.4946	0.1723	0.7600	0.7601	0.7600	0.7600
2.6016	6.8522	3200	4.0736	6.2654	2.5031	1.5923	0.1127	0.7456	0.7518	0.7431	0.7431
2.5155	7.0664	3300	3.9459	6.0688	2.4635	1.5211	0.1405	0.7584	0.7597	0.7575	0.7575
2.6918	7.2805	3400	3.9312	6.0072	2.4510	1.5271	0.1492	0.7541	0.7534	0.7551	0.7551

Framework versions

Transformers 4.40.2
Pytorch 2.1.2
Datasets 2.18.0
Tokenizers 0.19.1

Amala3/bertweet-base_ordinal_7_seed42_EN-NL