You can deploy this model and start earning money today!

faodl/
model_g20_multilabel_MiniLM-L12-v2_15_sample

Name: model_g20_multilabel_MiniLM-L12-v2_15_sample
Availability: InStock

setfit

safetensors

bert

sentence-transformers

text-classification

generated_from_setfit_trainer

2209.11055

sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

finetune

Views

No views yet

SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 as the Sentence Transformer embedding model. A OneVsRestClassifier instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Classification head: a OneVsRestClassifier instance
Maximum Sequence Length: 128 tokens

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

python

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("faodl/model_g20_multilabel_MiniLM-L12-v2_15_sample")
# Run inference
preds = model("13 (Deroga delle norme in materia di riconoscimento delle qualifiche professionali sanitarie) 1. 93 (Disposizioni in materia di autoservizi pubblici non di linea) 1. 4 (Disciplina delle aree sanitarie temporanee) 1.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	3	93.9143	1651

Training Hyperparameters

batch_size: (8, 8)
num_epochs: (4, 4)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 20
body_learning_rate: (2e-05, 2e-05)
head_learning_rate: 2e-05
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
l2_weight: 0.01
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0001	1	0.2185	-
0.0068	50	0.1579	-
0.0136	100	0.1625	-
0.0204	150	0.1649	-
0.0272	200	0.1511	-
0.0340	250	0.1263	-
0.0408	300	0.1335	-
0.0476	350	0.1276	-
0.0544	400	0.1143	-
0.0612	450	0.1095	-
0.0680	500	0.1029	-
0.0748	550	0.1161	-
0.0816	600	0.114	-
0.0884	650	0.0945	-
0.0952	700	0.0903	-
0.1020	750	0.0793	-
0.1088	800	0.0848	-
0.1156	850	0.0802	-
0.1224	900	0.0819	-
0.1293	950	0.0802	-
0.1361	1000	0.0879	-
0.1429	1050	0.0738	-
0.1497	1100	0.0737	-
0.1565	1150	0.0761	-
0.1633	1200	0.0715	-
0.1701	1250	0.0633	-
0.1769	1300	0.06	-
0.1837	1350	0.06	-
0.1905	1400	0.0641	-
0.1973	1450	0.057	-
0.2041	1500	0.0554	-
0.2109	1550	0.0552	-
0.2177	1600	0.0447	-
0.2245	1650	0.0442	-
0.2313	1700	0.0547	-
0.2381	1750	0.0358	-
0.2449	1800	0.0503	-
0.2517	1850	0.0366	-
0.2585	1900	0.0421	-
0.2653	1950	0.0332	-
0.2721	2000	0.0429	-
0.2789	2050	0.0316	-
0.2857	2100	0.0382	-
0.2925	2150	0.0456	-
0.2993	2200	0.0327	-
0.3061	2250	0.0286	-
0.3129	2300	0.0295	-
0.3197	2350	0.0305	-
0.3265	2400	0.0223	-
0.3333	2450	0.0228	-
0.3401	2500	0.0305	-
0.3469	2550	0.0294	-
0.3537	2600	0.0342	-
0.3605	2650	0.0275	-
0.3673	2700	0.0181	-
0.3741	2750	0.0267	-
0.3810	2800	0.0229	-
0.3878	2850	0.0213	-
0.3946	2900	0.0203	-
0.4014	2950	0.0281	-
0.4082	3000	0.025	-
0.4150	3050	0.0233	-
0.4218	3100	0.0306	-
0.4286	3150	0.0159	-
0.4354	3200	0.0246	-
0.4422	3250	0.0266	-
0.4490	3300	0.0242	-
0.4558	3350	0.0103	-
0.4626	3400	0.0191	-
0.4694	3450	0.0237	-
0.4762	3500	0.0216	-
0.4830	3550	0.0103	-
0.4898	3600	0.0097	-
0.4966	3650	0.0158	-
0.5034	3700	0.0156	-
0.5102	3750	0.0152	-
0.5170	3800	0.0187	-
0.5238	3850	0.0129	-
0.5306	3900	0.0157	-
0.5374	3950	0.0161	-
0.5442	4000	0.0131	-
0.5510	4050	0.0119	-
0.5578	4100	0.0213	-
0.5646	4150	0.0086	-
0.5714	4200	0.0086	-
0.5782	4250	0.0121	-
0.5850	4300	0.0168	-
0.5918	4350	0.0147	-
0.5986	4400	0.019	-
0.6054	4450	0.0151	-
0.6122	4500	0.0298	-
0.6190	4550	0.0187	-
0.6259	4600	0.013	-
0.6327	4650	0.0184	-
0.6395	4700	0.0249	-
0.6463	4750	0.0157	-
0.6531	4800	0.0081	-
0.6599	4850	0.0229	-
0.6667	4900	0.0227	-
0.6735	4950	0.0166	-
0.6803	5000	0.0222	-
0.6871	5050	0.0066	-
0.6939	5100	0.0135	-
0.7007	5150	0.0134	-
0.7075	5200	0.0134	-
0.7143	5250	0.0077	-
0.7211	5300	0.0106	-
0.7279	5350	0.0086	-
0.7347	5400	0.0169	-
0.7415	5450	0.0123	-
0.7483	5500	0.0085	-
0.7551	5550	0.0087	-
0.7619	5600	0.0143	-
0.7687	5650	0.0112	-
0.7755	5700	0.0185	-
0.7823	5750	0.0064	-
0.7891	5800	0.0077	-
0.7959	5850	0.0116	-
0.8027	5900	0.0063	-
0.8095	5950	0.0166	-
0.8163	6000	0.01	-
0.8231	6050	0.0088	-
0.8299	6100	0.0121	-
0.8367	6150	0.0214	-
0.8435	6200	0.009	-
0.8503	6250	0.0133	-
0.8571	6300	0.0062	-
0.8639	6350	0.0077	-
0.8707	6400	0.0201	-
0.8776	6450	0.0163	-
0.8844	6500	0.0071	-
0.8912	6550	0.0138	-
0.8980	6600	0.0131	-
0.9048	6650	0.0126	-
0.9116	6700	0.0042	-
0.9184	6750	0.0152	-
0.9252	6800	0.0194	-
0.9320	6850	0.0068	-
0.9388	6900	0.0154	-
0.9456	6950	0.0077	-
0.9524	7000	0.009	-
0.9592	7050	0.0053	-
0.9660	7100	0.0128	-
0.9728	7150	0.011	-
0.9796	7200	0.0039	-
0.9864	7250	0.0076	-
0.9932	7300	0.018	-
1.0	7350	0.0215	-
1.0068	7400	0.0022	-
1.0136	7450	0.01	-
1.0204	7500	0.0061	-
1.0272	7550	0.0039	-
1.0340	7600	0.0052	-
1.0408	7650	0.0053	-
1.0476	7700	0.0093	-
1.0544	7750	0.0099	-
1.0612	7800	0.0076	-
1.0680	7850	0.0094	-
1.0748	7900	0.0065	-
1.0816	7950	0.0083	-
1.0884	8000	0.007	-
1.0952	8050	0.0056	-
1.1020	8100	0.0112	-
1.1088	8150	0.0087	-
1.1156	8200	0.0055	-
1.1224	8250	0.0051	-
1.1293	8300	0.0096	-
1.1361	8350	0.0038	-
1.1429	8400	0.0055	-
1.1497	8450	0.0051	-
1.1565	8500	0.01	-
1.1633	8550	0.0058	-
1.1701	8600	0.0112	-
1.1769	8650	0.003	-
1.1837	8700	0.0094	-
1.1905	8750	0.0069	-
1.1973	8800	0.0131	-
1.2041	8850	0.0089	-
1.2109	8900	0.0061	-
1.2177	8950	0.0109	-
1.2245	9000	0.008	-
1.2313	9050	0.0122	-
1.2381	9100	0.0081	-
1.2449	9150	0.0014	-
1.2517	9200	0.0046	-
1.2585	9250	0.0049	-
1.2653	9300	0.0147	-
1.2721	9350	0.0105	-
1.2789	9400	0.0126	-
1.2857	9450	0.0031	-
1.2925	9500	0.0039	-
1.2993	9550	0.0038	-
1.3061	9600	0.0047	-
1.3129	9650	0.0037	-
1.3197	9700	0.0103	-
1.3265	9750	0.0007	-
1.3333	9800	0.0053	-
1.3401	9850	0.0018	-
1.3469	9900	0.0057	-
1.3537	9950	0.0044	-
1.3605	10000	0.0109	-
1.3673	10050	0.0056	-
1.3741	10100	0.0081	-
1.3810	10150	0.008	-
1.3878	10200	0.0081	-
1.3946	10250	0.0033	-
1.4014	10300	0.0055	-
1.4082	10350	0.0019	-
1.4150	10400	0.0033	-
1.4218	10450	0.0033	-
1.4286	10500	0.0058	-
1.4354	10550	0.0047	-
1.4422	10600	0.0068	-
1.4490	10650	0.0052	-
1.4558	10700	0.0033	-
1.4626	10750	0.001	-
1.4694	10800	0.0101	-
1.4762	10850	0.0011	-
1.4830	10900	0.008	-
1.4898	10950	0.0038	-
1.4966	11000	0.0033	-
1.5034	11050	0.0031	-
1.5102	11100	0.0107	-
1.5170	11150	0.004	-
1.5238	11200	0.0009	-
1.5306	11250	0.0034	-
1.5374	11300	0.0033	-
1.5442	11350	0.0011	-
1.5510	11400	0.0081	-
1.5578	11450	0.0025	-
1.5646	11500	0.0065	-
1.5714	11550	0.0069	-
1.5782	11600	0.0053	-
1.5850	11650	0.0031	-
1.5918	11700	0.0059	-
1.5986	11750	0.006	-
1.6054	11800	0.0007	-
1.6122	11850	0.0027	-
1.6190	11900	0.003	-
1.6259	11950	0.0052	-
1.6327	12000	0.0065	-
1.6395	12050	0.0032	-
1.6463	12100	0.0054	-
1.6531	12150	0.0063	-
1.6599	12200	0.0155	-
1.6667	12250	0.0105	-
1.6735	12300	0.0067	-
1.6803	12350	0.0034	-
1.6871	12400	0.0076	-
1.6939	12450	0.0042	-
1.7007	12500	0.003	-
1.7075	12550	0.0096	-
1.7143	12600	0.0054	-
1.7211	12650	0.005	-
1.7279	12700	0.0039	-
1.7347	12750	0.0061	-
1.7415	12800	0.0027	-
1.7483	12850	0.0033	-
1.7551	12900	0.0028	-
1.7619	12950	0.0038	-
1.7687	13000	0.0083	-
1.7755	13050	0.0074	-
1.7823	13100	0.0015	-
1.7891	13150	0.0037	-
1.7959	13200	0.0041	-
1.8027	13250	0.0007	-
1.8095	13300	0.0046	-
1.8163	13350	0.0007	-
1.8231	13400	0.0019	-
1.8299	13450	0.0051	-
1.8367	13500	0.0007	-
1.8435	13550	0.0013	-
1.8503	13600	0.0045	-
1.8571	13650	0.0006	-
1.8639	13700	0.0028	-
1.8707	13750	0.0028	-
1.8776	13800	0.001	-
1.8844	13850	0.001	-
1.8912	13900	0.0075	-
1.8980	13950	0.0041	-
1.9048	14000	0.0115	-
1.9116	14050	0.0007	-
1.9184	14100	0.0069	-
1.9252	14150	0.0017	-
1.9320	14200	0.005	-
1.9388	14250	0.0028	-
1.9456	14300	0.0029	-
1.9524	14350	0.0052	-
1.9592	14400	0.0023	-
1.9660	14450	0.0046	-
1.9728	14500	0.001	-
1.9796	14550	0.0009	-
1.9864	14600	0.0059	-
1.9932	14650	0.0075	-
2.0	14700	0.003	-
2.0068	14750	0.0088	-
2.0136	14800	0.0073	-
2.0204	14850	0.0023	-
2.0272	14900	0.0104	-
2.0340	14950	0.0024	-
2.0408	15000	0.0059	-
2.0476	15050	0.0041	-
2.0544	15100	0.0079	-
2.0612	15150	0.0011	-
2.0680	15200	0.0038	-
2.0748	15250	0.0009	-
2.0816	15300	0.0057	-
2.0884	15350	0.0025	-
2.0952	15400	0.0033	-
2.1020	15450	0.0093	-
2.1088	15500	0.0006	-
2.1156	15550	0.0024	-
2.1224	15600	0.0044	-
2.1293	15650	0.0069	-
2.1361	15700	0.0051	-
2.1429	15750	0.008	-
2.1497	15800	0.0047	-
2.1565	15850	0.0012	-
2.1633	15900	0.001	-
2.1701	15950	0.0019	-
2.1769	16000	0.0024	-
2.1837	16050	0.0066	-
2.1905	16100	0.0025	-
2.1973	16150	0.0037	-
2.2041	16200	0.0033	-
2.2109	16250	0.0023	-
2.2177	16300	0.0013	-
2.2245	16350	0.0033	-
2.2313	16400	0.0029	-
2.2381	16450	0.0038	-
2.2449	16500	0.0015	-
2.2517	16550	0.0007	-
2.2585	16600	0.0031	-
2.2653	16650	0.0061	-
2.2721	16700	0.0011	-
2.2789	16750	0.0049	-
2.2857	16800	0.0012	-
2.2925	16850	0.0036	-
2.2993	16900	0.004	-
2.3061	16950	0.0005	-
2.3129	17000	0.0019	-
2.3197	17050	0.003	-
2.3265	17100	0.0006	-
2.3333	17150	0.0009	-
2.3401	17200	0.0013	-
2.3469	17250	0.0018	-
2.3537	17300	0.0007	-
2.3605	17350	0.001	-
2.3673	17400	0.0054	-
2.3741	17450	0.0004	-
2.3810	17500	0.0028	-
2.3878	17550	0.0005	-
2.3946	17600	0.0003	-
2.4014	17650	0.0004	-
2.4082	17700	0.0031	-
2.4150	17750	0.0004	-
2.4218	17800	0.0013	-
2.4286	17850	0.0017	-
2.4354	17900	0.0013	-
2.4422	17950	0.0025	-
2.4490	18000	0.0004	-
2.4558	18050	0.0029	-
2.4626	18100	0.0023	-
2.4694	18150	0.0027	-
2.4762	18200	0.0015	-
2.4830	18250	0.0006	-
2.4898	18300	0.0024	-
2.4966	18350	0.0021	-
2.5034	18400	0.0005	-
2.5102	18450	0.0004	-
2.5170	18500	0.0042	-
2.5238	18550	0.0005	-
2.5306	18600	0.0012	-
2.5374	18650	0.005	-
2.5442	18700	0.0032	-
2.5510	18750	0.0079	-
2.5578	18800	0.001	-
2.5646	18850	0.0008	-
2.5714	18900	0.0042	-
2.5782	18950	0.001	-
2.5850	19000	0.001	-
2.5918	19050	0.0009	-
2.5986	19100	0.0003	-
2.6054	19150	0.0003	-
2.6122	19200	0.0003	-
2.6190	19250	0.0035	-
2.6259	19300	0.0006	-
2.6327	19350	0.0035	-
2.6395	19400	0.0003	-
2.6463	19450	0.0021	-
2.6531	19500	0.0005	-
2.6599	19550	0.004	-
2.6667	19600	0.0023	-
2.6735	19650	0.0006	-
2.6803	19700	0.004	-
2.6871	19750	0.0015	-
2.6939	19800	0.0008	-
2.7007	19850	0.0022	-
2.7075	19900	0.001	-
2.7143	19950	0.0007	-
2.7211	20000	0.0013	-
2.7279	20050	0.0004	-
2.7347	20100	0.001	-
2.7415	20150	0.0013	-
2.7483	20200	0.0004	-
2.7551	20250	0.0035	-
2.7619	20300	0.0006	-
2.7687	20350	0.001	-
2.7755	20400	0.0003	-
2.7823	20450	0.0006	-
2.7891	20500	0.0012	-
2.7959	20550	0.0003	-
2.8027	20600	0.0031	-
2.8095	20650	0.0005	-
2.8163	20700	0.0008	-
2.8231	20750	0.0006	-
2.8299	20800	0.0005	-
2.8367	20850	0.0004	-
2.8435	20900	0.0002	-
2.8503	20950	0.0011	-
2.8571	21000	0.0002	-
2.8639	21050	0.0033	-
2.8707	21100	0.0024	-
2.8776	21150	0.0004	-
2.8844	21200	0.0002	-
2.8912	21250	0.0029	-
2.8980	21300	0.0004	-
2.9048	21350	0.0003	-
2.9116	21400	0.0024	-
2.9184	21450	0.0027	-
2.9252	21500	0.0003	-
2.9320	21550	0.0006	-
2.9388	21600	0.0002	-
2.9456	21650	0.0021	-
2.9524	21700	0.0011	-
2.9592	21750	0.0006	-
2.9660	21800	0.0002	-
2.9728	21850	0.0004	-
2.9796	21900	0.0008	-
2.9864	21950	0.0028	-
2.9932	22000	0.0004	-
3.0	22050	0.0002	-
3.0068	22100	0.0002	-
3.0136	22150	0.0026	-
3.0204	22200	0.0002	-
3.0272	22250	0.0004	-
3.0340	22300	0.0005	-
3.0408	22350	0.0005	-
3.0476	22400	0.0022	-
3.0544	22450	0.0006	-
3.0612	22500	0.0009	-
3.0680	22550	0.0004	-
3.0748	22600	0.0002	-
3.0816	22650	0.0003	-
3.0884	22700	0.0002	-
3.0952	22750	0.0002	-
3.1020	22800	0.0002	-
3.1088	22850	0.0041	-
3.1156	22900	0.0014	-
3.1224	22950	0.0019	-
3.1293	23000	0.0023	-
3.1361	23050	0.0003	-
3.1429	23100	0.0005	-
3.1497	23150	0.0003	-
3.1565	23200	0.0009	-
3.1633	23250	0.0023	-
3.1701	23300	0.0003	-
3.1769	23350	0.0002	-
3.1837	23400	0.0003	-
3.1905	23450	0.0003	-
3.1973	23500	0.0015	-
3.2041	23550	0.0002	-
3.2109	23600	0.0004	-
3.2177	23650	0.0004	-
3.2245	23700	0.0009	-
3.2313	23750	0.0002	-
3.2381	23800	0.0003	-
3.2449	23850	0.0002	-
3.2517	23900	0.0001	-
3.2585	23950	0.0003	-
3.2653	24000	0.0002	-
3.2721	24050	0.0019	-
3.2789	24100	0.0002	-
3.2857	24150	0.0002	-
3.2925	24200	0.0002	-
3.2993	24250	0.0002	-
3.3061	24300	0.0003	-
3.3129	24350	0.0007	-
3.3197	24400	0.0009	-
3.3265	24450	0.0006	-
3.3333	24500	0.0003	-
3.3401	24550	0.0008	-
3.3469	24600	0.0007	-
3.3537	24650	0.0003	-
3.3605	24700	0.0002	-
3.3673	24750	0.0001	-
3.3741	24800	0.0001	-
3.3810	24850	0.0002	-
3.3878	24900	0.0009	-
3.3946	24950	0.0005	-
3.4014	25000	0.0001	-
3.4082	25050	0.0003	-
3.4150	25100	0.0001	-
3.4218	25150	0.0002	-
3.4286	25200	0.0002	-
3.4354	25250	0.0003	-
3.4422	25300	0.0002	-
3.4490	25350	0.0004	-
3.4558	25400	0.0005	-
3.4626	25450	0.0005	-
3.4694	25500	0.0002	-
3.4762	25550	0.0003	-
3.4830	25600	0.0001	-
3.4898	25650	0.0003	-
3.4966	25700	0.0006	-
3.5034	25750	0.0002	-
3.5102	25800	0.0003	-
3.5170	25850	0.0005	-
3.5238	25900	0.0002	-
3.5306	25950	0.0003	-
3.5374	26000	0.0002	-
3.5442	26050	0.0004	-
3.5510	26100	0.0001	-
3.5578	26150	0.0001	-
3.5646	26200	0.0002	-
3.5714	26250	0.0001	-
3.5782	26300	0.0005	-
3.5850	26350	0.0002	-
3.5918	26400	0.0001	-
3.5986	26450	0.0001	-
3.6054	26500	0.0003	-
3.6122	26550	0.0002	-
3.6190	26600	0.0002	-
3.6259	26650	0.0001	-
3.6327	26700	0.0001	-
3.6395	26750	0.0001	-
3.6463	26800	0.0005	-
3.6531	26850	0.0001	-
3.6599	26900	0.0002	-
3.6667	26950	0.0001	-
3.6735	27000	0.0001	-
3.6803	27050	0.0002	-
3.6871	27100	0.0002	-
3.6939	27150	0.0001	-
3.7007	27200	0.0001	-
3.7075	27250	0.0002	-
3.7143	27300	0.0002	-
3.7211	27350	0.0001	-
3.7279	27400	0.0008	-
3.7347	27450	0.0002	-
3.7415	27500	0.0008	-
3.7483	27550	0.0005	-
3.7551	27600	0.0002	-
3.7619	27650	0.0003	-
3.7687	27700	0.0002	-
3.7755	27750	0.0007	-
3.7823	27800	0.0003	-
3.7891	27850	0.0001	-
3.7959	27900	0.0006	-
3.8027	27950	0.0002	-
3.8095	28000	0.0001	-
3.8163	28050	0.0001	-
3.8231	28100	0.0002	-
3.8299	28150	0.0001	-
3.8367	28200	0.0001	-
3.8435	28250	0.0004	-
3.8503	28300	0.0001	-
3.8571	28350	0.0001	-
3.8639	28400	0.0001	-
3.8707	28450	0.0005	-
3.8776	28500	0.0004	-
3.8844	28550	0.0001	-
3.8912	28600	0.0002	-
3.8980	28650	0.0002	-
3.9048	28700	0.0003	-
3.9116	28750	0.0001	-
3.9184	28800	0.0002	-
3.9252	28850	0.0001	-
3.9320	28900	0.0001	-
3.9388	28950	0.0002	-
3.9456	29000	0.0002	-
3.9524	29050	0.0001	-
3.9592	29100	0.0001	-
3.9660	29150	0.0002	-
3.9728	29200	0.0002	-
3.9796	29250	0.0003	-
3.9864	29300	0.0001	-
3.9932	29350	0.0007	-
4.0	29400	0.0007	-

Framework Versions

Python: 3.11.13
SetFit: 1.1.2
Sentence Transformers: 4.1.0
Transformers: 4.52.4
PyTorch: 2.6.0+cu124
Datasets: 3.6.0
Tokenizers: 0.21.2

Citation

BibTeX

bibtex

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}