You can deploy this model and start earning money today!

faodl/
model_cca_multilabel_MiniLM-L12-70prop-data-augmented-v02

Name: model_cca_multilabel_MiniLM-L12-70prop-data-augmented-v02
Availability: InStock

setfit

safetensors

bert

sentence-transformers

text-classification

generated_from_setfit_trainer

2209.11055

sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

finetune

text-embeddings-inference

Views

No views yet

SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 as the Sentence Transformer embedding model. A OneVsRestClassifier instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Classification head: a OneVsRestClassifier instance
Maximum Sequence Length: 128 tokens

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

python

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("faodl/model_cca_multilabel_MiniLM-L12-70prop-data-augmented-v02")
# Run inference
preds = model("Mechanization investment will be integrated with soil health and water management programs, ensuring appropriate machinery selection to avoid soil compaction and water wastage.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	55.4334	951

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (2, 2)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 20
body_learning_rate: (2e-05, 2e-05)
head_learning_rate: 2e-05
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
l2_weight: 0.01
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0001	1	0.2114	-
0.0045	50	0.2069	-
0.0091	100	0.2029	-
0.0136	150	0.2025	-
0.0181	200	0.1984	-
0.0226	250	0.1848	-
0.0272	300	0.1784	-
0.0317	350	0.176	-
0.0362	400	0.1743	-
0.0408	450	0.1579	-
0.0453	500	0.149	-
0.0498	550	0.1532	-
0.0543	600	0.1551	-
0.0589	650	0.1483	-
0.0634	700	0.1474	-
0.0679	750	0.1444	-
0.0725	800	0.1363	-
0.0770	850	0.1269	-
0.0815	900	0.1541	-
0.0861	950	0.1256	-
0.0906	1000	0.1457	-
0.0951	1050	0.131	-
0.0996	1100	0.1224	-
0.1042	1150	0.1357	-
0.1087	1200	0.1341	-
0.1132	1250	0.1371	-
0.1178	1300	0.1305	-
0.1223	1350	0.1165	-
0.1268	1400	0.1191	-
0.1313	1450	0.1247	-
0.1359	1500	0.1209	-
0.1404	1550	0.129	-
0.1449	1600	0.1161	-
0.1495	1650	0.1215	-
0.1540	1700	0.1213	-
0.1585	1750	0.1193	-
0.1630	1800	0.1126	-
0.1676	1850	0.1253	-
0.1721	1900	0.1135	-
0.1766	1950	0.1032	-
0.1812	2000	0.0998	-
0.1857	2050	0.116	-
0.1902	2100	0.1088	-
0.1947	2150	0.104	-
0.1993	2200	0.1139	-
0.2038	2250	0.1084	-
0.2083	2300	0.1043	-
0.2129	2350	0.1149	-
0.2174	2400	0.1022	-
0.2219	2450	0.1106	-
0.2264	2500	0.1028	-
0.2310	2550	0.0986	-
0.2355	2600	0.0965	-
0.2400	2650	0.1047	-
0.2446	2700	0.1007	-
0.2491	2750	0.0979	-
0.2536	2800	0.0967	-
0.2582	2850	0.0999	-
0.2627	2900	0.1025	-
0.2672	2950	0.0938	-
0.2717	3000	0.0923	-
0.2763	3050	0.0885	-
0.2808	3100	0.0953	-
0.2853	3150	0.0931	-
0.2899	3200	0.095	-
0.2944	3250	0.0945	-
0.2989	3300	0.0919	-
0.3034	3350	0.0975	-
0.3080	3400	0.0906	-
0.3125	3450	0.0977	-
0.3170	3500	0.0952	-
0.3216	3550	0.0851	-
0.3261	3600	0.0883	-
0.3306	3650	0.0852	-
0.3351	3700	0.082	-
0.3397	3750	0.0901	-
0.3442	3800	0.0778	-
0.3487	3850	0.0819	-
0.3533	3900	0.0804	-
0.3578	3950	0.083	-
0.3623	4000	0.0855	-
0.3668	4050	0.0828	-
0.3714	4100	0.0899	-
0.3759	4150	0.0875	-
0.3804	4200	0.0816	-
0.3850	4250	0.09	-
0.3895	4300	0.0782	-
0.3940	4350	0.0831	-
0.3986	4400	0.0795	-
0.4031	4450	0.0807	-
0.4076	4500	0.0809	-
0.4121	4550	0.0763	-
0.4167	4600	0.08	-
0.4212	4650	0.0731	-
0.4257	4700	0.0759	-
0.4303	4750	0.0758	-
0.4348	4800	0.0791	-
0.4393	4850	0.0731	-
0.4438	4900	0.0774	-
0.4484	4950	0.0781	-
0.4529	5000	0.0783	-
0.4574	5050	0.0852	-
0.4620	5100	0.0771	-
0.4665	5150	0.0813	-
0.4710	5200	0.0795	-
0.4755	5250	0.0725	-
0.4801	5300	0.0751	-
0.4846	5350	0.0756	-
0.4891	5400	0.0715	-
0.4937	5450	0.0643	-
0.4982	5500	0.0675	-
0.5027	5550	0.0769	-
0.5072	5600	0.0761	-
0.5118	5650	0.0739	-
0.5163	5700	0.0716	-
0.5208	5750	0.0706	-
0.5254	5800	0.0719	-
0.5299	5850	0.0721	-
0.5344	5900	0.068	-
0.5389	5950	0.0626	-
0.5435	6000	0.0679	-
0.5480	6050	0.0713	-
0.5525	6100	0.0692	-
0.5571	6150	0.0728	-
0.5616	6200	0.0622	-
0.5661	6250	0.0686	-
0.5707	6300	0.073	-
0.5752	6350	0.0563	-
0.5797	6400	0.0621	-
0.5842	6450	0.0699	-
0.5888	6500	0.0691	-
0.5933	6550	0.0676	-
0.5978	6600	0.0621	-
0.6024	6650	0.0693	-
0.6069	6700	0.0708	-
0.6114	6750	0.0672	-
0.6159	6800	0.0728	-
0.6205	6850	0.0629	-
0.625	6900	0.0694	-
0.6295	6950	0.063	-
0.6341	7000	0.0591	-
0.6386	7050	0.0663	-
0.6431	7100	0.0722	-
0.6476	7150	0.0576	-
0.6522	7200	0.0604	-
0.6567	7250	0.0632	-
0.6612	7300	0.0709	-
0.6658	7350	0.0649	-
0.6703	7400	0.0611	-
0.6748	7450	0.0597	-
0.6793	7500	0.0712	-
0.6839	7550	0.0668	-
0.6884	7600	0.0664	-
0.6929	7650	0.0664	-
0.6975	7700	0.0622	-
0.7020	7750	0.0601	-
0.7065	7800	0.0582	-
0.7111	7850	0.0622	-
0.7156	7900	0.0648	-
0.7201	7950	0.064	-
0.7246	8000	0.0624	-
0.7292	8050	0.0622	-
0.7337	8100	0.0596	-
0.7382	8150	0.0633	-
0.7428	8200	0.0532	-
0.7473	8250	0.0565	-
0.7518	8300	0.0724	-
0.7563	8350	0.0559	-
0.7609	8400	0.064	-
0.7654	8450	0.0603	-
0.7699	8500	0.059	-
0.7745	8550	0.0543	-
0.7790	8600	0.0568	-
0.7835	8650	0.0638	-
0.7880	8700	0.0578	-
0.7926	8750	0.0692	-
0.7971	8800	0.0608	-
0.8016	8850	0.0652	-
0.8062	8900	0.061	-
0.8107	8950	0.0581	-
0.8152	9000	0.0627	-
0.8197	9050	0.0656	-
0.8243	9100	0.0579	-
0.8288	9150	0.0626	-
0.8333	9200	0.0587	-
0.8379	9250	0.0625	-
0.8424	9300	0.051	-
0.8469	9350	0.0553	-
0.8514	9400	0.0507	-
0.8560	9450	0.0521	-
0.8605	9500	0.0548	-
0.8650	9550	0.0536	-
0.8696	9600	0.0517	-
0.8741	9650	0.0569	-
0.8786	9700	0.0572	-
0.8832	9750	0.0553	-
0.8877	9800	0.0567	-
0.8922	9850	0.0594	-
0.8967	9900	0.0598	-
0.9013	9950	0.0548	-
0.9058	10000	0.0563	-
0.9103	10050	0.0466	-
0.9149	10100	0.0561	-
0.9194	10150	0.0533	-
0.9239	10200	0.0569	-
0.9284	10250	0.0484	-
0.9330	10300	0.0563	-
0.9375	10350	0.0597	-
0.9420	10400	0.0556	-
0.9466	10450	0.0542	-
0.9511	10500	0.0528	-
0.9556	10550	0.0652	-
0.9601	10600	0.0541	-
0.9647	10650	0.0581	-
0.9692	10700	0.0529	-
0.9737	10750	0.0497	-
0.9783	10800	0.0591	-
0.9828	10850	0.055	-
0.9873	10900	0.0464	-
0.9918	10950	0.0529	-
0.9964	11000	0.0562	-
1.0009	11050	0.0508	-
1.0054	11100	0.0447	-
1.0100	11150	0.0493	-
1.0145	11200	0.0526	-
1.0190	11250	0.0476	-
1.0236	11300	0.0469	-
1.0281	11350	0.0465	-
1.0326	11400	0.0484	-
1.0371	11450	0.0492	-
1.0417	11500	0.0518	-
1.0462	11550	0.0554	-
1.0507	11600	0.0533	-
1.0553	11650	0.0558	-
1.0598	11700	0.0517	-
1.0643	11750	0.0536	-
1.0688	11800	0.051	-
1.0734	11850	0.0512	-
1.0779	11900	0.0531	-
1.0824	11950	0.0496	-
1.0870	12000	0.0471	-
1.0915	12050	0.0492	-
1.0960	12100	0.0444	-
1.1005	12150	0.0441	-
1.1051	12200	0.053	-
1.1096	12250	0.048	-
1.1141	12300	0.044	-
1.1187	12350	0.0482	-
1.1232	12400	0.0486	-
1.1277	12450	0.0475	-
1.1322	12500	0.0484	-
1.1368	12550	0.0512	-
1.1413	12600	0.0534	-
1.1458	12650	0.0532	-
1.1504	12700	0.0422	-
1.1549	12750	0.0487	-
1.1594	12800	0.0489	-
1.1639	12850	0.0558	-
1.1685	12900	0.0506	-
1.1730	12950	0.0497	-
1.1775	13000	0.0593	-
1.1821	13050	0.0476	-
1.1866	13100	0.0435	-
1.1911	13150	0.0553	-
1.1957	13200	0.0527	-
1.2002	13250	0.0535	-
1.2047	13300	0.0469	-
1.2092	13350	0.054	-
1.2138	13400	0.0476	-
1.2183	13450	0.0507	-
1.2228	13500	0.0497	-
1.2274	13550	0.0519	-
1.2319	13600	0.0444	-
1.2364	13650	0.0516	-
1.2409	13700	0.0494	-
1.2455	13750	0.0425	-
1.25	13800	0.0487	-
1.2545	13850	0.0509	-
1.2591	13900	0.0523	-
1.2636	13950	0.0446	-
1.2681	14000	0.0478	-
1.2726	14050	0.0439	-
1.2772	14100	0.0518	-
1.2817	14150	0.0505	-
1.2862	14200	0.0497	-
1.2908	14250	0.0554	-
1.2953	14300	0.0482	-
1.2998	14350	0.0427	-
1.3043	14400	0.05	-
1.3089	14450	0.0477	-
1.3134	14500	0.0463	-
1.3179	14550	0.0448	-
1.3225	14600	0.045	-
1.3270	14650	0.0437	-
1.3315	14700	0.0467	-
1.3361	14750	0.0435	-
1.3406	14800	0.043	-
1.3451	14850	0.0486	-
1.3496	14900	0.049	-
1.3542	14950	0.0471	-
1.3587	15000	0.0455	-
1.3632	15050	0.0428	-
1.3678	15100	0.0425	-
1.3723	15150	0.0465	-
1.3768	15200	0.0452	-
1.3813	15250	0.0487	-
1.3859	15300	0.045	-
1.3904	15350	0.0431	-
1.3949	15400	0.0512	-
1.3995	15450	0.0411	-
1.4040	15500	0.0495	-
1.4085	15550	0.0406	-
1.4130	15600	0.0445	-
1.4176	15650	0.0496	-
1.4221	15700	0.0497	-
1.4266	15750	0.0466	-
1.4312	15800	0.0479	-
1.4357	15850	0.0448	-
1.4402	15900	0.0453	-
1.4447	15950	0.0486	-
1.4493	16000	0.0519	-
1.4538	16050	0.06	-
1.4583	16100	0.045	-
1.4629	16150	0.0421	-
1.4674	16200	0.0501	-
1.4719	16250	0.0525	-
1.4764	16300	0.0444	-
1.4810	16350	0.0422	-
1.4855	16400	0.0428	-
1.4900	16450	0.0398	-
1.4946	16500	0.0412	-
1.4991	16550	0.0482	-
1.5036	16600	0.046	-
1.5082	16650	0.0502	-
1.5127	16700	0.048	-
1.5172	16750	0.0447	-
1.5217	16800	0.0419	-
1.5263	16850	0.0429	-
1.5308	16900	0.0533	-
1.5353	16950	0.0482	-
1.5399	17000	0.0519	-
1.5444	17050	0.0503	-
1.5489	17100	0.0432	-
1.5534	17150	0.0388	-
1.5580	17200	0.0537	-
1.5625	17250	0.0477	-
1.5670	17300	0.0444	-
1.5716	17350	0.0407	-
1.5761	17400	0.0463	-
1.5806	17450	0.0417	-
1.5851	17500	0.0403	-
1.5897	17550	0.0481	-
1.5942	17600	0.0485	-
1.5987	17650	0.0462	-
1.6033	17700	0.0383	-
1.6078	17750	0.0429	-
1.6123	17800	0.0413	-
1.6168	17850	0.0421	-
1.6214	17900	0.0409	-
1.6259	17950	0.0436	-
1.6304	18000	0.0468	-
1.6350	18050	0.0446	-
1.6395	18100	0.0389	-
1.6440	18150	0.0443	-
1.6486	18200	0.0489	-
1.6531	18250	0.0489	-
1.6576	18300	0.0498	-
1.6621	18350	0.044	-
1.6667	18400	0.0392	-
1.6712	18450	0.0441	-
1.6757	18500	0.0429	-
1.6803	18550	0.0369	-
1.6848	18600	0.0409	-
1.6893	18650	0.0496	-
1.6938	18700	0.052	-
1.6984	18750	0.0377	-
1.7029	18800	0.0403	-
1.7074	18850	0.0473	-
1.7120	18900	0.0474	-
1.7165	18950	0.0447	-
1.7210	19000	0.0498	-
1.7255	19050	0.0427	-
1.7301	19100	0.0454	-
1.7346	19150	0.0478	-
1.7391	19200	0.0438	-
1.7437	19250	0.0378	-
1.7482	19300	0.0442	-
1.7527	19350	0.0453	-
1.7572	19400	0.0407	-
1.7618	19450	0.0487	-
1.7663	19500	0.0408	-
1.7708	19550	0.0444	-
1.7754	19600	0.0426	-
1.7799	19650	0.044	-
1.7844	19700	0.04	-
1.7889	19750	0.0484	-
1.7935	19800	0.0478	-
1.7980	19850	0.0495	-
1.8025	19900	0.037	-
1.8071	19950	0.0472	-
1.8116	20000	0.0411	-
1.8161	20050	0.0368	-
1.8207	20100	0.042	-
1.8252	20150	0.0359	-
1.8297	20200	0.0452	-
1.8342	20250	0.0504	-
1.8388	20300	0.0513	-
1.8433	20350	0.048	-
1.8478	20400	0.0497	-
1.8524	20450	0.0408	-
1.8569	20500	0.0448	-
1.8614	20550	0.0411	-
1.8659	20600	0.0401	-
1.8705	20650	0.0445	-
1.875	20700	0.0468	-
1.8795	20750	0.0488	-
1.8841	20800	0.0401	-
1.8886	20850	0.0408	-
1.8931	20900	0.0384	-
1.8976	20950	0.0496	-
1.9022	21000	0.0436	-
1.9067	21050	0.0434	-
1.9112	21100	0.041	-
1.9158	21150	0.0452	-
1.9203	21200	0.0454	-
1.9248	21250	0.0447	-
1.9293	21300	0.0476	-
1.9339	21350	0.0399	-
1.9384	21400	0.0397	-
1.9429	21450	0.0399	-
1.9475	21500	0.0462	-
1.9520	21550	0.0452	-
1.9565	21600	0.046	-
1.9611	21650	0.0368	-
1.9656	21700	0.043	-
1.9701	21750	0.0413	-
1.9746	21800	0.0459	-
1.9792	21850	0.0442	-
1.9837	21900	0.0381	-
1.9882	21950	0.0411	-
1.9928	22000	0.0519	-
1.9973	22050	0.0445	-

Framework Versions

Python: 3.12.12
SetFit: 1.1.3
Sentence Transformers: 5.1.2
Transformers: 4.57.1
PyTorch: 2.8.0+cu126
Datasets: 4.0.0
Tokenizers: 0.22.1

Citation

BibTeX

bibtex

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}