You can deploy this model and start earning money today!

faodl/
model_cca_multilabel_MiniLM-L12-50prop

Name: model_cca_multilabel_MiniLM-L12-50prop
Availability: InStock

setfit

safetensors

bert

sentence-transformers

text-classification

generated_from_setfit_trainer

2209.11055

sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

finetune

text-embeddings-inference

Views

No views yet

SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 as the Sentence Transformer embedding model. A OneVsRestClassifier instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Classification head: a OneVsRestClassifier instance
Maximum Sequence Length: 128 tokens

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

python

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("faodl/model_cca_multilabel_MiniLM-L12-50prop")
# Run inference
preds = model("School and workplace nutrition programs will promote healthier choices by removing sugar-rich products from regular offerings, expanding water access, and integrating nutrition education that addresses SSBs, portion sizes, and overall diet quality.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	78.4753	951

Training Hyperparameters

batch_size: (16, 16)
num_epochs: (2, 2)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 20
body_learning_rate: (2e-05, 2e-05)
head_learning_rate: 2e-05
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
l2_weight: 0.01
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0002	1	0.3075	-
0.0087	50	0.2066	-
0.0173	100	0.1932	-
0.0260	150	0.1878	-
0.0347	200	0.1824	-
0.0434	250	0.1682	-
0.0520	300	0.1566	-
0.0607	350	0.1487	-
0.0694	400	0.1542	-
0.0781	450	0.1553	-
0.0867	500	0.1513	-
0.0954	550	0.1329	-
0.1041	600	0.1551	-
0.1127	650	0.1428	-
0.1214	700	0.1414	-
0.1301	750	0.1152	-
0.1388	800	0.1283	-
0.1474	850	0.1305	-
0.1561	900	0.1303	-
0.1648	950	0.1257	-
0.1735	1000	0.1103	-
0.1821	1050	0.1183	-
0.1908	1100	0.1151	-
0.1995	1150	0.1129	-
0.2082	1200	0.1039	-
0.2168	1250	0.1126	-
0.2255	1300	0.1188	-
0.2342	1350	0.114	-
0.2428	1400	0.1094	-
0.2515	1450	0.1078	-
0.2602	1500	0.1018	-
0.2689	1550	0.1136	-
0.2775	1600	0.1004	-
0.2862	1650	0.1018	-
0.2949	1700	0.0929	-
0.3036	1750	0.0986	-
0.3122	1800	0.0951	-
0.3209	1850	0.0939	-
0.3296	1900	0.0898	-
0.3382	1950	0.095	-
0.3469	2000	0.0885	-
0.3556	2050	0.0941	-
0.3643	2100	0.1028	-
0.3729	2150	0.0945	-
0.3816	2200	0.0924	-
0.3903	2250	0.0846	-
0.3990	2300	0.0839	-
0.4076	2350	0.0927	-
0.4163	2400	0.0839	-
0.4250	2450	0.0799	-
0.4337	2500	0.0862	-
0.4423	2550	0.0872	-
0.4510	2600	0.0905	-
0.4597	2650	0.0857	-
0.4683	2700	0.0791	-
0.4770	2750	0.0829	-
0.4857	2800	0.0776	-
0.4944	2850	0.0775	-
0.5030	2900	0.088	-
0.5117	2950	0.0824	-
0.5204	3000	0.0871	-
0.5291	3050	0.0731	-
0.5377	3100	0.0799	-
0.5464	3150	0.0763	-
0.5551	3200	0.0725	-
0.5637	3250	0.0789	-
0.5724	3300	0.0893	-
0.5811	3350	0.0714	-
0.5898	3400	0.0802	-
0.5984	3450	0.0725	-
0.6071	3500	0.0756	-
0.6158	3550	0.0778	-
0.6245	3600	0.0735	-
0.6331	3650	0.0738	-
0.6418	3700	0.0733	-
0.6505	3750	0.0696	-
0.6592	3800	0.0732	-
0.6678	3850	0.0757	-
0.6765	3900	0.0652	-
0.6852	3950	0.0662	-
0.6938	4000	0.0796	-
0.7025	4050	0.0709	-
0.7112	4100	0.0678	-
0.7199	4150	0.0698	-
0.7285	4200	0.0636	-
0.7372	4250	0.0679	-
0.7459	4300	0.073	-
0.7546	4350	0.0685	-
0.7632	4400	0.074	-
0.7719	4450	0.0717	-
0.7806	4500	0.0615	-
0.7892	4550	0.0671	-
0.7979	4600	0.0655	-
0.8066	4650	0.0658	-
0.8153	4700	0.0585	-
0.8239	4750	0.0619	-
0.8326	4800	0.0615	-
0.8413	4850	0.0593	-
0.8500	4900	0.0596	-
0.8586	4950	0.063	-
0.8673	5000	0.0591	-
0.8760	5050	0.0685	-
0.8846	5100	0.0651	-
0.8933	5150	0.0623	-
0.9020	5200	0.0605	-
0.9107	5250	0.0618	-
0.9193	5300	0.0683	-
0.9280	5350	0.0631	-
0.9367	5400	0.0651	-
0.9454	5450	0.0578	-
0.9540	5500	0.0646	-
0.9627	5550	0.054	-
0.9714	5600	0.0638	-
0.9801	5650	0.0592	-
0.9887	5700	0.0632	-
0.9974	5750	0.0573	-
1.0061	5800	0.0568	-
1.0147	5850	0.0554	-
1.0234	5900	0.0519	-
1.0321	5950	0.0555	-
1.0408	6000	0.0487	-
1.0494	6050	0.0659	-
1.0581	6100	0.0463	-
1.0668	6150	0.0604	-
1.0755	6200	0.0553	-
1.0841	6250	0.0484	-
1.0928	6300	0.0475	-
1.1015	6350	0.0489	-
1.1101	6400	0.0544	-
1.1188	6450	0.051	-
1.1275	6500	0.05	-
1.1362	6550	0.0578	-
1.1448	6600	0.0518	-
1.1535	6650	0.0499	-
1.1622	6700	0.0512	-
1.1709	6750	0.054	-
1.1795	6800	0.0596	-
1.1882	6850	0.0445	-
1.1969	6900	0.0546	-
1.2056	6950	0.0605	-
1.2142	7000	0.0518	-
1.2229	7050	0.0535	-
1.2316	7100	0.0643	-
1.2402	7150	0.0509	-
1.2489	7200	0.0477	-
1.2576	7250	0.0421	-
1.2663	7300	0.0558	-
1.2749	7350	0.0431	-
1.2836	7400	0.0527	-
1.2923	7450	0.0512	-
1.3010	7500	0.049	-
1.3096	7550	0.0489	-
1.3183	7600	0.0515	-
1.3270	7650	0.0537	-
1.3356	7700	0.0556	-
1.3443	7750	0.0445	-
1.3530	7800	0.0509	-
1.3617	7850	0.0571	-
1.3703	7900	0.0582	-
1.3790	7950	0.0488	-
1.3877	8000	0.0482	-
1.3964	8050	0.0564	-
1.4050	8100	0.0487	-
1.4137	8150	0.0605	-
1.4224	8200	0.0539	-
1.4310	8250	0.0463	-
1.4397	8300	0.0468	-
1.4484	8350	0.0485	-
1.4571	8400	0.0569	-
1.4657	8450	0.0601	-
1.4744	8500	0.0545	-
1.4831	8550	0.0471	-
1.4918	8600	0.0472	-
1.5004	8650	0.0464	-
1.5091	8700	0.0511	-
1.5178	8750	0.0477	-
1.5265	8800	0.0464	-
1.5351	8850	0.0497	-
1.5438	8900	0.0493	-
1.5525	8950	0.0555	-
1.5611	9000	0.0523	-
1.5698	9050	0.0563	-
1.5785	9100	0.0473	-
1.5872	9150	0.0455	-
1.5958	9200	0.0469	-
1.6045	9250	0.0456	-
1.6132	9300	0.048	-
1.6219	9350	0.0498	-
1.6305	9400	0.0568	-
1.6392	9450	0.0501	-
1.6479	9500	0.0509	-
1.6565	9550	0.0482	-
1.6652	9600	0.0479	-
1.6739	9650	0.0442	-
1.6826	9700	0.0528	-
1.6912	9750	0.0453	-
1.6999	9800	0.041	-
1.7086	9850	0.0507	-
1.7173	9900	0.0495	-
1.7259	9950	0.0517	-
1.7346	10000	0.052	-
1.7433	10050	0.047	-
1.7520	10100	0.052	-
1.7606	10150	0.0565	-
1.7693	10200	0.0458	-
1.7780	10250	0.0409	-
1.7866	10300	0.0487	-
1.7953	10350	0.0516	-
1.8040	10400	0.049	-
1.8127	10450	0.0511	-
1.8213	10500	0.0498	-
1.8300	10550	0.0449	-
1.8387	10600	0.047	-
1.8474	10650	0.0463	-
1.8560	10700	0.0457	-
1.8647	10750	0.0495	-
1.8734	10800	0.0454	-
1.8820	10850	0.0486	-
1.8907	10900	0.049	-
1.8994	10950	0.0502	-
1.9081	11000	0.0454	-
1.9167	11050	0.0478	-
1.9254	11100	0.0509	-
1.9341	11150	0.0518	-
1.9428	11200	0.0445	-
1.9514	11250	0.043	-
1.9601	11300	0.0414	-
1.9688	11350	0.0452	-
1.9775	11400	0.0468	-
1.9861	11450	0.0426	-
1.9948	11500	0.0457	-

Framework Versions

Python: 3.12.12
SetFit: 1.1.3
Sentence Transformers: 5.1.1
Transformers: 4.57.1
PyTorch: 2.8.0+cu126
Datasets: 4.0.0
Tokenizers: 0.22.1

Citation

BibTeX

bibtex

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}