You can deploy this model and start earning money today!

faodl/
model_g20_multilabel_30sample

Name: model_g20_multilabel_30sample
Availability: InStock

setfit

safetensors

bert

sentence-transformers

text-classification

generated_from_setfit_trainer

2209.11055

sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

finetune

Views

No views yet

SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 as the Sentence Transformer embedding model. A OneVsRestClassifier instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Classification head: a OneVsRestClassifier instance
Maximum Sequence Length: 128 tokens

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

python

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("faodl/model_g20_multilabel_30sample")
# Run inference
preds = model("Housing and Community Amenities 
 

133.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	41.0925	506

Training Hyperparameters

batch_size: (8, 8)
num_epochs: (4, 4)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 20
body_learning_rate: (2e-05, 2e-05)
head_learning_rate: 2e-05
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
l2_weight: 0.01
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0001	1	0.2661	-
0.0068	50	0.1923	-
0.0136	100	0.1856	-
0.0204	150	0.1927	-
0.0272	200	0.1708	-
0.0340	250	0.1706	-
0.0408	300	0.156	-
0.0476	350	0.1597	-
0.0544	400	0.149	-
0.0612	450	0.1488	-
0.0680	500	0.1375	-
0.0748	550	0.1234	-
0.0816	600	0.1339	-
0.0884	650	0.126	-
0.0952	700	0.1347	-
0.1020	750	0.1323	-
0.1088	800	0.1159	-
0.1156	850	0.1236	-
0.1224	900	0.1218	-
0.1293	950	0.1323	-
0.1361	1000	0.1258	-
0.1429	1050	0.1206	-
0.1497	1100	0.1127	-
0.1565	1150	0.1211	-
0.1633	1200	0.1234	-
0.1701	1250	0.1178	-
0.1769	1300	0.1009	-
0.1837	1350	0.11	-
0.1905	1400	0.1103	-
0.1973	1450	0.1015	-
0.2041	1500	0.0926	-
0.2109	1550	0.099	-
0.2177	1600	0.1079	-
0.2245	1650	0.0979	-
0.2313	1700	0.1001	-
0.2381	1750	0.1039	-
0.2449	1800	0.0838	-
0.2517	1850	0.0941	-
0.2585	1900	0.0929	-
0.2653	1950	0.0851	-
0.2721	2000	0.0956	-
0.2789	2050	0.075	-
0.2857	2100	0.1067	-
0.2925	2150	0.0891	-
0.2993	2200	0.0939	-
0.3061	2250	0.0908	-
0.3129	2300	0.0847	-
0.3197	2350	0.0812	-
0.3265	2400	0.0918	-
0.3333	2450	0.0935	-
0.3401	2500	0.0792	-
0.3469	2550	0.0669	-
0.3537	2600	0.0883	-
0.3605	2650	0.0829	-
0.3673	2700	0.0656	-
0.3741	2750	0.0752	-
0.3810	2800	0.0825	-
0.3878	2850	0.0813	-
0.3946	2900	0.0852	-
0.4014	2950	0.0903	-
0.4082	3000	0.0902	-
0.4150	3050	0.0739	-
0.4218	3100	0.0786	-
0.4286	3150	0.083	-
0.4354	3200	0.0648	-
0.4422	3250	0.0704	-
0.4490	3300	0.0798	-
0.4558	3350	0.0651	-
0.4626	3400	0.0705	-
0.4694	3450	0.0653	-
0.4762	3500	0.0767	-
0.4830	3550	0.0747	-
0.4898	3600	0.0738	-
0.4966	3650	0.055	-
0.5034	3700	0.0741	-
0.5102	3750	0.0688	-
0.5170	3800	0.0699	-
0.5238	3850	0.0787	-
0.5306	3900	0.0673	-
0.5374	3950	0.0629	-
0.5442	4000	0.0639	-
0.5510	4050	0.0809	-
0.5578	4100	0.0694	-
0.5646	4150	0.0696	-
0.5714	4200	0.0577	-
0.5782	4250	0.0707	-
0.5850	4300	0.0542	-
0.5918	4350	0.0541	-
0.5986	4400	0.0462	-
0.6054	4450	0.0675	-
0.6122	4500	0.0561	-
0.6190	4550	0.056	-
0.6259	4600	0.0556	-
0.6327	4650	0.0552	-
0.6395	4700	0.0566	-
0.6463	4750	0.0578	-
0.6531	4800	0.0488	-
0.6599	4850	0.0419	-
0.6667	4900	0.0485	-
0.6735	4950	0.0477	-
0.6803	5000	0.0566	-
0.6871	5050	0.0571	-
0.6939	5100	0.0531	-
0.7007	5150	0.0563	-
0.7075	5200	0.0452	-
0.7143	5250	0.0459	-
0.7211	5300	0.039	-
0.7279	5350	0.0382	-
0.7347	5400	0.0679	-
0.7415	5450	0.0465	-
0.7483	5500	0.0493	-
0.7551	5550	0.0489	-
0.7619	5600	0.0443	-
0.7687	5650	0.0591	-
0.7755	5700	0.0441	-
0.7823	5750	0.0501	-
0.7891	5800	0.0497	-
0.7959	5850	0.0543	-
0.8027	5900	0.05	-
0.8095	5950	0.0449	-
0.8163	6000	0.0432	-
0.8231	6050	0.0491	-
0.8299	6100	0.0507	-
0.8367	6150	0.0405	-
0.8435	6200	0.0426	-
0.8503	6250	0.0528	-
0.8571	6300	0.0428	-
0.8639	6350	0.0534	-
0.8707	6400	0.0512	-
0.8776	6450	0.049	-
0.8844	6500	0.0386	-
0.8912	6550	0.0468	-
0.8980	6600	0.0505	-
0.9048	6650	0.0538	-
0.9116	6700	0.0484	-
0.9184	6750	0.044	-
0.9252	6800	0.0431	-
0.9320	6850	0.0456	-
0.9388	6900	0.0342	-
0.9456	6950	0.0445	-
0.9524	7000	0.0499	-
0.9592	7050	0.0589	-
0.9660	7100	0.0409	-
0.9728	7150	0.04	-
0.9796	7200	0.0443	-
0.9864	7250	0.0373	-
0.9932	7300	0.0306	-
1.0	7350	0.0303	-
1.0068	7400	0.0317	-
1.0136	7450	0.0364	-
1.0204	7500	0.0349	-
1.0272	7550	0.0388	-
1.0340	7600	0.0466	-
1.0408	7650	0.0334	-
1.0476	7700	0.0512	-
1.0544	7750	0.0413	-
1.0612	7800	0.0399	-
1.0680	7850	0.0412	-
1.0748	7900	0.0341	-
1.0816	7950	0.0395	-
1.0884	8000	0.045	-
1.0952	8050	0.0385	-
1.1020	8100	0.038	-
1.1088	8150	0.0376	-
1.1156	8200	0.0434	-
1.1224	8250	0.0323	-
1.1293	8300	0.0364	-
1.1361	8350	0.033	-
1.1429	8400	0.025	-
1.1497	8450	0.0461	-
1.1565	8500	0.033	-
1.1633	8550	0.0317	-
1.1701	8600	0.047	-
1.1769	8650	0.0344	-
1.1837	8700	0.0388	-
1.1905	8750	0.0359	-
1.1973	8800	0.0429	-
1.2041	8850	0.0355	-
1.2109	8900	0.0421	-
1.2177	8950	0.0351	-
1.2245	9000	0.0359	-
1.2313	9050	0.035	-
1.2381	9100	0.0331	-
1.2449	9150	0.0337	-
1.2517	9200	0.0376	-
1.2585	9250	0.0366	-
1.2653	9300	0.0369	-
1.2721	9350	0.0353	-
1.2789	9400	0.0439	-
1.2857	9450	0.0439	-
1.2925	9500	0.0288	-
1.2993	9550	0.0404	-
1.3061	9600	0.0355	-
1.3129	9650	0.0375	-
1.3197	9700	0.0452	-
1.3265	9750	0.0408	-
1.3333	9800	0.0369	-
1.3401	9850	0.0337	-
1.3469	9900	0.0294	-
1.3537	9950	0.0341	-
1.3605	10000	0.0356	-
1.3673	10050	0.0394	-
1.3741	10100	0.0387	-
1.3810	10150	0.0276	-
1.3878	10200	0.0345	-
1.3946	10250	0.037	-
1.4014	10300	0.0272	-
1.4082	10350	0.0341	-
1.4150	10400	0.033	-
1.4218	10450	0.0517	-
1.4286	10500	0.0297	-
1.4354	10550	0.0388	-
1.4422	10600	0.0312	-
1.4490	10650	0.0283	-
1.4558	10700	0.0287	-
1.4626	10750	0.0319	-
1.4694	10800	0.0343	-
1.4762	10850	0.033	-
1.4830	10900	0.0444	-
1.4898	10950	0.0239	-
1.4966	11000	0.0294	-
1.5034	11050	0.0313	-
1.5102	11100	0.0344	-
1.5170	11150	0.0304	-
1.5238	11200	0.0339	-
1.5306	11250	0.0342	-
1.5374	11300	0.0291	-
1.5442	11350	0.0301	-
1.5510	11400	0.0309	-
1.5578	11450	0.0346	-
1.5646	11500	0.0406	-
1.5714	11550	0.034	-
1.5782	11600	0.0273	-
1.5850	11650	0.0316	-
1.5918	11700	0.0404	-
1.5986	11750	0.0295	-
1.6054	11800	0.0385	-
1.6122	11850	0.0373	-
1.6190	11900	0.0384	-
1.6259	11950	0.0307	-
1.6327	12000	0.0222	-
1.6395	12050	0.0257	-
1.6463	12100	0.0313	-
1.6531	12150	0.0293	-
1.6599	12200	0.0312	-
1.6667	12250	0.0299	-
1.6735	12300	0.0284	-
1.6803	12350	0.042	-
1.6871	12400	0.031	-
1.6939	12450	0.0295	-
1.7007	12500	0.0339	-
1.7075	12550	0.0385	-
1.7143	12600	0.0355	-
1.7211	12650	0.0291	-
1.7279	12700	0.0366	-
1.7347	12750	0.0337	-
1.7415	12800	0.0268	-
1.7483	12850	0.0373	-
1.7551	12900	0.0404	-
1.7619	12950	0.025	-
1.7687	13000	0.0282	-
1.7755	13050	0.0282	-
1.7823	13100	0.0341	-
1.7891	13150	0.0338	-
1.7959	13200	0.0342	-
1.8027	13250	0.035	-
1.8095	13300	0.0399	-
1.8163	13350	0.035	-
1.8231	13400	0.0367	-
1.8299	13450	0.0294	-
1.8367	13500	0.0382	-
1.8435	13550	0.0261	-
1.8503	13600	0.0301	-
1.8571	13650	0.0258	-
1.8639	13700	0.0301	-
1.8707	13750	0.0306	-
1.8776	13800	0.0242	-
1.8844	13850	0.0258	-
1.8912	13900	0.0296	-
1.8980	13950	0.0338	-
1.9048	14000	0.0315	-
1.9116	14050	0.0282	-
1.9184	14100	0.0325	-
1.9252	14150	0.0286	-
1.9320	14200	0.0355	-
1.9388	14250	0.0317	-
1.9456	14300	0.0314	-
1.9524	14350	0.031	-
1.9592	14400	0.03	-
1.9660	14450	0.0262	-
1.9728	14500	0.0275	-
1.9796	14550	0.0356	-
1.9864	14600	0.0369	-
1.9932	14650	0.0364	-
2.0	14700	0.0344	-
2.0068	14750	0.0248	-
2.0136	14800	0.0273	-
2.0204	14850	0.0282	-
2.0272	14900	0.023	-
2.0340	14950	0.0278	-
2.0408	15000	0.0355	-
2.0476	15050	0.0258	-
2.0544	15100	0.0258	-
2.0612	15150	0.0322	-
2.0680	15200	0.0266	-
2.0748	15250	0.0279	-
2.0816	15300	0.0282	-
2.0884	15350	0.0289	-
2.0952	15400	0.024	-
2.1020	15450	0.0268	-
2.1088	15500	0.0348	-
2.1156	15550	0.0281	-
2.1224	15600	0.0282	-
2.1293	15650	0.0218	-
2.1361	15700	0.0201	-
2.1429	15750	0.0207	-
2.1497	15800	0.0308	-
2.1565	15850	0.0261	-
2.1633	15900	0.0292	-
2.1701	15950	0.0308	-
2.1769	16000	0.0298	-
2.1837	16050	0.0308	-
2.1905	16100	0.0359	-
2.1973	16150	0.0265	-
2.2041	16200	0.0351	-
2.2109	16250	0.0223	-
2.2177	16300	0.0322	-
2.2245	16350	0.0261	-
2.2313	16400	0.0206	-
2.2381	16450	0.0384	-
2.2449	16500	0.0381	-
2.2517	16550	0.0238	-
2.2585	16600	0.0261	-
2.2653	16650	0.0323	-
2.2721	16700	0.0296	-
2.2789	16750	0.0256	-
2.2857	16800	0.0287	-
2.2925	16850	0.0272	-
2.2993	16900	0.0285	-
2.3061	16950	0.0245	-
2.3129	17000	0.0299	-
2.3197	17050	0.0193	-
2.3265	17100	0.0234	-
2.3333	17150	0.0308	-
2.3401	17200	0.0239	-
2.3469	17250	0.0309	-
2.3537	17300	0.0331	-
2.3605	17350	0.0316	-
2.3673	17400	0.0292	-
2.3741	17450	0.0337	-
2.3810	17500	0.0338	-
2.3878	17550	0.0288	-
2.3946	17600	0.031	-
2.4014	17650	0.0251	-
2.4082	17700	0.0288	-
2.4150	17750	0.0249	-
2.4218	17800	0.0281	-
2.4286	17850	0.0284	-
2.4354	17900	0.0268	-
2.4422	17950	0.0303	-
2.4490	18000	0.0233	-
2.4558	18050	0.0297	-
2.4626	18100	0.0265	-
2.4694	18150	0.0306	-
2.4762	18200	0.0286	-
2.4830	18250	0.0278	-
2.4898	18300	0.0254	-
2.4966	18350	0.0278	-
2.5034	18400	0.0257	-
2.5102	18450	0.0272	-
2.5170	18500	0.0297	-
2.5238	18550	0.0262	-
2.5306	18600	0.0309	-
2.5374	18650	0.0259	-
2.5442	18700	0.0212	-
2.5510	18750	0.026	-
2.5578	18800	0.0252	-
2.5646	18850	0.0228	-
2.5714	18900	0.0304	-
2.5782	18950	0.0278	-
2.5850	19000	0.0263	-
2.5918	19050	0.0305	-
2.5986	19100	0.0315	-
2.6054	19150	0.0288	-
2.6122	19200	0.0221	-
2.6190	19250	0.022	-
2.6259	19300	0.0299	-
2.6327	19350	0.0302	-
2.6395	19400	0.0282	-
2.6463	19450	0.0308	-
2.6531	19500	0.0306	-
2.6599	19550	0.0327	-
2.6667	19600	0.0284	-
2.6735	19650	0.0185	-
2.6803	19700	0.0248	-
2.6871	19750	0.0212	-
2.6939	19800	0.0254	-
2.7007	19850	0.0276	-
2.7075	19900	0.027	-
2.7143	19950	0.0261	-
2.7211	20000	0.0307	-
2.7279	20050	0.0225	-
2.7347	20100	0.0189	-
2.7415	20150	0.0325	-
2.7483	20200	0.0304	-
2.7551	20250	0.0351	-
2.7619	20300	0.0274	-
2.7687	20350	0.0318	-
2.7755	20400	0.0266	-
2.7823	20450	0.0211	-
2.7891	20500	0.0388	-
2.7959	20550	0.0245	-
2.8027	20600	0.0307	-
2.8095	20650	0.0346	-
2.8163	20700	0.0251	-
2.8231	20750	0.0289	-
2.8299	20800	0.0338	-
2.8367	20850	0.0228	-
2.8435	20900	0.0248	-
2.8503	20950	0.0176	-
2.8571	21000	0.0277	-
2.8639	21050	0.0312	-
2.8707	21100	0.0271	-
2.8776	21150	0.0251	-
2.8844	21200	0.0253	-
2.8912	21250	0.0304	-
2.8980	21300	0.0321	-
2.9048	21350	0.0223	-
2.9116	21400	0.0269	-
2.9184	21450	0.0326	-
2.9252	21500	0.0226	-
2.9320	21550	0.0347	-
2.9388	21600	0.0223	-
2.9456	21650	0.0256	-
2.9524	21700	0.0256	-
2.9592	21750	0.0322	-
2.9660	21800	0.0281	-
2.9728	21850	0.0318	-
2.9796	21900	0.0279	-
2.9864	21950	0.0303	-
2.9932	22000	0.0349	-
3.0	22050	0.0254	-
3.0068	22100	0.0185	-
3.0136	22150	0.0241	-
3.0204	22200	0.0285	-
3.0272	22250	0.0257	-
3.0340	22300	0.0247	-
3.0408	22350	0.023	-
3.0476	22400	0.0335	-
3.0544	22450	0.0302	-
3.0612	22500	0.0249	-
3.0680	22550	0.029	-
3.0748	22600	0.0312	-
3.0816	22650	0.0303	-
3.0884	22700	0.0225	-
3.0952	22750	0.0271	-
3.1020	22800	0.0275	-
3.1088	22850	0.0264	-
3.1156	22900	0.0202	-
3.1224	22950	0.0247	-
3.1293	23000	0.0292	-
3.1361	23050	0.0235	-
3.1429	23100	0.019	-
3.1497	23150	0.0247	-
3.1565	23200	0.0219	-
3.1633	23250	0.0217	-
3.1701	23300	0.0236	-
3.1769	23350	0.0223	-
3.1837	23400	0.0237	-
3.1905	23450	0.0307	-
3.1973	23500	0.0275	-
3.2041	23550	0.0192	-
3.2109	23600	0.0198	-
3.2177	23650	0.0322	-
3.2245	23700	0.0195	-
3.2313	23750	0.019	-
3.2381	23800	0.0266	-
3.2449	23850	0.0287	-
3.2517	23900	0.0205	-
3.2585	23950	0.025	-
3.2653	24000	0.0282	-
3.2721	24050	0.0261	-
3.2789	24100	0.0275	-
3.2857	24150	0.0273	-
3.2925	24200	0.0195	-
3.2993	24250	0.0265	-
3.3061	24300	0.0276	-
3.3129	24350	0.0277	-
3.3197	24400	0.0224	-
3.3265	24450	0.0231	-
3.3333	24500	0.0275	-
3.3401	24550	0.0333	-
3.3469	24600	0.0181	-
3.3537	24650	0.0266	-
3.3605	24700	0.0268	-
3.3673	24750	0.0177	-
3.3741	24800	0.0185	-
3.3810	24850	0.023	-
3.3878	24900	0.0281	-
3.3946	24950	0.0202	-
3.4014	25000	0.0206	-
3.4082	25050	0.0224	-
3.4150	25100	0.0275	-
3.4218	25150	0.0272	-
3.4286	25200	0.0221	-
3.4354	25250	0.0259	-
3.4422	25300	0.0244	-
3.4490	25350	0.034	-
3.4558	25400	0.0258	-
3.4626	25450	0.0271	-
3.4694	25500	0.0291	-
3.4762	25550	0.0204	-
3.4830	25600	0.0248	-
3.4898	25650	0.0225	-
3.4966	25700	0.0347	-
3.5034	25750	0.0243	-
3.5102	25800	0.031	-
3.5170	25850	0.024	-
3.5238	25900	0.0199	-
3.5306	25950	0.0278	-
3.5374	26000	0.0318	-
3.5442	26050	0.0267	-
3.5510	26100	0.027	-
3.5578	26150	0.0191	-
3.5646	26200	0.0233	-
3.5714	26250	0.0239	-
3.5782	26300	0.0203	-
3.5850	26350	0.0243	-
3.5918	26400	0.0246	-
3.5986	26450	0.0233	-
3.6054	26500	0.0364	-
3.6122	26550	0.0273	-
3.6190	26600	0.0269	-
3.6259	26650	0.0206	-
3.6327	26700	0.0316	-
3.6395	26750	0.023	-
3.6463	26800	0.0257	-
3.6531	26850	0.0263	-
3.6599	26900	0.0218	-
3.6667	26950	0.0257	-
3.6735	27000	0.0228	-
3.6803	27050	0.0256	-
3.6871	27100	0.0239	-
3.6939	27150	0.0225	-
3.7007	27200	0.0294	-
3.7075	27250	0.0187	-
3.7143	27300	0.02	-
3.7211	27350	0.0261	-
3.7279	27400	0.0201	-
3.7347	27450	0.0253	-
3.7415	27500	0.0265	-
3.7483	27550	0.0303	-
3.7551	27600	0.0239	-
3.7619	27650	0.0246	-
3.7687	27700	0.0249	-
3.7755	27750	0.023	-
3.7823	27800	0.0237	-
3.7891	27850	0.0197	-
3.7959	27900	0.0268	-
3.8027	27950	0.0246	-
3.8095	28000	0.029	-
3.8163	28050	0.0248	-
3.8231	28100	0.0275	-
3.8299	28150	0.0241	-
3.8367	28200	0.027	-
3.8435	28250	0.0252	-
3.8503	28300	0.0245	-
3.8571	28350	0.0241	-
3.8639	28400	0.0264	-
3.8707	28450	0.0233	-
3.8776	28500	0.0319	-
3.8844	28550	0.0236	-
3.8912	28600	0.0277	-
3.8980	28650	0.0178	-
3.9048	28700	0.0209	-
3.9116	28750	0.0263	-
3.9184	28800	0.0236	-
3.9252	28850	0.0216	-
3.9320	28900	0.0209	-
3.9388	28950	0.0283	-
3.9456	29000	0.0307	-
3.9524	29050	0.0276	-
3.9592	29100	0.0277	-
3.9660	29150	0.031	-
3.9728	29200	0.0304	-
3.9796	29250	0.0332	-
3.9864	29300	0.0277	-
3.9932	29350	0.0233	-
4.0	29400	0.0237	-

Framework Versions

Python: 3.11.13
SetFit: 1.1.2
Sentence Transformers: 4.1.0
Transformers: 4.52.4
PyTorch: 2.6.0+cu124
Datasets: 3.6.0
Tokenizers: 0.21.1

Citation

BibTeX

bibtex

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}