This is a sentence-transformers model finetuned from hyrinmansoor/changAI-nomic-embed-text-v1.5-finetuned. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: hyrinmansoor/changAI-nomic-embed-text-v1.5-finetuned
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Supported Modality: Text

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'ModernBertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'revenue from warenty claim items vs non-claimed lst yr',
    '[FIELD] item_code | [TABLE] tabWarranty Claim | desc: links to the Item master; identifies the product subject to a warranty claim',
    'Support_027',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.6816, -0.0659],
#         [ 0.6816,  1.0000,  0.0028],
#         [-0.0659,  0.0028,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 262,905 training samples
Columns: anchor, positive, qid, and module

Approximate statistics based on the first 1000 samples:

	anchor	positive	qid	module
type	string	string	string	string
details	min: 7 tokens mean: 16.09 tokens max: 45 tokens	min: 6 tokens mean: 28.94 tokens max: 71 tokens	min: 5 tokens mean: 6.44 tokens max: 8 tokens	min: 3 tokens mean: 3.97 tokens max: 5 tokens

Samples:

anchor	positive	qid	module
`how many days has the work order WO-999 been open?`	`[FIELD] name \| [TABLE] tabWork Order \| desc: Unique identifier for the production order.`	`Manufacturing_931`	`Manufacturing`
`Whats the total amount invoiced for product XYZ?`	`[FIELD] item_code \| [TABLE] tabInvoice Item \| desc: Code of the item on the invoice line.`	`Selling_2982`	`Selling`
`Which quality meetings are missing their corrective actions?`	`[FIELD] quality_meeting \| [TABLE] tabQuality Action \| desc: Links a quality action item back to the specific quality meeting where it was generated; if NULL for a meeting, indicates no actions were planned for it.`	`Quality Management_233`	`Quality Management`

Loss: CachedMultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "mini_batch_size": 4, 5 "gather_across_devices": false, 6 "directions": [ 7 "query_to_doc" 8 ], 9 "partition_mode": "joint", 10 "hardness_mode": null, 11 "hardness_strength": 0.0 12}

Evaluation Dataset

Unnamed Dataset

Size: 13,837 evaluation samples
Columns: anchor, positive, qid, and module

Approximate statistics based on the first 1000 samples:

	anchor	positive	qid	module
type	string	string	string	string
details	min: 7 tokens mean: 16.25 tokens max: 58 tokens	min: 6 tokens mean: 29.28 tokens max: 300 tokens	min: 5 tokens mean: 6.39 tokens max: 8 tokens	min: 3 tokens mean: 3.97 tokens max: 5 tokens

Samples:

anchor	positive	qid	module
`Opportunities automatically closed based on CRM setting?`	`[FIELD] transaction_date \| [TABLE] tabOpportunity \| desc: the date the opportunity was recorded or transacted; compare with expected closing date and CRM setting`	`CRM_791`	`CRM`
`purchase amount last year vs year before`	`[FIELD] purchase_date \| [TABLE] tabAsset \| desc: the date the asset was acquired; used to filter assets by acquisition year`	`Assets_123`	`Assets`
`Count how many tickets were resolved by Sarah.`	`[TABLE] tabUser \| desc: Master data for users.`	`Support_273`	`Support`

Loss: CachedMultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "mini_batch_size": 4, 5 "gather_across_devices": false, 6 "directions": [ 7 "query_to_doc" 8 ], 9 "partition_mode": "joint", 10 "hardness_mode": null, 11 "hardness_strength": 0.0 12}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 12
per_device_eval_batch_size: 12
learning_rate: 2e-05
num_train_epochs: 1
warmup_steps: 0.1
fp16: True
load_best_model_at_end: True

All Hyperparameters

Click to expand

do_predict: False
prediction_loss_only: True
per_device_train_batch_size: 12
per_device_eval_batch_size: 12
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 1
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: None
warmup_steps: 0.1
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
enable_jit_checkpoint: False
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
use_cpu: False
seed: 42
data_seed: None
bf16: False
fp16: True
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: -1
ddp_backend: None
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_for_metrics: []
eval_do_concat_batches: True
auto_find_batch_size: False
full_determinism: False
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
use_cache: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss
0.0023	50	0.2635	-
0.0046	100	0.2359	-
0.0068	150	0.1566	-
0.0091	200	0.1481	-
0.0114	250	0.1677	-
0.0137	300	0.1388	-
0.0160	350	0.1351	-
0.0183	400	0.1027	-
0.0205	450	0.1594	-
0.0228	500	0.0934	0.2806
0.0251	550	0.0720	-
0.0274	600	0.0712	-
0.0297	650	0.1208	-
0.0320	700	0.0668	-
0.0342	750	0.1037	-
0.0365	800	0.0771	-
0.0388	850	0.1067	-
0.0411	900	0.0887	-
0.0434	950	0.1118	-
0.0456	1000	0.0506	0.2427
0.0479	1050	0.0407	-
0.0502	1100	0.0567	-
0.0525	1150	0.0681	-
0.0548	1200	0.0734	-
0.0571	1250	0.0486	-
0.0593	1300	0.0546	-
0.0616	1350	0.0621	-
0.0639	1400	0.0941	-
0.0662	1450	0.0451	-
0.0685	1500	0.0489	0.2237
0.0707	1550	0.0674	-
0.0730	1600	0.0724	-
0.0753	1650	0.0913	-
0.0776	1700	0.0769	-
0.0799	1750	0.0512	-
0.0822	1800	0.1051	-
0.0844	1850	0.0338	-
0.0867	1900	0.0597	-
0.0890	1950	0.0326	-
0.0913	2000	0.0738	0.2170
0.0936	2050	0.0576	-
0.0959	2100	0.0414	-
0.0981	2150	0.0566	-
0.1004	2200	0.0855	-
0.1027	2250	0.0969	-
0.1050	2300	0.0458	-
0.1073	2350	0.0531	-
0.1095	2400	0.0353	-
0.1118	2450	0.0491	-
0.1141	2500	0.0620	0.2198
0.1164	2550	0.0341	-
0.1187	2600	0.0648	-
0.1210	2650	0.0395	-
0.1232	2700	0.0603	-
0.1255	2750	0.0572	-
0.1278	2800	0.0707	-
0.1301	2850	0.0810	-
0.1324	2900	0.0782	-
0.1346	2950	0.0380	-
0.1369	3000	0.0437	0.2119
0.1392	3050	0.0376	-
0.1415	3100	0.0321	-
0.1438	3150	0.0537	-
0.1461	3200	0.0701	-
0.1483	3250	0.0453	-
0.1506	3300	0.0519	-
0.1529	3350	0.0378	-
0.1552	3400	0.0454	-
0.1575	3450	0.0405	-
0.1598	3500	0.0432	0.2307
0.1620	3550	0.0756	-
0.1643	3600	0.0585	-
0.1666	3650	0.0676	-
0.1689	3700	0.0446	-
0.1712	3750	0.0409	-
0.1734	3800	0.0500	-
0.1757	3850	0.0641	-
0.1780	3900	0.0717	-
0.1803	3950	0.0406	-
0.1826	4000	0.0250	0.2051
0.1849	4050	0.0318	-
0.1871	4100	0.0401	-
0.1894	4150	0.0374	-
0.1917	4200	0.0651	-
0.1940	4250	0.0474	-
0.1963	4300	0.0756	-
0.1985	4350	0.0318	-
0.2008	4400	0.0407	-
0.2031	4450	0.0338	-
0.2054	4500	0.0409	0.2004
0.2077	4550	0.0639	-
0.2100	4600	0.0599	-
0.2122	4650	0.0457	-
0.2145	4700	0.0390	-
0.2168	4750	0.0380	-
0.2191	4800	0.0402	-
0.2214	4850	0.0295	-
0.2237	4900	0.0617	-
0.2259	4950	0.0231	-
0.2282	5000	0.0525	0.2119
0.2305	5050	0.0403	-
0.2328	5100	0.0489	-
0.2351	5150	0.0467	-
0.2373	5200	0.0318	-
0.2396	5250	0.0514	-
0.2419	5300	0.0478	-
0.2442	5350	0.0251	-
0.2465	5400	0.0505	-
0.2488	5450	0.0525	-
0.2510	5500	0.0372	0.2049
0.2533	5550	0.0379	-
0.2556	5600	0.0396	-
0.2579	5650	0.0240	-
0.2602	5700	0.0458	-
0.2624	5750	0.0305	-
0.2647	5800	0.0469	-
0.2670	5850	0.0521	-
0.2693	5900	0.0351	-
0.2716	5950	0.0349	-
0.2739	6000	0.0385	0.2081
0.2761	6050	0.0521	-
0.2784	6100	0.0294	-
0.2807	6150	0.0382	-
0.2830	6200	0.0626	-
0.2853	6250	0.0269	-
0.2876	6300	0.0355	-
0.2898	6350	0.0400	-
0.2921	6400	0.0405	-
0.2944	6450	0.0269	-
0.2967	6500	0.0218	0.2027
0.2990	6550	0.0300	-
0.3012	6600	0.0396	-
0.3035	6650	0.0591	-
0.3058	6700	0.0336	-
0.3081	6750	0.0467	-
0.3104	6800	0.0460	-
0.3127	6850	0.0258	-
0.3149	6900	0.0354	-
0.3172	6950	0.0559	-
0.3195	7000	0.0367	0.1875
0.3218	7050	0.0246	-
0.3241	7100	0.0382	-
0.3263	7150	0.0361	-
0.3286	7200	0.0257	-
0.3309	7250	0.0448	-
0.3332	7300	0.0374	-
0.3355	7350	0.0440	-
0.3378	7400	0.0461	-
0.3400	7450	0.0532	-
0.3423	7500	0.0664	0.1859
0.3446	7550	0.0256	-
0.3469	7600	0.0413	-
0.3492	7650	0.0255	-
0.3515	7700	0.0314	-
0.3537	7750	0.0332	-
0.3560	7800	0.0236	-
0.3583	7850	0.0590	-
0.3606	7900	0.0262	-
0.3629	7950	0.0319	-
0.3651	8000	0.0542	0.1967
0.3674	8050	0.0362	-
0.3697	8100	0.0230	-
0.3720	8150	0.0333	-
0.3743	8200	0.0408	-
0.3766	8250	0.0295	-
0.3788	8300	0.0200	-
0.3811	8350	0.0192	-
0.3834	8400	0.0316	-
0.3857	8450	0.0230	-
0.3880	8500	0.0356	0.2008
0.3903	8550	0.0447	-
0.3925	8600	0.0255	-
0.3948	8650	0.0257	-
0.3971	8700	0.0546	-
0.3994	8750	0.0564	-
0.4017	8800	0.0309	-
0.4039	8850	0.0708	-
0.4062	8900	0.0427	-
0.4085	8950	0.0242	-
0.4108	9000	0.0330	0.1990
0.4131	9050	0.0416	-
0.4154	9100	0.0420	-
0.4176	9150	0.0336	-
0.4199	9200	0.0511	-
0.4222	9250	0.0152	-
0.4245	9300	0.0284	-
0.4268	9350	0.0374	-
0.4290	9400	0.0421	-
0.4313	9450	0.0478	-
0.4336	9500	0.0399	0.1907
0.4359	9550	0.0442	-
0.4382	9600	0.0227	-
0.4405	9650	0.0329	-
0.4427	9700	0.0220	-
0.4450	9750	0.0549	-
0.4473	9800	0.0465	-
0.4496	9850	0.0482	-
0.4519	9900	0.0409	-
0.4542	9950	0.0288	-
0.4564	10000	0.0333	0.1824
0.4587	10050	0.0555	-
0.4610	10100	0.0457	-
0.4633	10150	0.0305	-
0.4656	10200	0.0240	-
0.4678	10250	0.0278	-
0.4701	10300	0.0222	-
0.4724	10350	0.0255	-
0.4747	10400	0.0293	-
0.4770	10450	0.0462	-
0.4793	10500	0.0586	0.1803
0.4815	10550	0.0424	-
0.4838	10600	0.0210	-
0.4861	10650	0.0333	-
0.4884	10700	0.0290	-
0.4907	10750	0.0467	-
0.4929	10800	0.0155	-
0.4952	10850	0.0293	-
0.4975	10900	0.0325	-
0.4998	10950	0.0452	-
0.5021	11000	0.0445	0.1841
0.5044	11050	0.0324	-
0.5066	11100	0.0343	-
0.5089	11150	0.0354	-
0.5112	11200	0.0182	-
0.5135	11250	0.0154	-
0.5158	11300	0.0266	-
0.5181	11350	0.0241	-
0.5203	11400	0.0469	-
0.5226	11450	0.0337	-
0.5249	11500	0.0229	0.1860
0.5272	11550	0.0427	-
0.5295	11600	0.0199	-
0.5317	11650	0.0561	-
0.5340	11700	0.0287	-
0.5363	11750	0.0175	-
0.5386	11800	0.0442	-
0.5409	11850	0.0332	-
0.5432	11900	0.0147	-
0.5454	11950	0.0372	-
0.5477	12000	0.0319	0.1844
0.5500	12050	0.0457	-
0.5523	12100	0.0235	-
0.5546	12150	0.0438	-
0.5568	12200	0.0282	-
0.5591	12250	0.0392	-
0.5614	12300	0.0226	-
0.5637	12350	0.0349	-
0.5660	12400	0.0208	-
0.5683	12450	0.0234	-
0.5705	12500	0.0152	0.1848
0.5728	12550	0.0280	-
0.5751	12600	0.0242	-
0.5774	12650	0.0301	-
0.5797	12700	0.0343	-
0.5820	12750	0.0291	-
0.5842	12800	0.0296	-
0.5865	12850	0.0300	-
0.5888	12900	0.0294	-
0.5911	12950	0.0324	-
0.5934	13000	0.0259	0.1762
0.5956	13050	0.0126	-
0.5979	13100	0.0416	-
0.6002	13150	0.0136	-
0.6025	13200	0.0396	-
0.6048	13250	0.0220	-
0.6071	13300	0.0243	-
0.6093	13350	0.0262	-
0.6116	13400	0.0440	-
0.6139	13450	0.0325	-
0.6162	13500	0.0135	0.1766
0.6185	13550	0.0228	-
0.6207	13600	0.0288	-
0.6230	13650	0.0136	-
0.6253	13700	0.0379	-
0.6276	13750	0.0251	-
0.6299	13800	0.0560	-
0.6322	13850	0.0157	-
0.6344	13900	0.0193	-
0.6367	13950	0.0344	-
0.6390	14000	0.0182	0.1828
0.6413	14050	0.0429	-
0.6436	14100	0.0230	-
0.6459	14150	0.0332	-
0.6481	14200	0.0327	-
0.6504	14250	0.0241	-
0.6527	14300	0.0273	-
0.6550	14350	0.0333	-
0.6573	14400	0.0409	-
0.6595	14450	0.0255	-
0.6618	14500	0.0297	0.1896
0.6641	14550	0.0309	-
0.6664	14600	0.0150	-
0.6687	14650	0.0344	-
0.6710	14700	0.0370	-
0.6732	14750	0.0583	-
0.6755	14800	0.0179	-
0.6778	14850	0.0163	-
0.6801	14900	0.0192	-
0.6824	14950	0.0140	-
0.6847	15000	0.0129	0.1859
0.6869	15050	0.0191	-
0.6892	15100	0.0251	-
0.6915	15150	0.0251	-
0.6938	15200	0.0397	-
0.6961	15250	0.0232	-
0.6983	15300	0.0220	-
0.7006	15350	0.0143	-
0.7029	15400	0.0464	-
0.7052	15450	0.0327	-
0.7075	15500	0.0262	0.1800
0.7098	15550	0.0362	-
0.7120	15600	0.0457	-
0.7143	15650	0.0134	-
0.7166	15700	0.0245	-
0.7189	15750	0.0517	-
0.7212	15800	0.0494	-
0.7234	15850	0.0339	-
0.7257	15900	0.0263	-
0.7280	15950	0.0348	-
0.7303	16000	0.0523	0.1757
0.7326	16050	0.0381	-
0.7349	16100	0.0283	-
0.7371	16150	0.0255	-
0.7394	16200	0.0231	-
0.7417	16250	0.0129	-
0.7440	16300	0.0145	-
0.7463	16350	0.0274	-
0.7486	16400	0.0556	-
0.7508	16450	0.0420	-
0.7531	16500	0.0364	0.1777
0.7554	16550	0.0169	-
0.7577	16600	0.0556	-
0.7600	16650	0.0243	-
0.7622	16700	0.0427	-
0.7645	16750	0.0201	-
0.7668	16800	0.0234	-
0.7691	16850	0.0175	-
0.7714	16900	0.0182	-
0.7737	16950	0.0333	-
0.7759	17000	0.0191	0.1785
0.7782	17050	0.0328	-
0.7805	17100	0.0254	-
0.7828	17150	0.0300	-
0.7851	17200	0.0264	-
0.7873	17250	0.0223	-
0.7896	17300	0.0489	-
0.7919	17350	0.0097	-
0.7942	17400	0.0218	-
0.7965	17450	0.0315	-
0.7988	17500	0.0212	0.1793
0.8010	17550	0.0107	-
0.8033	17600	0.0133	-
0.8056	17650	0.0254	-
0.8079	17700	0.0247	-
0.8102	17750	0.0232	-
0.8125	17800	0.0161	-
0.8147	17850	0.0200	-
0.8170	17900	0.0423	-
0.8193	17950	0.0153	-
0.8216	18000	0.0288	0.1730
0.8239	18050	0.0163	-
0.8261	18100	0.0106	-
0.8284	18150	0.0082	-
0.8307	18200	0.0216	-
0.8330	18250	0.0294	-
0.8353	18300	0.0256	-
0.8376	18350	0.0062	-
0.8398	18400	0.0416	-
0.8421	18450	0.0203	-
0.8444	18500	0.0107	0.1750
0.8467	18550	0.0236	-
0.8490	18600	0.0313	-
0.8512	18650	0.0136	-
0.8535	18700	0.0274	-
0.8558	18750	0.0303	-
0.8581	18800	0.0208	-
0.8604	18850	0.0309	-
0.8627	18900	0.0355	-
0.8649	18950	0.0279	-
0.8672	19000	0.0297	0.1705
0.8695	19050	0.0209	-
0.8718	19100	0.0295	-
0.8741	19150	0.0207	-
0.8764	19200	0.0242	-

The bold row denotes the saved checkpoint.

Training Time

Training: 5.8 hours

Framework Versions

Python: 3.12.13
Sentence Transformers: 5.4.1
Transformers: 5.0.0
PyTorch: 2.10.0+cu128
Accelerate: 1.13.0
Datasets: 4.0.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CachedMultipleNegativesRankingLoss

bibtex

@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}