This is a sentence-transformers model finetuned from microsoft/harrier-oss-v1-270m. It maps sentences & paragraphs to a 640-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: microsoft/harrier-oss-v1-270m
Maximum Sequence Length: 128 tokens
Output Dimensionality: 640 dimensions
Similarity Function: Cosine Similarity
Supported Modality: Text

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'embedding_dimension': 640, 'pooling_mode': 'lasttoken', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Instruct: Represent this Buddhist passage for semantic retrieval.\nQuery: Bhagavā taṃ paṭiggahetvā ānandattheraṃ āṇāpesi – ‘‘ imaṃ phalaṃ madditvā pānaṃ dehī ’’ ti. Thero tathā akāsi. Bhagavā ambarasaṃ pivitvā ambaṭṭhiṃ uyyānapālassa datvā ‘‘ imaṃ ropehī ’’ ti āha. So vālukaṃ viyūhitvā taṃ ropesi, ānandatthero kuṇḍikāya udakaṃ āsiñci.',
    'The Blessed One accepted it and instructed Venerable Ānanda: “Crush this fruit and give it as a drink.” The Elder did so. The Blessed One, having drunk the mango juice, gave the mango seed to the gardener and said, “Plant this.” He dug up the sand and planted it, and Venerable Ānanda poured water from a vessel.',
    '“The foremost of my disciples, monks, who are very learned. Who are mindful. Who are clear-headed. Who are resolute. Who are attendants, is Ānanda” –',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 640]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.8314, 0.1940],
#         [0.8314, 1.0000, 0.2902],
#         [0.1940, 0.2902, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 744,489 training samples
Columns: anchor and positive
Approximate statistics based on the first 1000 samples:
anchor positive
type string string
details
min: 22 tokens
mean: 90.93 tokens
max: 128 tokens
min: 9 tokens
mean: 55.09 tokens
max: 128 tokens

Samples:

anchor	positive
`Instruct: Represent this Buddhist passage for semantic retrieval.[object Object]Query: Namo tassa bhagavato arahato sammāsambuddhassa`	`Homage to the Blessed One, the Perfected One, the Fully Self-Awakened Buddha`
`Instruct: Represent this Buddhist passage for semantic retrieval.[object Object]Query: 1. Evaṃ me sutaṃ – ekaṃ samayaṃ bhagavā sāvatthiyaṃ viharati jetavane anāthapiṇḍikassa ārāme. Tatra kho bhagavā bhikkhū āmantesi – ‘‘ bhikkhavo ’’ ti. ‘‘ Bhadante ’’ ti te bhikkhū bhagavato paccassosuṃ. Bhagavā etadavoca –`	`Thus have I heard — on one occasion the Blessed One was dwelling at Sāvatthī in Jeta's Grove, in Anāthapiṇḍika’s Park. There the Blessed One addressed the monks: “Monks!” “Venerable sir,” those monks replied to the Blessed One. The Blessed One said:`
`Instruct: Represent this Buddhist passage for semantic retrieval.[object Object]Query: 2.‘‘ Nāhaṃ, bhikkhave, aññaṃ ekasaddampi samanupassāmi yaṃ evaṃ purisassa cittaṃ pariyādāya tiṭṭhati yathayidaṃ, bhikkhave, itthisaddo. Itthisaddo, bhikkhave, purisassa cittaṃ pariyādāya tiṭṭhatī ’’ ti. Dutiyaṃ.`	`“Monks, I do not see a single sound that invades his mind and remains in a person as much as the sound of a woman. The sound of a woman, monks, overpowers a man’s mind.” The second.`

Loss: MultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false, 5 "directions": [ 6 "query_to_doc" 7 ], 8 "partition_mode": "joint", 9 "hardness_mode": null, 10 "hardness_strength": 0.0 11}

Evaluation Dataset

Unnamed Dataset

Size: 15,180 evaluation samples
Columns: anchor and positive
Approximate statistics based on the first 1000 samples:
anchor positive
type string string
details
min: 21 tokens
mean: 82.69 tokens
max: 128 tokens
min: 7 tokens
mean: 58.92 tokens
max: 128 tokens

Samples:

anchor	positive
`Instruct: Represent this Buddhist passage for semantic retrieval.[object Object]Query: ‘‘ Nāhaṃ, bhikkhave, aññaṃ ekarūpampi samanupassāmi yaṃ evaṃ purisassa cittaṃ pariyādāya tiṭṭhati yathayidaṃ, bhikkhave, itthirūpaṃ. Itthirūpaṃ, bhikkhave, purisassa cittaṃ pariyādāya tiṭṭhatī ’’ ti. Paṭhamaṃ.`	`“Monks, I do not see a single sight that invades his mind and remains in a person as much as the sight of a woman. The sight of a woman, monks, overpowers a man’s mind.” The first.`
`Instruct: Represent this Buddhist passage for semantic retrieval.[object Object]Query: Paṇihitaacchavaggo pañcamo.`	`The Chapter on the Directed and Clear is the fifth.`
`Instruct: Represent this Buddhist passage for semantic retrieval.[object Object]Query: 8. Kalyāṇamittādivaggo`	`8. Chapter on Good Friendship`

Loss: MultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false, 5 "directions": [ 6 "query_to_doc" 7 ], 8 "partition_mode": "joint", 9 "hardness_mode": null, 10 "hardness_strength": 0.0 11}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 16
gradient_accumulation_steps: 2
learning_rate: 2e-05
num_train_epochs: 1
lr_scheduler_type: cosine
warmup_steps: 0.05
fp16: True
remove_unused_columns: False
load_best_model_at_end: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

do_predict: False
prediction_loss_only: True
per_device_train_batch_size: 16
per_device_eval_batch_size: 8
gradient_accumulation_steps: 2
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 1
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: None
warmup_ratio: None
warmup_steps: 0.05
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
enable_jit_checkpoint: False
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
use_cpu: False
seed: 42
data_seed: None
bf16: False
fp16: True
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: -1
ddp_backend: None
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
disable_tqdm: False
remove_unused_columns: False
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_for_metrics: []
eval_do_concat_batches: True
auto_find_batch_size: False
full_determinism: False
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
use_cache: False
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss
0.0000	1	2.4117	-
0.0021	50	2.2977	-
0.0043	100	1.4074	-
0.0064	150	0.9612	-
0.0086	200	0.6430	-
0.0107	250	0.4462	-
0.0129	300	0.3157	-
0.0150	350	0.2660	-
0.0172	400	0.2222	-
0.0193	450	0.1691	-
0.0215	500	0.1565	-
0.0236	550	0.1109	-
0.0258	600	0.1512	-
0.0279	650	0.1317	-
0.0301	700	0.1211	-
0.0322	750	0.0998	-
0.0344	800	0.0884	-
0.0365	850	0.0988	-
0.0387	900	0.0813	-
0.0408	950	0.0744	-
0.0430	1000	0.0733	-
0.0451	1050	0.0689	-
0.0473	1100	0.0577	-
0.0494	1150	0.0637	-
0.0516	1200	0.0645	-
0.0537	1250	0.0491	-
0.0559	1300	0.0614	-
0.0580	1350	0.0512	-
0.0602	1400	0.0452	-
0.0623	1450	0.0577	-
0.0645	1500	0.0518	-
0.0666	1550	0.0366	-
0.0688	1600	0.0529	-
0.0709	1650	0.0412	-
0.0731	1700	0.0313	-
0.0752	1750	0.0397	-
0.0774	1800	0.0361	-
0.0795	1850	0.0428	-
0.0817	1900	0.0423	-
0.0838	1950	0.0288	-
0.0860	2000	0.0490	0.0196
0.0881	2050	0.0379	-
0.0903	2100	0.0309	-
0.0924	2150	0.0347	-
0.0946	2200	0.0327	-
0.0967	2250	0.0442	-
0.0989	2300	0.0244	-
0.1010	2350	0.0337	-
0.1032	2400	0.0308	-
0.1053	2450	0.0271	-
0.1075	2500	0.0342	-
0.1096	2550	0.0344	-
0.1118	2600	0.0285	-
0.1139	2650	0.0334	-
0.1161	2700	0.0359	-
0.1182	2750	0.0309	-
0.1203	2800	0.0455	-
0.1225	2850	0.0301	-
0.1246	2900	0.0229	-
0.1268	2950	0.0236	-
0.1289	3000	0.0327	-
0.1311	3050	0.0148	-
0.1332	3100	0.0189	-
0.1354	3150	0.0213	-
0.1375	3200	0.0246	-
0.1397	3250	0.0230	-
0.1418	3300	0.0246	-
0.1440	3350	0.0204	-
0.1461	3400	0.0207	-
0.1483	3450	0.0339	-
0.1504	3500	0.0202	-
0.1526	3550	0.0268	-
0.1547	3600	0.0252	-
0.1569	3650	0.0225	-
0.1590	3700	0.0279	-
0.1612	3750	0.0233	-
0.1633	3800	0.0204	-
0.1655	3850	0.0212	-
0.1676	3900	0.0256	-
0.1698	3950	0.0211	-
0.1719	4000	0.0209	0.0132
0.1741	4050	0.0245	-
0.1762	4100	0.0176	-
0.1784	4150	0.0184	-
0.1805	4200	0.0293	-
0.1827	4250	0.0256	-
0.1848	4300	0.0185	-
0.1870	4350	0.0115	-
0.1891	4400	0.0199	-
0.1913	4450	0.0145	-
0.1934	4500	0.0158	-
0.1956	4550	0.0238	-
0.1977	4600	0.0267	-
0.1999	4650	0.0222	-
0.2020	4700	0.0166	-
0.2042	4750	0.0175	-
0.2063	4800	0.0168	-
0.2085	4850	0.0188	-
0.2106	4900	0.0182	-
0.2128	4950	0.0108	-
0.2149	5000	0.0202	-
0.2171	5050	0.0128	-
0.2192	5100	0.0148	-
0.2214	5150	0.0182	-
0.2235	5200	0.0124	-
0.2257	5250	0.0137	-
0.2278	5300	0.0099	-
0.2300	5350	0.0203	-
0.2321	5400	0.0128	-
0.2343	5450	0.0168	-
0.2364	5500	0.0185	-
0.2386	5550	0.0143	-
0.2407	5600	0.0148	-
0.2428	5650	0.0290	-
0.2450	5700	0.0133	-
0.2471	5750	0.0146	-
0.2493	5800	0.0192	-
0.2514	5850	0.0229	-
0.2536	5900	0.0234	-
0.2557	5950	0.0114	-
0.2579	6000	0.0098	0.0111
0.2600	6050	0.0163	-
0.2622	6100	0.0119	-
0.2643	6150	0.0187	-
0.2665	6200	0.0184	-
0.2686	6250	0.0129	-
0.2708	6300	0.0131	-
0.2729	6350	0.0129	-
0.2751	6400	0.0138	-
0.2772	6450	0.0122	-
0.2794	6500	0.0198	-
0.2815	6550	0.0231	-
0.2837	6600	0.0150	-
0.2858	6650	0.0173	-
0.2880	6700	0.0156	-
0.2901	6750	0.0212	-
0.2923	6800	0.0159	-
0.2944	6850	0.0250	-
0.2966	6900	0.0144	-
0.2987	6950	0.0181	-
0.3009	7000	0.0123	-
0.3030	7050	0.0222	-
0.3052	7100	0.0155	-
0.3073	7150	0.0263	-
0.3095	7200	0.0216	-
0.3116	7250	0.0143	-
0.3138	7300	0.0092	-
0.3159	7350	0.0070	-
0.3181	7400	0.0203	-
0.3202	7450	0.0174	-
0.3224	7500	0.0262	-
0.3245	7550	0.0239	-
0.3267	7600	0.0126	-
0.3288	7650	0.0132	-
0.3310	7700	0.0145	-
0.3331	7750	0.0170	-
0.3353	7800	0.0125	-
0.3374	7850	0.0124	-
0.3396	7900	0.0205	-
0.3417	7950	0.0108	-
0.3439	8000	0.0156	0.0099
0.3460	8050	0.0151	-
0.3482	8100	0.0188	-
0.3503	8150	0.0059	-
0.3525	8200	0.0152	-
0.3546	8250	0.0202	-
0.3568	8300	0.0131	-
0.3589	8350	0.0145	-
0.3610	8400	0.0135	-
0.3632	8450	0.0147	-
0.3653	8500	0.0080	-
0.3675	8550	0.0118	-
0.3696	8600	0.0064	-
0.3718	8650	0.0204	-
0.3739	8700	0.0101	-
0.3761	8750	0.0125	-
0.3782	8800	0.0129	-
0.3804	8850	0.0062	-
0.3825	8900	0.0120	-
0.3847	8950	0.0112	-
0.3868	9000	0.0131	-
0.3890	9050	0.0166	-
0.3911	9100	0.0114	-
0.3933	9150	0.0157	-
0.3954	9200	0.0133	-
0.3976	9250	0.0145	-
0.3997	9300	0.0042	-
0.4019	9350	0.0142	-
0.4040	9400	0.0138	-
0.4062	9450	0.0214	-
0.4083	9500	0.0095	-
0.4105	9550	0.0067	-
0.4126	9600	0.0094	-
0.4148	9650	0.0063	-
0.4169	9700	0.0120	-
0.4191	9750	0.0116	-
0.4212	9800	0.0105	-
0.4234	9850	0.0222	-
0.4255	9900	0.0142	-
0.4277	9950	0.0121	-
0.4298	10000	0.0091	0.0080
0.4320	10050	0.0173	-
0.4341	10100	0.0098	-
0.4363	10150	0.0195	-
0.4384	10200	0.0117	-
0.4406	10250	0.0091	-
0.4427	10300	0.0146	-
0.4449	10350	0.0143	-
0.4470	10400	0.0132	-
0.4492	10450	0.0125	-
0.4513	10500	0.0116	-
0.4535	10550	0.0106	-
0.4556	10600	0.0099	-
0.4578	10650	0.0118	-
0.4599	10700	0.0051	-
0.4621	10750	0.0079	-
0.4642	10800	0.0086	-
0.4664	10850	0.0094	-
0.4685	10900	0.0065	-
0.4707	10950	0.0182	-
0.4728	11000	0.0160	-
0.4750	11050	0.0094	-
0.4771	11100	0.0129	-
0.4793	11150	0.0119	-
0.4814	11200	0.0183	-
0.4835	11250	0.0208	-
0.4857	11300	0.0125	-
0.4878	11350	0.0063	-
0.4900	11400	0.0106	-
0.4921	11450	0.0136	-
0.4943	11500	0.0086	-
0.4964	11550	0.0085	-
0.4986	11600	0.0115	-
0.5007	11650	0.0137	-
0.5029	11700	0.0141	-
0.5050	11750	0.0064	-
0.5072	11800	0.0123	-
0.5093	11850	0.0094	-
0.5115	11900	0.0090	-
0.5136	11950	0.0053	-
0.5158	12000	0.0086	0.0076
0.5179	12050	0.0093	-
0.5201	12100	0.0063	-
0.5222	12150	0.0121	-
0.5244	12200	0.0103	-
0.5265	12250	0.0066	-
0.5287	12300	0.0112	-
0.5308	12350	0.0127	-
0.5330	12400	0.0161	-
0.5351	12450	0.0071	-
0.5373	12500	0.0096	-
0.5394	12550	0.0088	-
0.5416	12600	0.0095	-
0.5437	12650	0.0075	-
0.5459	12700	0.0113	-
0.5480	12750	0.0121	-
0.5502	12800	0.0059	-
0.5523	12850	0.0099	-
0.5545	12900	0.0064	-
0.5566	12950	0.0094	-
0.5588	13000	0.0108	-
0.5609	13050	0.0074	-
0.5631	13100	0.0038	-
0.5652	13150	0.0043	-
0.5674	13200	0.0082	-
0.5695	13250	0.0112	-
0.5717	13300	0.0178	-
0.5738	13350	0.0073	-
0.5760	13400	0.0059	-
0.5781	13450	0.0042	-
0.5803	13500	0.0121	-
0.5824	13550	0.0093	-
0.5846	13600	0.0087	-
0.5867	13650	0.0169	-
0.5889	13700	0.0064	-
0.5910	13750	0.0047	-
0.5932	13800	0.0099	-
0.5953	13850	0.0086	-
0.5975	13900	0.0173	-
0.5996	13950	0.0141	-
0.6017	14000	0.0071	0.0070
0.6039	14050	0.0066	-
0.6060	14100	0.0069	-
0.6082	14150	0.0143	-
0.6103	14200	0.0084	-
0.6125	14250	0.0119	-
0.6146	14300	0.0142	-
0.6168	14350	0.0067	-
0.6189	14400	0.0186	-
0.6211	14450	0.0164	-
0.6232	14500	0.0060	-
0.6254	14550	0.0175	-
0.6275	14600	0.0060	-
0.6297	14650	0.0137	-
0.6318	14700	0.0161	-
0.6340	14750	0.0064	-
0.6361	14800	0.0067	-
0.6383	14850	0.0153	-
0.6404	14900	0.0056	-
0.6426	14950	0.0149	-
0.6447	15000	0.0052	-
0.6469	15050	0.0153	-
0.6490	15100	0.0069	-
0.6512	15150	0.0155	-
0.6533	15200	0.0060	-
0.6555	15250	0.0079	-
0.6576	15300	0.0075	-
0.6598	15350	0.0174	-
0.6619	15400	0.0084	-
0.6641	15450	0.0053	-
0.6662	15500	0.0117	-
0.6684	15550	0.0110	-
0.6705	15600	0.0131	-
0.6727	15650	0.0080	-
0.6748	15700	0.0099	-
0.6770	15750	0.0073	-
0.6791	15800	0.0030	-
0.6813	15850	0.0085	-
0.6834	15900	0.0118	-
0.6856	15950	0.0075	-
0.6877	16000	0.0113	0.0060

Training Time

Training: 6.1 hours

Framework Versions

Python: 3.12.13
Sentence Transformers: 5.4.1
Transformers: 5.0.0
PyTorch: 2.10.0+cu128
Accelerate: 1.13.0
Datasets: 5.0.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

bibtex

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}

	anchor	positive
type	string	string
details	min: 22 tokens mean: 90.93 tokens max: 128 tokens	min: 9 tokens mean: 55.09 tokens max: 128 tokens

dhammanana/harrier-tipitaka-v1

SentenceTransformer based on microsoft/harrier-oss-v1-270m

Model Details

Model Description

Model Sources

Full Model Architecture

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

Unnamed Dataset

Evaluation Dataset

Unnamed Dataset

Training Hyperparameters

Non-Default Hyperparameters

All Hyperparameters

Training Logs

Training Time

Framework Versions

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss

dhammanana/
harrier-tipitaka-v1