This is a PyLate model finetuned from ozayezerceli/ettin-encoder-150M-TR on the msmarco-tr dataset. It maps sentences & paragraphs to sequences of 128-dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator.

Model Details

Model Description

Model Type: PyLate model
Base model: ozayezerceli/ettin-encoder-150M-TR
Document Length: 180 tokens
Query Length: 32 tokens
Output Dimensionality: 128 tokens
Similarity Function: MaxSim
Training Dataset:
- msmarco-tr
Language: tr

Model Sources

Documentation: PyLate Documentation
Repository: PyLate on GitHub
Hugging Face: PyLate models on Hugging Face

Full Model Architecture

ColBERT(
  (0): Transformer({'max_seq_length': 179, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Dense({'in_features': 768, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'use_residual': False})
)

Usage

First install the PyLate library:

pip install -U pylate

Retrieval

Use this model with PyLate to index and retrieve documents. The index uses FastPLAID for efficient similarity search.

Indexing documents

Load the ColBERT model and initialize the PLAID index, then encode and index your documents:

python

from pylate import indexes, models, retrieve

# Step 1: Load the ColBERT model
model = models.ColBERT(
    model_name_or_path="pylate_model_id",
)

# Step 2: Initialize the PLAID index
index = indexes.PLAID(
    index_folder="pylate-index",
    index_name="index",
    override=True,  # This overwrites the existing index if any
)

# Step 3: Encode the documents
documents_ids = ["1", "2", "3"]
documents = ["document 1 text", "document 2 text", "document 3 text"]

documents_embeddings = model.encode(
    documents,
    batch_size=32,
    is_query=False,  # Ensure that it is set to False to indicate that these are documents, not queries
    show_progress_bar=True,
)

# Step 4: Add document embeddings to the index by providing embeddings and corresponding ids
index.add_documents(
    documents_ids=documents_ids,
    documents_embeddings=documents_embeddings,
)

Note that you do not have to recreate the index and encode the documents every time. Once you have created an index and added the documents, you can re-use the index later by loading it:

python

# To load an index, simply instantiate it with the correct folder/name and without overriding it
index = indexes.PLAID(
    index_folder="pylate-index",
    index_name="index",
)

Retrieving top-k documents for queries

Once the documents are indexed, you can retrieve the top-k most relevant documents for a given set of queries. To do so, initialize the ColBERT retriever with the index you want to search in, encode the queries and then retrieve the top-k documents to get the top matches ids and relevance scores:

python

# Step 1: Initialize the ColBERT retriever
retriever = retrieve.ColBERT(index=index)

# Step 2: Encode the queries
queries_embeddings = model.encode(
    ["query for document 3", "query for document 1"],
    batch_size=32,
    is_query=True,  #  # Ensure that it is set to False to indicate that these are queries
    show_progress_bar=True,
)

# Step 3: Retrieve top-k documents
scores = retriever.retrieve(
    queries_embeddings=queries_embeddings,
    k=10,  # Retrieve the top 10 matches for each query
)

Reranking

If you only want to use the ColBERT model to perform reranking on top of your first-stage retrieval pipeline without building an index, you can simply use rank function and pass the queries and documents to rerank:

python

from pylate import rank, models

queries = [
    "query A",
    "query B",
]

documents = [
    ["document A", "document B"],
    ["document 1", "document C", "document B"],
]

documents_ids = [
    [1, 2],
    [1, 3, 2],
]

model = models.ColBERT(
    model_name_or_path="pylate_model_id",
)

queries_embeddings = model.encode(
    queries,
    is_query=True,
)

documents_embeddings = model.encode(
    documents,
    is_query=False,
)

reranked_documents = rank.rerank(
    documents_ids=documents_ids,
    queries_embeddings=queries_embeddings,
    documents_embeddings=documents_embeddings,
)

Training Details

Training Dataset

msmarco-tr

Dataset: msmarco-tr at ffad30a
Size: 910,904 training samples
Columns: query, positive, and negative
Approximate statistics based on the first 1000 samples:
query positive negative
type string string string
details
min: 6 tokens
mean: 17.07 tokens
max: 32 tokens
min: 25 tokens
mean: 31.99 tokens
max: 32 tokens
min: 21 tokens
mean: 31.99 tokens
max: 32 tokens

Samples:

query	positive	negative
`sinir dokusundaki miyelin kılıfı nerede`	`Miyelin, bir tabaka oluşturan akson dielektrik (elektriksel olarak yalıtkan) malzemeyi çevreleyen yağlı bir beyaz maddedir, miyelin kılıfı, genellikle sadece bir nöronun aksonu etrafında bulunur. Sinir sisteminin düzgün çalışması için gereklidir. Bir tür glial hücrenin bir dış büyümesidir. Miyelin kılıfının üretimi miyelinasyon olarak adlandırılır. İnsanlarda, miyelin kılıfı 14'üncü haftada başlar.`	`İnsanlarda, dört temel doku tipi vardır: epitel dokusu, bağ dokusu, kas dokusu ve sinir dokusu. Her genel doku tipi içinde, belirli doku tipleri vardır. Bunu bir futbol takımı gibi düşünün.Her biri sahada kendi 'iş' olan bireysel oyuncular vardır.n insanlar, dört temel doku tipi vardır: epitel dokusu, bağ dokusu, kas dokusu ve sinir dokusu. Bu genel doku tipinde, her bir genel doku tipinde vardır.`
`Okulların Makine Mühendisliğini Sundukları Şeyler`	`Makine Mühendisliği Teknolojisi Dereceleri için Üst Okullar. Pennsylvania Eyalet Üniversitesi - Harrisburg, Purdue Üniversitesi ve Houston Üniversitesi, makine mühendisliği teknolojisi (MET) alanında lisans derecesi sunan üç okuldur. Bu üniversitelerdeki MET programları hakkında daha fazla bilgi edinmek için okumaya devam edin.`	`Mühendis tanımı, motorların veya makinelerin tasarımında, yapımında ve kullanımında veya çeşitli mühendislik dallarından herhangi birinde eğitimli ve yetenekli bir kişi: bir makine mühendisi; bir inşaat mühendisi. Daha fazla bilgi için bkz.`
`kim navigatör karıştırma valfleri taşır`	`BRADLEY THERMOSTATIC MIXING VANAS. Bradley Corporation, armatür ve sıhhi tesisat ürünlerinin üretiminde lider, dört hat üretir. termostatik karıştırma valfleri (TMVs). Bradley Navigator Yüksek Düşük termostatik karıştırma valfleri vardır. Dıştan gelen talebin çok düşük olduğu uygulamalar için idealdir.`	`Hidrolik Valfler. Eaton valfleri, tüm dünyadaki pazarlarda müşterilerimiz için rekabet avantajı sağlar. Geniş bir seçenek yelpazesinde benzersiz kalite sunan yüksek değerli hidrolik valf ürünlerimiz, gerçek endüstri liderlerinin tüm özelliklerini ve performans seviyelerini içerir. Endüstriyel Valfler.`

Loss: pylate.losses.contrastive.Contrastive

Evaluation Dataset

msmarco-tr

Dataset: msmarco-tr at ffad30a
Size: 9,202 evaluation samples
Columns: query, positive, and negative
Approximate statistics based on the first 1000 samples:
query positive negative
type string string string
details
min: 6 tokens
mean: 16.9 tokens
max: 32 tokens
min: 29 tokens
mean: 32.0 tokens
max: 32 tokens
min: 28 tokens
mean: 31.99 tokens
max: 32 tokens

Samples:

query	positive	negative
`Ermin hangi hayvandır`	`1 Aslında ermine kelimesi beyaz kürklü bir hayvanı ifade ederken, sırt üstü kahverengi kürklü ve karnında baş ve beyaz kürklü bireyler için stoat kullanılır.`	`Dünyada kaç hayvan türü var? İşte kaba bir sayım ve bilim adamlarının sayılara nasıl ulaştıklarına dair kısa bir açıklama. Dünyada kaç hayvan türü var? İşte kaba bir sayım ve bilim adamlarının sayılara nasıl ulaştıklarına dair kısa bir açıklama. Kaç hayvan türü var? [object Object] Strauss, Bob.`
`Abacus nereden çıktı`	`Abacus: Kısa Bir Tarih. Abacus, kökeni Yunanca abax veya abakon (masa veya tablet anlamına gelir) kelimelerinden gelen ve muhtemelen kum anlamına gelen Semitik abq kelimesinden kaynaklanan Latince bir kelimedir. Abacus, büyük sayıları saymak için kullanılan birçok sayma cihazından biridir.`	`Hücre apeksinde, bir flagellum için çapa alanı olan bazal gövdedir. Bazal cisimler, dokuz periferik mikrotübül üçlüsü ile centrioles'inkine benzer bir alt yapıya sahiptir (görüntünün alt merkezindeki yapıya bakınız).`
`Başın arkasında radyasyon tedavisi yüz kızarıklığına neden olur mu`	`Radyasyon Terapisinin En Yaygın Yan Etkileri. Cilt reaksiyonu: Radyasyon tedavisinin yaygın bir yan etkisi, tedavi edilen vücut bölgesinde cilt tahrişidir. Cilt reaksiyonu, hafif kızarıklık ve kuruluktan (güneş yanığına benzer) bazı hastalarda cildin şiddetli soyulmasına (desquamation) kadar değişebilir.`	`Bu açıklama amfizemi işaret edebilir. Bu, sigara içme geçmişiniz varsa daha da muhtemeldir. Radyasyon terapisi bilinen nedenlerden biri değildir. Bu konuda daha fazla cevap almak ve semptomlarınızı çözmeye yardımcı olmak için bir pulmonologla takip etmenizi isteyeceğim. Umarım bu, sorgunuzu tamamen ele alır. Sigara içme geçmişiniz varsa, daha da fazla umut eder. Radyasyon terapisi, bu sorunun çözümüne yardımcı olmanızı ve bu sorunun cevabını takip etmenizi isterim.`

Loss: pylate.losses.contrastive.Contrastive

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 32
per_device_eval_batch_size: 32
learning_rate: 3e-06
num_train_epochs: 1
fp16: True

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 3e-06
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 1
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss
0.0000	1	4.1579
0.0035	100	3.3423
0.0070	200	1.9776
0.0105	300	1.3823
0.0141	400	1.1908
0.0176	500	1.0101
0.0211	600	0.9372
0.0246	700	0.8878
0.0281	800	0.8133
0.0316	900	0.8001
0.0351	1000	0.7634
0.0386	1100	0.7144
0.0422	1200	0.7239
0.0457	1300	0.7352
0.0492	1400	0.7019
0.0527	1500	0.7021
0.0562	1600	0.6415
0.0597	1700	0.6469
0.0632	1800	0.6384
0.0667	1900	0.5887
0.0703	2000	0.6121
0.0738	2100	0.6309
0.0773	2200	0.5957
0.0808	2300	0.5758
0.0843	2400	0.6137
0.0878	2500	0.606
0.0913	2600	0.5516
0.0948	2700	0.6184
0.0984	2800	0.5571
0.1019	2900	0.5462
0.1054	3000	0.62
0.1089	3100	0.5548
0.1124	3200	0.564
0.1159	3300	0.5299
0.1194	3400	0.5464
0.1230	3500	0.5554
0.1265	3600	0.5818
0.1300	3700	0.5608
0.1335	3800	0.5641
0.1370	3900	0.5337
0.1405	4000	0.5381
0.1440	4100	0.5308
0.1475	4200	0.5036
0.1511	4300	0.5292
0.1546	4400	0.5007
0.1581	4500	0.5442
0.1616	4600	0.5201
0.1651	4700	0.5331
0.1686	4800	0.4986
0.1721	4900	0.5312
0.1756	5000	0.4578
0.1792	5100	0.5328
0.1827	5200	0.488
0.1862	5300	0.5007
0.1897	5400	0.4652
0.1932	5500	0.4924
0.1967	5600	0.5278
0.2002	5700	0.4527
0.2038	5800	0.4902
0.2073	5900	0.4796
0.2108	6000	0.4582
0.2143	6100	0.491
0.2178	6200	0.4789
0.2213	6300	0.4728
0.2248	6400	0.4737
0.2283	6500	0.4565
0.2319	6600	0.4512
0.2354	6700	0.4813
0.2389	6800	0.4638
0.2424	6900	0.5149
0.2459	7000	0.4587
0.2494	7100	0.4805
0.2529	7200	0.4847
0.2564	7300	0.5415
0.2600	7400	0.5055
0.2635	7500	0.4437
0.2670	7600	0.4712
0.2705	7700	0.4736
0.2740	7800	0.4242
0.2775	7900	0.4598
0.2810	8000	0.4398
0.2845	8100	0.4577
0.2881	8200	0.4501
0.2916	8300	0.4757
0.2951	8400	0.4386
0.2986	8500	0.4636
0.3021	8600	0.4282
0.3056	8700	0.44
0.3091	8800	0.4413
0.3127	8900	0.4548
0.3162	9000	0.4515
0.3197	9100	0.461
0.3232	9200	0.4299
0.3267	9300	0.4161
0.3302	9400	0.4567
0.3337	9500	0.4036
0.3372	9600	0.3965
0.3408	9700	0.4746
0.3443	9800	0.4078
0.3478	9900	0.4234
0.3513	10000	0.4423
0.3548	10100	0.4151
0.3583	10200	0.4298
0.3618	10300	0.4246
0.3653	10400	0.4292
0.3689	10500	0.4258
0.3724	10600	0.4153
0.3759	10700	0.4239
0.3794	10800	0.4286
0.3829	10900	0.4364
0.3864	11000	0.4276
0.3899	11100	0.3974
0.3935	11200	0.4334
0.3970	11300	0.4343
0.4005	11400	0.4039
0.4040	11500	0.4148
0.4075	11600	0.4022
0.4110	11700	0.4203
0.4145	11800	0.4188
0.4180	11900	0.4265
0.4216	12000	0.4125
0.4251	12100	0.3977
0.4286	12200	0.3811
0.4321	12300	0.4199
0.4356	12400	0.4062
0.4391	12500	0.3842
0.4426	12600	0.4063
0.4461	12700	0.4069
0.4497	12800	0.3984
0.4532	12900	0.4161
0.4567	13000	0.3827
0.4602	13100	0.4201
0.4637	13200	0.3808
0.4672	13300	0.4335
0.4707	13400	0.3975
0.4742	13500	0.4059
0.4778	13600	0.4077
0.4813	13700	0.4164
0.4848	13800	0.4146
0.4883	13900	0.3857
0.4918	14000	0.3952
0.4953	14100	0.416
0.4988	14200	0.413
0.5024	14300	0.403
0.5059	14400	0.4139
0.5094	14500	0.4237
0.5129	14600	0.3944
0.5164	14700	0.4237
0.5199	14800	0.357
0.5234	14900	0.3983
0.5269	15000	0.4182
0.5305	15100	0.4478
0.5340	15200	0.3948
0.5375	15300	0.4127
0.5410	15400	0.4161
0.5445	15500	0.3815
0.5480	15600	0.3875
0.5515	15700	0.3859
0.5550	15800	0.399
0.5586	15900	0.3936
0.5621	16000	0.3927
0.5656	16100	0.3961
0.5691	16200	0.3657
0.5726	16300	0.3743
0.5761	16400	0.3881
0.5796	16500	0.3815
0.5832	16600	0.3719
0.5867	16700	0.365
0.5902	16800	0.4117
0.5937	16900	0.3819
0.5972	17000	0.3821
0.6007	17100	0.3589
0.6042	17200	0.367
0.6077	17300	0.3768
0.6113	17400	0.3652
0.6148	17500	0.3766
0.6183	17600	0.3952
0.6218	17700	0.3786
0.6253	17800	0.3975
0.6288	17900	0.3944
0.6323	18000	0.3884
0.6358	18100	0.3664
0.6394	18200	0.3755
0.6429	18300	0.3692
0.6464	18400	0.3747
0.6499	18500	0.3646
0.6534	18600	0.3856
0.6569	18700	0.3698
0.6604	18800	0.3872
0.6639	18900	0.3801
0.6675	19000	0.3736
0.6710	19100	0.4025
0.6745	19200	0.3689
0.6780	19300	0.39
0.6815	19400	0.4083
0.6850	19500	0.3403
0.6885	19600	0.332
0.6921	19700	0.3585
0.6956	19800	0.4102
0.6991	19900	0.3708
0.7026	20000	0.4117
0.7061	20100	0.3862
0.7096	20200	0.3819
0.7131	20300	0.3653
0.7166	20400	0.3518
0.7202	20500	0.3804
0.7237	20600	0.3539
0.7272	20700	0.3782
0.7307	20800	0.3846
0.7342	20900	0.3412
0.7377	21000	0.38
0.7412	21100	0.3799
0.7447	21200	0.368
0.7483	21300	0.3733
0.7518	21400	0.3979
0.7553	21500	0.3677
0.7588	21600	0.362
0.7623	21700	0.3897
0.7658	21800	0.3481
0.7693	21900	0.39
0.7729	22000	0.366
0.7764	22100	0.4224
0.7799	22200	0.3745
0.7834	22300	0.3616
0.7869	22400	0.3855
0.7904	22500	0.3742
0.7939	22600	0.3605
0.7974	22700	0.3467
0.8010	22800	0.3614
0.8045	22900	0.3484
0.8080	23000	0.3769
0.8115	23100	0.3324
0.8150	23200	0.3276
0.8185	23300	0.3684
0.8220	23400	0.3595
0.8255	23500	0.3884
0.8291	23600	0.3586
0.8326	23700	0.3614
0.8361	23800	0.3505
0.8396	23900	0.3665
0.8431	24000	0.3771
0.8466	24100	0.3935
0.8501	24200	0.3489
0.8536	24300	0.3406
0.8572	24400	0.3901
0.8607	24500	0.3444
0.8642	24600	0.3873
0.8677	24700	0.3668
0.8712	24800	0.3452
0.8747	24900	0.3497
0.8782	25000	0.3314
0.8818	25100	0.3878
0.8853	25200	0.3705
0.8888	25300	0.3564
0.8923	25400	0.3826
0.8958	25500	0.3674
0.8993	25600	0.3591
0.9028	25700	0.3525
0.9063	25800	0.3603
0.9099	25900	0.3439
0.9134	26000	0.3636
0.9169	26100	0.3552
0.9204	26200	0.3697
0.9239	26300	0.3957
0.9274	26400	0.373
0.9309	26500	0.3878
0.9344	26600	0.3868
0.9380	26700	0.3648
0.9415	26800	0.3399
0.9450	26900	0.344
0.9485	27000	0.3834
0.9520	27100	0.3623
0.9555	27200	0.3464
0.9590	27300	0.3666
0.9626	27400	0.3657
0.9661	27500	0.3495
0.9696	27600	0.3527
0.9731	27700	0.3484
0.9766	27800	0.362
0.9801	27900	0.3879
0.9836	28000	0.3558
0.9871	28100	0.3666
0.9907	28200	0.3456
0.9942	28300	0.3358
0.9977	28400	0.342

Framework Versions

Python: 3.12.12
Sentence Transformers: 5.1.1
PyLate: 1.3.4
Transformers: 4.56.2
PyTorch: 2.8.0+cu126
Accelerate: 1.10.1
Datasets: 4.0.0
Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084"
}

PyLate

bibtex

@misc{PyLate,
title={PyLate: Flexible Training and Retrieval for Late Interaction Models},
author={Chaffin, Antoine and Sourty, Raphaël},
url={https://github.com/lightonai/pylate},
year={2024}
}

	query	positive	negative
type	string	string	string
details	min: 6 tokens mean: 17.07 tokens max: 32 tokens	min: 25 tokens mean: 31.99 tokens max: 32 tokens	min: 21 tokens mean: 31.99 tokens max: 32 tokens

newmindai/col-ettin-encoder-150M-TR

PyLate model based on ozayezerceli/ettin-encoder-150M-TR

Model Details

Model Description

Model Sources

Full Model Architecture

Usage

Retrieval

Indexing documents

Retrieving top-k documents for queries

Reranking

Training Details

Training Dataset

msmarco-tr

Evaluation Dataset

msmarco-tr

Training Hyperparameters

Non-Default Hyperparameters

All Hyperparameters

Training Logs

Framework Versions

Citation

BibTeX

Sentence Transformers

PyLate

newmindai/
col-ettin-encoder-150M-TR