This is a sentence-transformers model finetuned from BAAI/bge-m3 on the arcanum-quests-queries-synthetic dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-m3
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 1024 dimensions
Similarity Function: Cosine Similarity
Supported Modality: Text
Training Dataset:
- arcanum-quests-queries-synthetic

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'XLMRobertaModel'})
  (1): Pooling({'embedding_dimension': 1024, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("pameydorke/arcanum-quests-base-retriever")
# Run inference
sentences = [
    'whiskey pickup tarant',
    "The Boil When chatting with Caleb Malloy in his pub, he will explain the geography of The Boil and how the gangs (or Clans) operate. If the player is unwilling to commit to joining one of the gangs, then Caleb can offer them some neutral work This is to retrieve a scheduled whiskey delivery that has been getting waylaid of late by one of the gangs Caleb's delivery boy is stalled on Garillon Bridge, so the party needs to go there to make the pickup Garillon Bridge The player can find Caleb's boy (actually Dwarf) on the bridge between The Boil and Tarant The party will get jumped by a rival gang as they approach. This may happen before or after actually getting the whiskey. Regardless of order, dispatch the gang and take possession of the whiskey The Boil Return to the pub and give Caleb the Whiskey He will give you his thanks, and a reward of 500 gp and 3100 exp",
    "Gladys in Dernholm will tell you that she wants a ring that used to belong to her son. Speak with Archibald by the dock and ask him about the ring. He is unwilling to hand it over to you. Talk to his son Bernard (just outside the house-on the docks), he will tell you its some sort of game between the two. Glady sends someone looking for the ring every so often. The ring belongs to Gladys son, who gave it to Bernard sister. Hints to a possible romantic connection between the two. Option 1: Pickpocket the ring from Archibald. Tell Glady he gave it to you. Then you'll receive 800 experience and some alignment. Option 2: Kill Archibald and Bernard for the ring, Gladys will be very upset with you. No reward. Option 3: Use Persuasion (at least 1+ point in skill) and talk to Bernard and Archibald to hand over the ring. Go back to Archibald and tell him you think Gladys is attracted to him, and he'll give you the ring. Then when you return it to Gladys you'll receive 1600 experience and a boost to your alignment.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.5063, 0.2837],
#         [0.5063, 1.0000, 0.4145],
#         [0.2837, 0.4145, 1.0000]])

Evaluation

Metrics

Information Retrieval

Dataset: arcanum-test
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.6884
cosine_accuracy@3	0.8333
cosine_accuracy@5	0.8768
cosine_accuracy@10	0.9348
cosine_precision@1	0.6884
cosine_precision@3	0.2778
cosine_precision@5	0.1754
cosine_precision@10	0.0935
cosine_recall@1	0.6884
cosine_recall@3	0.8333
cosine_recall@5	0.8768
cosine_recall@10	0.9348
cosine_ndcg@10	0.8099
cosine_mrr@10	0.7701
cosine_map@100	0.7735

Training Details

Training Dataset

arcanum-quests-queries-synthetic

Dataset: arcanum-quests-queries-synthetic at 69805bb
Size: 549 training samples
Columns: anchor and positive
Approximate statistics based on the first 549 samples:
anchor positive
type string string
details
min: 5 tokens
mean: 9.99 tokens
max: 23 tokens
min: 6 tokens
mean: 171.09 tokens
max: 1037 tokens

Samples:

anchor	positive
`gnome and ring quest`	So, you've got this ring you got from a gnome, and you want to find out about it. The first person to ask would be the gnome standing a bit to the northeast of the town well. He will ask you about the crash if you talk to him, and will be especially interested in the ring if you tell him about it. This little bastard is obviously not to be trusted, so don't give him the ring. He'll attack you for it later, but he's a wussy little gnome so you have nothing to fear. The next person to ask about the ring is Ristezze of Ristezze's Imported Goods, right next to the blacksmith. Ask him all about the ring. He will tell you everything, EXCEPT for where P. Schuyler & Sons, the store that originally sold it, is located. There's a couple of ways you can get this info from him: Trade him Bessie Toone’s boot (found in Bessie Toone’s mine) or the camera found on the body of Isaac Zapruder (crash site) for it. In the full game, I suggest you hang on to the camera, as it might help later on. A charism...
`sorcerous beast amulet`	`From talking with the guard outside the isle of despair fort, you will learn that there is a sorcerous beast that has been threatening the locals. The guard will give you an amulet to dispose of it. Follow the shoreline until you find footprints and follow them back to its lair. The amulet increases you healing power with respect to poison but causes people to have a negative reaction to you.`
`Find Liam Cameron`	1 - Starting Point2 - Liam's WorkshopThere are two things to loot inside: the chest at the foot of the bed (for a magickal trap and a scroll of disperse magick) and the dresser beside the bed (for Liam Cameron's Journal). The journal tells of a portal that has been releasing strange creatures into the area, and Liam's attempt to destroy it. Hang onto the journal so you can return it to Mrs. Cameron.3 - PathFollow the path away from Liam's Workshop. You'll have to fight some void creatures along the way, but eventually you'll stumble upon Liam's body. You'll find another magickal trap if you loot it. So obviously Liam didn't close the portal.West of Liam's body you'll find the portal. If you had an easy time with the void creatures along the path, then you might want to hang around the portal for a while killing void creatures (the portal will keep producing them).When you want to destroy the portal, there are two ways to do it. If you have a technological aptitude then you should use t...

Loss: GISTEmbedLoss with these parameters:
json
1{ 2 "guide": "SentenceTransformer('BAAI/bge-m3')", 3 "temperature": 0.01, 4 "margin_strategy": "absolute", 5 "margin": 0.0, 6 "contrast_anchors": true, 7 "contrast_positives": true, 8 "gather_across_devices": false 9}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 4
per_device_eval_batch_size: 4
gradient_accumulation_steps: 4
warmup_steps: 0.1
fp16: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 4
per_device_eval_batch_size: 4
gradient_accumulation_steps: 4
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: None
warmup_steps: 0.1
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
enable_jit_checkpoint: False
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
use_cpu: False
seed: 42
data_seed: None
bf16: False
fp16: True
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: -1
ddp_backend: None
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_for_metrics: []
eval_do_concat_batches: True
auto_find_batch_size: False
full_determinism: False
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
use_cache: False
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	arcanum-test_cosine_ndcg@10
0.2899	10	0.2124	-
0.5797	20	0.2401	-
0.8696	30	0.3161	-
1.1449	40	0.1497	-
1.4348	50	0.0967	-
1.7246	60	0.1864	-
2.0	70	0.1121	-
2.2899	80	0.0535	-
2.5797	90	0.0365	-
2.8696	100	0.0068	0.8099

Training Time

Training: 11.4 minutes

Framework Versions

Python: 3.12.13
Sentence Transformers: 5.4.0
Transformers: 5.0.0
PyTorch: 2.10.0+cu128
Accelerate: 1.13.0
Datasets: 4.0.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

GISTEmbedLoss

bibtex

@misc{solatorio2024gistembed,
    title={GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning},
    author={Aivin V. Solatorio},
    year={2024},
    eprint={2402.16829},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

	anchor	positive
type	string	string
details	min: 5 tokens mean: 9.99 tokens max: 23 tokens	min: 6 tokens mean: 171.09 tokens max: 1037 tokens