This is a sentence-transformers model finetuned from hkunlp/instructor-xl. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: hkunlp/instructor-xl
Maximum Sequence Length: 128 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False, 'architecture': 'T5EncoderModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': False})
  (2): Dense({'in_features': 1024, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ahmedHamdi/narrative-similarity-pt-en-instructor-xl")
# Run inference
sentences = [
    'Represent the plot: One year after the events of the previous film, Eggsy has officially joined Kingsman, taken the title of Galahad from his late mentor Harry Hart, and begun a relationship with Tilde, the Crown Princess of Sweden. On his way home, he is ambushed by Charlie Hesketh, a former Kingsman candidate rejected by the previous film who lost his arm and vocal cords (but was inadvertently saved by Eggsy). Eggsy escapes Charlie and his henchmen in a car chase through London, but Charlie\'s severed cybernetic arm hacks into Kingsman\'s servers via the computer in Eggsy\'s car. With the information obtained, Poppy Adams, leader of the world\'s largest drug cartel, the Golden Circle, launches missiles that destroy Kingsman headquarters and all facilities in Great Britain, killing all agents except Eggsy and Merlin. Eggsy and Merlin follow an emergency Doomsday protocol that leads them to Statesman, the American version of Kingsman, posing as a..." A distillery based in Kentucky. There, they discover that Harry survived Valentine\'s shooting thanks to Statesman technology, but has amnesia. Statesman leader Champagne offers support to take down the Golden Circle. Statesman agent Tequila later develops a blue rash and shows signs of mania and is replaced by another agent, Whiskey, as Eggsy\'s partner. Eggsy plants a tracking device on Charlie\'s ex-girlfriend, Clara Von Gluckfberg, during a sexual encounter at the Glastonbury Festival, but when he tells Princess Tilde beforehand what he has to do, she ends their relationship. Later, Eggsy manages to cure Harry\'s amnesia by threatening to shoot a Cairn Terrier puppy that resembles Harry\'s deceased dog. Poppy broadcasts a message announcing that she has added an artificial virus to all her drugs, which causes users to develop symptoms like Tequila\'s, before progressing to paralysis and finally death. She offers the antidote to the world if... The President of the United States ends up legalizing drugs and granting immunity to the cartel. The president negotiates publicly, but secretly decides to let all those affected die, as he is more than happy to get rid of all drug users and put Poppy out of business. Eggsy, Harry, and Whiskey accompany Clara to a Golden Circle facility in Italy. Eggsy manages to steal a sample of the antidote, but is broken by Whiskey during an attack by Golden Circle henchmen, leading Harry to suspect he was working for the enemy. Harry shoots Whiskey in the head, but Eggsy, believing Harry is delirious due to incomplete recovery, saves him with the same technology Statesman used to save Harry. A delirious Princess Tilde calls out to Eggsy, showing that she has the blue rash. Eggsy, Harry, and Merlin discover the location of Poppy\'s hideout in Cambodia, Poppy Land, and fly there. As they approach Poppy\'s hideout, Eggsy steps on a mine, But Merlin sacrifices himself to save him and Harry. Eggsy and Harry break into the hideout; Eggsy kills Charlie, while Harry destroys Poppy\'s robotic dogs with the help of Elton John, who had been kidnapped by Poppy. They retrieve the laptop controlling the drones that are supposed to deliver the antidote worldwide and inject a concentrated dose of contaminated heroin into Poppy. Poppy gives them the laptop\'s password for the antidote, but she overdoses and dies. Whiskey interrupts them before they can enter the password, revealing that he wants all drug users dead after his pregnant wife was killed in a robbery committed by two methamphetamine users. Eggsy and Harry fight and kill Whiskey, then activate the drones. In the aftermath, the president is deposed for wanting to let the victims die, and Statesman buys a distillery in Scotland to help rebuild Kingsman. Statesman\'s tech support specialist, Ginger Ale, is appointed to the position previously held by Whiskey, Eggsy marries Princess Tilde, and Tequila joins the Kingsman, who acquire a new tailor shop in London.',
    "Represent the plot:  One year after defeating Richmond Valentine, Gary Eggsy Unwin has officially joined the Kingsman intelligence service, taking his late mentor Harry Hart's title of Galahad, and began a relationship with Tilde, Crown Princess of Sweden. He is ambushed by Charlie Hesketh, a former Kingsman trainee who lost his arm and vocal cords during the Valentine incident, replacing them with cybernetics. Eggsy evades Charlie and his henchmen in a car chase across London, but Charlie's severed cybernetic arm hacks into the Kingsman database through Eggsy's car. Charlie passes the information to Poppy Adams – leader of the world's largest drug cartel, the Golden Circle – who launches a volley of missiles that destroy the Kingsman headquarters and wipe out all its agents in Britain, aside from Eggsy and Merlin, Kingsman's tech support. Kingsman's emergency protocol leads Eggsy and Merlin to Statesman, an American private intelligence service posing as a Bourbon whiskey distillery in Kentucky, as well as Kingsman's American counterpart. There, they discover that Harry survived his shooting thanks to Statesman nanotechnology, but is suffering from amnesia. They also learn about the Golden Circle from Statesman leader Champagne, and the two agencies agree to work together to bring the cartel down. Statesman agent Tequila later develops a blue rash and shows signs of mania, and is replaced by agent Whiskey as Eggsy's partner. Eggsy plants a tracking device on Charlie's ex-girlfriend Clara Von Gluckfberg during a sexual encounter at Glastonbury Festival, but his revelation of the mission to Tilde strains their relationship. Eggsy cures Harry's amnesia by threatening to shoot a Cairn Terrier puppy that resembles Harry's late dog, shortly before Poppy broadcasts a message to the world about a toxin she has secretly added to all her drugs, which causes users to develop symptoms like Tequila's before paralysis and, ultimately, death. She offers the antidote if the President of the United States ends his country's war on drugs and grants her cartel immunity from prosecution. The President decides to quarantine everyone affected, intending to let them all die to put Poppy out of business and rid the world of drug users, whom he sees as criminals. Eggsy, Harry, and Whiskey track Clara to the Golden Circle's antidote factory in the mountains of Italy. Eggsy steals an antidote sample, which is broken by Whiskey during an ambush by the Golden Circle's henchmen, leading Harry to suspect him of working against them. Harry shoots Whiskey in the head, but Eggsy, believing Harry to be delusional as a result of his recovery, saves him with nanotechnology as Charlie destroys the antidote factory, killing Clara for unwittingly betraying the Golden Circle. Tilde calls Eggsy in a manic state, revealing she has been affected by the blue rash after taking drugs due to her stress over Eggsy's infidelity. Eggsy, Harry, and Merlin discover the location of Poppy's hideout, Poppy Land, in Cambodia and fly there to steal the laptop which can deploy the drones carrying the antidote. Arriving at Poppy Land, Eggsy steps on a land mine, but Merlin switches places with Eggsy and sacrifices himself instead. Storming the lair, Eggsy kills Charlie while Harry destroys Poppy's robotic attack dogs with the help of Elton John, who had been kidnapped by Poppy. They secure the laptop and inject Poppy with a concentrated dose of her own toxin-infused heroin. Poppy gives them the laptop password in a drugged state but dies from an overdose due to the heroin. Whiskey interrupts them before they can deploy the drones, revealing that he wants all drug users to die after his pregnant wife was killed in a robbery by meth addicts. Eggsy and Harry fight and kill Whiskey by pushing him headfirst into an industrial shredder, then deploy the drones to release the antidote, saving the affected victims throughout the world. The President is impeached for conspiring to commit genocide against drug victims, and Statesman purchases a distillery in Scotland to help rebuild Kingsman. Statesman tech support specialist Ginger Ale is appointed to Whiskey's position, Eggsy marries Princess Tilde, and Tequila takes an exchange assignment at Kingsman, which acquires a new tailor shop in London.",
    "Represent the plot: The film is based on the true story of Barbara Daly Baekeland (Moore), her husband Brooks Baekeland (Dillane), heir to the Bakelite plastics fortune, and their only child Antony (Redmayne), who was diagnosed with schizophrenia. The story begins with Antony's birth and follows the family to the time of his arrest for the murder of his mother.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.8590, -0.1198],
#         [ 0.8590,  1.0000, -0.1240],
#         [-0.1198, -0.1240,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 7,104 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1
type string string
details
min: 14 tokens
mean: 101.58 tokens
max: 128 tokens
min: 7 tokens
mean: 125.37 tokens
max: 128 tokens

Samples:

sentence_0	sentence_1
`Represent the plot: During World War II, Barry Kane, an aircraft factory worker, is accused of starting a fire that kills his friend. Kane becomes a fugitive and tries to uncover the truth, but ends up entangled in situations surrounded by enemies and traitors.`	Represent the plot: Barry Kane is falsely accused of torching Stewart Aircraft Works in Glendale, California, an act of sabotage that incinerates one of his co-workers. Barry believes the culprit is a man named Fry, but investigators find no such name on the employee list. Thus Barry, under suspicion, becomes the target of a manhunt. While eluding capture, he remembers Fry's address from an envelope, so he thumbs a truck ride to a huge ranch in the High Desert. While there, he learns the whereabouts of Fry. He also discovers the ranch's owner, Charles Tobin, is collaborating with Fry and other saboteurs. Barry escapes the ranch, later taking refuge with a blind man whose niece, Patricia Martin, attempts to betray him to police. Barry insists he is innocent and kidnaps Patricia. This leads to a series of adventures that take the couple from one end of the country to the other. Barry and Patricia stow away on a circus caravan, whose members conceal the pair from authorities. Eventually t...
`Represent the plot: Roy Miller is a secret agent who accidentally meets June Havens at a Boston airport. She's unlucky in love, and he ends up involving her in a mission to protect a powerful battery, the key to an infinite source of energy.`	Represent the plot: Returning home to Boston from buying car parts in Wichita, June Havens (Cameron Diaz) collides with Roy Miller (Tom Cruise) at the airport, twice, and is bumped to a later flight. CIA Agent John Fitzgerald (Peter Sarsgaard), believing she works with Roy, puts her back on the plane. While she is in the plane restroom, Roy subdues the passengers and flight crew, agents sent by Fitzgerald, and crash-lands the plane in a cornfield. Drugging the shocked June, he warns that agents will come after her. Waking at home, June prepares for her sister April's (Maggie Grace) wedding, and learns she wants to sell their father's 1966 Pontiac GTO tri-power, which June had planned to restore as a wedding gift. She is picked up by Fitzgerald and his agents. Roy appears, kills numerous agents in a highway chase and rescues her. She flees to her ex-boyfriend Rodney (Marc Blucas), a firefighter, before Roy arrives and pretends to take June hostage, shooting Rodney in the process. Roy co...
`Represent the plot: The story takes place at Bunker Hill, a fictional military academy with a long tradition of training school-aged boys. Knowing that the school is about to close, Cadet Major Brian Moreland leads his classmates in an armed rebellion against the construction of a private housing development on the site.`	Represent the plot: Cadet Brian Moreland meets privately with Bunker Hill Military Academy commander General Harlan Bache, who promotes him to Cadet Major, the highest cadet rank. The next day Bache announces that the school's board of trustees will sell the school to real estate developers but that they will remain open for one more year to allow the seniors to graduate and the underclassmen to find other schools. Bache and the cadets hope this will be a chance to save the school. A dance is held at the academy after commencement. Local teenagers outside the gates harass some cadets and a brawl breaks out. When Bache attempts to end the fight, his service pistol is seized by one of the local boys and the weapon discharges, killing one of the local boys. Although the magazine was removed, a round was still in the chamber. Bache is held responsible and after he is arrested has a heart attack which leaves him in critical condition at the hospital. The board decides to close the school im...

Loss: MultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false 5}

Training Hyperparameters

Non-Default Hyperparameters

multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 8
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.5631	500	0.0717
1.1261	1000	0.0224
1.6892	1500	0.0157
2.2523	2000	0.0079
2.8153	2500	0.0044

Framework Versions

Python: 3.9.21
Sentence Transformers: 5.1.2
Transformers: 4.57.6
PyTorch: 2.8.0+cu128
Accelerate: 1.10.1
Datasets: 4.5.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

bibtex

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

	sentence_0	sentence_1
type	string	string
details	min: 14 tokens mean: 101.58 tokens max: 128 tokens	min: 7 tokens mean: 125.37 tokens max: 128 tokens