Views
No views yet
SentenceTransformer(
(0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'NomicBertModel'})
(1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
)pip install -U sentence-transformers1from sentence_transformers import SentenceTransformer
2
3# Download from the 🤗 Hub
4model = SentenceTransformer("MohammedKamal23/nomic_ai_nomic_embed_text_v1_5-matryoshka-genre")
5# Run inference
6sentences = [
7 'In the small, picturesque town of Willow Creek, nestled between the rolling hills and the vast expanse of the prairie, life moved at a slower pace. The people here were close-knit, their lives intertwined through generations of shared memories and traditions. There was a saying in Willow Creek: "In every generation, there is a new beginning."\n\nOur story begins with the arrival of a new family in town, the Johnson family. The Johnsons were a bit different from the rest of the townspeople - they were loud, boisterous, and had an air of sophistication that was quite unusual for the simple folk of Willow Creek. The townspeople, however, were welcoming and eager to make new friends.\n\nThe Johnson family consisted of the patriarch, Mr. Johnson, a retired music producer; his wife, Mrs. Johnson, a former opera singer; their son, Thomas, a young aspiring musician; and their daughter, Emily, a curious and intelligent teenager.\n\nThe townspeople soon learned about the Johnson\'s past. Mr. Johnson, once a renowned music producer, had discovered and nurtured the talents of many up-and-coming musicians. Mrs. Johnson, a gifted opera singer, had performed on the grandest stages of Europe and Asia. Their son, Thomas, was a prodigy, already composing music that was being hailed as revolutionary.\n\nAs the years passed, the Johnson family became more and more ingrained in the community. The townspeople, initially fascinated by the Johnson\'s glamorous past, soon learned to appreciate their talents. Mrs. Johnson would often perform at the local community center, while Mr. Johnson would share his knowledge of music production with the younger generation. Thomas and Emily also became actively involved in the town\'s events, from the annual Harvest Festival to the Winter Carnival.\n\nHowever, there was a subtle undercurrent of tension that began to brew among the townspeople. The younger generation, who had grown up listening to mainstream pop and rock music, found the classical melodies of the Johnson family to be out of touch with the times. The older generation, on the other hand, cherished the music and the memories it evoked.\n\nThis generation gap became increasingly evident during the annual Willow Creek Music Festival. The Johnson family, naturally, would showcase their classical and traditional music, while the younger generation would perform their contemporary hits.\n\nOne day, as the townspeople were gathered at the community center, a new face appeared. He was a tall, thin man with slicked-back hair and an air of mystery. His name was Vincent, and he claimed to be a distant relative of the Johnson family. He had come to Willow Creek to learn the secrets of the Johnson\'s musical legacy and, as he claimed, to reconnect with his roots.\n\nVincent quickly became a central figure in the town, especially among the younger generation. He introduced them to the world of pop and rock music, teaching them the latest dance moves and the hottest songs. The younger generation, eager to learn and please their newfound mentor, quickly abandoned the traditional melodies of the Johnson family in favor of the catchy tunes of the modern world.\n\nAs the years passed, the rift between the generations grew wider. The older generation clung to their classical melodies, while the younger generation reveled in the new sounds of pop and rock. The annual Willow Creek Music Festival became a battleground, with each side determined to prove their superiority in the realm of music.\n\nIt was during one such festival that a twist of fate changed the course of the town\'s history. As the sun set and the stage lights glittered in the twilight, the two sides prepared to face each other in a musical showdown. The older generation, led by Mr. Johnson, stood resolute on one side of the stage, while the younger generation, with Vincent as their leader, stood on the other side.\n\nAs the two sides began their performance, the tension in the air was palpable. The older generation played their classical melodies, each note a testament to their love for tradition. The younger generation, on the other hand, played their contemporary tunes, their energy and enthusiasm infectious.\n\nSuddenly, amidst the cacophony of sounds, a voice boomed through the speakers, interrupting the performance. It was Vincent, his voice filled with an urgency that silenced the crowd.\n\n"Ladies and gentlemen of Willow Creek, I stand before you today to tell you a story. A story of a time when the music of the old and the new came together in perfect harmony."\n\nAs the crowd leaned in to listen, Vincent continued his tale.\n\n"In the time of our ancestors, the people of Willow Creek were united by a single, unifying force - their love for music. The young and the old, the rich and the poor, would gather together to celebrate their shared heritage, their voices blending together in a symphony of sound."\n\nVincent paused, allowing the crowd to absorb his words.\n\n"But as the years went by, the people of Willow Creek began to drift apart. The younger generation, drawn to the allure of the new and the unknown, turned away from the classical melodies of their ancestors. The older generation, clinging to their traditions, refused to embrace the changing tides of music."\n\nAs he spoke, the crowd could feel the weight of the generations\' division in their very souls.\n\n"And so, the people of Willow Creek found themselves trapped in a never-ending cycle of discord, their hearts divided by the very thing that had once united them - their love for music."\n\nVincent\'s voice grew more passionate as he continued his tale.\n\n"But there was a time, my friends, when the people of Willow Creek discovered that they could bridge the generation gap through the power of music. It was during the time when the Johnson family had just arrived in town, when the old and the new, the classical and the contemporary, came together in a beautiful, harmonious blend."\n\nThe crowd listened in rapt attention, their hearts swelling with the hope that Vincent\'s story could be their salvation.\n\n"It was during the first Willow Creek Music Festival, when the Johnson family shared their music with the town. The older generation, captivated by the beauty of the classical melodies, began to embrace the new sounds that their children and grandchildren so cherished. The younger generation, in turn, learned to appreciate the timeless beauty of the music that had once been the lifeblood of their ancestors."\n\nAs Vincent\'s story reached its climax, the crowd could feel the tension in the air beginning to dissipate. The older and younger generations, who had been standing on opposite sides of the stage, slowly began to mingle, their hands reaching out to one another in a gesture of unity.\n\nVincent, his voice now filled with the hope of a brighter future, concluded his tale.\n\n"And so, my friends, it is up to you to find the harmony that once existed in this town. To embrace the music of the old and the new, and to unite as one."\n\nAs the sun dipped below the horizon, casting its golden glow over the town, the people of Willow Creek came together on the stage, their voices blending together in a symphony of sound that had once been lost to the sands of time. The generation gap, which had threatened to tear the town apart, was now a distant memory, replaced by a newfound appreciation for the power of music to bring people together.\n\nAnd as the final notes of the melody faded into the night, the people of Willow Creek knew that they had truly found the harmony that had once been the soul of their town.',
8 "Grandpa sat in his old armchair, the radio buzzing with his favorite jazz tunes. The creaking floorboards under his feet brought him back to the days of his youth. He cherished these moments, lost in the past.\n\n His grandson, on the other hand, sat on the couch with his noise-canceling headphones, listening to the latest electronic music. His world was in vibrant colors, full of technology and innovation.\n\n One day, Grandpa decided to share his old radio with his grandson. Excitedly, the grandson plugged in his headphones into the radio's auxiliary port. The jazz melody filled the room, and the grandson's eyes lit up.\n\n They sat there, side by side, the old radio and the new headphones. Two different generations, connected by music, a shared language that transcended time.",
9 'In the quiet, picturesque town of Willow Creek, nestled between the lush green hills, lived a group of eccentric yet beloved artists. The town was renowned for its beautiful landscapes and the vibrant art scene that flourished there. Among these artists was a talented painter named Vincent, who was known for his incredible ability to capture the essence of nature on canvas.\n\n Vincent\'s latest masterpiece, "The Whispering Willows," was a breathtaking depiction of the town\'s beautiful willow trees that whispered secrets to the wind. The painting had been on display at the local art gallery for weeks, drawing visitors from all over the world. The gallery owner, Mr. Charles, was ecstatic about the attention the painting had garnered and was eager to sell it to the highest bidder.\n\n One fateful evening, just as the sun dipped below the horizon, casting a golden glow on the willow trees lining the banks of the River Willow, the painting mysteriously vanished from its frame. The entire town was abuzz with speculation and whispers of who could have possibly stolen the priceless masterpiece.\n\n The local police department, led by the witty and resourceful Detective Samantha, took up the case. Along with her partner, the methodical and observant Detective Thomas, they began their investigation. The duo was determined to recover the painting and bring the culprit to justice.\n\n As the investigation unfolded, a series of peculiar events began to unfold. A mysterious figure, cloaked in darkness, seemed to be watching the art gallery from a distance. The detectives were intrigued by this shadowy presence and decided to follow the figure one night.\n\n The figure led them to a hidden, underground gallery, where a collection of stolen artworks was displayed. To their astonishment, they discovered that the mastermind behind the art heists was none other than the town\'s beloved artist, Vincent.\n\n Vincent had been struggling to make ends meet, and his desperate attempts to sell his artwork had driven him to steal the works of other artists. He had planned to replace "The Whispering Willows" with a forgery he had painted, which would then be sold to the highest bidder.\n\n The detectives confronted Vincent, who was crestfallen and ashamed of his actions. He tearfully confessed to his crimes and agreed to return the stolen artworks to their rightful owners.\n\n The town of Willow Creek was in an uproar as the news of Vincent\'s misdeeds spread like wildfire. The once-beloved artist was ostracized by the community, and his reputation was in tatters. Vincent, devastated by the loss of his friends and the love he had once received from the townspeople, retreated into solitude, vowing to make amends for his actions.\n\n In an unexpected twist of fate, a wealthy art collector from a distant city heard about the stolen masterpiece and offered to buy "The Whispering Willows" for a record-breaking price. The proceeds from the sale would be used to fund an art restoration project in Willow Creek, reviving the town\'s vibrant art scene.\n\n The townspeople, touched by Vincent\'s remorse and his commitment to making amends, forgave him, and the community rallied behind the restoration project. The town\'s art scene flourished once again, and the people of Willow Creek came to appreciate the value of art and its power to unite them.\n\n The story of Vincent, the talented yet flawed artist, became a cautionary tale for the people of Willow Creek, reminding them of the importance of integrity and the consequences of their actions. The town continued to thrive, and the Mystery of the Misplaced Masterpiece lived on as a cherished memory of the resilience and strength of the human spirit.',
10]
11embeddings = model.encode(sentences)
12print(embeddings.shape)
13# [3, 768]
14
15# Get the similarity scores for the embeddings
16similarities = model.similarity(embeddings, embeddings)
17print(similarities)
18# tensor([[1.0000, 0.6600, 0.1270],
19# [0.6600, 1.0000, 0.1260],
20# [0.1270, 0.1260, 1.0000]])anchor, positive, and negative| anchor | positive | negative | |
|---|---|---|---|
| type | string | string | string |
| modality | text | text | text |
| details |
|
|
|
| anchor | positive | negative |
|---|---|---|
In the year 2234, Earth was no longer inhabited by humans. It had been transformed into a vast, lush garden under the care of the Celestial Council. The Council, made up of twelve powerful beings from across the universe, had decided that Earth needed a chance to heal after centuries of human exploitation.[object Object][object Object] In this new era, Earth became known as the Crystal Realm. It was a place where no human could ever return, but where the essence of human spirit and innovation lived on. The Crystal Realm was a beautiful, glowing sphere, filled with towering crystal spires, and covered in a soft, ethereal mist. Its vibrant energy was a source of wonder and fascination for the inhabitants of the universe.[object Object][object Object] In the heart of the Crystal Realm, there was a massive crystal palace known as the Crystal Citadel. It was the home of the Celestial Council and the center of their power. The Citadel was the only place on Earth where the beings of the Council could manifest their physical forms, as their t... | In a world where magic and science coexist, the Forgotten Realm is a forgotten land. It's a realm that was once a part of a great kingdom that ruled the world. This kingdom was known for its magic, technology, and the prosperity it brought to its people. However, due to a great cataclysm, the Forgotten Realm was sealed away from the rest of the world.[object Object][object Object]In this realm, there lived a young man named Aradon. He was the son of the greatest mage and the greatest scientist of their time. Aradon was known for his intelligence, bravery, and wit. He was destined to become a hero, but little did he know that he was about to embark on a journey that would change his destiny and the fate of the Forgotten Realm.[object Object][object Object]As the story unfolds, Aradon learns about his true heritage. He discovers that his father was the one who sealed the Forgotten Realm away to protect it from a great evil, an entity known as the Shadowlord, who sought to conquer the world and enslave all living beings. The Shadowlord's army,... | Once upon a time, in a small, secluded village nestled deep within the heart of a lush forest, there lived a young woman named Aria. Aria was an extraordinary individual, for she possessed a rare gift: the ability to communicate with the supernatural beings that dwelled within the enchanted realm that lay hidden behind the veil of the forest.[object Object][object Object]Aria's gift had been passed down through generations of women in her family, and she was the last remaining person in the village with this unique ability. The villagers had long forgotten about the enchanted realm, and Aria's gift was regarded as nothing more than an old wives' tale. However, Aria knew the truth and she longed to explore the realm and discover its hidden secrets.[object Object][object Object]One fateful day, while wandering deep into the forest, Aria stumbled upon the entrance to the enchanted realm - a shimmering pool of water, shimmering with a myriad of colors. As she gazed into the water, she noticed that it was not only reflecting the surrounding fore... |
Once upon a time in a small town called "Mundaneville," a mild-mannered man named Mr. Patterson lived a simple life. Mr. Patterson was a man of routine, a man of predictability, a man of… pants. His pants, to be more specific.[object Object][object Object] Mr. Patterson had a peculiar obsession with his pants. He had a collection of 1,001 pairs of pants, each more unique and colorful than the last. Every morning, he would spend hours meticulously selecting the perfect pair of pants to match his shirt and tie, all the while ensuring that they were perfectly aligned with his belt and shoes.[object Object][object Object] One fateful day, as Mr. Patterson was going about his usual routine, he received a mysterious letter in the mail. The envelope was plain and unassuming, with no return address or any indication of its contents. Intrigued, Mr. Patterson opened the letter and read aloud the cryptic message inside:[object Object][object Object] "Dear Mr. Patterson,[object Object][object Object] We have learned of your obsession with pants and have decided to take advantage of your peculiar... | In the bustling city of New York, nestled in the heart of Brooklyn, stood a small soda fountain known as "The Frothy Scoop." This unassuming establishment was the heartbeat of the community, a place where friends gathered, neighbors chatted, and children delighted in the sweet, fizzy treats.[object Object][object Object] At the center of this quaint soda fountain was a group of unlikely heroes: a barber named Sam, an out-of-work actress named Sarah, a struggling musician named Max, and a kind-hearted janitor named Tim. Each of these individuals had their own unique story, and together, they formed an unbreakable bond that would change the course of their lives forever.[object Object][object Object] Sam, the barber, was a tall, lanky man in his early fifties. He was well-known for his uncanny ability to tell the most hilarious stories while cutting hair. Sam had a knack for finding humor in the most mundane situations, and his infectious laughter was contagious. Sam's wife, Sarah, was an out-of-work actress with a flair for the dramati... | In the seedy underbelly of the city of Newhope, nestled between the towering skyscrapers and the crooked alleys, lived a man known as Mr. C. Mr. C was a seasoned private detective with a knack for unraveling the most twisted of mysteries. He was tall, dark, and handsome, with a face that could hide a thousand secrets, and eyes that could pierce through the darkest of lies.[object Object][object Object] One day, Mr. C received a mysterious letter from an anonymous client. The letter read, "Mr. C, I have a job for you. It's a case so puzzling that even your sharp mind may not be able to solve it. But I assure you, the prize is worth your time. Meet me at the old lighthouse by the docks, at midnight."[object Object][object Object] Intrigued by the challenge, Mr. C decided to accept the case. As the clock struck twelve, Mr. C arrived at the old lighthouse, only to find a single lamp flickering in the darkness. He stepped closer, and the lamp went out. The wind began to howl, and the rain started to pour. Suddenly, the lamp flared back to ... |
Once upon a time, in a small town named Elmswood, lived a culinary genius named Albert. He was not just a fantastic cook, but also a kind-hearted and adventurous man who had an unusual passion: the paranormal. Albert's life took a turn when he discovered a peculiar recipe in an old cookbook he had found in an antique shop. This recipe, titled "The Ghostly Gourmet," was said to have the power to summon spirits in the form of delicious dishes.[object Object][object Object]As a self-taught chef, Albert was always looking for new ways to enhance his cooking, so he decided to give this recipe a try. To his surprise, the recipe worked. As soon as he prepared the dish according to the recipe's instructions, the most incredible aroma filled the air, and a spirit appeared. The spirit was that of a charming, witty gentleman named Edgar who was known to be the best chef in the afterlife.[object Object][object Object]Edgar and Albert quickly became friends and started to cook incredible meals together, with Edgar offering his otherworldly culinary skil... | Once upon a time in the small, peaceful town of Willowbrook, a young boy named Timmy lived. Timmy was an ordinary boy who loved to explore the woods behind his house with his best friends, Tommy, and Lucy. The woods were a magical place where they often discovered strange and mystical creatures.[object Object][object Object] One sunny day, while playing in the woods, Timmy found a peculiar, glowing mushroom. He had never seen anything like it before, so he picked it up and took it home. His parents, Mr. and Mrs. Johnson, warned him that touching strange things could be dangerous, but Timmy was too curious to heed their warning.[object Object][object Object] That night, Timmy couldn't resist the temptation to touch the mushroom, and as soon as his fingers made contact, he felt a sudden surge of energy. The next morning, Timmy woke up with the astonishing ability to understand and communicate with animals.[object Object][object Object] Overjoyed with his newfound gift, Timmy shared his secret with Tommy and Lucy. Together, they decided to use their powers for g... | Once upon a time, in the small and picturesque town of Crestwood, there lived an extraordinary man named Albert. Albert was a philosopher, a thinker, and a dreamer who spent his days pondering the great questions of life, such as "What is the meaning of life?" and "How can we find happiness in an imperfect world?" However, despite his intellectual prowess, Albert was far from the typical philosopher. You see, Albert was a dog.[object Object][object Object]Albert's life took an unexpected turn one fateful morning when he stumbled upon a mysterious, old book while exploring the nearby woods. The book, titled "The Philosophical Comedy," claimed to hold the secrets to the ultimate happiness and the key to unlocking the true potential of all living beings. Intrigued and eager to learn, Albert decided to embark on a grand adventure to uncover the truth behind the book's claims and bring the wisdom it contained back to his beloved town.[object Object][object Object]As Albert began his journey, he was accompanied by a motley crew of unlikely heroes... |
MatryoshkaLoss with these parameters:
1{
2 "loss": "MultipleNegativesRankingLoss",
3 "matryoshka_dims": [
4 768,
5 512,
6 256,
7 128
8 ],
9 "matryoshka_weights": [
10 1,
11 1,
12 1,
13 1
14 ],
15 "n_dims_per_step": -1
16}per_device_train_batch_size: 4gradient_accumulation_steps: 2learning_rate: 0.0003warmup_ratio: 0.1fp16: Trueoverwrite_output_dir: Falsedo_predict: Falseprediction_loss_only: Trueper_device_train_batch_size: 4per_device_eval_batch_size: 8per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 2eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 0.0003weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 3max_steps: -1lr_scheduler_type: linearlr_scheduler_kwargs: Nonewarmup_ratio: 0.1warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falsebf16: Falsefp16: Truefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonelocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Falseignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}parallelism_config: Nonedeepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torch_fusedoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthproject: huggingfacetrackio_space_id: trackioddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Nonehub_always_push: Falsehub_revision: Nonegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseinclude_for_metrics: []eval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters:auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: noneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseuse_liger_kernel: Falseliger_kernel_config: Noneeval_use_gather_object: Falseaverage_tokens_across_devices: Trueprompts: Nonebatch_sampler: batch_samplermulti_dataset_batch_sampler: proportionalrouter_mapping: {}learning_rate_mapping: {}1@inproceedings{reimers-2019-sentence-bert,
2 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
3 author = "Reimers, Nils and Gurevych, Iryna",
4 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
5 month = "11",
6 year = "2019",
7 publisher = "Association for Computational Linguistics",
8 url = "https://arxiv.org/abs/1908.10084",
9}1@misc{kusupati2024matryoshka,
2 title={Matryoshka Representation Learning},
3 author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
4 year={2024},
5 eprint={2205.13147},
6 archivePrefix={arXiv},
7 primaryClass={cs.LG}
8}1@misc{oord2019representationlearningcontrastivepredictive,
2 title={Representation Learning with Contrastive Predictive Coding},
3 author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
4 year={2019},
5 eprint={1807.03748},
6 archivePrefix={arXiv},
7 primaryClass={cs.LG},
8 url={https://arxiv.org/abs/1807.03748},
9}