This repository contains evaluation code for the "Contextualized Embeddings" project.
pip install -e .
pip install git+
https://github.com/jina-ai/late-chunking --no-deps # for late-chunking with jina
Refer to scripts/evaluation.py for an example of how to use the code.
from datasets import load_dataset
from cde_benchmark.embedders.sentence_transformer_embedder import SentenceTransformerEmbedder
from… See the full description on the dataset page:
https://huggingface.co/datasets/vidore/baselines-v2.
