Is this your dataset? Claim it with the Hugging Face account that owns it.

CohereLabs/
fineweb-edu-100B-index

Name: fineweb-edu-100B-index
Creator: CohereLabs
License: https://spdx.org/licenses/CC-BY-NC-SA-4.0.html

cc-by-nc-sa-4.0

Views

No views yet

This dataset contains the embeddings for the 100B variant of fineweb-edu, embedded with the Cohere Embed V3 model. You can search on this dataset with just 500MB of memory using DiskVectorIndex.

Installation & Usage

Get your free Cohere API key from cohere.com. You must set this API key as an environment variable: export COHERE_API_KEY=your_api_key

Install the package: pip install DiskVectorIndex

You can then search via: from DiskVectorIndex import DiskVectorIndex

index =… See the full description on the dataset page: https://huggingface.co/datasets/CohereLabs/fineweb-edu-100B-index.