A sparse retrieval encoder that transforms text into interpretable, indexable sparse vectors using BERT's MLM predictions. Trained with a novel two-pass distillation methodology: ranking-heavy first (α=0.7), vocabulary-heavy second (α=0.3, low LR).
FLES-2 v32 produces sparse vectors over a 30,522-dimensional vocabulary space (BERT WordPiece). Each dimension corresponds to a vocabulary term, and the weight indicates how strongly that term is predicted for the input text. The result is a bag-of-expanded-terms representation that can be indexed with standard inverted indices.
1from fles1_encoder import FLES1Encoder
2
3# Load model
4encoder = FLES1Encoder.from_pretrained("mindoval/fles2-v32")
5
6# Encode text to sparse vector
7sparse_vec = encoder.encode("What is machine learning?")
8# Returns: {"machine": 1.82, "learning": 1.65, "artificial": 0.94, "intelligence": 0.87, ...}
9
10# Batch encode
11vectors = encoder.encode_batch(["query 1", "query 2"], batch_size=32)
1@misc{fles2v32,
2 title={FLES-2: Two-Pass Sparse Self-Distillation for Learned Sparse Retrieval},
3 author={Tavarez, Golvis},
4 year={2026},
5 publisher={Mindoval, Inc.},
6 url={https://huggingface.co/mindoval/fles2-v32}
7}
Built by Mindoval, Inc. Training compute provided by Microsoft Corporation (Azure ML, H100 GPUs).
