Views
No views yet
text-embedding-ada-002 embeddings
(dim 1539). Decides per chunk whether it belongs in the answer set —
a variable-size selection instead of a fixed top-k.
| bench | groups | P | R | F1 |
|---|---|---|---|---|
| gold_easy | 9 | 1.0 | 0.3793 | 0.55 |
| gold_medium | 2 | 1.0 | 0.9474 | 0.973 |
| gold_hard | 5 | 0.9886 | 0.9667 | 0.9775 |
| llm_held_out | 29 | 0.6735 | 0.4125 | 0.5116 |
gold_* tiers are hand-curated deterministic labels (easy = section how-to,
medium = single-doc, hard = table/matrix incl. reverse lookups); rephrasing
variants of those questions are in train, so they measure learned question
types. llm_held_out is strict generalization on unseen questions.1{
2 "train_groups": 1184,
3 "gold_train_groups": 48,
4 "test_groups": 45,
5 "gold_test_groups": 16,
6 "train_candidates": 27544,
7 "train_pass": 5567,
8 "train_fail": 21977,
9 "pass_ratio": 0.202,
10 "avg_candidates_per_train_group": 23.3,
11 "avg_pass_per_train_group": 4.7
12}