ConvMemory v2 Evidence Reranker
This is the ConvMemory v0.5.0 protected top-10 token-evidence reranker checkpoint.
It is intended to be attached to the base ConvMemory LoCoMo/MPNet checkpoint:
Purdy0228/ConvMemory-LoCoMo-MPNet.
Usage
1from convmemory import ConvMemory
2
3model = ConvMemory.from_pretrained("Purdy0228/ConvMemory-LoCoMo-MPNet")
4model.load_evidence_reranker("Purdy0228/ConvMemory-v2-Evidence-Reranker")
5
6ranked = model.retrieve(
7 query=query,
8 memories=memories,
9 evidence_reranker="v2",
10 top_k=10,
11)
The repository layout is compatible with convmemory.EvidenceReranker.from_pretrained:
evidence_reranker_config.jsonMANIFEST.jsoncross_encoder/ SentenceTransformers CrossEncoder checkpoint
What It Does
ConvMemory v2 preserves the exact ConvMemory v1 top-10 candidate set and only
reorders that protected prefix using token-level query/memory evidence. It cannot
recover a gold memory that v1 failed to retrieve into top-10.
Training
- Source experiment:
experiments/v361_top10_evidence_reranker.py
- Seed: 7
- Base model:
cross-encoder/ms-marco-MiniLM-L-6-v2
- Training target: gold-only listwise retrieval cross-entropy
- Teacher weight: 0.0
- Candidate pool: ConvMemory v1 top-10 from dense MPNet top-500
Headline Evaluation
Canonical v361 5-seed headline, reported in the ConvMemory repository:
- ConvMemory v1 FULL MRR: 0.5824
- ConvMemory v2 FULL MRR: 0.6560
- Delta: +0.0734, paired bootstrap 95% CI [+0.0645, +0.0827]
The v364 load-bearing ablation retrained the same full-text arm in an ablation
harness and obtained FULL MRR 0.6677. Text perturbations collapsed:
- no memory text: 0.2966
- random other-query text: 0.2506
- shuffled memory text: 0.2731
- scalar only: 0.5792
Anti-Leak Discipline
The public inference API rejects gold-defining or post-hoc fields such as:
gold, gold_ids, is_current, is_latest, is_stale, stale, answer,
answer_text, ce_score, mxbai_score, teacher_score, gpt_label,
entity_id, and slot_id.
Inference inputs are query text, candidate id/text, optional candidate position
or time metadata, and the protected v1 top-10 candidate set.
Limitations
- LoCoMo-specific fine-tuning; validate or retrain before using cross-domain.
- Recall-preserving over v1 top-10, not a replacement for candidate generation.
- Not a full top-500 cross-encoder. It is a bounded precision stage after v1.
