pyrrho-v1-nano-g5 is a small multitask RAG governance co-processor for anti-hallucination
and retrieval-quality pipelines. It reads a user question plus retrieved source
passages, then returns a calibrated evidence-state decision and auxiliary signals
that fitz-sage can use before answer generation.
It is not an answer generator and not an open-world fact checker. It sits between
retrieval and generation, or beside a retrieval package as a fast evidence
quality layer. Compared with pyrrho-v1-nano-g5-alpha, this package expands the V10 retrieval-obligation training block from the initial 5/5/5 set to the full target-10 fitz-gov V10.0.0 set and uses a completed 3-seed run.
Governance Labels
Label
Meaning
ABSTAIN
The retrieved sources do not contain enough evidence to answer the question.
DISPUTED
The retrieved sources conflict on the answer.
TRUSTWORTHY
The retrieved sources consistently support answering the question.
Multitask Heads
Head
Labels / values
Intended use
governance
ABSTAIN, DISPUTED, TRUSTWORTHY
Post-retrieval evidence sufficiency and conflict decision.
Query-only hint for the preferred retrieval substrate.
retrieval_obligation
31 V10 obligation labels
Query-only target/closure obligation for corpus-aware retrieval planning.
Outputs
This is a custom multitask package, not a standard single-head
AutoModelForSequenceClassification artifact. The recommended runtime is
pyrrho.multitask_inference.PyrrhoMultiTaskPredictor from the pyrrho repository.
The predictor returns a structured object:
Field
Meaning
governance.final_label
Final calibrated label after the TRUSTWORTHY threshold rule.
governance.raw_label
Highest-probability governance label before threshold calibration.
governance.probabilities
Probability distribution over ABSTAIN, DISPUTED, TRUSTWORTHY.
governance.threshold
TRUSTWORTHY probability threshold used by the package.
The model does not generate answers, citations, source spans, retrieval results,
or natural-language explanations. It classifies and scores the (query, retrieved_contexts) evidence state.
Intended Use
Use this model when a RAG or retrieval package needs fast local signals about:
whether retrieved evidence is enough to answer,
whether retrieved evidence conflicts,
what kind of evidence the query needs before retrieval,
which semantic/domain route the query belongs to,
which fitz-gov support/failure pattern is active,
what retrieval action and gap type the evidence state suggests,
whether retrieval should retry, broaden, or escalate.
This model is not intended to write answers, verify facts outside the provided
sources, replace a retriever, or replace human review in high-stakes settings.
Quick Start
Install the pyrrho package from the repository that contains this runtime, then
load the package with the multitask predictor:
python scripts/package_multitask_encoder.py verify --package-dir models/pyrrho-v1-nano-g5 --device cpu
Release Selection
Seed: 7
TRUSTWORTHY threshold: 0.34
Selection reason: Seed 7 was selected because it had the strongest held-out retrieval-obligation macro F1 while staying inside the governance release gates.
Held-Out Test Metrics
Metric
Result
Governance accuracy
0.9744
False-TRUSTWORTHY rate
0.0129
Query-contract accuracy
0.8875
Query-contract macro F1
0.8624
Route accuracy
0.9346
Route macro F1
0.9340
Taxonomy accuracy
0.7823
Taxonomy macro F1
0.7929
Scalar MAE
0.0691
Retrieval-action macro F1
0.8680
Gap-type macro F1
0.8333
Answerability-shape macro F1
0.9287
Retrieval-modality macro F1
0.8601
Retrieval-obligation macro F1
0.8313
Three-seed headline from the local release summary:
Metric
Mean +/- std
Governance accuracy
97.39 +/- 0.13%
False-TRUSTWORTHY rate
1.09 +/- 0.25%
Query-contract macro F1
86.27 +/- 0.14%
Route accuracy
93.65 +/- 0.14%
Taxonomy accuracy
78.42 +/- 0.22%
Scalar MAE
0.0689 +/- 0.0002
Retrieval-action macro F1
86.67 +/- 0.18%
Gap-type macro F1
83.47 +/- 0.18%
Answerability-shape macro F1
93.27 +/- 0.32%
Retrieval-modality macro F1
85.98 +/- 0.10%
Retrieval-obligation macro F1
82.48 +/- 0.47%
Training Data
Trained on the fitz-gov V10.0.0 row set: published V9.0.0 plus 12,748 V10 target-10 retrieval-planning rows. The V10 block combines 6,058 initial 5/5/5 rows with 6,690 target-10 continuation rows; after repair, blind-label QA scored 12,748/12,748 agreement with 0 triage, 0 missing, 0 invalid, and 0 error rows. The g5 training prep used local splits train=42,826 / validation=5,372 / test=5,305 over the same 53,503 rows; the public fitz-gov V10.0.0 dataset split manifest is train=42,814 / validation=5,346 / test=5,343. The release package records the local training config in
training_config.yaml and detailed metrics in reports/summary.json.
Limitations
This is a governance and routing co-processor, not a generator.
The auxiliary heads are useful signals, not ground-truth explanations.
Query-contract and route predictions are query-only and can be wrong when the
user query is underspecified.
Taxonomy and scalar outputs are trained on fitz-gov labels/signals and should
be treated as decision-support metadata, not universal factual judgments.
The retrieval-obligation head is trained only on the V10 rows; V6-V9 rows are masked for that head.
Retrieval obligation is the main remaining weak head. Low-support or fine-grained obligations such as row-key lookup, column-value lookup, stale-row versioning, and mixed-modality obligations should be treated as planning hints, not hard guarantees.
The license is CC BY-NC 4.0. Commercial use requires a separate license.
