pyrrho-v1-nano-g5.5 is a small multitask RAG governance co-processor for anti-hallucination
and retrieval-quality pipelines. It reads a user question plus retrieved source
passages, then returns a calibrated evidence-state decision and auxiliary signals
that fitz-sage can use before answer generation.
It is not an answer generator and not an open-world fact checker. It sits between
retrieval and generation, or beside a retrieval package as a fast evidence
quality layer. Compared with pyrrho-v1-nano-g5, this package trains the same multitask surface on the official fitz-gov V11.0.0 repair release, adding targeted strict-owner retrieval-planning rows for class obligations, failure-focused cases, and larger retrieval evidence packs.
Governance Labels
Label
Meaning
ABSTAIN
The retrieved sources do not contain enough evidence to answer the question.
DISPUTED
The retrieved sources conflict on the answer.
TRUSTWORTHY
The retrieved sources consistently support answering the question.
Multitask Heads
Head
Labels / values
Intended use
governance
ABSTAIN, DISPUTED, TRUSTWORTHY
Post-retrieval evidence sufficiency and conflict decision.
Query-only hint for the preferred retrieval substrate.
retrieval_obligation
31 V10 obligation labels
Query-only target/closure obligation for corpus-aware retrieval planning.
Outputs
This is a custom multitask package, not a standard single-head
AutoModelForSequenceClassification artifact. The recommended runtime is
pyrrho.multitask_inference.PyrrhoMultiTaskPredictor from the pyrrho repository.
The predictor returns a structured object:
Field
Meaning
governance.final_label
Final calibrated label after the TRUSTWORTHY threshold rule.
governance.raw_label
Highest-probability governance label before threshold calibration.
governance.probabilities
Probability distribution over ABSTAIN, DISPUTED, TRUSTWORTHY.
governance.threshold
TRUSTWORTHY probability threshold used by the package.
The model does not generate answers, citations, source spans, retrieval results,
or natural-language explanations. It classifies and scores the (query, retrieved_contexts) evidence state.
Intended Use
Use this model when a RAG or retrieval package needs fast local signals about:
whether retrieved evidence is enough to answer,
whether retrieved evidence conflicts,
what kind of evidence the query needs before retrieval,
which semantic/domain route the query belongs to,
which fitz-gov support/failure pattern is active,
what retrieval action and gap type the evidence state suggests,
whether retrieval should retry, broaden, or escalate.
This model is not intended to write answers, verify facts outside the provided
sources, replace a retriever, or replace human review in high-stakes settings.
Quick Start
Install the pyrrho package from the repository that contains this runtime, then
load the package with the multitask predictor:
python scripts/package_multitask_encoder.py verify --package-dir models/pyrrho-v1-nano-g5.5 --device cpu
Release Selection
Seed: 1337
TRUSTWORTHY threshold: 0.43
Selection reason: Seed 1337 was selected because it has the lowest held-out false-TRUSTWORTHY rate among the three g5.5 seeds while also having the strongest held-out retrieval-obligation macro F1 and clearing all governance gates.
Held-Out Test Metrics
Metric
Result
Governance accuracy
0.9800
False-TRUSTWORTHY rate
0.0089
Query-contract accuracy
0.8964
Query-contract macro F1
0.8759
Route accuracy
0.9458
Route macro F1
0.9449
Taxonomy accuracy
0.8282
Taxonomy macro F1
0.8256
Scalar MAE
0.0638
Retrieval-action macro F1
0.8844
Gap-type macro F1
0.8635
Answerability-shape macro F1
0.9485
Retrieval-modality macro F1
0.8938
Retrieval-obligation macro F1
0.8698
Three-seed headline from the local release summary:
Metric
Mean +/- std
Governance accuracy
97.98 +/- 0.04%
False-TRUSTWORTHY rate
0.92 +/- 0.05%
Query-contract macro F1
87.68 +/- 0.07%
Route accuracy
94.63 +/- 0.06%
Taxonomy accuracy
82.51 +/- 0.25%
Scalar MAE
0.0637 +/- 0.0001
Retrieval-action macro F1
88.44 +/- 0.10%
Gap-type macro F1
86.27 +/- 0.10%
Answerability-shape macro F1
94.97 +/- 0.14%
Retrieval-modality macro F1
89.18 +/- 0.17%
Retrieval-obligation macro F1
86.38 +/- 0.44%
Training Data
Trained on the published fitz-gov V11.0.0 Hugging Face release with official query-grouped splits. Total prepared rows: 60,883 = 2,980 V6 rows + 7,520 V7 rows + 14,092 V8 rows + 16,163 V9 rows + 12,748 V10 rows + 7,380 V11 rows. Splits are train=48,800 / validation=6,028 / test=6,055. Split assignments come from v11/split_assignments.jsonl at dataset commit 580809e42376d84284043689c702de4c500bca85. The release package records the local training config in
training_config.yaml and detailed metrics in reports/summary.json.
Limitations
This is a governance and routing co-processor, not a generator.
The auxiliary heads are useful signals, not ground-truth explanations.
Query-contract and route predictions are query-only and can be wrong when the
user query is underspecified.
Taxonomy and scalar outputs are trained on fitz-gov labels/signals and should
be treated as decision-support metadata, not universal factual judgments.
The retrieval-obligation head is trained only on rows with a concrete retrieval obligation; rows with retrieval_obligation=none are masked for that head.
Retrieval obligation and retrieval modality are planning heads. Low-confidence fine-grained obligations should be treated as retrieval hints, not hard guarantees.
This package is trained against the official V11 benchmark contract; fitz-sage integration still needs a separate strict-owner benchmark run before declaring a production upgrade.
The license is CC BY-NC 4.0. Commercial use requires a separate license.
