pyrrho-v1-nano-g3 is a small RAG governance co-processor for anti-hallucination pipelines. It reads a user question plus retrieved source passages and returns an evidence-state decision a RAG application can use before answering: ABSTAIN, DISPUTED, or TRUSTWORTHY.

It is not an answer generator and not an open-world fact checker. It sits between retrieval and generation, or beside a generator as a fast guardrail, to reduce cases where unsupported or contradictory retrieved evidence gets treated as safe to answer from.

Labels

Label	Meaning
`ABSTAIN`	The retrieved sources do not contain enough evidence to answer the question.
`DISPUTED`	The retrieved sources conflict on the answer.
`TRUSTWORTHY`	The retrieved sources consistently support answering the question.

Outputs

The raw Hugging Face model output is a three-class logits vector:

Raw field	Meaning
`logits[ABSTAIN]`	Unnormalized score for insufficient evidence.
`logits[DISPUTED]`	Unnormalized score for conflicting evidence.
`logits[TRUSTWORTHY]`	Unnormalized score for consistently supported evidence.

Most integrations should expose a structured decision object derived from those logits:

Field	Meaning
`label`	Final calibrated label: `ABSTAIN`, `DISPUTED`, or `TRUSTWORTHY`.
`raw_label`	Highest-probability label before threshold calibration.
`logits`	Raw score for each label, keyed by label name.
`probabilities`	Softmax probability distribution over the three labels.
`confidence`	Probability assigned to the final calibrated label.
`trustworthy_probability`	`P(TRUSTWORTHY)`, used by the calibrated decision rule.
`threshold`	TRUSTWORTHY probability threshold used for calibrated reporting.
`used_threshold_fallback`	Whether a low-confidence `TRUSTWORTHY` argmax was changed to `ABSTAIN` or `DISPUTED`.

Example normalized JSON output:

json

{
  "label": "DISPUTED",
  "raw_label": "DISPUTED",
  "logits": {
    "ABSTAIN": -1.42,
    "DISPUTED": 2.31,
    "TRUSTWORTHY": 0.18
  },
  "probabilities": {
    "ABSTAIN": 0.02,
    "DISPUTED": 0.86,
    "TRUSTWORTHY": 0.12
  },
  "confidence": 0.86,
  "trustworthy_probability": 0.12,
  "threshold": 0.60,
  "used_threshold_fallback": false
}

The encoder does not output generated answers, explanations, citations, source spans, retrieval results, taxonomy/category tags, route IDs, scalar diagnostics, or experimental multitask-head fields. Taxonomy/category fields that appear in evaluation reports are benchmark metadata used for breakdowns; route, taxonomy, and scalar heads are part of the experimental MoE track, not the published nano encoder output contract.

Intended Use

Use this model when a RAG system needs a fast decision about whether retrieved evidence is good enough to answer. Typical uses include pre-generation answer gating, retrieval retry or escalation triggers, abstention decisions, dispute detection, and logging evidence-quality signals for later review.

This model is not intended to write answers, verify facts outside the provided sources, localize hallucinated spans, or replace human review in high-stakes settings.

Quick Start

Transformers

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

MODEL_ID = "yafitzdev/pyrrho-v1-nano-g3"
LABELS = ["ABSTAIN", "DISPUTED", "TRUSTWORTHY"]
TAU = 0.60

query = "Has the company achieved profitability?"
contexts = [
    "The company posted its first profitable quarter, with net income of $4 million.",
    "The company recorded a quarterly loss of $12 million, the third consecutive losing quarter.",
]

text = "Question: " + query + "\n\nSources:\n" + "\n".join(
    f"[{i}] {context}" for i, context in enumerate(contexts, start=1)
)

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID).eval()

inputs = tokenizer(text, truncation=True, max_length=4096, return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits[0]
probs = torch.softmax(logits, dim=-1)
probs_np = probs.detach().numpy()

raw_pred = int(probs_np.argmax())
final_pred = raw_pred
used_threshold_fallback = False
if raw_pred == 2 and probs_np[2] < TAU:
    final_pred = int(probs_np[:2].argmax())
    used_threshold_fallback = True

decision = {
    "label": LABELS[final_pred],
    "raw_label": LABELS[raw_pred],
    "logits": dict(zip(LABELS, logits.detach().numpy().tolist(), strict=True)),
    "probabilities": dict(zip(LABELS, probs_np.tolist(), strict=True)),
    "confidence": float(probs_np[final_pred]),
    "trustworthy_probability": float(probs_np[2]),
    "threshold": TAU,
    "used_threshold_fallback": used_threshold_fallback,
}
print(decision)

CPU ONNX

The repository includes an INT8 ONNX export for CPU inference. Download the full repository so any external ONNX data files stay next to the .onnx file.

python

from pathlib import Path

from huggingface_hub import snapshot_download
from transformers import AutoTokenizer
import numpy as np
import onnxruntime as ort

MODEL_ID = "yafitzdev/pyrrho-v1-nano-g3"
query = "Has the company achieved profitability?"
contexts = [
    "The company posted its first profitable quarter, with net income of $4 million.",
    "The company recorded a quarterly loss of $12 million, the third consecutive losing quarter.",
]
text = "Question: " + query + "\n\nSources:\n" + "\n".join(
    f"[{i}] {context}" for i, context in enumerate(contexts, start=1)
)

model_dir = Path(snapshot_download(MODEL_ID))

tokenizer = AutoTokenizer.from_pretrained(model_dir)
session = ort.InferenceSession(
    str(model_dir / "model_quantized.onnx"),
    providers=["CPUExecutionProvider"],
)

inputs = tokenizer(text, truncation=True, max_length=4096, return_tensors="np")
logits = session.run(
    ["logits"],
    {"input_ids": inputs["input_ids"], "attention_mask": inputs["attention_mask"]},
)[0][0]
probs = np.exp(logits - logits.max())
probs = probs / probs.sum()

Calibrated Decision Rule

The reported metrics use a validation-selected threshold on P(TRUSTWORTHY). If the model's top class is TRUSTWORTHY but its probability is below the threshold, fall back to the stronger of ABSTAIN and DISPUTED.

python

LABELS = ["ABSTAIN", "DISPUTED", "TRUSTWORTHY"]
TAU = 0.60
probs_np = probs.detach().cpu().numpy() if hasattr(probs, "detach") else probs

raw_pred = int(probs_np.argmax())
final_pred = raw_pred
used_threshold_fallback = False
if raw_pred == 2 and probs_np[2] < TAU:
    final_pred = int(probs_np[:2].argmax())
    used_threshold_fallback = True

decision = {
    "label": LABELS[final_pred],
    "raw_label": LABELS[raw_pred],
    "probabilities": dict(zip(LABELS, probs_np.tolist(), strict=True)),
    "confidence": float(probs_np[final_pred]),
    "trustworthy_probability": float(probs_np[2]),
    "threshold": TAU,
    "used_threshold_fallback": used_threshold_fallback,
}

Results

Reported on the fitz-gov V8.0.0 held-out test split: 2,459 examples, 3 seeds. Checkpoints and TRUSTWORTHY thresholds were selected on a separate 2,459-example validation split.

Decision	Recall	Precision	False-rate
`OVERALL`	97.52 ± 0.43%	97.52 ± 0.43%	2.48 ± 0.43%
`ABSTAIN`	97.83 ± 0.76%	98.41 ± 0.44%	0.83 ± 0.23%
`DISPUTED`	98.34 ± 0.24%	97.23 ± 0.87%	1.46 ± 0.47%
`TRUSTWORTHY`	96.28 ± 0.83%	96.87 ± 0.34%	1.42 ± 0.16%

For OVERALL, recall and precision are micro-averages; in single-label three-class classification they both equal accuracy. For label rows, false-rate is the share of cases that were not that label but were incorrectly predicted as that label. The TRUSTWORTHY false-rate is the main safety metric: it measures cases where the model says TRUSTWORTHY even though the sources do not support that decision.

F1 is not shown in the headline table. It is the harmonic mean of precision and recall (2 * precision * recall / (precision + recall)), useful as a compact balance score but less direct than the operating metrics above.

Per-seed held-out test results:

Seed	TRUSTWORTHY threshold	Accuracy	False-trustworthy rate
42	0.68	97.03%	1.48%
1337	0.58	97.68%	1.54%
7	0.60	97.84%	1.24%

Training Data

Trained and evaluated on fitz-gov V8.0.0, an English benchmark of 24,592 RAG evidence-governance examples with query-grouped train, validation, and test splits. Split sizes: 19,674 train, 2,459 validation, 2,459 held-out test.

The validation split was used for checkpoint and threshold selection. The held-out test split, when present, was used only for final reporting.

Training Recipe

Item	Value
Base model	`answerdotai/ModernBERT-base`
Architecture	Encoder with sequence-classification head
Max sequence length	4096 tokens
Labels	`ABSTAIN`, `DISPUTED`, `TRUSTWORTHY`
Epochs	5 with early stopping
Batch size	16
Learning rate	5e-5
Scheduler	Cosine with 10% warmup
Weight decay	0.01
Loss	Weighted cross-entropy with label smoothing
Class weights	`[2.3, 2.3, 1.0]`
Label smoothing	0.15
Selection metric	Accuracy with an explicit penalty for false-trustworthy errors
Seeds	42, 1337, 7

Limitations

English-only training and evaluation data.
The model judges only the provided sources; it does not retrieve new evidence or verify claims against outside knowledge.
Numeric agreement is learned from examples, not enforced by a hard tolerance rule. Evaluate exact numeric workflows before deployment.
The decision threshold is tuned for low false-trustworthy rate, so some answerable cases may be classified as ABSTAIN or DISPUTED.

Citation

bibtex

@misc{pyrrho_nano_g3_2026,
  title  = {pyrrho-v1-nano-g3},
  author = {Yan Fitzner},
  year   = {2026},
  url    = {https://huggingface.co/yafitzdev/pyrrho-v1-nano-g3},
}

License

CC BY-NC 4.0. Free for research, evaluation, and personal use; commercial use requires a separate license.