RQA is a judge model designed to evaluate the quality of reasoning in text.
It does not generate, rewrite, or explain content — instead, it assesses whether a text contains logical problems, and if so, what kind.

RQA is a judge, not a teacher and not a generator.

🔍 What Problem Does RQA Solve?

Texts written by humans or LLMs can:

sound coherent,
use correct vocabulary,
appear persuasive,

…but still contain logical problems that are:

implicit,
structural,
hidden in argumentation.

RQA focuses strictly on reasoning quality, not on style, sentiment, or factual correctness.

🧩 Model Overview

Property	Value
Model Type	Judge / Evaluator
Base Encoder	XLM-RoBERTa Large
Pooling	Mean pooling
Heads	2 (binary + multi-label)
Language	Russian 🇷🇺
License	MIT

🧠 What the Model Predicts

RQA produces two independent signals that are combined at inference time:

1️⃣ Logical Issue Detection (Binary)

has_issue ∈ {false, true}
Calibrated probability available
Designed to answer:
“Does this text contain a reasoning problem?”

2️⃣ Error Type Signals (Multi-label)

The model estimates probabilities for specific error types:

false_causality
unsupported_claim
overgeneralization
missing_premise
contradiction
circular_reasoning

⚠️ Important
Error type probabilities are diagnostic signals, not mandatory labels.
They are surfaced only if has_issue == true during inference.

🟡 Hidden Logical Problems (Key Concept)

RQA explicitly distinguishes between:

🔴 Explicit Logical Errors

Clearly identifiable fallacies:

invalid causal inference
circular reasoning
contradictions
unsupported claims

🟡 Hidden Logical Problems

Texts that are:

argumentative or persuasive,
structurally incomplete,
reliant on implicit assumptions,

but do not contain a cleanly classifiable fallacy.

Examples:

missing or unstated premises
rhetorical generalizations
context-dependent claims

Hidden problems are not misclassifications —
they are an intended diagnostic category.

⚖️ Inference Logic (Important)

The model uses decision logic on top of raw logits:

Binary head decides whether a problem exists
Error heads provide type-level evidence
If:
- has_issue == false
- but error probabilities are non-zero
  → the text may be flagged as borderline or hidden problem

This prevents:

false positive error labels,
incoherent outputs,
over-triggering on clean factual texts.

🏗️ Architecture Details

Encoder: XLM-RoBERTa Large (pretrained weights preserved)
Pooling: Mean pooling (robust for long texts)
Two independent projections:
- binary reasoning head
- multi-label error head
Separate dropout and projections to reduce negative transfer

🎓 Training Philosophy

🔒 Strict Data Contract

Logical texts contain no errors
Hidden-problem texts contain no explicit fallacies
Invalid samples are removed, not auto-corrected

⚖️ Balanced Difficulty

Hidden problems ≤ 30% of problematic texts
Prevents collapse into vague uncertainty detection

🎯 Loss Design

Binary BCE for issue detection
Masked multi-label loss for error types
Stability-oriented multi-task optimization

🌡️ Confidence Calibration

RQA applies post-hoc temperature scaling:

Separate calibration for:
- has_issue
- each error type
Enables:
- meaningful probabilities
- safe threshold tuning
- production use without retraining

🚀 Intended Use

✅ Recommended for:

Reasoning quality evaluation
LLM output auditing
AI safety pipelines
Argumentation analysis
Pre-filtering / routing systems

❌ Not intended for:

Text generation
Error correction
Explanation or tutoring
Grammar or style analysis
Fact checking

🧪 Model Behavior

Conservative by design
Optimized for low false positives
Explicitly robust to:
- topic changes
- writing style
- emotional tone

RQA judges logical structure, not persuasion quality.

📚 Training Data (High-level)

Custom-built dataset
Thousands of long-form argumentative texts
Multiple domains and reasoning styles
Carefully controlled balance of:
- logical texts
- explicit errors
- hidden problems

The dataset was designed specifically for judge behavior, not for text generation.

⚠️ Limitations

Logical validity ≠ factual correctness
Purely descriptive texts may still trigger diagnostic signals
Highly rhetorical or persuasive texts can be flagged as hidden problems
Philosophical disagreement is not always a logical error

🧩 Philosophy

Good reasoning is not about sounding convincing —
it is about what actually follows from what.

RQA is built around this principle.

🔧 Implementation Details

Custom Hugging Face architecture (modeling_rqa.py)
Requires:
- trust_remote_code=True
Uses safetensors
No .bin weights (this is expected behavior)

🚀 Quick Start

python

import torch
from transformers import AutoTokenizer, AutoModel

device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = AutoTokenizer.from_pretrained(
    "skatzR/RQA-R1",
    trust_remote_code=True
)

model = AutoModel.from_pretrained(
    "skatzR/RQA-R1",
    trust_remote_code=True
).to(device)

model.eval()

🧠 Reference Inference Logic

RQA is designed to be used with explicit post-processing logic, including:

temperature scaling
thresholding
disagreement diagnostics
hidden-problem detection

A fully working reference implementation is provided here:

👉 📄 inference.py — Reference Inference Implementation

👉 🤖Try it live — Instant logical error detection in your browser. Powered by Streamlit Cloud.

✅ Example

📄 Текст:
После того как в городе открыли новый торговый центр, увеличилось количество разводов. 
Следовательно, открытие торгового центра разрушает семьи.

🔎 Обнаружена проблема: ДА (100.00%)

❌ Явные логические ошибки:
  • Ложная причинно-следственная связь — 95.95%

📊 Disagreement: 0.034

📜 License

MIT