This repository contains a verifier-guarded financial question answering system built on top of the PIT-4B point-in-time financial language model family.

The project combines:

A merged GRPO-aligned PIT generator
Numeric verification
NLI-based semantic grounding verification
Abstention behavior for unsupported answers

The objective is to improve reliability and reduce hallucinations in financial QA over earnings-call transcripts and financial documents.

Model Overview

Base Model

Diamegs/PIT-4B-FT-202212

Alignment

The model was aligned using:

reasoning supervision
GRPO-style preference optimization
LoRA fine-tuning
hallucination-aware financial QA data

The LoRA adapter was merged into the final model weights.

Final Merged Model

NurErtug/pit-finance-grpo-merged

Intended Use

The system is designed for:

grounded financial QA
earnings-call reasoning
hallucination reduction
numeric consistency
abstention on unsupported questions
verifier-guided inference

Verifier Architecture

The verifier is implemented as an inference-time guardrail wrapper.

The verifier is NOT merged into model weights.

The pipeline operates as follows:

text

User Question
      ↓
Merged PIT Generator
      ↓
Numeric Verifier
      ↓
NLI Verifier
      ↓
Abstention Policy
      ↓
Final Answer

Verification Components

1. Numeric Verifier

Checks whether numeric claims generated by the model are supported by the evidence/context.

Examples:

unsupported dollar amounts
hallucinated percentages
fabricated growth figures

2. NLI Verifier

Uses:

MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli

to verify semantic support between:

evidence/context
generated answer

Possible outcomes:

entailment
neutral
contradiction

3. Abstention Policy

If the verifier determines:

insufficient evidence
unsupported claims
semantic contradiction

the system returns:

NOT ENOUGH INFORMATION.

instead of hallucinating an answer.

Installation

Clone the repository:

bash

git clone https://huggingface.co/NurErtug/pit-finance-grpo-merged
cd pit-finance-grpo-merged

Install dependencies:

pip install -r requirements.txt

Loading the Model

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_name = "NurErtug/pit-finance-grpo-merged"

tokenizer = AutoTokenizer.from_pretrained(
    model_name,
    trust_remote_code=True,
    use_fast=False,
)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    trust_remote_code=True,
    torch_dtype=torch.float16,
    device_map="auto",
)

model.eval()

Using the Verifier Wrapper

python

from verifier_wrapper import VerifierWrapper

wrapper = VerifierWrapper()

context = """
Revenue increased 18% year-over-year.
Alternative minimum tax credits totaled $125 million.
"""

question = "What was the revenue growth?"

result = wrapper.answer(
    context=context,
    question=question,
)

print(result)

Example Output

python

{
    "initial_answer": "Revenue increased 18% year-over-year.",
    "final_answer": "Revenue increased 18% year-over-year.",
    "rejected": False,
    "reward": 1.8
}

Example rejected output:

python

{
    "initial_answer": "$250 million",
    "final_answer": "NOT ENOUGH INFORMATION.",
    "rejected": True,
    "reject_reason": "verifier_failed"
}

Repository Contents

text

README.md
verifier_wrapper.py
requirements.txt
config.json
generation_config.json
model.safetensors
tokenizer files

Research Motivation

Financial QA systems are especially vulnerable to:

hallucinated financial figures
unsupported claims
temporal leakage
fabricated reasoning

This project explores verifier-guided inference as a lightweight alternative to:

expensive RLHF pipelines
larger judge-model architectures
fully integrated verifier training

The verifier wrapper provides an interpretable post-generation filtering mechanism for improving reliability in financial language models.

Limitations

The system may still:

over-abstain
reject partially correct answers
fail under highly ambiguous contexts
struggle with extremely long contexts

The verifier improves robustness but does not guarantee factual correctness.

Citation

bibtex

@misc{pit_finance_verifier_2026,
  title={PIT Finance GRPO + Verifier System},
  author={Nur Ertug},
  year={2026},
  publisher={Hugging Face}
}

License

Apache-2.0