LoRA adapter that adds DPO (Direct Preference Optimization) on top of the earnings-call SFT model jdecim/PIT-4B-FT-202212-earnings-SFT. Base model: Diamegs/PIT-4B-FT-202212.

The SFT stage taught the model to answer earnings-call questions; the DPO stage teaches it to prefer factually-grounded answers over loopy / hallucinated / wrong-number alternatives.

⚠️ PIT compliance

This adapter was retrained on a date-filtered preference set after a PIT leak was discovered in the original DPO pipeline: linked_exchanges.jsonl spans all years (2006–2024) and was being read directly by generate_financial_dpo.py, so future-dated exchanges (2023–2024) were leaking into the DPO training even though the underlying SFT splits were correctly partitioned chronologically.

This release uses only pairs with mostimportantdateutc < 2023-01-01 (3066 rows, 113 future-dated rows dropped from the 3179-row source). Confirmed at training time by an in-script guard that aborts if any training row has a date ≥ 2023-01-01.

Patched scripts: src/dpo_generation/improve_dpo_dataset.py and src/dpo_generation/generate_financial_dpo.py both now honor a --pit-date-horizon flag end-to-end.

Model details

Base model: Diamegs/PIT-4B-FT-202212 (PIT horizon 2022-12)
Adapter chain: Earnings SFT → DPO (this adapter)
Method: DPO via TRL, β=0.1, learning rate 5e-6, 1 epoch, LoRA r=16
Training preferences: jdecim/pit-earnings-call-qa @ 202212/dpo.jsonl (3066 PIT-safe pairs)

Preference-data composition

Tiered "rejected" responses so the model has to learn what's wrong with a candidate, not just whether it loops:

Tier	Rejected source	Approx. share
`easy`	Raw loopy PIT-base generations (kept verbatim)	17% (523)
`medium`	Qwen-shortened loops (1–2 repeats kept)	34% (1048)
`hard`	Qwen-cleaned loops, factual errors preserved	16% (501)
`nur`	Imported from `NurErtug/finance-dpo-pairs-verified`, date-filtered to <2022-01-01	32% (994)

All rejected text is constructed from PIT-safe material — no future knowledge is introduced.

Prompt format

Identical to the SFT model:

<|user|>
Context:
{context}

Question: {question}
<|assistant|>
{answer}<|end|>

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base = "Diamegs/PIT-4B-FT-202212"
adapter = "jdecim/PIT-4B-FT-202212-earnings-SFT-DPO"

tok = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    base, torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)
model.eval()

Evaluation

General NLP benchmarks (0-shot, lm-eval-harness)

Reported as acc_norm where the task defines it, acc otherwise — the standard lm-eval-harness convention.

Task	Base	+ Earnings SFT	+ SFT-DPO (this model, PIT-safe)
HellaSwag (acc_norm)	0.645	0.659	0.686
PIQA (acc_norm)	0.774	0.758	0.770
Winogrande (acc)	0.634	0.618	0.619
BoolQ (acc)	0.609	0.634	0.678
ARC-Easy (acc_norm)	0.618	0.550	0.579
ARC-Challenge (acc_norm)	0.347	0.317	0.339
OpenBookQA (acc_norm)	0.396	0.366	0.394
General avg	0.575	0.557	0.581

DPO recovers and slightly exceeds the base on general benchmarks (58.1% vs 57.5%) while preserving the in-domain earnings-call gains. The PIT-safe retrain matches the prior leaky run within statistical noise (58.1% vs 58.2% avg), confirming the 113 future-dated rows were not driving performance.

Earnings-call benchmark

Per-bucket metrics on jdecim/pit-earnings-call-qa @ 202212/benchmark_synthetic.jsonl (500 forward_synthetic + 500 unanswerable; all rows transcript_date ≥ 2023 → forward eval) and @ 202212/benchmark_1k.jsonl (4-bucket clean benchmark) pending. See src/evaluate_benchmark_metrics.py and runai/run_benchmark_eval.sh.

Related artifacts

SFT model: jdecim/PIT-4B-FT-202212-earnings-SFT
Dataset (SFT splits, DPO pairs, benchmark): jdecim/pit-earnings-call-qa

Limitations

2048-token context limit.
Strict 2022-12 PIT horizon.
DPO is a preference signal, not a factuality guarantee — the model can still hallucinate on out-of-distribution contexts.

Citation

Built atop the PIT-4B-FT family from Kelly et al. (2026) "Scaling Point-in-Time Language Models". DPO methodology: Rafailov et al., Direct Preference Optimization (2023). Preference-data inspiration: NurErtug/finance-dpo-pairs-verified.