Qwen2.5-3B-Instruct, QLoRA + DPO, trained to synthesize a final answer from
multi-tool evidence (RAG/WEB/STOCK) for an investment research agent.
Trained on 402 preference pairs (396 on-policy + 6 hand-crafted), scored
against a rule-based reward function checking citation format, fabrication,
conflict-flagging, and coverage.
Results
Preference accuracy on held-out pairs: 72.5% (up from 55% baseline/smoke test)
Single-source RAG synthesis: solid (3.00/3.00 on eval scenarios)
Multi-evidence synthesis (mixing RAG+WEB+STOCK): 2.1-2.4/3.00 - known gap,
model occasionally fabricates page-style citations for WEB/STOCK evidence
that structurally has no page numbers, or drops citation formatting under
bullet-point output structure.
Known limitation
System prompt currently asks for [Company Year, Page X] citation uniformly
across all evidence types, which WEB/STOCK evidence can't actually satisfy.
v2 fix: restrict that citation format to RAG evidence only in the prompt.
